Is an MCP gateway the same as an AI gateway or LLM gateway?

They are related. An LLM gateway sits between applications and model providers and handles routing, budgets and logging of model calls. An MCP gateway does the same for tool calls. Several products, including Walma AI Hub, combine both so policy, budgets and logs cover the whole agent.

Do I need an MCP gateway for a small team?

Usually not below ten or so people using one or two clients. Above that, or as soon as MCP servers touch customer data or production systems, the gateway is where governance becomes possible without slowing developers down.

Can an MCP gateway stop prompt injection?

It cannot make injection impossible, but it is the best place to reduce it: it can strip write tools from sessions that read untrusted content, flag results that look like instructions, and require approval for risky actions, all centrally.

Where should an MCP gateway run?

In the region where your data has to stay. For EU companies that means an EU cloud region, ideally inside your own tenant, so that prompts, tool arguments and logs never leave your legal jurisdiction.

What is an MCP gateway, and when do you need one?

An MCP gateway is a single control point between the AI clients your people use and the MCP servers those clients talk to. Instead of every client connecting to every server directly, they connect to the gateway. The gateway decides what is allowed, holds the credentials, keeps the traffic in your region, and writes the log.

If that sounds like an API gateway, that is the right intuition. It is the same architectural pattern, applied to agent traffic. This guide explains what it does, when a company actually needs one, and what to check before choosing one. It assumes you know what MCP is.

The problem a gateway solves

MCP without a gateway looks like this. Each developer configures servers in each client. Credentials live in environment variables and JSON files on laptops. Nobody has a list of which servers are in use. Tool permissions are whatever the client's approval dialog defaulted to. Logs are scattered across clients and servers and cannot be joined.

That is fine for one person. It does not survive the questions security, legal and finance ask when an agent touches production or customer data:

Which servers are approved, and who approved them?
Can this user's agent call delete_issue, send_message, run_query?
Where do the tokens live, and how do we revoke them?
Does our data leave the EU when the agent calls this server?
What did the agent do at 14:32 last Tuesday?

A gateway is the place where all five have an answer.

What an MCP gateway does

Server allowlist. The gateway exposes only the servers the company has approved, at pinned versions with reviewed tool descriptions. Clients cannot reach anything else through it. This closes the tool-poisoning and typosquat problems described in our MCP security guide.

Per-user and per-team tool policy. The same server can look different to different people. A developer sees read and write tools on GitHub; an analyst sees read-only. The Jira server's delete_issue is hidden unless a policy grants it. Policies live in the gateway, so there is nothing on the client for a user to switch off.

Credential injection. The client authenticates once to the gateway. The gateway holds the OAuth clients and API keys for each server and attaches the right, minimally scoped credential to each call. Developer laptops stop being token stores.

Approval and risk rules. Write, send, delete and payment tools can require a human approval that the gateway records. Sessions that read untrusted content (public issues, web pages, inbound email) can be run with write tools removed.

Result inspection. Because every tool result passes through, the gateway can flag content that looks like injected instructions before it reaches the model, and can redact secrets or personal data on the way out.

Regional hosting. A gateway in your own EU region, ideally in your own cloud tenant, means prompts, tool arguments and logs are processed under your jurisdiction. Internal MCP servers can run next to it, so they never need a public endpoint.

One log. Every call, with user, client, server, tool, arguments, approval decision and timestamp, in one place, exportable to the SOC.

When you need one

A rough rule from the rollouts we have done:

Situation	Gateway?
One developer, a few local servers, personal projects	No
A team under ten, one client, servers with read-only scopes	Optional
Multiple clients (Claude, Cursor, Copilot) across teams	Yes
Any MCP server that reaches customer data or production	Yes
Regulated sector, GDPR transfer questions, audit requirements	Yes, in your region
Internal MCP servers wrapping ERP, CRM, data warehouse	Yes, and host them behind it

The trigger is rarely security alone. It is usually the moment someone in legal or the CISO's office asks "where does this data go?" and nobody can answer.

What to look for

Not every product called a gateway does all of the above. Questions worth asking:

Does it speak MCP natively? Some products proxy HTTP and stop there. A real gateway understands tool lists, can filter tools per user, and can inspect results.
Does it cover the model calls too? An agent's risk surface is model plus tools. A gateway that also routes LLM traffic can enforce budgets, model allowlists and logging in the same policy. Products that only do one half leave a gap.
Where does it run? Vendor cloud in the US, vendor cloud in the EU, or your own tenant. For most European companies only the last two are acceptable, and the last one is the one that satisfies data protection officers without a debate.
Can it host internal servers? If you are going to write MCP servers for your own systems, they should run behind the gateway, not on the public internet.
Is policy enforced server-side? If a user can edit a config file on their laptop and bypass the policy, it is not a policy.
What does the log contain, and can you export it? Ask to see a real log line.
How does it handle credentials? Per user, per server, minimally scoped, revocable centrally.

MCP gateway vs LLM gateway vs AI gateway

The vocabulary is still settling.

An LLM gateway proxies calls to model providers: routing, failover, budgets, logging of prompts and completions.
An MCP gateway proxies tool calls between clients and MCP servers.
An AI gateway is the umbrella term, used by different vendors to mean either or both.

For an agent, both halves matter. A model policy without a tool policy still lets the agent act freely; a tool policy without a model policy leaves cost and data residency of the model calls unmanaged. Products that combine the two, with one identity, one policy engine and one log, are what most companies end up wanting.

How Walma does it

Walma AI Hub is an AI gateway in the combined sense. It runs in the customer's own Azure tenant in an EU region and sits in front of Claude, GPT, Codex, Cursor and the MCP servers the company approves. Policies for models, budgets and tools live in the same place, and every model call and tool call lands in the same log.

Internal MCP servers, for example a read-only server over the ERP, run next to the gateway inside the tenant. Developers install one signed client and get everything through one key. There is no client-side switch to turn policy off.

If you are working out whether you need this layer, a 20-minute walkthrough with an engineer is the fastest way to find out. Book one here.