MCP Gateway: why you need it between ChatGPT and your services

1. Why have yet another layer at all?

Almost everyone starts the same way: there is one MCP server, it describes a couple of tools, ChatGPT talks to it directly over HTTPS, and everything seems great. The notional architecture looks like this:

ChatGPT  →  your MCP server  →  database / external APIs

At the “pet project” stage this is indeed a reasonable option. But as soon as the application starts gaining functionality and the development team grows, problems surface quickly.

First, the MCP server turns into a “God object.” It simultaneously contains gift recommendation tools, checkout, analytics, and something else from the series “hey, let's also cram reporting in here.” Different parts of the code have different SLAs and different security requirements, but they are glued into one process.

Second, ChatGPT and other clients are forced to know the topology of your services. If after six months you add another MCP server for commerce, you will have to reconnect clients, change configs and descriptions. Instead of “a single entry point,” you end up with a zoo of URLs.

Third, it becomes unclear where to implement what is common to all services: authentication, logging, metrics, rate limiting, token verification, localization, and regional routing. If you scatter this across all MCP/Agent services, you will get a lot of duplication and different behavior in different services.

To break this coupling and at the same time hide internal complexity from ChatGPT, MCP Gateway comes into play — a network gateway and a single entry point for all MCP traffic.

2. What is an MCP Gateway in the context of a ChatGPT App

Formally, an MCP Gateway is a proxy layer and a single entry point between MCP clients (ChatGPT, MCP Jam, your internal tools) and a set of your backend services — usually REST/HTTP APIs, microservices, Agents services, commerce backends, and so on.

The gateway implements the MCP protocol to the outside (it looks like a single MCP server to ChatGPT), and inside it simply calls regular REST endpoints over HTTP/gRPC.

For a tools/list request, the gateway does not proxy the call further but returns its own list of tools: it is either hardcoded or assembled from configuration. Each tool is bound to a specific REST endpoint and data schema. For a tools/call request, the gateway takes the tool name, finds the corresponding REST route, and invokes it via a fetch/HTTP client.

Schematically, it can be represented like this:

flowchart LR
    ChatGPT["ChatGPT / model"] --> |MCP JSON-RPC| Gateway["MCP Gateway<br/>(single MCP server)"]
    Gateway --> GiftAPI["Gift REST API<br/>/ gifts microservice"]
    Gateway --> CommerceAPI["Commerce REST API<br/>/ ACP / payments"]
    Gateway --> AnalyticsAPI["Analytics Service<br/>/ events and metrics"]

For ChatGPT this is a single server: one URL, one set of tools, one event stream. For you — a flexible traffic routing point to different cold and hot services.

3. MCP Gateway in the GiftGenius architecture

To be less abstract and show the gateway “in a live system,” let’s continue our GiftGenius example — an app that recommends gifts and can place orders via ACP/Instant Checkout.

In the simple version we had one MCP server that handled both suggest_gifts and checkout_start. Now that the app has grown, we split responsibilities:

• Gift REST API — search and gift recommendations, working with the catalog and feed (a regular HTTP/REST service).
• Commerce REST API — ACP, checkout sessions, order statuses, integration with the payment provider.
• Analytics Service / REST API — event and metrics collection (which recommendations are opened, what is purchased).
• A separate Agents service (if needed) — complex multi-step scenarios. It is also available via HTTP/REST, not via MCP.

The MCP Gateway becomes a single entry point for all these components. It:

• for tools/list requests returns a single list of tools that it defines itself: each tool is bound to a specific REST endpoint of one of the services;
• for tools/call requests looks at the tool name (params.name), determines via a routing table which REST service to call, and invokes the corresponding HTTP method (via fetch, axios, etc.).

If a tools/call comes in with the name suggest_gifts, the gateway calls the corresponding REST endpoint in the Gift REST API. If it’s checkout_start, the request goes to the Commerce REST API.

A small TypeScript pseudocode in an Express style might look like this:

// Very simplified MCP request handler
app.post("/mcp", async (req, res) => {
  const mcpReq = req.body as { method: string; params?: any };
  const ctx = buildContextFromHeaders(req); // auth, locale, etc.

  const toolName = mcpReq.params?.name;
  const backendRes = await callBackend(toolName, mcpReq, ctx);

  res.json(backendRes);
});

Inside pickBackend you can rely on the method name, the tool name, the user’s locale, and even the service version (for canary and blue/green releases, which we’ll discuss later in the module).

4. Responsibilities of the MCP Gateway: what it definitely does

We looked at how the gateway fits into the GiftGenius architecture. Now let’s explicitly fix what responsibilities it has as a separate layer, regardless of the specific application. It’s important to think of the gateway as a network and cross-service layer. Its job is not to handle gift business logic, but to solve the infrastructure tasks around it.

Request routing

The first role is a router. The gateway receives an MCP request and, based on its content, the user context, and its configuration, selects the target service.

For example, in GiftGenius you can set up a simple routing table:

const TOOL_ROUTES: Record<string, "gift" | "commerce" | "analytics"> = {
  suggest_gifts: "gift",
  get_similar_gifts: "gift",
  checkout_start: "commerce",
  get_order_status: "commerce",
  log_event: "analytics",
};

And then use it like this:

function pickBackend(req: McpRequest, ctx: GatewayContext): Backend {
  if (req.method === "tools/list") return "aggregator";
  if (req.method === "tools/call") {
    const toolName = req.params?.name;
    const group = TOOL_ROUTES[toolName] ?? "gift";
    return group === "commerce" ? commerceBackend : giftBackend;
  }
  return giftBackend;
}

In our case, giftBackend, commerceBackend, analyticsBackend are ordinary REST services: each has a base URL ("https://gift-api.internal", "https://commerce-api.internal", …). The gateway does not forward MCP inside; it translates the MCP call into an HTTP request to the needed REST endpoint.

Perimeter authentication and authorization

The second key function is perimeter protection. The gateway is a convenient place to validate a token, understand who the user is, which organization they came from, and what permissions they have.

It can, for example, accept an OAuth token from ChatGPT or your MCP auth server, validate it (preferably using a vetted library rather than homegrown cryptography), and turn it into a neat context object:

type GatewayContext = {
  userId: string | null;
  tenantId: string | null;
  locale: string;
};

function buildContextFromHeaders(req: Request): GatewayContext {
  const token = req.headers["authorization"]; // "Bearer ..."
  const claims = token ? verifyJwt(token) : null;

  return {
    userId: claims?.sub ?? null,
    tenantId: claims?.tenant ?? null,
    locale: (req.headers["x-openai-locale"] as string) || "en-US",
  };
}

Internal backend/REST services then don’t have to parse raw HTTP headers and tokens; they receive a normalized context with userId, tenantId, and locale. MCP docs explicitly recommend: don’t implement token validation “from scratch,” use proven libraries and short-lived tokens.

Logging, tracing, and metrics

The third role is observability. The gateway sees all incoming MCP requests and responses, so it’s the ideal place to set a correlation ID, log tool parameters (without sensitive data), record response time and status.

The simplest idea:

app.use((req, res, next) => {
  const requestId = crypto.randomUUID();
  (req as any).requestId = requestId;

  const start = Date.now();
  res.on("finish", () => {
    const ms = Date.now() - start;
    console.log(
      `[${requestId}] ${req.method} ${req.url} -> ${res.statusCode} in ${ms}ms`
    );
  });

  next();
});

Later in the observability module you’ll be able to send this data not just to console.log, but to a structured store and build dashboards on top of it.

Basic load control

The fourth, but also important task is initial load control. The gateway is a convenient place to put call counters by users, organizations, tools, and endpoints, to prevent a single “rogue” client from burning your cluster and model budget.

In this module we only capture the idea for now: rate limiting and queues live at the gateway level; implementation details (Redis, token buckets, leaky buckets) will be covered in the next lecture on perimeter protection.

Context enrichment for requests

Finally, the gateway is a good place to turn the raw context from an MCP client into tidy arguments for internal tools.

For example, ChatGPT can pass the user’s locale via openai/locale and _meta["openai/userLocation"]. The gateway can:

• choose the needed regional service (ru server, en server, etc.);
• add locale to the tool call arguments, even if the tool didn’t explicitly request it in the JSON Schema (for example, as an optional field).

Conceptually:

function enrichToolArgs(args: any, ctx: GatewayContext) {
  return {
    ...args,
    locale: args.locale ?? ctx.locale,
    tenantId: ctx.tenantId,
  };
}

As a result, the Gift API immediately receives a “rich context” and can, for example, fetch Russian gift descriptions for "ru-RU" and English ones for "en-US".

5. What an MCP Gateway should NOT do

When developers get a “magical place through which everything passes,” there’s a natural desire to shove everything that used to live in separate services into it. This is how a gateway risks turning into a monster.

There are several things that generally should not live in this layer.

First, complex business logic. Gift selection, discount rules, shipping cost calculation, ACP logic — all of that should remain inside specialized backend/commerce services. The gateway can, at most, do light pre-validation (e.g., check that a price is not negative), but it should not select SKUs or compute tax per region.

Second, long-lived user state. The gateway is a typical stateless service. It should scale horizontally, not rely on local memory, and be safe to restart. If you start storing, say, the state of a checkout wizard or temporary cart contents in it, you’ll quickly run into pain syncing between instances.

Third, service-specific functionality that makes more sense inside the backend services themselves (Gift API, Commerce API). For example, if the Gift backend wants to cache gift search results, let it do so itself, possibly using Redis. The gateway doesn’t have to know about this internal optimization. We’ll separately talk about perimeter protection, and we emphasize there: a gateway is about network and cross-service concerns, not about business rules for recommendations.

Fourth, heavy computations. If inside the gateway you start invoking LLM models, performing complex transformations and aggregations, it will stop being a “thin” front and turn into yet another fat backend that is hard to scale and debug.

6. Gateway, localization, and service versions

We covered the gateway’s core responsibilities and what not to put into it. Now let’s look at a couple of typical “advanced” tasks that are convenient to solve at this layer: localization and service versioning. Another interesting role of the gateway is smart routing by locale and service versions.

When ChatGPT calls your App, it already has some understanding of the user’s language (openai/locale) and, often, their geolocation (_meta["openai/userLocation"]). The gateway can use this information to send requests to the appropriate backend services.

For example, you can build an architecture of “one gateway — many monolingual backend servers”:

• ru-Gift API — Russian-only catalog and texts.
• en-Gift API — English-only.
• jp-Gift API — Japanese (when you decide to take over the world).

In this case, the gateway acts as the MCP server for ChatGPT and, based on locale and userLocation, selects the required internal service.

Conceptually:

function pickGiftBackendByLocale(ctx: GatewayContext): Backend {
  if (ctx.locale.startsWith("ru")) return giftRuBackend;
  if (ctx.locale.startsWith("ja")) return giftJpBackend;
  return giftEnBackend;
}

It’s also convenient to implement simple canary routing there. In this production architecture module we suggest using the gateway to send a portion of traffic to a new service cluster and the rest to the old one.

A very crude canary example:

function pickGiftBackendCanary(ctx: GatewayContext): Backend {
  const hash = hashUser(ctx.userId ?? "anonymous");
  const bucket = hash % 100;
  return bucket < 5 ? giftBackendV2 : giftBackendV1; // 5% of traffic goes to v2
}

This allows you to safely roll out a new version of the Gift API, watching metrics and errors without breaking the entire prod at once.

7. Typical architectures: from “all-in-one” to Gateway

Earlier in the course you already saw several variants of production architecture for a ChatGPT App. In this module we highlight three basic topologies that are sufficient in 90% of cases.

The first is “all-in-one.” The App widget (Next.js), MCP server, Agents logic, and a simple commerce backend live in one service, often in one repository and even in one Vercel app. The plus of this approach is almost no DevOps, simple deploy, minimal latency. The minus — it’s hard to scale individual parts, a single hot feature can take down the entire app, and boundaries between components are blurred.

The second is App + MCP Gateway + multiple backend services. Here the Next.js widget lives separately (e.g., on Vercel), and all MCP traffic goes through the Gateway, which routes requests to the Gift REST API, Commerce REST API, Agents service, ACP backend, and others. This is exactly the scheme we are analyzing now within GiftGenius, and it fits 90% of real production cases.

The third is the same, but across multiple regions (multi-region), with a global load balancer in front of the gateway. Then a user from Europe lands in the eu cluster, from the US — in the us cluster, and each region is built as “Gateway + multiple backend services.” This is a story for fairly large projects with a global audience.

For us it is important not so much to memorize all options as to get used to thinking of the gateway as a separate logical component of the architecture, even if at early stages its role is played by, say, a single MCP monolith or your App’s backend.

8. Where an MCP Gateway physically lives

Good news: an MCP Gateway is not necessarily a huge standalone service on Kubernetes. Most often it goes through several stages of maturity.

At the smallest scale, the gateway role can be performed by the MCP server itself. In this case you just need to carefully structure the code: move routing, authentication, and logging into one module, and tool logic into others. In this module we explicitly note that in small systems the gateway functions can live inside the MCP server or the App’s backend part (for example, in a Next.js API route).

The next step is a separate Node/TypeScript service. It can be an Express/Fastify app that listens on "/mcp" and calls into several internal HTTP services. For many teams this is a comfortable option: it fits well with familiar DevOps tooling.

Simplest skeleton of such a service:

const app = express();
app.use(express.json());

app.post("/mcp", handleMcpRequest); // all gateway magic lives here

app.listen(4000, () => {
  console.log("MCP Gateway listening on :4000");
});

At an even more mature stage, the gateway can be implemented on top of managed solutions: AWS API Gateway, Cloudflare Workers/Routes, NGINX/Envoy with routing config and Lua/JS scripts. It’s important to understand this is a change in implementation, not the concept. Architecturally, ChatGPT still talks to a single point, and the gateway handles all the details.

9. Mini example: a simple MCP Gateway for GiftGenius

We already looked separately at routing, context, and handling tools/list. Now let’s assemble everything into one clear but small example. Suppose we have two internal REST services:

• GIFT_API_BASE = "https://gift-api.internal";
• COMMERCE_API_BASE = "https://commerce-api.internal".

And a single gateway that ChatGPT will call at "https://gateway.giftgenius.com/mcp".

First, let’s define a couple of types:

type Backend = "gift" | "commerce";

type ToolRoute = {
  backend: Backend;
  method: "GET" | "POST";
  path: string;
};

const TOOL_ROUTES: Record<string, ToolRoute> = {
  suggest_gifts: {
    backend: "gift",
    method: "POST",
    path: "/api/gifts/suggest",
  },
  checkout_start: {
    backend: "commerce",
    method: "POST",
    path: "/api/checkout/start",
  },
  get_order_status: {
    backend: "commerce",
    method: "GET",
    path: "/api/orders/status",
  },
};

Next, implement backend selection and the call:

async function callBackend(toolName: string, mcpReq: McpRequest, ctx: GatewayContext) {
  const route = TOOL_ROUTES[toolName];
  if (!route) {
    throw new Error(`Unknown tool: ${toolName}`);
  }

  const base =
    route.backend === "gift" ? GIFT_API_BASE : COMMERCE_API_BASE;

  const url = base + route.path;

  // args that arrived in the MCP tools/call invocation
  const args = {
    ...(mcpReq.params?.arguments ?? {}),
    locale: ctx.locale,
  };

  const res = await fetch(url, {
    method: route.method,
    headers: { "content-type": "application/json" },
    body: route.method === "POST" ? JSON.stringify(args) : undefined,
  });

  const data = await res.json();

  // Wrap the REST service response into an MCP response
  return {
    result: data,
  } satisfies McpResponse;
}

And finally, the main handler that:

1. Builds context from headers (auth, locale).
2. Selects a backend.
3. Either aggregates tools/list or proxies tools/call.

app.post("/mcp", async (req, res) => {
  const mcpReq = req.body as McpRequest;
  const ctx = buildContextFromHeaders(req);

  if (mcpReq.method === "tools/list") {
    // The gateway declares the tools and their schemas itself
    const tools = [
      {
        name: "suggest_gifts",
        description: "Selects gifts by budget and interests.",
        inputSchema: { /* ... JSON Schema ... */ },
      },
      {
        name: "checkout_start",
        description: "Creates a draft order and starts checkout.",
        inputSchema: { /* ... */ },
      },
      // ...
    ];

    return res.json({ result: { tools } });
  }

  if (mcpReq.method === "tools/call") {
    const toolName = mcpReq.params?.name;
    const backendRes = await callBackend(toolName, mcpReq, ctx);
    return res.json(backendRes);
  }

  res.status(400).json({ error: { message: "Unsupported MCP method" } });
});

This is, of course, a simplified scheme, but it already shows the key ideas:

• the gateway does not know how the Gift API selects gifts;
• it merely routes carefully, enriches arguments, and, if desired, logs and rate-limits calls.

10. How all this ties into the module’s upcoming topics

The MCP Gateway is a foundational part of everything we’ll discuss in the remaining lectures of the module:

• Next we’ll talk about perimeter protection: rate limiting, queues, and backpressure. All of this primarily lives at the gateway level, because it sees all incoming traffic and can “cut off the excess” before requests overwhelm the backend.
• Then we’ll discuss resilience: timeouts, circuit breakers, bulkheads. The gateway is the point where it’s convenient to set timeouts for outbound calls centrally and to enable/disable problematic services (for example, temporarily “cut off” the Commerce API if it starts erroring).
• Finally, when talking about scaling and deployment, we’ll look at the gateway as a separate cluster that can be load-balanced, rolled out using blue/green and canary schemes, and rolled back independently of internal MCP services.

Essentially, if you previously thought “I have an App and an MCP server,” the scheme now expands to “I have an App, an MCP Gateway, several backend/Agents clusters, and a commerce backend.” And it’s precisely the gateway that allows you not to complicate the configuration for ChatGPT — it still sees a single MCP point.

11. Common mistakes when working with an MCP Gateway

Mistake #1: turning the gateway into a “business monster.”
A common trap: since everything goes through the gateway, why not add discount calculation, SKU selection, complex category rules, or promo code validation there. As a result, you get a fat service that is hard to scale and change, and you lose the point of splitting into Gift API, Commerce API, and other specialized components. It’s better to keep the gateway as a thin network layer and leave all domain-specific logic inside the specialized services.

Mistake #2: storing long-lived user state in the gateway.
The idea “let’s store the user’s cart right in the gateway’s memory” sounds tempting while you have a single instance. As soon as a second one appears, the pain begins: where is the real cart — on instance A or B? What happens after a restart? The gateway must remain stateless: at most a small cache of handshakes or configs, and all session/order state is stored in a database or specialized services.

Mistake #3: making ChatGPT aware of the internal service topology.
If you start exposing multiple API servers to ChatGPT (Gift API separately, Commerce API separately) and only use the gateway “in places,” you lose the main advantage: a single entry and centralized control. When the topology changes, you’ll have to change configuration in several places. It’s much simpler to configure the MCP Gateway once as the official endpoint for the App and hide all internal changes behind it.

Mistake #4: duplicating cross-service logic in all backends.
Sometimes teams try to implement authentication, rate limiting, logging, and localization in each REST service separately. As a result, the rights and limits policy in the Gift API is one thing, in the Commerce API — another, and the App behavior becomes unpredictable. The gateway exists to centralize these things: validate the token, determine tenant and locale, log the call, apply limits, and then go to the specific service.

Mistake #5: overloading the gateway with heavy computations and LLM calls.
Technically nothing prevents you from calling yet another LLM model from the gateway, doing complex aggregations or long batch operations. But this quickly turns it into another heavy backend that is impossible to scale and isolate cleanly. The gateway should remain fast and predictable: at most light transformation and routing. All heavy work — inside REST services or into queues/workers, which we’ll discuss later in the module.

Mistake #6: overcomplicating the infrastructure too early.
The opposite extreme is to immediately build a separate Kubernetes cluster, an NGINX stack, Cloudflare Workers, and lots of complex configs for a small educational App. There’s no point until you have real load and reliability requirements. It’s perfectly fine to start with a single MCP monolith or a simple Node gateway and only later, as you grow, move separate components into clusters and managed services.