Introduction to agents: roles, run cycle, determinism, idempotency

1. What an agent is and why you might want one

Agents are not a mandatory part of the ChatGPT App: you can build as many great apps as you want without using LLM agents. However, I have three strong reasons to tell you about them.

Agents are an excellent way to add intelligence to your app’s backend. Smart gift selection, analysis of a user’s textual preferences. Complex scenarios of searching, analyzing, processing, and summarizing—this is all very easy to do with LLM agents.

ChatGPT has released its Agents SDK for TS and Python. It’s very good. Agent orchestration comes out of the box. Not just one agent but a whole team can tackle a complex task. This is a very promising direction.

There’s also a learning objective. ChatGPT calls mcp-tools the same way LLM agents call their tools. Once you understand how LLM agents work, it becomes clear how to, for example, build a state machine on the model side inside your app. Likewise, studying the Agents SDK gives you a sense of how the ChatGPT SDK will work in the future.

So let’s get started.

What is an LLM agent

If the ChatGPT App is the nice, convenient “front end” of your service inside ChatGPT, and the MCP server is the “engine” with tools and business logic, then an agent is like a smart dispatcher that can:

• read the goal;
• decide on its own which tools to call and in what order;
• ask for additional data if needed;
• repeat steps if there were errors;
• reach a clear final result.

In wording close to the Agents SDK documentation, an agent is a program. With access to an LLM and a set of tools, it can autonomously plan steps to achieve a goal and execute them via tool calls.

If we draw a parallel with what you already have:

• In a typical ChatGPT App, the ChatGPT model orchestrates calls to your MCP tools directly.
• An LLM agent on the backend also has a task description and a set of tools and decides by itself which tools to call, how many steps to take, when to stop, and what result to return.

In the context of our GiftGenius, this might look like:

• App without an agent: the model directly calls searchGifts, then filterByBudget, then getDetails—re-thinking each time;
• App with an agent: ChatGPT calls an MCP tool, and the backend gives the agent a task: “Find the top 5 gifts for this profile.” The agent makes several steps: gathers extra information, calls different search tools, filters, sorts, builds final cards, and returns a ready structured answer.

ChatGPT and an LLM agent on the backend are like a company’s director and an employee. ChatGPT has much more freedom: it talks to the user and decides which strategic tasks to launch (call MCP tools). The LLM agent works only on the backend, doesn’t interact with the user, but it also “thinks” and can call its own tool. Think of it as a lightweight ChatGPT.

2. What an agent consists of: LLM, instructions, tools, and state

It’s convenient to think about an agent as several layers.

First, under the hood it’s still the same LLM. It could be GPT‑5.1 or another model used by the Agents SDK. It generates text, plans steps, chooses tools—in short, it handles the “thinking,” but now in your orchestration context.

Second, on top of the model you have instructions. This is the agent’s system prompt that sets its role, boundaries, style, and ways to use tools. You already did something similar for the ChatGPT App, but now it’s applied to a separate agent.

Third, there’s a set of agent tools. These could be:

• TypeScript functions (classic function calling);
• HTTP/REST tools;
• wrappers around your MCP tools so the agent can access the same backend as the ChatGPT App;
• OpenAI’s own built‑in “hosted” tools (for example, web search, if you enable them).

And finally, there are rules for handling state and steps: how session state is stored, how intermediate results are saved, how loops are limited. We’ll go deeper into this in the next lecture on memory and state, but even now it’s useful to keep in mind that an agent is not a “one‑off request” but potentially a long process that persists progress.

If you look at this through a TypeScript developer’s eyes, you mentally get an object roughly like this (pseudocode, close to the Agents SDK in TS):

const giftAgent = new Agent({
  model: "gpt-5.1",
  systemPrompt: giftAgentPrompt,
  tools: { searchGifts, filterGifts, checkoutDraft },
  // here as well — settings for memory, step limits, etc.
});

We won’t dive into exact APIs now; the point is the picture: model, instructions, tools, and behavior settings in one place.

3. Message roles: system / user / assistant / tool in an agent’s world

You’re already familiar with the classic roles system, user, assistant, and tool from Chat Completions. In the Agents SDK they remain, but get a slightly more applied meaning.

The system role sets the agent’s personality and mission. For a GiftGenius agent, for example, it could be: “You are a gift selection agent. Your task is to find 3–7 relevant gift options based on the recipient profile and budget with a minimum number of steps, then prepare structured JSON for the widget.” This is also where you set constraints: what it must not do (e.g., not make real purchases without a separate step) and how it should work with tools.

The user role in the agent context is not necessarily a “live person.” Most often it’s a “job” for the agent: a goal formulated by your App, a service, or another agent. For example, the ChatGPT App can invoke the agent with a user message: “Pick 5 gift ideas for a developer colleague, budget $50, occasion—birthday.”

The assistant role is what the model “says” inside the agent. These can be intermediate thoughts and plans or the final answer. Your task is to craft the system prompt so that these messages are useful and logged if needed.

The tool role (or its analogues in a particular SDK) describes tool call results: “50 products found via MCP,” “API returned a timeout error,” “DB returned the user profile.” These messages, together with assistant messages, form the history of an agent’s run cycle.

It’s handy to summarize this in a small table:

system

user

assistant

tool

This structure matters because it’s exactly what the run cycle—the star of today’s lecture—is built on.

4. How the LLM calls functions on your backend

When you’re used to the “question–answer” mode, it feels like the LLM works by a simple scheme: text comes in → the model outputs text. In reality, under the hood it’s a bit more complex, and that’s precisely why function calling works.

The model doesn’t receive a single question; it receives a list of messages—a conversation history. All prior turns are there: system instructions (“who you are and what’s allowed”), your messages, the model’s past answers, tool results. At each step the model looks at this entire thread like a chat log and decides: “What next message should be appended to the end?”.

This is the key idea: an LLM always takes one step—appends the next message to the end of the history. It doesn’t “change the past” and doesn’t edit old messages; it just continues the list. You write a question; the model answers. You add a second question; the model answers again, but with the whole history (all messages) in mind.

Function calling is built on the same principle. Instead of directly “running a function,” the model does the following:

• it sees the list of available tools and their descriptions along with the conversation history;
• it decides: “It’s more logical now not just to answer with text but first to call such-and-such tool.”;
• and as the next message it appends to the history not a regular textual answer but a special message of the form “I want to call this tool with the following arguments.”

Then it’s not the model but your backend that reads this new message at the end of the history, understands that it’s a request to call a function, and calls the desired tool. It then adds another message to the history—with the tool result—and sends the full list of messages back to the model. The model again looks at the entire thread and appends the next step: either another call or a final human‑readable answer.

That is:

• for ordinary Q&A: “next message” = a textual answer;
• for function calling: “next message” = an instruction to call a function or the response after using a function.

There’s no separate magical “call a function” command—it’s simply a special kind of next message that the model appends to the end of the chain.

The model doesn’t call your backend functions via a public API. It simply “writes in the chat” that it wants to call a function with parameters. Your backend calls the local function, and its answer is appended to the chat. And it all starts again.

5. The agent’s run cycle: how it “thinks” step by step

An LLM agent is essentially an object/algorithm on your server that runs the agent run cycle—an extended loop of “question → think → possibly act → think again → … → final answer.” In OpenAI’s documentation this is sometimes called the agent loop or the ReAct pattern (Reason + Act + Observe).

Conceptually, a single agent run looks like this:

1. The agent receives input: system instructions, the job (user message), and possibly the current state.
2. The model generates a step: either a textual answer, or plans, and a decision to call one or more tools.
3. If the model chose a tool call, the agent invokes the corresponding tool in code (this could be a local function, an MCP tool, an HTTP request, DB access, etc.).
4. Tool results are added to the history as tool messages.
5. The cycle returns to the model with new context. The model decides what to do next: continue planning, call another tool, or finish with a final answer.
6. When the model explicitly or under stopping conditions ends the run, the agent returns the final result to the caller.

As a small diagram, it could look like this:

flowchart TD
    A[Run start: goal + system] --> B[Call the model]
    B --> C{Does the model want
to answer with text
or call a tool?}
    C --> D["Text answer
(assistant)"]
    D --> E{Is the task complete?}
    E -->|Yes| F[Final result]
    E -->|No| B

    C --> G["Tool call
(call description)"]
    G --> H[Invoke function / MCP / HTTP]
    H --> I["Tool result
(tool message)"]
    I --> B

If we translate this into simplified TypeScript pseudocode (far from the real API but logically accurate), it would look like:

async function runAgent(goal: string) {
  let context = buildInitialContext(goal);

  while (!isFinished(context)) {
    const decision = await callLLM(context); // agent step

    if (decision.type === "tool_call") {		// call a function?
      const toolResult = await callTool(decision.tool, decision.args);	// invoke a local function
      context = appendToolResult(context, toolResult);		// append the result to the end of the list
    } else {
      context = appendAssistantMessage(context, decision.message); 
    }

    enforceLimits(context); // step/time/loop limits
  }

  return extractFinalResult(context);
}

The Agents SDK takes care of most of this routine: storing history, marshalling tool calls, retry logic, etc. You configure it and implement the tools.

Run vs step

It’s important to distinguish two concepts:

• run—a single agent execution for some goal: “select gifts for this occasion”;
• step—one step of the run cycle: a specific model call that can result in a text answer or a tool call.

In monitoring you will see multiple steps inside a single run, and safety/cost limits are often set either “per run” or “per step.”

Now that it’s clear how an agent lives within a single run and moves through the run cycle, let’s look at where this is actually worth doing and where simple tools are enough.

5. Where a GiftGenius agent is needed and where it’s overkill

Before rushing to write an agent for everything, it’s useful to ask yourself an honest question: “Do we even need one here?”

A good scenario for an agent is a multi‑step task with branching, retries, and logic that’s inconvenient to keep solely in prompts.

In GiftGenius, such a task could be a “smart gift selection wizard” that:

• can ask for important details (recipient’s gender, hobbies, closeness level);
• can access multiple product sources (different vendors via MCP tools);
• filters and ranks results;
• retries on source errors or follows a fallback path;
• returns not just a list of texts but a structured list of candidates with explanations and links to SKUs from the product feed.

Here the agent is truly useful as an “orchestrator,” especially if later you want to add a voice/Realtime scenario or complex commerce (ACP).

But for a simple getGiftDetails(giftId) call, you don’t need an agent: a regular MCP tool invoked directly from ChatGPT fully covers the case. Same for basic “single‑step” scenarios like “describe this gift based on the product card text.”

A sound rule of thumb: if you can describe the scenario as “one reasonable tool,” you probably don’t need an agent. If you start explicitly laying out a multi‑step workflow with checks and retries, chances are an agent will serve you well.

6. Determinism: how to make agent behavior predictable

Determinism in the world of LLM agents is tricky. Theoretically, with identical input and identical settings you want to get the same plan and the same sequence of tool calls. In practice the model remains stochastic, but you have several levers to control predictability.

First, the classic: temperature and other generation parameters. The lower the temperature, the less creativity and the more “obedience.” For a gift selection agent, you’ll likely want some freedom but not too much, otherwise the model will invent a new way to call the same tool every morning.

Second, clear system instructions. If you vaguely describe behavior like “you can call different tools and do what you want,” don’t be surprised when the agent jumps between APIs or tries to answer “off the top of its head.” It’s much better to spell out exactly when to call a tool, what parameters are allowed, how to interpret errors, and in which cases to finish the job.

For example, the GiftGenius agent’s system prompt could include the fragment:

If you don’t have a complete recipient profile (age, gender, occasion, approximate budget),
first ask clarifying questions through the user-facing channel and wait for answers.
Only then call the search_gifts tool with a completed profile.
Do not invent products; always rely on tool results.

Such instructions reduce decision variability and make behavior more deterministic.

Third, the design of the tools themselves. If you have three tools that “roughly do the same thing” to search for gifts, the model will inevitably choose different ones from time to time. It’s better to design tools with clear, non‑overlapping responsibilities and document that in their descriptions.

Finally, you can use guardrails—rules and schemas that validate the agent’s actions and the model’s outputs. The Agents SDK has built‑in support for checks and constraints, including on output structure. If the model tries to generate something off‑schema, you can gently correct it or even repeat the step.

Mini example: pin down the output format

Suppose you need the agent to always return JSON with a gifts field, and inside it objects with id, title, and score. You can:

• describe this schema at the agent level;
• require the final output to match it;
• if violated—repeat the step or return a safe error.

Pseudocode:

const giftResultSchema = z.object({
  gifts: z.array(z.object({
    id: z.string(),
    title: z.string(),
    score: z.number().min(0).max(1),
  }))
});

// In the agent config
const agent = new Agent({
  /* ... */
  outputSchema: giftResultSchema,
});

When the model tries to return something odd, the runner will report a validation error, and you can either re‑prompt the model or log the incident.

7. Idempotency: why your agent might call your API twice

If determinism is about “the same plan for the same inputs,” then idempotency is about safe retries. In the context of agents, it’s critical for two reasons.

First, you get one more retry layer: not only HTTP clients and load balancers can retry, but the agent itself can decide to repeat a tool call if it received an error or a partial result. Second, real production scenarios add webhooks, queues, and streaming channels—and you can accidentally process the same logical step multiple times.

You’ve already discussed idempotency at the MCP tool level: don’t charge twice, don’t create the same order twice, use idempotency keys in requests. Now it’s the same, multiplied by the agent’s multi‑step nature.

Imagine GiftGenius has a create_checkout_session tool that creates a checkout draft in ACP/Stripe from a list of selected gifts. If the agent decides to repeat this call due to a network error, you really don’t want two separate orders and two charges.

Therefore, you should:

• come up with an external idempotency key for each logical action (for example, runId + stepIndex or an explicitly generated checkoutDraftId);
• pass it into your backend/ACP endpoint;
• on the backend, check whether you have already processed this key and return the saved result instead of executing again.

Pseudo‑example in TypeScript:

async function createCheckoutDraft(runId: string, payload: DraftPayload) {
  const key = `gift-checkout-${runId}`;

  const existing = await findDraftByKey(key);
  if (existing) return existing;

  const draft = await stripe.checkout.sessions.create({
    /* ... */,
    idempotencyKey: key, // or your own layer on top
  });

  await saveDraftWithKey(key, draft);
  return draft;
}

Now, even if for some reason the agent calls this tool twice with the same runId, your code remains idempotent: the same logical step → the same actual result.

“Check first, then act”

A second common idempotency pattern is to check state before acting. For example, before creating an order, verify whether an order with the same clientReferenceId or the same set of parameters already exists. This is especially convenient in long workflows where the agent may “forget” it already did something in a previous step.

Safe mode/Fake mode

During development it’s useful to have a “safe mode” for dangerous tools: instead of the real action they only log what would have been done and return a fake result. For agents this is a convenient way to run the run cycle in a production‑like environment without risking money or data.

8. Mini practice: describe the GiftGenius agent in plain language

We’ve discussed the run cycle, determinism, and tool idempotency. Let’s step away from code for a minute and check how this all comes together in a real‑life scenario.

It’s useful now to do a small exercise on paper (or mentally), without code.

Imagine you’re describing a simple agent:

• system

  : you are a gift‑selection assistant; you always clarify important details, never invent products, and rely only on tool results.

• user

  : I want a gift for a colleague up to $50.

Describe in words which steps such an agent should take.

A typical scenario might look like this.

1. First, the agent checks whether the information is sufficient. If not, it asks clarifying questions: what the colleague roughly does (designer, developer, manager), any taboos (alcohol, gag gifts), delivery constraints. The answers go either into session state or the tool call parameters.
2. Then the agent calls the search_gifts tool with a filled profile: “developer colleague, budget 50, category—gadgets and office.” The tool returns a list of candidates with prices, categories, and product IDs.
3. Next the agent can call an additional tool, filter_gifts_by_constraints, if it turns out some products can’t be shipped to the required region, or it can filter within its prompt. After that it sorts options by relevance and price, possibly adding short comments (“good if the colleague likes coffee,” “a solid option for remote work”).
4. Finally, the agent prepares a structured final response for the ChatGPT App: a list of 5–7 gifts with brief descriptions, usage tips, and links to Checkout (or the next step—creating a checkout draft).

Where are tool calls needed here? Obviously in product search and filtering, availability checks, and creating a checkout draft. Which steps must be idempotent? First and foremost everything related to orders and money—creating the checkout draft and possibly writing the history to the DB.

9. Common mistakes when getting started with agents

Mistake #1: treating the agent like “a second ChatGPT without constraints.”
Sometimes you want to just give the model yet another prompt and call it an “agent.” The result is something that generates a lot of text, calls tools chaotically, and is hard to control. To avoid this, clearly define the agent’s role in system, restrict the tool list, and think of it as an orchestrator with a concrete mission, not as “another universe of text generation.”

Mistake #2: lack of idempotency in tools.
Developers often move old HTTP handlers under an agent “as is,” not accounting for the runner now automatically retrying calls. With payments and orders, this can lead to very unpleasant consequences. The right approach is to design tools so that a repeated call with the same logical key does not trigger the action again.

Mistake #3: overly creative model settings.
High temperature is great for writing toasts and poems, but for an agent that must reliably orchestrate multi‑step processes it leads to unpredictable behavior: the model will choose different tools each time, generate different plans, and sometimes even forget it has tools. Treat agents as “service” entities and keep them in a stricter mode.

Mistake #4: one “do‑everything” tool.
Sometimes you want to make a single universal tool like execute_any_sql or do_anything_with_orders and then hand it to the agent. Combined with LLM creativity, this is an almost guaranteed security risk. It’s much better to have several narrow, specialized tools with clear contracts and permissions than one “divine” tool with full rights to everything.

Mistake #5: no explicit criteria for ending a run.
If you don’t tell the agent when to stop, it can drift into infinite or semi‑infinite loops: check results again, ask the user again, try to call the tool again for the same error. This often shows up only under load when a dependency is unstable. The right way is to set limits on the number of steps, run time, and retries for the same error, and to describe in system that the agent must “honestly give up” when it has exhausted reasonable options.

Mistake #6: stuffing everything into agent state.
Since the Agents SDK simplifies working with session state, it’s tempting to put everything there: large documents, raw logs, sensitive data. This bloats context, increases cost, and creates security risks. Agent state should store only what’s truly needed to continue; everything else belongs in databases, logs, and other layers, with privacy in mind.

Mistake #7: trying to use an agent where a simple MCP tool is enough.
Sometimes developers start with an agent, even if the task is just to call one function and return the result. This adds complexity where it isn’t needed: a run cycle, state, extra logs, and potential failure points. If the scenario fits a single tool call without a complex workflow, keep it that way and introduce an agent only when true multi‑step logic appears.

1

Task

ChatGPT Apps, level 12, lesson 0

Locked

Agent message feed (roles + tool-call vs tool-result)

Agent message feed (roles + tool-call vs tool-result)

Open

1

Task

ChatGPT Apps, level 12, lesson 0

Locked

Agent mini-runtime (run cycle: model → tool-call → tool → final)

Agent mini-runtime (run cycle: model → tool-call → tool → final)

Open