Agent memory and state: session vs persistent, checkpoints

1. Why an agent needs separate memory at all

This part builds on earlier lessons of module 12 about agents: we already discussed the basic architecture, the run cycle, and tools; here the focus is on memory and state.

If you draw an analogy with a typical web app, the LLM here is a very smart CPU that can execute complex “text programs”. And the agent’s state is a combination of RAM and SSD: short‑lived session data and long‑term storage.

In a classic ChatGPT chat without your code, “memory” is just the list of messages system/user/assistant/tool that the model sees in the current request. For an agent, that’s not enough because:

• it needs to remember the progress of complex processes: which workflow step is already completed, which gift options have been filtered out, what the user has confirmed;
• it needs to know long‑term facts about the user: preferences, shipping address, history of past orders;
• it needs to survive failures: if the server crashes in the middle of picking a gift, the user shouldn’t have to re‑enter everything.

If you try to keep all of this only in the prompt context, you quickly hit the context window limit and keep paying tokens for the same facts. In parallel, you risk security: too much unnecessary data regularly gets sent to the model. That’s why agent systems always have explicit state — object(s) that live outside the message history and are managed by you.

2. Agent state layers: context, session, persistent

Let’s start by breaking it into layers. An agent typically has at least three levels of “memory”:

1. Message history (dialog context).
2. Session state.
3. Persistent state.

It’s important not to lump these concepts together.

Message history: “dirty memory”

The message history is what the LLM sees on each step: system instructions, user requests, agent responses, and tool results.

The advantage is that you don’t need to manage it manually — Agents SDK and the platform itself handle it via a Session/Conversation entity.

The downside is that it’s “dirty” memory: lots of extra words, repetitions, and accidental user data. This data is expensive in tokens and poorly structured. You don’t want a 200‑item list of already filtered gifts to be read to the model every time as plain text.

Session state: working short‑term memory

Session state is a structured object that lives within a single agent session/conversation. A good analogy for a frontend developer is useState or a Redux store that lives while the tab is open.

What lives there includes:

• the current process step (for example, "collecting_profile" or "filtering_candidates");
• a temporary cache of tool results;
• session parameters: locale, selected channel, flags like “the user accepted the terms”.

This state can be stored somewhere close to the agent — in Redis, in an in‑memory KV store, or via a built‑in SessionService of a specific SDK. The key point is: don’t try to stuff all of this into the system prompt.

Persistent state: long‑term data

Persistent state lives long: across sessions, checkouts, and devices. It’s the user’s profile, orders, saved wishlists, settings.

The key idea: the agent doesn’t “remember” persistent data magically; it “reads” it via tools — for example, get_user_profile, get_past_orders. No hidden global variables inside the agent; always an explicit call.

Comparison table

3. Session state: what it is and how to store it

Imagine that the GiftGenius agent runs a multi‑step process:

1. Collects the recipient’s profile.
2. Generates a list of candidates.
3. Filters them by budget, shipping, region.
4. Prepares the final selection.

Along the way it constantly interacts with the user and calls tools. Everything related to “the progress of this specific gift‑selection session” logically belongs in session state.

Example structure of GiftGenius session state

Let’s describe a session state type in TypeScript:

// State within a single "gift selection"
export type GiftSessionState = {
  step:
    | "collecting_profile"
    | "generating_candidates"
    | "filtering"
    | "finalizing";

  // recipient profile draft
  profileDraft?: {
    recipientType?: string;
    ageRange?: string;
    interests?: string[];
    dislikes?: string[];
  };

  // candidate product ids returned by backend
  candidateIds?: string[];

  // gift selected by the user
  selectedGiftId?: string;

  // technical flags
  locale?: string;
};

Here we deliberately don’t put full product objects — only their IDs. Let full data live in the DB; when needed, the agent calls the tool get_gift_details(gift_id).

Session in Agents SDK (conceptually)

Many agent SDKs have a session abstraction that takes care of storing message history and additionally lets you store structured state. In pseudo‑code it might look like this:

import { createRunner, OpenAIConversationsSession } from "@openai/agents";
// type GiftSessionState from the example above

const session = new OpenAIConversationsSession<GiftSessionState>({
  sessionId: "chatgpt-thread-id-or-random",
});

const runner = createRunner({ agent });

const result = await runner.run({
  session,
  input: "I want a gift for a colleague up to $50",
});

Under the hood, the SDK will:

• load the message history for this session;
• add the new user message;
• pass it to the model and the toolchain;
• save the updated state back (including session.state).

You work with session.state like with a regular object.

Updating session state from tools

A typical pattern: a tool that computes something also updates the session state. For example, a tool that builds the recipient profile from user answers:

export async function updateProfileDraft(
  session: GiftSessionState,
  answers: { questionId: string; value: string }
): Promise<GiftSessionState> {
  const next: GiftSessionState = { ...session };

  if (!next.profileDraft) {
    next.profileDraft = {};
  }

  if (answers.questionId === "interests") {
    next.profileDraft.interests = answers.value.split(",").map((s) => s.trim());
  }

  // ...other fields

  next.step = "generating_candidates";
  return next;
}

Here we pass not the entire Session from the SDK into the tool but only its state (type GiftSessionState). In real code, it makes sense to name such an argument, for example, currentState so you don’t confuse it with the Session object.

The agent calls this tool, receives a new state object, and saves it back into session.state.

4. Persistent state: the agent’s long‑term memory

Now remember that GiftGenius works not just in a single chat. The user can come back a week later, from another device, and say: “Pick a gift for the same friend as last time, but the budget has increased.”

This information should not live in session state but in persistent storage: in the database, commerce/ACP backend (the commerce layer to be covered in a separate module), etc.

Example persistent model

Let’s describe a recipient profile model in the DB (simplified, as a TypeScript type):

// What is stored in the DB
export type RecipientProfile = {
  id: string;
  userId: string;
  label: string; // "marketing colleague"
  recipientType: string;
  ageRange?: string;
  interests: string[];
  dislikes: string[];
  lastUsedAt: string; // ISO date
};

And a repository (for now, a simple Map — in reality you’d build an ORM/SQL layer):

const profiles = new Map<string, RecipientProfile>();

export const RecipientRepo = {
  async findByUser(userId: string): Promise<RecipientProfile[]> {
    return [...profiles.values()].filter((p) => p.userId === userId);
  },

  async save(profile: RecipientProfile): Promise<void> {
    profiles.set(profile.id, profile);
  },
};

The agent accesses persistent via tools

It’s important that the agent doesn’t poke the DB directly but works through tools. Then it remains a “clean” entity: LLM and planning logic in one place, integrations in another.

For example, the get_recipient_profiles tool:

export async function getRecipientProfilesTool(input: {
  userId: string;
}): Promise<{ profiles: RecipientProfile[] }> {
  const profiles = await RecipientRepo.findByUser(input.userId);

  return {
    profiles,
  };
}

In the tool description, the agent reads: “use this tool to get saved recipient profiles for the current user.” It decides on its own when exactly to call it.

Bottom line: session state is about the progress of a specific conversation and temporary caches that can be lost without pain. Persistent data is what must survive sessions and devices: profiles, orders, wishlists. The agent always reads them via tools, not by “magically remembering.”

5. How session and persistent work together in the run cycle

Now let’s put it all together. On each step of the agent run cycle we have a short sequence:

1. Load session state by sessionId.
2. If needed, load relevant persistent data from the DB via tools.
3. Form the context for the model (messages + structured state).
4. The model decides: answer with text or call tools.
5. Tools update either session state or persistent data (via the DB).
6. Save the new session state and, if needed, create a checkpoint (more on this below).
7. Return a response to the user.

Mermaid diagram:

flowchart TD
    A[Get user input] --> B["Load Session (state + messages)"]
    B --> C{Need persistent data?}
    C -- Yes --> D[Call tools: get_user_profile, get_recipient_profiles]
    C -- No --> E[Build context for LLM]
    D --> E
    E --> F["Call the model (LLM)"]
    F --> G{Does the model want to call a tool?}
    G -- Yes --> H[Execute tool, update session/persistent]
    G -- No --> I[Prepare final answer]
    H --> J[Create checkpoint and save Session]
    I --> J
    J --> K[Response to the user]

This cycle makes the agent’s behavior reproducible: at each step we explicitly know what the state was before the model call and what changed after.

6. Checkpoints: snapshots of the agent state

Checkpoints are saved “state snapshots” of the agent at an important step of the process. It’s not just “current session state,” but a recorded fact in external storage: at step N we had such state, such tool results, and such user input.

Why they’re needed:

• recovery after errors and crashes;
• user ability to “continue later”;
• debugging: reproducibility of a problematic run;
• audit: what exactly the agent did before, for example, creating an order.

What typically goes into a checkpoint

A typical checkpoint contains:

• identifiers: runId, userId, workflowId, stepId;
• the session state at that moment;
• key identifiers of persistent entities (for example, the id of an order draft);
• metadata: creation time, agent version.

It’s important not to pull in the entire text of the dialog. Below, in the memory hygiene section, we’ll return to what exactly is worth saving and what is not.

It’s better to store a reference to the Session or a brief summary of steps.

7. Designing checkpoints for GiftGenius

Let’s take our gift selection process and decide where we want checkpoints. For example:

• after collecting the recipient profile;
• after generating and initially filtering candidates;
• before offering the user the final choice.

Types for checkpoint and workflow state

Let’s describe the workflow state (very similar to GiftSessionState, but this is already a “snapshot” for checkpoints):

export type GiftWorkflowStep =
  | "profile_collected"
  | "candidates_generated"
  | "filtered"
  | "final_choice_made";

export type GiftCheckpoint = {
  id: string;
  runId: string;
  userId: string;

  step: GiftWorkflowStep;

  // the part of session state
  // we need for recovery
  sessionState: GiftSessionState;

  // which candidate ids were generated
  candidateIds: string[];

  createdAt: string; // ISO
  agentVersion: string;
};

Checkpoint storage (simplified)

As before, we’ll make a simple Map instead of a real DB:

const checkpoints = new Map<string, GiftCheckpoint>();

export const GiftCheckpointRepo = {
  async save(cp: GiftCheckpoint) {
    checkpoints.set(cp.id, cp);
  },

  async findByRun(runId: string): Promise<GiftCheckpoint[]> {
    return [...checkpoints.values()].filter((c) => c.runId === runId);
  },

  async findLastByUser(userId: string): Promise<GiftCheckpoint | undefined> {
    return [...checkpoints.values()]
      .filter((c) => c.userId === userId)
      .sort((a, b) => b.createdAt.localeCompare(a.createdAt))[0];
  },
};

Creating a checkpoint from agent code

Imagine a helper that we call after an important step:

import { randomUUID } from "crypto";

export async function createCheckpoint(params: {
  runId: string;
  userId: string;
  step: GiftWorkflowStep;
  sessionState: GiftSessionState;
  candidateIds: string[];
}) {
  const checkpoint: GiftCheckpoint = {
    id: randomUUID(),
    runId: params.runId,
    userId: params.userId,
    step: params.step,
    sessionState: params.sessionState,
    candidateIds: params.candidateIds,
    createdAt: new Date().toISOString(),
    agentVersion: "v1.3.0",
  };

  await GiftCheckpointRepo.save(checkpoint);
}

The agent can call it at the right moment:

await createCheckpoint({
  runId,
  userId,
  step: "filtered",
  sessionState,
  candidateIds,
});

For recovery we:

1. Find the last checkpoint by runId or userId.
2. Restore session.state from checkpoint.sessionState.
3. If necessary, pull fresh data from the DB by candidateIds.

8. Where to store session, persistent, and checkpoints technically

At the infrastructure level you usually have three different classes of storage:

• In‑memory — for dev/demos, fast but ephemeral memory.
• Redis (or another KV store) — for session state.
• Relational/NoSQL DB — for persistent data and checkpoints.

In‑memory store for local development

For local dev mode, a simple in‑memory store is often enough. For example, a mini store with TTL for sessions:

type StoredSession<T> = {
  state: T;
  expiresAt: number;
};

const sessions = new Map<string, StoredSession<GiftSessionState>>();

export function saveSession(sessionId: string, state: GiftSessionState) {
  sessions.set(sessionId, {
    state,
    expiresAt: Date.now() + 30 * 60 * 1000, // 30 minutes
  });
}

export function loadSession(sessionId: string): GiftSessionState | undefined {
  const stored = sessions.get(sessionId);
  if (!stored) return undefined;
  if (stored.expiresAt < Date.now()) {
    sessions.delete(sessionId);
    return undefined;
  }
  return stored.state;
}

This works great for local dev, but in prod with horizontal scaling (multiple instances) it won’t work anymore.

Redis for session state

In production, storing session state in Redis is convenient:

• fast writes/reads;
• TTL “out of the box”;
• accessible to all service instances.

Simplified pseudo‑example:

// Wrapper around the Redis client
export async function saveSessionToRedis(
  sessionId: string,
  state: GiftSessionState
) {
  const json = JSON.stringify(state);
  await redis.set(`session:${sessionId}`, json, "EX", 60 * 30); // 30 minutes
}

export async function loadSessionFromRedis(
  sessionId: string
): Promise<GiftSessionState | undefined> {
  const json = await redis.get(`session:${sessionId}`);
  return json ? (JSON.parse(json) as GiftSessionState) : undefined;
}

Postgres/another DB for persistent data and checkpoints

Persistent state and checkpoints are “serious” entities that need transactions, migrations, indexes, and other joys of life. They are placed in Postgres, MySQL, Firestore, etc.

Architecture pattern here is simple:

• session in Redis with TTL;
• persistent data and checkpoints in the DB without TTL (or with a business‑dependent retention policy).

9. Memory hygiene: sizes, privacy, separation of concerns

Agent memory isn’t just “let’s put an object somewhere and go.” There are several important rules that save money and your sleep.

Don’t cram everything into messages

Message history is an expensive resource:

• its length heavily affects the cost of a model request;
• it generally has a lot of “noise.”

Therefore:

• try to extract facts from the history into structured state as early as possible;
• use summarization for older parts of the history;
• if you store textual history in checkpoints, do it separately from what’s sent to the model.

Privacy and PII

Especially for commerce scenarios, it’s important not to store sensitive data in places where it shouldn’t end up. Memory architecture docs directly emphasize that PII should not be kept in messages or checkpoints without sanitization.

Practical rules:

• don’t put email/phone/address directly into session state unless the agent needs it to function;
• in logs and checkpoints, prefer identifiers (userId, recipientProfileId) over raw strings;
• if you need to carry PII across multiple steps, use separate protected fields in persistent storage and pass only a key in the state.

Separating business data and chat log

A good pattern is to treat state as “clean” memory, and messages as “dirty.”

That is:

• business entities (profiles, orders, carts) always live in the DB;
• state/checkpoints contain the minimum needed to resume the process;
• logs/chat history are stored separately (for example, in a vector store) and used for analytics, but not mixed into every model request.

10. Mini practice: what would you save?

To solidify the difference between memory layers, let’s step away from theory and think through a concrete case. You don’t have to write code — it’s enough to sketch the structures on paper or in your head.

Imagine your GiftGenius agent had the following dialog with a user:

• User: “I need a gift for a developer colleague, budget up to $50; he likes board games and caffeine.”
• Agent: asks a couple of follow‑up questions.
• User: “He hates mugs and is already buried in notebooks.”
• Agent: generates a list of 10 ideas, the user picks one, but says: “I’ll come back later to finish.”

Think about:

1. What would you put into session state (which may expire in 30 minutes)?
2. What goes into persistent storage so the user can return a week later?
3. What would the checkpoint look like after choosing an idea but before placing the order?

Try to sketch the corresponding TypeScript types and functions saveSessionState, savePersistentState, createGiftIdeaCheckpoint by analogy with the examples in this lecture. If you want, you can jot down these types and functions in an editor following the examples above — that will be a good mini checkpoint before the next lecture.

11. Common mistakes when working with agent memory

Mistake #1: trying to keep everything only in the message history.
A developer rejoices: “Well, the model already sees the entire dialog, why invent any additional state?” As a result, after a few dozen messages the context window is clogged with junk, tokens cost like a new MacBook, and the agent’s behavior becomes unstable — it simply doesn’t see important older facts. This problem should be solved by explicitly separating session state and persistent storage, not by increasing limits.

Mistake #2: mixing session and persistent into a single object.
It’s sometimes tempting to create one large AgentState entity, throw everything into it, and save it “as is” in the database. This blurs the boundary between temporary data of a specific conversation and the user’s long‑term data. You get stories like “after a deploy all sessions mysteriously restored from last year’s data” or “one user’s session accidentally picked up someone else’s persistent profile.” Separate the levels intentionally.

Mistake #3: storing too much in checkpoints.
A common mistake is to write the entire JSON of tool responses, the entire dialog history, raw integration data, and more into a checkpoint. After a couple of weeks of operation the checkpoint database balloons to an indecent size, backups take an hour, and DB queries slow down. A checkpoint should contain only the facts truly needed to continue the process, plus minimal metadata.

Mistake #4: forgetting about TTL and cleaning up session state.
If session states don’t have an expiration time, any random user experiment in Dev Mode stays in Redis forever. After a couple of months you look at monitoring and see a pile of “forgotten” sessions hogging memory. The session level should be designed with an explicit TTL, and the persistent level — with a well‑thought‑out retention policy.

Mistake #5: storing PII in state and checkpoints without need.
It’s especially dangerous when email, address, card number are carelessly put into session state, and then that object gets serialized into logs, flows into analytics, and into checkpoints. This creates serious risks from a regulatory and security perspective. It’s better to store safe identifiers and, if necessary, resolve them into real data through separate protected tools.

Mistake #6: no recovery strategy from checkpoints.
Some teams dutifully write checkpoints but don’t think through how the agent should recover from them. As a result, when “something goes wrong,” developers look at a table with pretty JSONs but don’t have code that can reconstruct a run from them. Checkpointing without a recovery scenario is just an expensive log, not a reliability tool.

Mistake #7: tightly coupling the agent to a specific storage implementation.
If the agent’s code directly accesses Redis/Postgres, it’s harder to migrate, test, and evolve. When the architecture changes (for example, MCP resources or a separate state service appear), you’ll have to rewrite agent logic. It’s much better when the agent sees only abstractions like Session and a set of tools, and it’s the tools that know where the data actually lives.