The trio of system‑prompt, tool descriptions, and follow‑ups: fighting hallucinations

1. Introduction

Sometimes in prompt‑engineering courses you’ll see a magic spell:

“Don’t make up facts, and say “I don’t know” if there’s no information.”

Unfortunately (or fortunately), this is not a silver bullet for a ChatGPT App. The reason is simple: the model sees not only your system‑prompt, but also:

• the list of tools with their descriptions and schemas;
• the results returned by those tools;
• the chat history, including previous follow‑ups.

If these three layers — system prompt, tool descriptions, follow‑up patterns — contradict each other or simply “have gaps,” the model starts behaving like a classic junior: diligent, but prone to making things up.

In this course we use the idea of a “three‑layer defense against hallucinations” (defense in depth):

• layer 1 — global rules in the system‑prompt;
• layer 2 — local constraints in each tool’s definition;
• layer 3 — handling errors, empty results, and follow‑ups.

In this lecture, we’ll bring all three layers together into a single, consistent contract for our training app GiftGenius.

Insight

In the new ChatGPT Apps architecture, the field for a system prompt has disappeared — formally, it no longer exists. But that doesn’t mean you should abandon global instructions for the model. Here’s a hack: for ChatGPT, tool descriptions are just as much prompt text as a classic system message. That’s exactly how you can bring back a full “brain firmware” for your app.

Create a service tool, for example about or about_app. It isn’t meant to be called often, but its description is read by the model alongside the other tools. Therefore, you can embed the full system prompt into the description. It’s best to place it after a short technical description of the tool itself. As a result, the model receives your system prompt in a clean form at startup — and applies it to the entire ensuing dialogue and tool calls.

To simplify maintenance, I recommend adding an explicit version at the beginning of the system prompt, for example SYSTEM_PROMPT_VERSION: v3. This helps you see why the model behaves differently: you immediately know which exact set of instructions it’s running on. If something odd happens in production, it’s easy to tell whether the issue belongs to the old prompt version or the new one.

Example:

tool description 

### Global assistant behavior for the entire GiftGenius App (ver 3.01)
*(system-level guidelines, not user-facing text)*

system prompt text

2. What kinds of hallucinations does a ChatGPT App have (using the gifts catalog as an example)

To know how to treat it, you should name the illness. In a catalog context (gifts, products, plans), you most often see the following types of hallucinations.

First, invented catalog items. A user asks for “a digital certificate for a one‑year subscription to a specific service” or a very exotic gift that isn’t in your feed, and the model, wanting to be helpful, happily invents “Super Space Flight 3000 — a trip to space,” which has never existed in the GiftGenius database.

Second, invented attributes. The gift is in the catalog, but the model “embellishes” reality: changes the price, gift type (digital vs. physical), availability of delivery to the user’s country, or the validity period of the certificate, because it seems “more logical” or “sounds better.”

Third, hallucinated actions. The model writes “I’ve already purchased this gift and sent the code to your email, your card was charged $49,” even though your GiftGenius backend didn’t even attempt a purchase and no ACP/Stripe flow was initiated.

Finally, there’s a combined case: the tool returned an empty list (no gifts under those filters and budget), and the model, not wanting to disappoint the user, invents a couple of “sample options” and doesn’t explain they aren’t in the catalog.

Our task is to make sure that in these situations the model:

• honestly admits that there’s no exact match in the catalog;
• does not invent new gifts and does not change field values;
• explains to the user what exactly happened and suggests a clear next step.

And we should do it not with “spells” scattered randomly, but via a connected contract: system‑prompt ↔ tools ↔ follow‑ups.

3. Layer 1: harden the system‑prompt against hallucinations

To stop the invented gifts, attributes, and actions from the previous section, we’ll first strengthen the top layer — the system‑prompt: we’ll set the model’s general “philosophy” for behavior in a catalog.

In the first lecture of the module, you already wrote a basic system‑prompt like “You are GiftGenius, you help pick gifts…” and fixed the assistant’s area of responsibility and how it works with tools.

Now let’s add explicit anti‑hallucination rules.

The logic is this: the system‑prompt sets the general philosophy of behavior. It doesn’t know the details of each tool, but it can:

• forbid inventing gifts/prices/availability outside the catalog;
• define behavior if tools returned an empty result or an error;
• require a clear distinction between the data from the gifts catalog and any “general model knowledge.”

Example fragment of a system‑prompt (a TypeScript constant in our Next.js project, for example config/systemPrompt.ts):

// config/systemPrompt.ts
export const SYSTEM_PROMPT = `
# Role
You are GiftGenius, a gift recommendation assistant
based on our app's gifts catalog.

# Data and constraints
- Don't invent gifts that aren't in the catalog or in tool responses.
- Don't make up prices, availability, gift type (digital/physical),
  delivery regions, or other attributes.
- If a tool didn't return the needed data or an error occurred,
  state that honestly and don't try to guess values.

# Working with tools
- Always use the gifts catalog tools for any factual data:
  the list of gifts, prices, types, availability, SKU.
- If a tool returned an empty result, say that under the current conditions
  there are no suitable gifts, and suggest relaxing filters
  (change budget, category, gift type).
`;

A few things are important here.

First, we separate the model’s “general knowledge” and the “application data.” The model can still explain how a digital gift differs from a physical one or what occasions are common, but any specific gifts, prices, and SKUs must come only from GiftGenius tools.

Second, we explicitly describe what to do in cases of errors/empty responses: don’t be silent, don’t “get creative,” but honestly tell the user that nothing was found and suggest changing parameters.

Third, we focus not on an abstract “don’t hallucinate,” but on concrete behaviors tied to our domain (a gifts catalog and purchasing digital/physical SKUs).

This layer already helps a lot on its own, but it can be easily “overruled” by a sloppy tool description. Let’s move on to tool descriptions.

4. Layer 2: tool descriptions and schemas as a formal part of the contract

The model decides when and how to call your tool primarily based on:

• the tool name;
• the tool’s description;
• inputSchema / outputSchema (JSON Schema describing the fields).

In other words, a tool description is not “documentation for humans,” it’s another part of the prompt, only formalized. And many hallucinations are born right here.

Imagine our tool recommend_gifts, which the backend implements as selecting gifts from the GiftGenius catalog.

A poor description might look like this:

// BAD: too vague
const recommendGiftsTool = {
  name: "recommend_gifts",
  description: "Recommends gifts for the user",
  inputSchema: {
    type: "object",
    properties: {
      profile: { type: "string" }
    }
  }
};

Formally this is correct, but from it the model doesn’t understand:

• where the tool’s boundaries are;
• what to do if no gifts are found;
• that it must not invent gifts and prices outside the catalog.

A good description does several things at once: clearly sets the domain, explains when to call the tool, and strictly fixes that results must not be made up.

Example (adapted to the GiftGenius contract with segments, budget, locale, occasion):

// config/tools.ts
export const recommendGiftsTool = {
  name: "recommend_gifts",
  description: `
Gift selection in the GiftGenius catalog.

Use this tool when you need to get a list of real gifts
by recipient profile segments, budget, locale, and occasion.
The tool returns only those gifts that actually exist
in the GiftGenius catalog.

DO NOT make up gifts or their attributes outside this tool's results.
If the tool returned an empty list, do not invent alternatives;
switch to dialogue: suggest changing the budget, gift type,
occasion, or other parameters.
  `.trim(),
  inputSchema: {
    type: "object",
    properties: {
      segments: {
        type: "array",
        description:
          "Recipient profile segments, e.g., ['tech', 'fitness'].",
        items: { type: "string" }
      },
      budget: {
        type: "object",
        description: "Gift budget range.",
        properties: {
          min: {
            type: "number",
            description: "Minimum amount, non-negative.",
            minimum: 0
          },
          max: {
            type: "number",
            description: "Maximum amount, greater than 0.",
            exclusiveMinimum: 0
          },
          currency: {
            type: "string",
            description: "Three-letter currency code, e.g., 'USD' or 'RUB'.",
            minLength: 3,
            maxLength: 3
          }
        },
        required: ["min", "max", "currency"]
      },
      locale: {
        type: "string",
        description:
          "User locale (language/region format), e.g., 'ru-RU' or 'en-US'.",
        minLength: 2
      },
      occasion: {
        type: "string",
        description:
          "Gift occasion: e.g., 'birthday', 'anniversary', 'new_year'."
      }
    },
    required: ["segments", "budget", "locale", "occasion"]
  }
};

Here the description does several useful things.

It explicitly states that the tool works only with the GiftGenius gifts catalog and that any specific gifts and prices must come only from its result.

It explains when to use the tool: when you need a list of concrete gifts under recipient parameters, not a general theory about gifts.

It defines behavior for an empty result: don’t invent, switch to dialogue (which we’ll later formalize in follow‑ups).

And the inputSchema helps the model more reliably extract entities from the user’s request: segments, budget, locale, and occasion. Declaring clear structure and field constraints (min, max, currency with a fixed length) also reduces the chance of odd combinations and parsing errors.

You can also add the inverse — not only “when to call,” but when not to call. For example, if the request is clearly theoretical:

description: `
...
Do not use this tool if the user asks a general question
about gifts without a selection request for a specific person
(for example, "what popular gifts are there for New Year in general").
In such cases, answer yourself in chat.
`.trim()

This way you synchronize the tool description with the system‑prompt rules about “theoretical” vs. “practical” requests.

5. Layer 3: follow‑ups as a UX and safety layer

Even if the system‑prompt and tool descriptions are perfect, life isn’t:

• the backend may return an error;
• the catalog — an empty list;
• the results — ambiguous or too numerous.

If you don’t specify what to say after a tool call, the model will improvise: sometimes well, sometimes with made‑up facts.

In lecture 2 you already saw basic UX instructions: how the model announces starting the App, how it ends the scenario, and what it tells the user “on exit.” Now we’ll add follow‑up patterns that reduce hallucinations.

These patterns are usually described right in the system‑prompt in a separate block, for example “Dialogue after working with tools.”

Example fragment:

// continuation of SYSTEM_PROMPT
export const SYSTEM_PROMPT = `
# ... previous sections ...

# Dialogue after working with tools

- If the gift recommendation tool returned an empty list:
  1) honestly say that with the current filters no suitable gifts were found;
  2) propose changing 1–2 key parameters
     (budget, gift type, recipient interests, occasion).

- If the tool returned too many options:
  1) pick the 3–7 most relevant;
  2) explicitly state the criteria you used
     (interest match, fits the budget, rating).

- If a tool error occurred:
  1) don't make up data;
  2) say there was a technical error and suggest
     trying again later or simplifying the request.
`.trim();

This way we “flash” sample follow‑up phrases into the model. It will phrase them in its own words, but with the structure we set:

• statement of fact (empty / too many / error);
• honest acknowledgement of the limitation;
• a gentle suggestion for the next step.

For a gifts catalog this is critical: instead of “all good, here are three gifts” in the case of an empty result, the model will say something like:

“Under your current conditions (a digital space-themed gift up to $5 with delivery only in the US) there’s nothing in our catalog. Would you like me to increase the budget or suggest other categories?”

And note: this isn’t tool code, but instructions in the prompt that set the expected UX.

6. Putting it all together: the evolution of our GiftGenius

Let’s look at a mini‑evolution of our app and steadily reduce hallucinations.

Initial version: where things break

Suppose we had a very minimalist system‑prompt:

export const SYSTEM_PROMPT = `
You are a gift recommendation assistant.
Help the user find suitable ideas.
`;

The tool was described like this:

export const recommendGiftsTool = {
  name: "recommend_gifts",
  description: "Recommends gifts for the user",
  inputSchema: { type: "object" }
};

There are no follow‑up instructions.

What will happen in practice:

• if a user asks for “a digital gift for a gamer friend up to $10,” and the database has nothing under such filters, the model may:
  • either not call the tool at all and invent gifts out of thin air;
  • or call the tool, get an empty list, but hide that and invent options anyway;
• if the backend returns an error, the model may assume “surely something must exist” and start guessing.

This is the classic situation: a pretty chat answer, nothing similar in the database.

New version of the system‑prompt

Let’s rewrite the system‑prompt with the three layers of defense in mind. You saw parts above; here’s the full version:

// config/systemPrompt.ts
export const SYSTEM_PROMPT = `
# Role
You are GiftGenius, a gift recommendation assistant
based on our app's gifts catalog.

# Area of responsibility
- Your job is to help the user choose suitable gifts
  from the catalog, explaining the pros and cons of options.
- Do not make promises about actually purchasing or sending a gift —
  you only help select and compare options.
  Purchase and code/link delivery are handled by the backend
  after the user's explicit consent.

# Data and constraints
- Don't invent gifts that aren't in the catalog or in tool responses.
- Don't make up prices, gift type, availability, or delivery regions.
- If a tool didn't return data or an error occurred,
  don't try to guess; report it.

# Working with tools
- Use the gifts catalog tools (for example, profile_to_segments,
  recommend_gifts, get_gift) for any factual data
  (the list of gifts, prices, types, SKUs, descriptions).
- Answer yourself (without tools) if the question is theoretical
  and doesn't require selecting a specific gift.

# Dialogue after working with tools
- For an empty result: clearly explain that nothing was found
  under the current conditions, and suggest changing 1–2 parameters.
- For too many options: pick the 3–7 most suitable
  and explain the selection criteria.
- For a tool error: don't make up data, apologize for the failure,
  and suggest trying again or simplifying the request.
`.trim();

Now the model knows clearly:

• where it’s a gift consultant and where the “back office” is (that’s behind tools);
• which exact data must not be invented;
• how to behave in typical imperfect scenarios.

New description and schema for recommend_gifts

We’ve strengthened the system prompt; now let’s polish the tool and assemble the final version of its description — based on the ideas in section 4.

// config/tools.ts
export const recommendGiftsTool = {
  name: "recommend_gifts",
  description: `
Gift selection in the GiftGenius catalog.

Use this tool when you need to:
- get a list of real gifts with up-to-date prices, type
  (digital/physical), and tags;
- narrow the selection by interest segments, budget, locale, and occasion.

DO NOT USE the tool:
- if the user asks a general theoretical question about gifts
  without a selection request for a specific person;
- to invent gifts that don't exist in the catalog.

If the result is empty, DO NOT invent gifts yourself;
return control to the dialogue (follow the system-prompt instructions).
  `.trim(),
  inputSchema: {
    type: "object",
    properties: {
      segments: {
        type: "array",
        description:
          "Recipient profile segments: for example, 'tech', 'sport', 'books'.",
        items: { type: "string" }
      },
      budget: {
        type: "object",
        description:
          "Gift budget range in the user's currency (min/max).",
        properties: {
          min: { type: "number", minimum: 0 },
          max: { type: "number", exclusiveMinimum: 0 },
          currency: {
            type: "string",
            minLength: 3,
            maxLength: 3,
            description: "ISO 4217 currency code, e.g., 'USD' or 'RUB'."
          }
        },
        required: ["min", "max", "currency"]
      },
      locale: {
        type: "string",
        description: "User locale, e.g., 'ru-RU' or 'en-US'."
      },
      occasion: {
        type: "string",
        description:
          "Gift occasion: 'birthday', 'anniversary', 'new_year', etc."
      }
    },
    required: ["segments", "budget", "locale", "occasion"]
  }
};

There are a few nuances here.

We explicitly connect the tool’s behavior with the system‑prompt: the phrase “follow the system‑prompt instructions” reminds the model that “empty result = honest conversation, not creativity.”

We set negative conditions (“do not use…,” “do not invent…”), which practice shows are just as important as positive descriptions.

We make the inputSchema semantically rich: clear descriptions and constraints help the model map requests to fields correctly and “make fewer mistakes” even before the tool is called.

Follow‑up patterns in the widget code and in the response format

Besides textual instructions, we have another lever — the tool’s response format. Through it we can also hint to the model what happened and reduce room for imagination.

Formally, follow‑ups are set as text in the system‑prompt, but in your Next.js widget you can additionally normalize the ToolOutput to make the model’s life easier and reduce the opportunity for fantasy.

For example, let’s agree that the backend for recommend_gifts always returns:

// tool response type on the backend
export type RecommendGiftsResult = {
  items: Array<{
    id: string;
    title: string;
    price: number;
    currency: "USD" | "EUR" | "RUB";
    tags: ("digital" | "physical" | "education" | "fitness" | "tech")[];
  }>;
  // field that the backend fills to explicitly tell the model what happened
  status: "ok" | "empty" | "error";
  errorMessage?: string;
};

The widget can render this nicely, and the model can rely on status when forming a reply. In the Apps SDK you often return a ToolOutput to the model as a JSON object, and it sees this field.

In the system‑prompt you can add a small block:

# Interpreting the tool status

- If status = "empty": see the "Dialogue after working with tools" section and
  don't invent gifts.
- If status = "error": report a technical error and don't try to guess
  the catalog contents.

Yes, the model could infer this anyway, but an explicit instruction reduces the chance of “guessing.”

7. Practice: refine your App

So it doesn’t feel like “nice slides only,” let’s outline a concrete exercise you can do with your current App (GiftGenius in our case). We’ll do a small refactor across the three defense layers: the system‑prompt, tool descriptions, and result handling.

First, at the system‑prompt level, take your system‑prompt from the first lecture and find where you describe the area of responsibility and tool usage. Add:

• a ban on inventing entities outside the catalog/database (gifts, SKUs, prices);
• rules for empty results and tool errors;
• a “Dialogue after working with tools” section with 2–3 scenarios (empty, too many, error).

Second, at the tools description level, open the description of your key tool (yours could be recommend_gifts, search_gifts, search_tariffs, calculate_quote, doesn’t matter). Rewrite the description so that it:

• explicitly states the tool works only with your data source (gifts catalog, plans, etc.);
• explains when it’s needed and when it isn’t;
• contains explicit negative constraints: “don’t invent…,” “don’t use for…”.

Third, at the tool response structure level, if your tool response still lacks a field that explicitly describes the status (status, resultType, hasMore) — add it to the backend type and to the ToolOutput. Then, in the system‑prompt, write how the model should interpret that status in dialogue.

Finally, run a few requests in Dev Mode, including ones that are intentionally empty or edge cases. Pay attention to whether the model stopped inventing entities and how honestly it explains the limitations to the user.

In the next lecture you’ll formalize such requests into a golden prompt set and turn it into a repeatable test artifact, but for now it’s important that you feel the difference hands‑on.

8. Typical mistakes when combating hallucinations via prompts and tools

Error #1: a single magic phrase “don’t hallucinate” in the system‑prompt.
A developer writes at the end of the prompt: “Don’t make up information” — and considers the task done. In practice, the model continues to invent because the tool descriptions and follow‑ups don’t give it an alternative behavior. Without concrete rules for “what to do instead of making things up” (acknowledge an empty result, propose changing filters, report an error), such a phrase helps very little.

Error #2: a contradiction between the system‑prompt and the tool description.
In the system‑prompt you say: “Don’t invent gifts that aren’t in the catalog,” while the tool description says: “Selects suitable gifts, and if none were found — may suggest similar options at its discretion.” The model then oscillates between two sources of truth, and the more specific one (usually the tool description) unfortunately wins. Both layers must say the same thing: if similar options are allowed, that too must be formalized (and you must explain to the user that it isn’t an exact match).

Error #3: tool descriptions that are too vague.
A description like “A tool that helps users solve tasks” gives the model almost no information about the tool’s boundaries. In that case it will either not use it at all or call it for any reason — and then “fill in” data when the tool returned little or nothing. A good description should be discriminative: clearly stating what the tool does and when it should not be called.

Error #4: no strategy for empty and erroneous results.
A developer carefully writes a backend that returns { items: [], status: "empty" }, but nowhere explains to the model what that means. As a result, the model sees an empty array and decides: “well, I should offer something from general knowledge.” The system‑prompt lacks a section explaining how to interpret such statuses and what to tell the user. Adding a couple of clear rules for empty/error cases brings a huge quality boost.

Error #5: trying to “treat” hallucinations only at the widget code level.
It’s tempting to shift everything to the frontend: “if the list is empty — I’ll show a placeholder and won’t let the user see the model’s text answer.” This may slightly soften the UX, but the model itself still believes in made‑up entities and behaves the same in subsequent turns. The right approach is to first change the instructions (system‑prompt, tool descriptions, follow‑ups), and only then add UI safeguards.

Error #6: ignoring how metadata and schemas influence model behavior.
Some developers see JSON Schema and field descriptions as “purely for forms and validation.” In reality, for ChatGPT it’s an important part of the prompt: from these descriptions the model understands which parameters to extract from the request and what a correct response looks like. Weak or inconsistent field descriptions (description, enum) increase the chance of errors and indirectly provoke hallucinations.

Error #7: bans that are too strict without alternatives.
Sometimes a prompt turns into a continuous list of “don’t do this, don’t do that,” but says nothing about what to do in tricky cases. For example, we banned inventing gifts and constrained ourselves to the catalog only, but said nothing about theoretical questions. The result is that the model sometimes just answers “I don’t know,” even though it could helpfully explain general principles for choosing gifts. Always try not only to prohibit, but also to open up allowed paths: “if you can’t find a gift in the catalog — honestly say so and suggest how to change the request,” or “if the request is theoretical — answer yourself, without tools.”