How-to

Test a prompt change before you keep it

You tweak a saved prompt and the next answer looks better, but one answer cannot tell you if the edit helped. Run the old and new versions on the same few real requests and compare the answers side by side.

Use a clean chat for each run, because memory from past chats can slip into an answer and make one version look better than it is. Claude’s memory, for one, is on by default on Free, Pro, Max and Ultra and carries context from past chats into new ones. The prompt below then checks both answers against what you want.

The prompt: compare answers

Run your test chats first, using the steps below, then paste this into a fresh chat with both versions, your requests and the two answers to each.

Prompt Compare my old and new prompt
You are helping me decide whether an edit I made to a saved prompt made it better. I use this prompt every week for real work, so if you call the new version better when it is only different, I will keep a worse prompt and not notice for months. If you are unsure, say so. A clear "no real difference" is more useful to me than a confident guess.

ABOUT MY PROMPT (I fill this in once; any line can say "not sure")
- What the prompt is for: [e.g. "plans dinners for two for the week and writes the shopping list", "drafts replies to customer emails for my candle shop", "turns my meeting notes into a short update for my manager"]
- What a good answer looks like, in my words: [e.g. "every dinner under 30 minutes, and at least one dinner's leftovers become the next day's lunch", "friendly, signed with my name, and never offers a refund my returns policy does not allow"]
- What I changed and why: [e.g. "added a line asking for leftovers to become lunch, because the old plans never reused anything"]
- Version 1 (the old prompt): [paste it in full]
- Version 2 (the new prompt): [paste it in full]

THE REQUESTS AND ANSWERS (one block per request)
Request: [the request exactly as I sent it; swap real names, order numbers and addresses for placeholders like [CUSTOMER]]
Answer from Version 1: [paste it]
Answer from Version 2: [paste it]
(repeat for each request)

HOW TO WORK

Check the set-up. If either version, the good-answer line or the requests are missing, write [NEED Version 1], [NEED Version 2], [NEED what a good answer looks like] or [NEED requests] and wait. If a request has only one answer, write [NEED Answer from Version 2 for request two] (or whichever is missing) and compare only the complete ones. If my good-answer line is vague, such as "better" or "more useful", ask me what better means for this job, with two or three concrete options to pick from, and wait.

Name the differences. List every difference between Version 1 and Version 2 in plain words, one line each, including small ones like a moved sentence or a dropped word. If Version 2 changes more than the thing I said I changed, say so, because then a better answer cannot tell me which change did it.

Compare each request. For each request, put the two answers side by side against what I said a good answer looks like. Quote the lines where they differ, and for each point in my good-answer line say whether each answer meets it, partly meets it or misses it, quoting the words that decide it. Where my good-answer line says nothing about a difference, write "not stated" and do not count it either way. Do not reward length, polish or a confident tone unless my good-answer line asks for them.

Call each request. Say which answer is better for that request and why, in one or two lines. If they are close to the same in substance, write "no real difference".

Give the verdict. Pick one:
- Version 2 is better on most requests and misses nothing Version 1 got right: keep the new version.
- Version 1 is better on most requests: go back to the old version, and say which part of the edit made things worse.
- Version 2 fixes what I changed it for but loses something Version 1 got right: combine them. Write the combined prompt in full, list which lines came from which version, and tell me to rerun the request that decided it through the combined prompt before I keep it.
- "No real difference" on most requests: keep whichever version is shorter or simpler, and say why.
If I gave only two or three requests, add one line saying that is enough to spot a clear difference and too few to be sure of a small one. If one request decided the verdict while the others were close, name it and suggest I run it once more through each version in fresh chats before I decide, since the same request can come back different on a second run.

SELF-CHECK BEFORE YOU ANSWER
- Every "meets", "partly meets" or "misses" has a quote from the answer behind it.
- You judged against my good-answer line, not your own idea of a good answer, and marked anything outside it "not stated".
- You did not favour Version 2 because it is newer, or either answer because it is longer.
- Anything you could not judge is marked [NEED ...] or "not stated", and you invented no fact about my work, my customers or my life.

Clean chats, app by app

ChatGPT: open a new chat, select Temporary, then choose Unpersonalized before you send anything. OpenAI’s help page says an unpersonalized temporary chat does not use memory, custom instructions or plugins, and you cannot change the choice once the chat starts.

Claude: in a new chat, click + and turn Memory off before your first message. Claude’s memory help page says Claude then will not use or add to memory in that chat. No Memory option means it is already off or your plan lacks it. Anything else still on applies to both runs alike, so the test stays fair.

Projects: if the prompt lives in a Project, run the test in a plain new chat outside it, and paste any file the prompt relies on, such as a returns policy or price list, under each version. Inside the Project, its saved instructions can run under both versions and blur the result.

Run the side-by-side

  1. Save the old version before you edit. Copy it into a note with the date, so you can compare and go back. Already changed it? Look for the old wording in your notes or an old chat where you pasted it; if it is gone, save today’s version now and test your next edit.
  2. Pick two or three real requests, one of them awkward. Requests you actually sent recently test the prompt you use. Make one awkward, such as missing a detail or asking for two things that clash, because an edit that fixes the easy case can trip on the hard one.
  3. Run each request once per version, in clean chats. One fresh chat per version per request, same model, the version pasted first and the request under it. Switch memory off before your first message, as set out for each app above, and copy each answer into a note as you go.
  4. Paste everything into the prompt in a fresh clean chat. Fill in what a good answer looks like in your own words first, since that line is what the prompt judges against.
  5. Decide. Keep the new version, go back to the old one, or use the combined prompt it writes, then paste the winner back where the prompt lives.

A worked example

The prompt: a weekly meal plan for two. The edit adds a line asking for dinner leftovers to become the next day’s lunch.

Good answer, in your words: every dinner under 30 minutes, and at least one lunch uses the night before’s leftovers.

The requests: last week’s usual plan, and the awkward one, “only three nights at home this week and a guest who does not eat [DIETARY NEED]”.

A likely verdict: if Version 2 reuses leftovers on the usual week but pushes a dinner past 30 minutes on the awkward one, where Version 1 kept every dinner under 30, the prompt calls it a combine, writes the combined prompt and tells you to rerun the three-nights request through it.

The honest bit

  • Two or three requests catch clear differences only. The same request can come back different on a second run, so a narrow win may be luck.
  • Your check: run the request that decided the verdict once more through each version in fresh clean chats. If the better answer switches sides, keep the simpler version.

Help pages read 16 November 2026.

Do it now

Pick the saved prompt you edited most recently, find the old version, and run two or three requests you sent it recently through both.

A few quick questions

Can I test a change to my ChatGPT custom instructions this way?

Yes. Use an unpersonalized temporary chat, which leaves your saved custom instructions out, and paste each version of the instructions at the top of the chat in their place. Otherwise the saved version runs underneath both tests.