Reflexion

In plain words

After a bad mock interview you write one line in your notebook, and the next mock goes better. Nothing about you was retrained. Reflexion does the same for an agent: when a run fails, the agent writes a short honest note about why, saves it, and starts the next run fresh with that note in its prompt. Learning happens in plain words, not in model weights.

How it flows

A tiny example

memory = []
for trial in range(3):
    result = actor(task, memory)
    if evaluator(result):
        break
    memory.append(reflect(task, result))
    memory = memory[-5:]

Notice that the retry starts fresh from the same task, and the only thing carried forward is the short list of notes.

What you will learn

• Why an agent that fails once will keep failing unless you make it write down what went wrong.
• The three parts of Reflexion: Actor, Evaluator, Self-Reflector.
• The three ways to judge a run: scalar, heuristic, and self-evaluation.
• When reflection helps, and when old notes start hurting you.

The problem, simply

Think about your first mock interview for placements. You froze on a DSA question, gave a brute-force answer, and walked out. If you go to the next mock without thinking about it, you will freeze the same way again.

But suppose you sit for five minutes after and write one line in your notebook: "I jumped to code before asking about input size." Next mock, you read that line before entering. Same brain, same syllabus, but now you ask about constraints first. Nothing about you was retrained. One sentence changed the outcome.

Agents have exactly this problem. An agent tries a task, fails, and the next run starts blank. It repeats the same mistake, again and again, because nothing from the failed run survives.

The old fix from machine learning is to retrain the model. That means thousands of attempts, GPUs, money, and time. For a normal product team fixing one bad behaviour, that is not realistic. So the question becomes: can the agent learn from failure using only words?

The idea

Reflexion says yes. After a failed attempt, the agent writes a short note about why it failed, saves that note, and starts the next attempt fresh but with the note pasted into its prompt. No weights change. The learning lives in plain English. This idea comes from a 2023 research paper called Reflexion, and people call it verbal reinforcement learning because the "reward" arrives as sentences instead of numbers.

Three parts and one notebook

Reflexion splits the agent into three roles.

• Actor — actually does the task. Usually a normal think-act-observe loop like ReAct (an agent style where the model alternates between reasoning and calling a tool).
• Evaluator — scores the attempt. Did it pass or fail?
• Self-Reflector — reads the failed attempt and writes one honest line about what went wrong.

And one data structure: episodic memory, which is just a small list of past reflections for this kind of task. Think of it as the agent's notebook.

A worked example

Suppose Priya builds an agent that answers questions about IRCTC train timings.

Trial 1: a user asks "which trains run from Hyderabad to Chennai on Friday morning?" The agent searches by station name only, ignores the day, and returns Sunday trains. The Evaluator compares with the real answer and marks it a fail.

The Self-Reflector writes: "I searched by station but dropped the day filter, so I returned trains for the wrong day. Next time, pull the day out of the question before searching."

Trial 2 starts completely fresh. The agent has no memory of the actual conversation, only that one line at the top of its prompt. It reads the line, extracts "Friday", filters correctly, and passes.

Notice what happened. The model is identical. Only the prompt grew by one sentence.

Three ways to score an attempt

1. Scalar — an outside signal that clearly says pass or fail. Unit tests passing, a task completing, a known correct answer matching. Strongest and cleanest signal, use it whenever you have it.
2. Heuristic — rules you write yourself for known bad shapes. "If the agent repeated the same action twice, mark it stuck." "If the run crossed 50 steps, mark it inefficient."
3. Self-evaluated — the model grades its own run. Needed when there is no correct answer to compare against, but it is the weakest, because a confidently wrong agent will happily give itself full marks.

Tip

Tip: In practice, mix them. Scalar when ground truth exists, self-evaluation when it does not, and heuristics sitting on top as safety rails.

Why this pattern is everywhere

Reflexion is less an algorithm and more a shape. Any agent that quietly gets better across runs is doing some version of it: score the run, write a lesson, feed the lesson into the next run.

IMPRemember: the reflection must be actionable. "I should be more careful" teaches the next run nothing. "I dropped the day filter" changes the next run's behaviour.

When it does not help

Reflection is useless when the agent already succeeds first try, and worse than useless when the failure was external. If Flipkart's API was down for two minutes, a reflection saying "the API was down" will sit in memory forever and confuse future runs. That is superstition, not learning.

There is also memory rot. Reflections pile up. Some go stale, some were wrong to begin with, and every run gets slower because the prompt keeps growing. Fix it by capping the buffer, giving reflections an expiry after N runs, or running a periodic cleanup pass that merges and deletes notes.

Build it

"""Reflexion on a toy puzzle: pick 3 numbers that add up to TARGET."""

TARGET = 21
MAX_NOTES = 5  # bounded notebook, so old reflections cannot pile up forever

def actor(memory):
    """A pretend 'model'. Starts from a fixed guess, nudges it using past notes."""
    guess = [3, 5, 8]
    for note in memory:
        gap = int(note.split("gap=")[1].split()[0])
        # It is cautious: it moves only about halfway towards what the note says.
        guess[-1] += (gap + 1) // 2 if gap > 0 else -((-gap + 1) // 2)
    return guess

def evaluator(guess):
    """Scalar evaluator: ground truth says pass or fail. No opinions."""
    return sum(guess) == TARGET

def self_reflector(trial, guess):
    """Writes one actionable line, not a vague 'try harder'."""
    gap = TARGET - sum(guess)
    return f"Trial {trial}: tried {guess}, sum={sum(guess)}, gap={gap} on the last number."

def run(use_reflection, trials=4):
    memory = []
    label = "WITH reflection" if use_reflection else "WITHOUT reflection"
    print(f"\n--- {label} ---")
    for t in range(1, trials + 1):
        guess = actor(memory if use_reflection else [])
        passed = evaluator(guess)
        print(f"  trial {t}: guess={guess} sum={sum(guess)} -> {'PASS' if passed else 'FAIL'}")
        if passed:
            print(f"  solved in {t} trials")
            return t
        note = self_reflector(t, guess)
        print(f"    reflection: {note}")
        memory.append(note)
        del memory[:-MAX_NOTES]  # keep the notebook small
    print("  never solved")
    return None

if __name__ == "__main__":
    run(use_reflection=False)
    run(use_reflection=True)

Run it with python3 file.py. Look at the two blocks. Without reflection, every trial prints the same guess and the same failure, forever. With reflection, the gap shrinks each trial and it passes. The model function never changed — only the notes it was allowed to read.

Where you will see this

• Claude Code keeping learnings in a project memory file that gets loaded at the start of every future session.
• Coding agents in editors like Cursor that store project rules you correct once and never have to repeat.
• Code-generating agents that run the test suite, read the failure, and retry with the error in context.
• Customer-support bots where escalated conversations get turned into notes that shape later replies.
• Any team that keeps a "known failures and fixes" file and pastes it into the agent's prompt.

Common mistakes

• Vague reflections. "Be more careful next time" adds tokens and zero information. Force the reflection to name the specific step that went wrong.
• Keeping every reflection forever. The notebook grows, runs get slower, and stale notes push the agent towards behaviour that is no longer correct.
• Reflecting on external failures. A timeout or an outage is not a lesson. Filter these out before saving.
• Trusting self-evaluation alone. Without ground truth the agent grades its own hallucination as correct and then writes a confident, wrong lesson.
• Carrying the failed conversation into the retry. The retry should start fresh with only the reflection. Otherwise the old wrong reasoning drags the new attempt down with it.

If they ask in an interview

Q: What is Reflexion and how is it different from normal reinforcement learning?

A: Reflexion makes an agent improve by writing a natural-language reflection after a failure and putting it into the next attempt's prompt. Normal reinforcement learning improves by updating model weights over thousands of trials. Reflexion needs no training, no GPUs, and can fix a failure mode within a few attempts.

Q: What are the parts of a Reflexion system?

A: An Actor that does the task, an Evaluator that scores the attempt, and a Self-Reflector that turns a failure into a short lesson. Those lessons live in episodic memory, a small buffer that gets prepended to the next attempt.

Q: What is the main risk in production?

A: Memory rot. Reflections accumulate, some become stale or were wrong to start with, and they slow down and mislead later runs. You control it with a bounded buffer, expiry on old notes, and periodic cleanup.

Try these

1. Change the Evaluator to return the distance from the target instead of just pass or fail, and feed that number into the reflection. Does it converge in fewer trials?
2. Give each reflection an expiry of two trials. Watch what happens when a useful note gets dropped too early.
3. Add a heuristic evaluator that marks a trial as stuck if the guess is identical to the previous guess. Print when it triggers.
4. Make the Actor ignore memory completely, then change only the wording of the reflection until the Actor is forced to use it. Note what wording finally worked.

Words, simply

Quick recap

• Failure plus one honest written note plus a fresh retry beats a blank retry, with zero training.
• The signal quality decides everything: a real pass or fail beats the model grading itself.
• Notes are not free. Cap them, expire them, and never save a lesson about a flaky network.