Memory Blocks and Sleep-Time Compute

In plain words

Your placement notebook has one first page you read every day, and a hundred messy pages you read rarely. An agent's memory works the same way. Memory blocks are the first page: small named pieces the agent always sees and can edit itself. Sleep-time compute is a second agent that tidies all of it later, when nobody is waiting for a reply.

How it flows

A tiny example

human = Block(label="human", value="", limit=120)

def primary_turn(user_line):
    reply = think(user_line, core=human.value)
    human.append(extract_fact(user_line))   # fast, messy
    return reply

def sleep_pass():
    # off the critical path, so a slower model is fine
    human.value = strong_model_summarise(human.value)

Notice that the user turn only appends, and every rewrite or deletion happens later inside sleep_pass.

What you will learn

• What a memory block is, and why an agent keeps some facts always in front of its eyes.
• The three memory layers an agent uses: core, recall, archival.
• What sleep-time compute means, and why a background agent can afford a slower, costlier model.
• How to build a tiny two-agent loop where one agent answers and the other cleans up memory.

The problem, simply

Think about your placement preparation notebook. On the very first page you write things you never want to forget: your branch, your target companies, your resume points. You see that page every time you open the notebook.

The rest is different. Random doubts, half-solved DP problems, the Amazon interview experience Rahul told you about. You go there only when needed.

Now here is the part that hurts. After two months the notebook is a mess. One page says "Infosys drive on 12th", another says "Infosys postponed". You never get time to fix it, because when you sit down you are busy solving, not organising.

An agent has exactly this problem. Memory keeps growing, old facts contradict new ones, and tidying up while the user waits makes replies slow. So we need two things: structure for the memory, and a quiet time to clean it.

The idea

Three layers of memory

Split the agent's memory into three layers, each with a clear job.

• Core — always inside the prompt. The agent sees it on every single turn. Small, precious space.
• Recall — the conversation history. Not always visible, but fetchable when needed.
• Archival — the big outside store. Any fact, any time, pulled in by search.

Core is your first page. Recall is the rest of the notebook. Archival is the seniors' Drive folder you search when stuck.

Memory blocks

A memory block is one named, editable section of core memory. Not a blob of text — a proper object with fields.

Each block carries an id, a label (like human), a value (the actual text), a limit (maximum characters allowed), and a description telling the model when it should edit this block.

The original idea, from a research project called MemGPT, had two blocks:

• Human block — facts about the user. Name, role, preferences, goals.
• Persona block — the agent's own identity. Who it is, its tone, its hard rules.

Letta, the platform that grew out of that project, opened this up: you can define any block you want. A task block for the current goal, a project block for codebase facts, a safety block for rules that must never break.

The agent edits blocks through tools, like any other tool call: append text, replace old with new, read a block, or summarise one nearing its limit.

IMP

Note: The limit is not decoration. Core memory shares space with the prompt. If a block grows without a cap, it eats the space your actual conversation needs.

Sleep-time compute

Now here is the trick. Run a second agent in the background, when the user is not waiting.

The main agent answers fast. It writes raw, messy notes into blocks and moves on. Later, between turns or overnight, the sleep-time agent wakes up, reads those notes and consolidates them: merges duplicates, summarises fat blocks, drops facts a newer fact has contradicted.

Three good things fall out of this design.

First, no latency cost. The user never waits for cleanup, because it happens off the critical path — the chain of steps between the user's question and the user's answer.

Second, you can use a stronger model. Nobody is staring at a spinner, so point the sleep agent at a slower, costlier, smarter model and let it think properly.

Third, it is the natural time to clean. Same as you: you attend the class, and things settle in your head at night.

A worked example

Suppose Priya is building a study-plan agent.

Turn 1, she says: "I am from CSE, targeting Amazon." The primary agent appends to the human block: CSE student. Target: Amazon.

Turn 2: "Actually I have TCS on the 5th, so Amazon prep is on hold." Primary appends again: TCS drive on 5th. Amazon on hold. The block now has 2 lines, one partly stale.

Turn 3: "Suggest today's problems." The agent answers using both lines. Fine, but the block is filling up and "Amazon" now means two different things.

Then Priya closes her laptop. The sleep-time agent runs, sees that "Target: Amazon" and "Amazon on hold" conflict, and rewrites the block to one clean line: CSE student. Immediate goal: TCS drive on 5th. Amazon deferred. Next morning the primary agent starts with clean memory and fewer characters used.

IMPRemember: The primary agent writes fast and dirty; the sleep-time agent is the one allowed to rewrite and delete.

About the thinking trace

One upgrade worth knowing for interviews. Earlier agent loops made the model type thoughts inside normal text, as a line starting with Thought:, plus a message-sending tool and a "heartbeat" flag to keep the loop alive. Newer versions dropped all that. Modern models emit reasoning on a separate channel, carried across turns.

The control loop is still ReAct — think, act, observe, repeat. Only the thinking is now a real structured field instead of text you begged the model to format correctly.

Build it

"""Toy memory blocks + a sleep-time consolidation pass. Standard library only."""

class Block:
    def __init__(self, label, value="", limit=120, description=""):
        self.label = label
        self.value = value
        self.limit = limit          # max characters allowed
        self.description = description

    def near_limit(self):
        return len(self.value) > 0.7 * self.limit

class BlockStore:
    def __init__(self):
        self.blocks = {}

    def add(self, block):
        self.blocks[block.label] = block

    def append(self, label, text):
        b = self.blocks[label]
        b.value = (b.value + " " + text).strip()[: b.limit]   # hard cap

    def show(self):
        for b in self.blocks.values():
            print(f"  [{b.label}] ({len(b.value)}/{b.limit}) {b.value}")

def primary_turn(store, user_line, fact):
    """Fast path: answer, then dump a raw note into core memory."""
    print(f"User: {user_line}")
    print(f"Agent: noted, I will keep that in mind.")
    store.append("human", fact)

def sleep_pass(store):
    """Off the critical path: merge, drop stale facts, shrink."""
    b = store.blocks["human"]
    facts = [f.strip() for f in b.value.split(".") if f.strip()]
    kept = []
    for f in facts:
        # a newer line about the same topic wins
        topic = f.split()[0].lower()
        kept = [k for k in kept if k.split()[0].lower() != topic]
        kept.append(f)
    b.value = ". ".join(kept) + "."
    print(f"[sleep] rewrote 'human' -> {len(facts)} facts became {len(kept)}")

store = BlockStore()
store.add(Block("human", limit=120, description="facts about the user"))
store.add(Block("persona", "I am a calm placement mentor.", limit=80))

primary_turn(store, "I am CSE, targeting Amazon.", "Amazon is my target company.")
primary_turn(store, "TCS drive is on the 5th.", "TCS drive is on the 5th.")
primary_turn(store, "Amazon prep is on hold.", "Amazon is deferred for now.")

print("\nBefore sleep:")
store.show()
sleep_pass(store)
print("\nAfter sleep:")
store.show()

Look at two things in the output. The human block before sleep holds three raw lines, one of them already outdated. After the sleep pass, the two Amazon lines collapse into the newer one, and the character count drops — free space bought back without the user waiting for it.

Where you will see this

• Coding agents like Claude Code and Cursor keeping a project-facts block: build command, folder layout, style rules.
• ChatGPT-style "memory" features, where a small set of facts about you is pinned and quietly updated later.
• Support bots keeping a customer block (plan, language, past complaint) always visible, while old tickets sit in archival.
• Swiggy or Flipkart style assistants remembering your usual address and diet preference in a tiny always-on block.
• Any team running a nightly job to dedupe and re-summarise their agent's knowledge store.

Common mistakes

• Letting blocks grow forever. Append, append, append, and one day the block eats your whole prompt budget. Wire a summariser that fires before a write crosses the limit.
• Doing consolidation on the critical path. If cleanup runs while the user waits, your slowest requests become horrible. Move it to the sleep pass.
• Silent drift. The sleep-time agent rewrites a block and the primary agent never realises its memory changed under it. Version every block and show the diff in the trace.
• Letting untrusted text reach the sleep agent. If a scraped web page flows straight into core memory, an attacker can plant instructions there. See the prompt injection lesson in Module 5.
• Treating memory as one flat pile. Without the core / recall / archival split you cannot say "always visible" versus "fetch only if asked".

If they ask in an interview

Q: What are memory blocks and how are they different from just stuffing facts in the system prompt?

A: A block is a typed, persistent piece of core memory with a label, value, character limit and description. The model edits it through tools during the run, so memory changes as the conversation goes. A system prompt is fixed text only the developer changes.

Q: What is sleep-time compute, and why can it use a bigger model?

A: A second agent that consolidates memory in the background while the user is not waiting. Since it sits off the critical path, latency does not matter, so you can run a costlier model that summarises and resolves contradictions better.

Q: Name the three memory tiers and one thing that lives in each.

A: Core is always visible in the prompt, like facts about the user and the agent's persona. Recall is the conversation history, fetched when needed. Archival is the external store of arbitrary facts, reached by search.

Try these

1. Add a summarise(label) step that fires when a block crosses 70 percent of its limit. Try 50, 70 and 90 percent and see which causes fewest rewrites without ever overflowing.
2. Add versioning: on every write store the previous value, and add history(label) so you can answer "why did the agent forget this?".
3. Write a dedupe step for the sleep pass that collapses two stored facts when most of their words overlap. Run it only in the sleep pass, never during a user turn.
4. Make the sleep agent an untrusted writer: if it wants to modify the persona or safety block, require a second check before the change is committed.

Words, simply

Quick recap

• Give memory structure: core is always visible, recall is history, archival is the big store. Blocks are the typed editable pieces of core.
• Keep cleanup off the critical path — a sleep-time agent can be slower and smarter because nobody is waiting on it.
• Cap every block, version every write, and never let untrusted text walk straight into core memory.