The Sleep Framework: A New Way for LLMs to Retain Knowledge in Continual Learning

Why a longer context is not the same as durable memory

Source: The sleep framework: A new way for LLMs to retain knowledge in continual learning, Google Research, October 27, 2026. Technical details are checked against the authors' paper, Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories.

A language model can appear to learn inside a conversation while retaining almost nothing after the session ends. The prompt, working notes, and retrieved passages act as temporary scaffolding. Once that context disappears, the model returns to its underlying parameters. Google Research calls this gap between temporary adaptation and persistent knowledge one of the central obstacles to continual learning.

The proposed Sleep framework borrows the organization of human sleep, but not its biology. A model alternates between an active waking phase, where it receives new information and writes to fast memory, and a sleep phase, where it pauses external input and consolidates selected knowledge into slower parametric memory. A second sleep process, called dreaming, generates synthetic practice so that the model can learn how to use what it has stored.

The important claim is architectural rather than anthropomorphic. The system is not conscious, does not experience rest, and does not discover a hidden biological mechanism inside a transformer. It is a training lifecycle that separates three operations that conventional post-training often mixes together: acquiring experience, stabilizing it, and improving behavior from it.

The paper's strongest idea is that continual learning is a memory-system problem, not merely a fine-tuning problem. If a fixed-capacity network must represent every new fact, skill, and interaction without interfering with earlier ones, forgetting is not an accidental bug. It is a direct consequence of asking too few parameters to absorb too many changes.

The failure mode: static weights versus temporary context

Modern LLMs acquire broad capabilities during pre-training and adapt quickly through in-context learning. Neither mechanism provides a permanent update path after deployment. The model may follow a new specification for the duration of a prompt, but the next session starts from the same checkpoint. A changed fact, a local operating rule, or a lesson learned during a long deployment remains outside the model unless it is retrieved again.

Repeated fine-tuning looks like the obvious answer. The difficulty is interference. Gradient updates that improve performance on a newly introduced task can overwrite representations needed by older tasks. In the extreme case, this is catastrophic forgetting: the model becomes better at the latest data while losing capabilities that were already present.

External memory does not remove the problem. A transcript, vector index, or wiki can preserve information, but the model must still find the right entry and reason with it at inference time. Retrieval also consumes context and may fail when the query is vague, when memories conflict, or when the relevant knowledge should influence a computation before the model has explicitly formulated a search. The Sleep framework therefore targets parametric consolidation: it changes the model's computation while attempting to protect earlier knowledge.

Two phases, but three different jobs

In the waking phase, the model receives external input and behaves like a deployed assistant or agent. It can keep recent tokens in a fast store, including a key-value cache, and use that material while the task is active. In the sleep phase, external input is intentionally reduced or removed. The system turns inward and works on the experiences already collected.

Within sleep, memory consolidation and dreaming have separate responsibilities:

1. Consolidation asks, “What from recent experience should become durable knowledge?” It transfers an abstraction from a fast, high-frequency memory block into a slower block with newly available capacity.
2. Dreaming asks, “Can the model learn to use that knowledge more effectively?” It generates synthetic practice, selects useful examples, and updates a controlled set of parameters based on measured improvement.

The ordering matters. Consolidation first establishes a more stable representation. Dreaming then rehearses and refines that representation. Running the second step directly on a fixed model would bring back the familiar risks of iterative self-training: errors can be reinforced, and new adaptations can erode prior capabilities.

Consolidation is an upward move through a memory hierarchy

The framework builds on the idea of a continuum memory system. Different blocks update at different frequencies. Fast blocks track recent context and change often. Slower blocks change less frequently and hold more durable abstractions. This is not a simple two-tier cache. Consolidation can happen repeatedly between neighboring levels, moving information from faster components toward increasingly stable ones.

Before each consolidation, the model adds capacity to the receiving block. The implementation pre-allocates low-rank mixture-of-experts parameters and masks the inactive experts during ordinary forward and backward passes. A sleep cycle can unlock one of those experts without changing the surrounding codebase or the nominal active parameter count. The low-rank matrices keep the new capacity smaller than a dense expansion.

For an update from memory block l - 1 to block l, the procedure follows a strict sequence:

1. Compute: calculate the prospective update for the faster block, but do not apply it to the live model yet.
2. Consolidate: treat the pre-update model as the teacher. Build a larger student that contains the prospective update and a newly activated low-rank expert in the slower block.
3. Update: train the new expert against the teacher, apply the prepared update, reset the temporary experts in the faster block, and leave the new slower expert active.

This ordering gives the teacher a stable target. The teacher represents the model before the transition. The student represents the intended post-transition model with enough new capacity to preserve that knowledge. Only the newly activated parameters are optimized, which limits interference with representations that were already stable.

Knowledge seeding: distillation for a larger, not smaller, student

Conventional distillation usually transfers knowledge from a large teacher to a smaller student. The paper calls its operation knowledge seeding because the student is the expanded network. A smaller active version of the self, often represented by inactive experts, seeds a larger-capacity version. The goal is not to compress the model for deployment. It is to preserve an abstraction while giving it more room to persist.

The distillation objective combines teacher-generated data with data sampled by the student. This is a generalized knowledge distillation setup. Teacher logits provide token-level guidance, while on-policy student samples expose the errors and trajectories that the expanded model would actually produce. The on-policy component reduces the mismatch between a clean teacher distribution and the model's own inference behavior.

The paper then adds learning to imitate (LTI), an RL-based objective intended to teach the student not only to contain the teacher's knowledge but to sample from it in a similar way. For a teacher-generated sequence and a student continuation, the reward combines semantic agreement with token-level edit similarity:

$$ r(\hat{y}; y) = \gamma r_{\mathrm{sem}}(\hat{y}; y) + (1-\gamma)r_{\mathrm{abs}}(\hat{y}; y) $$

The semantic term is supplied by a frozen reward model and rewards matching meaning. The absolute term uses Levenshtein distance, normalized by the longer sequence, and drops to zero once the edit distance exceeds a threshold:

$$ r_{\mathrm{abs}}(\hat{y}; y)= \begin{cases} 1-\dfrac{\operatorname{Levenshtein}(\hat{y},y)}{\max(|\hat{y}|,|y|)}, & z(\hat{y},y)\leq z_0,\\ 0, & \text{otherwise}. \end{cases} $$

This two-part reward is useful because meaning and surface form fail in different ways. A student might preserve the general idea but drift into a different answer format. Another student might match several tokens while changing a decisive relation. Semantic scoring and edit similarity push against both failure modes.

The final knowledge-seeding objective trades off imitation reward against divergence from the teacher:

$$ \mathcal{L}_{\mathrm{KS}}= \mathbb{E}_{x\sim\mathcal{D}} \left[ (1-\alpha)\mathbb{E}_{y\sim \mathrm{LM}_{\mathrm{student}}}[r(y)] -\alpha\mathbb{E}_{y\sim \mathrm{LM}_{\mathrm{student}}} \left[\mathcal{D}(\mathrm{LM}_{\mathrm{teacher}}\Vert \mathrm{LM}_{\mathrm{student}})(y\mid x)\right] \right] $$

Here, alpha controls how much the update follows the teacher's distribution versus the imitation reward. The reward decides what behavior should be preserved. The divergence term constrains how far the expanded student moves.

Pruning makes the hierarchy reusable

After the knowledge has been transferred, the temporary experts in the faster block are reset. The paper compares this to synaptic pruning: temporary connections are removed after their useful abstraction has been moved to a more stable level. The reset also frees capacity for the next waking period.

This lifecycle is more interesting than a one-time adapter merge. The model is not simply accumulating permanent modules forever. It is designed to grow selectively at slower levels and recycle capacity at faster levels. Whether that schedule remains healthy over months of updates is an open engineering question, but the architecture at least makes the question explicit.

Dreaming explores combinations that waking data may miss

Dreaming is the self-improvement stage. Given a task context C and a way to evaluate downstream performance, the model generates a set of synthetic samples, or dreams. A normal generator tends to reproduce familiar patterns. The Sleep framework deliberately perturbs that process by having each mixture-of-experts router select an additional random expert while generating a dream.

The random expert can bring unrelated knowledge into the sample. That may sound like noise injection, but the purpose is to widen the set of gradient directions explored during rehearsal. The model can rehearse a new fact alongside a distantly related skill, expose an unexpected connection, or discover a path that its ordinary high-probability sampling would never produce.

Generation alone is not enough, because most synthetic samples may be easy, redundant, or wrong. The framework scores each dream by the gradient of a supervised language-modeling objective. It keeps the top-scoring samples and a smaller random set for diversity. The selected dreams are then tested through isolated LoRA fine-tuning updates. A dream receives reward when the fine-tuned model improves on the downstream task; otherwise its reward is zero.

The process resembles a learned curriculum:

1. Generate several candidate experiences with controlled randomness.
2. Estimate which samples carry a strong learning signal.
3. Keep the best candidates plus some random exploration.
4. Test each candidate through a small parameter update.
5. Reward the generator when that update improves downstream behavior.

The paper builds this loop on SEAL-style self-adaptation and uses the ReST^EM algorithm for optimization. The key difference is the surrounding memory hierarchy. Dreaming is not asked to carry the entire burden of continual learning. It operates after consolidation has created a protected, expanded representation.

Why the method can help, according to the theory

The paper offers two complementary explanations. For consolidation, knowledge seeding is treated as structured low-rank compression. The model does not copy every token from its fast memory into a slower block. It searches for a compact set of directions that preserve both its own on-policy behavior and the semantic properties rewarded by LTI. The new expert supplies the dimensions needed to write those directions without overwriting older knowledge.

For dreaming, random expert routing broadens the set of explored updates. Gradient-based filtering then emphasizes samples that produce a useful learning signal. The intended result is a better-conditioned optimization process: learning is less dominated by a few directions, and the model can improve without collapsing into a narrow pattern of self-generated data.

These arguments do not make the method safe by construction. They explain why the design may reduce interference. They do not prove that every generated dream is true, that semantic reward captures every important constraint, or that an expanded expert will remain interpretable. Those remain empirical and operational issues.

What the experiments show

The reported results span class-incremental learning, continual translation of unseen languages, long-context question answering, factual knowledge incorporation, mathematical reasoning, and few-shot abstract reasoning. The experiments are primarily based on Llama and Qwen language models, with a Hope-style continuum memory backbone.

In class-incremental classification, the full Sleep model outperforms the same memory architecture without sleep-time consolidation and external baselines such as InCA and EWC. The benchmark names are CLINC, Banking, and DBpedia:

Benchmark	Sleep	No-sleep memory baseline	Reported gap
CLINC	97%	93%	+4 points
Banking	92%	84%	+8 points
DBpedia	89%	82%	+7 points

The continual-translation experiment is more revealing because two unseen languages are learned in sequence. In-context learning performs well while one language is active, then collapses after the second is introduced. The paper reports Manchu-to-English ChRF falling from 20.9 to 10.1 under that sequence. Sleep-3 reaches 23.9 on Manchu and 38.1 on Kalamang after learning both languages. That is not merely better retention than ICL. It is close to the model's single-language result, suggesting that consolidation preserved useful translation behavior across the sequence.

On BABILong, small long-context models such as Titans and ARMT remain competitive up to roughly one million tokens but degrade sharply beyond that point. Hope with sleep-time consolidation remains stable at extreme lengths up to ten million tokens. The result is notable because the method does not attempt to keep ten million tokens in an active context. It compresses task-relevant structure into parametric memory during sleep.

For factual incorporation on SQuAD, the complete four-level Sleep model reaches 48.9% after a single passage and 46.2% under continued pretraining with 200 passages. The base model scores 31.9% in both settings, and SEAL scores 46.7% and 43.2%. The ablation is equally important: removing dreaming reduces accuracy to 35.7% and 36.2%. Consolidation without rehearsal is not doing the whole job.

On mathematical reasoning, Sleep improves over base, SFT, GRPO, and on-policy self-distillation for both tested Qwen sizes. The Qwen3-8B results are 79.2% on AIME-24, 69.0% on AIME-25, and 46.1% on HMMT-25. The smaller Qwen3-1.7B model reaches 53.2%, 40.2%, and 29.3%, respectively.

Evaluation	Base or no dreaming	Sleep	What changes
SQuAD, one passage	31.9% base	48.9%	New fact is consolidated and rehearsed
SQuAD, 200 passages	32.0% fine-tuned, no dreaming	46.2%	Dreaming prevents consolidation from being brittle
AIME-24, Qwen3-8B	73.8% base	79.2%	Consolidation also improves reasoning
BABILong	Degrades after about 1M tokens	Stable to 10M tokens	Long context is compressed during sleep

The efficiency result is mixed and should be read carefully. When compared step for step, ordinary SFT is about four times faster. When the comparison targets the same AIME or HMMT score, SFT is reported to require 3.6 to 4.8 times as much wall-clock time to catch up. Sleep is therefore expensive per update but potentially more efficient per unit of final capability.

What this could mean for long-lived embodied agents

The source evaluates language-model tasks, not robot policies. It does not demonstrate a robot that improves after a night of operation, and it does not test manipulation, navigation, sim-to-real transfer, or safety under physical uncertainty. That distinction should remain explicit.

There is nevertheless a useful systems lesson for embodied AI. A long-running robot accumulates shift logs, sensor summaries, interventions, failed grasps, localization exceptions, and operator corrections. Keeping all of that in a growing retrieval store is workable but not sufficient. Some lessons need to change the model's prior behavior, not wait for a future query to retrieve them.

A Sleep-like robotics loop could separate four stages:

1. Wake and record: collect observations, actions, outcomes, uncertainty, and human corrections without immediately changing safety-critical parameters.
2. Consolidate: compress validated experience into a low-rank memory expert attached to a slower policy or planning level.
3. Dream with constraints: generate synthetic variations through a simulator or world model, keeping only samples whose updates improve held-out tasks.
4. Validate before promotion: run replay, simulation, shadow mode, and hardware gates before the new expert can affect physical actions.

The random-expert idea is especially sensitive in robotics. Mixing distant knowledge could uncover a useful transfer, but it could also propose physically invalid or unsafe behavior. In an embodied deployment, the dreaming objective would need hard constraints, verified state transitions, conservative action filters, and a reversible promotion path. Improvement on a language-model reward is not evidence that an action policy is safe.

The useful analogy is not “let the robot sleep and hope it becomes wiser.” It is “separate experience collection from durable model change, and never promote a dreamed change without external validation.”

The limits that matter

Self-generated practice can reinforce errors. Random experts widen exploration, while semantic rewards and downstream tests filter some bad samples. Neither mechanism supplies a complete truth signal. In long deployments, a wrong but internally consistent lesson could become durable.

Capacity expansion is not free. Pre-allocating and masking parameters makes activation easier, but storage, routing, checkpointing, and evaluation become more complex. The paper keeps the active parameter count unchanged during the experiments, yet the system still stores a growing set of experts. Operational systems would need a retirement policy for experts that are rarely useful.

The sleep schedule is another policy to tune. When should consolidation happen? How often should the model dream? Which memory level should receive the update? The experiments show that more consolidation stages help some tasks, and that making the most persistent memory too adaptive weakens retention. There is no universal schedule.

Forgetting has several different meanings. A model can lose a capability, overwrite a fact, retrieve the wrong memory, or change the style of its output. The reported benchmarks cover several of these behaviors, but not privacy deletion, provenance, malicious memory injection, or conflicting operator instructions.

The robotics bridge remains unproven. Language benchmarks test text prediction and reasoning. They do not measure contact-rich control, real-time constraints, or the cost of a wrong action. A robot would need a separate safety case before the paradigm could be used beyond offline policy improvement.

The design lesson: memory needs a lifecycle

The most durable contribution of the Sleep framework is not the label. It is the lifecycle. The system distinguishes fast context, protected consolidation, exploratory rehearsal, and measured promotion. It treats forgetting as a capacity and interference problem while also recognizing that merely adding capacity is not enough. The model must learn to use what it has stored.

That combination of knowledge transfer and behavioral rehearsal is why the results are more persuasive than a simple adapter-growth story. Dreaming turns stored knowledge into practice. Consolidation prevents that practice from immediately colliding with everything the model already knows. The two stages are complementary, and the ablations show that removing either one weakens the result.

For RobotWorld readers, the article is best treated as foundational memory infrastructure. It does not yet provide a recipe for a continuously learning robot. It does provide a concrete framework for asking what a deployed agent should remember, where that memory should live, how it should be stabilized, and what evidence should be required before a new memory is allowed to change behavior.

Read the original Google Research post at research.google. The peer-facing technical report is available as arXiv:2606.03979.