The sleep framework puts forgetting at the centre of AI learning

A language model that learns a new fact has not necessarily retained it. It might answer correctly while the fact remains in the prompt, then lose access when the session ends. Alternatively, training might make the answer persist while degrading an earlier skill. The sleep framework targets the harder problem: changing what a model can use later while preserving what it could already do.

Google Research’s October 27 post describes alternating active interaction with offline consolidation and synthetic practice. The underlying research predates that post: arXiv records show a June 2 submission and July 10 revision, while the authors identify a version public on OpenReview from September 2025.

The technical contribution is a procedure for allocating capacity and transferring information between memory components. Its evidence comes from defined learning and evaluation settings. The words “sleep” and “dreaming” name computational stages; they do not establish consciousness or a biological sleep process in the model.

TL;DR Sleep combines periodic consolidation, added low-rank capacity and synthetic training. Evaluate its gains together with retention of earlier tasks, the training protocol and total computation. A longer benchmark sequence is not automatically a larger general-purpose context window.

Four meanings of remembering

A model can use information through several mechanisms. They differ in where the information lives, how it is supplied and what must happen before the model can use it again. Calling every mechanism “memory” obscures those differences.

External storage can preserve a document indefinitely without making its contents part of a model’s parameters. Parameter updates can alter behaviour without preserving a verbatim source. Context can contain a correct answer while the model fails to locate or apply it. These are different failure modes and require different measurements.

Sleep concerns the movement of useful information into parameters through a learning procedure. That makes its claims stronger than merely saving a transcript, but it also creates a harder responsibility. The model’s behaviour changes after the update, so the evaluation must consider both the material being learned and capabilities that should remain stable.

A simple diagnostic is to remove the original information before asking the retention question. If the passage remains available, success could come from ordinary context use. If a retriever supplies the passage again, success could come from external memory. Neither result is unimportant; it simply answers a different question from whether the update internalised usable knowledge.

The next diagnostic is sequential. Learning one passage successfully does not establish that the model can accumulate knowledge through many updates. The later update can interfere with earlier behaviour. A continual-learning result needs an account of the sequence and of what survives after it.

A scheduled transfer between memory components

In Language Models Need Sleep, Ali Behrouz, Farnoosh Hashemi, Adel Javanmard and Vahab Mirrokni describe memory components operating at different update frequencies. Faster components adapt more readily; slower components provide more persistent storage. Consolidation transfers information between them before the faster state is replaced.

The ordering is important. The procedure computes a prospective update but preserves the pre-update model as a teacher. A student incorporates the prospective change and additional low-rank capacity. The new capacity learns to preserve relevant teacher behaviour before the update is committed and the faster component’s temporary experts are reset.

This is more precise than saying that a model learns while idle. It specifies which state supplies the teaching signal, which state is being prepared for future use and which parameters receive the consolidation update. If the teacher were overwritten first, the procedure could lose the very behaviour it was supposed to preserve.

The design also separates two kinds of change. The system can reuse capacity in a fast component while accumulating information in a slower one. These operations need not happen at the same frequency. That gives the learning procedure a schedule rather than treating every incoming piece of information as an equally permanent update.

The analytical trade-off is stability versus adaptability. A persistent component can protect useful information from rapid changes, but it must eventually accommodate genuinely new knowledge. A fast component can respond to new material, but that flexibility makes its current state less durable. The schedule determines when information crosses that boundary.

The framework does not eliminate the need to decide what is worth preserving. Distillation and reward define the behaviour the transfer tries to retain. Those choices can preserve task-relevant abstractions without reproducing every detail of the input, but the resulting compression is only useful if the later questions depend on what survived.

Low-rank capacity is a specific resource

The phrase “parameter expansion” can suggest an unrestricted redesign of the network. The paper describes adding low-rank experts and notes that an implementation can preallocate parameters and mask them until activation. The architecture and update rules remain designed by people. Expansion refers to capacity made available to the learning process, not a system freely rewriting its own code.

The original LoRA implementation provides useful background for low-rank adaptation. It freezes original weights and learns smaller matrix factors representing an update. The PEFT conceptual guide explains how rank and the shape of the adapted matrix determine the added trainable parameters. These resources explain the building block; they are not separate replications of Sleep.

A worked matrix example makes the resource concrete. Suppose a square weight matrix has dimension 4,096, and an update is represented by two factors with rank 64. The full matrix contains 16,777,216 entries. The two factors contain 524,288 entries in total. This arithmetic illustrates parameterisation only; it is not a measurement of the Sleep implementation’s memory use or runtime.

The smaller update is constrained. It cannot represent every possible full-rank change through that one factorisation. That constraint can make adaptation economical, but it also means that rank is part of the capacity decision. Increasing rank changes the number of parameters and the range of possible updates.

Total stored parameters also differ from parameters activated during a particular forward pass. Sparse routing can leave some stored capacity inactive for a given input. A claim about unchanged active computation therefore does not establish unchanged checkpoint size, training state or consolidation cost.

For a growing memory system, both quantities matter over time. The useful resource report records stored capacity, active capacity and the work required to update it. A single parameter total can conceal the distinction that makes the approach possible.

Frozen weights can produce changed behaviour

Preserving old numerical weights is a structural property. Preserving old answers is a behavioural property. The former can help the latter, but they are not equivalent because the model’s output depends on the complete computation, including any newly active parameters and routing decisions.

A small hypothetical example illustrates the distinction. Suppose an original system assigns scores of 4 and 3 to two possible answers, so the first answer wins. An added component contributes 0 to the first and 2 to the second. The original scores remain unchanged, but the combined scores are now 4 and 5. The selected answer changes. This arithmetic is not a model of the paper’s exact router; it demonstrates why unchanged components do not guarantee unchanged outputs.

The new contribution may be desirable. Perhaps the earlier answer was wrong and the update corrected it. Or it may represent interference, where useful earlier behaviour is displaced. The outcome depends on the task and evidence, rather than on whether the old parameters were physically overwritten.

This distinction also clarifies what a retention target should preserve. Keeping every previous response unchanged would prevent correction of a mistaken answer. A useful learner instead needs to preserve established capabilities while changing responses when reliable new information warrants it. That requires distinguishing obsolete knowledge from knowledge that remains valid.

A hypothetical changing fact makes the conflict explicit. A catalogue can identify one item as available before an update and unavailable afterward. Repeating the earlier answer would count as retention under a naive unchanged-answer test, yet be incorrect for the current catalogue. A historical question about the earlier date should still receive the earlier answer. The desired behaviour is conditional on time and context.

Such cases motivate separate evaluation sets for durable facts, superseded facts and unaffected skills. They also show why a learner needs more than one overall forgetting score. An update may correctly replace an outdated answer while accidentally damaging an unrelated capability. Combining those changes can hide both.

Sleep’s architectural protection therefore addresses one part of a broader learning problem. The practical evidence must still show how the resulting model behaves after consolidation. It should identify which answers were expected to change, which were expected to remain, and how those expectations were determined. This turns retention from a vague promise into a testable account of appropriate change.

Distillation follows the student’s own paths

Knowledge distillation trains a student to approximate a teacher. In sequence generation, the student may encounter its own prefixes rather than the teacher’s preferred continuation. A procedure trained only on teacher-generated sequences can therefore receive supervision in a different distribution from the one it produces during use.

The TRL Generalized Knowledge Distillation guide explains the on-policy alternative: the student generates sequences and receives token-level feedback from the teacher on those sequences. The documented training controls distinguish the fraction of student-generated data from the divergence used to compare distributions. These are separate design choices.

Sleep builds its consolidation objective around on-policy distillation and an imitation component. The latter asks the student to complete prefixes of teacher-generated text and rewards similarity, including semantic and token-level terms. That combination aims to preserve usable teacher behaviour, rather than merely make selected internal information available somewhere in the network.

The distinction matters because storing information and using it are different capabilities. A model might preserve a representation while failing to generate the answer when needed. Behavioural training can target that gap, but it inherits the strengths and limitations of the teacher and reward.

Teacher agreement is not an independent fact check. If a teacher’s response contains an error, successful imitation can preserve that error. A semantic reward can also accept two statements as equivalent when a later task depends on a distinction between them. These are reasons to evaluate retained knowledge against independent answers, not arguments that distillation lacks value.

For an application, the key question is what the teaching signal represents. It may capture reliable knowledge, a task-specific strategy or a transient mistaken response. Consolidation makes selected behaviour more durable. The quality of that selection remains consequential after the original context disappears.

Dreaming is a training-data procedure

The framework’s second stage generates synthetic examples, selects candidates and evaluates updates derived from them. The paper describes random expert selection during generation, gradient-based selection and an improvement-based reward after candidate fine-tuning. “Dreaming” is a compact name for this search and training process.

The related SEAL repository describes generating self-edits, including fine-tuning data and update directives. Its knowledge-incorporation implementation makes the inner loop concrete: a temporary low-rank adapter is trained and evaluated, and the resulting performance informs training rewards. Sleep builds on this general adaptation pattern while adding its consolidation design.

This clarifies what self-improvement means in the experiment. A candidate example is useful if the update it induces improves a specified evaluation. The model does not receive an unrestricted instruction to improve itself according to an unknowable internal goal. The procedure supplies the task, update method and signal used to choose among candidates.

The inner loop creates an important cost. Generating a candidate is only one part of the work. The procedure may also train a candidate adaptation and evaluate the resulting model. Reporting only the size of the final synthetic dataset can conceal the larger search that produced it.

Selection also determines what the system learns to value. If improvement is measured on a narrow set of tasks, synthetic examples may become effective for those tasks without establishing broader gains. Independent held-out evaluation is needed to distinguish a useful adaptation rule from increasingly specialised optimisation of the selection signal.

The meaningful question is therefore which behaviour improves under which feedback. The metaphor becomes less mysterious when the report identifies the generated artifact, the update it triggers and the independent result used to assess it.

The long-context result has a training condition

The paper reports strong BABILong results out to 10 million tokens. Appendix B makes a crucial distinction: the small models are fine-tuned under the benchmark’s training protocol, while the figure also contains zero-shot results from large models. It states that the small models, including Hope, drop substantially without fine-tuning.

The BABILong repository explains the benchmark construction. Task-relevant facts from bAbI are interspersed with unrelated book text. Questions can require one or several supporting facts. This tests retrieval and reasoning amid distractors, with a controllable increase in sequence length.

That is a valuable stress test of a memory procedure. It does not establish equally reliable comprehension of every kind of ten-million-token document. A sequence containing sparse synthetic facts creates a different information problem from a document whose every paragraph contributes to an answer or whose meaning depends on subtle cross-references.

Nor is the result automatically a serving specification. An API context limit describes what an interface accepts under a defined configuration. A benchmark result describes performance on tasks after a specified preparation and training process. Consolidating a long stream into parameters is different from placing the entire stream in a conventional attention window.

The distinction is reinforced by RULER’s benchmark design. Its configurable tasks vary retrieval demands, multi-hop tracing, aggregation and question answering. Length is one axis; task complexity and the kind of information to preserve are others.

A strong follow-up evaluation would vary those axes separately. It would identify whether failure comes from locating information, retaining several facts, updating a fact or applying the retained information. Such a result would explain the mechanism’s boundary more clearly than one maximum sequence length.

New knowledge and old capability need separate scores

The paper’s factual-incorporation experiment measures answers without supplying the original passages at evaluation. It reports both a single-passage setting and a continued-pretraining setting involving 200 passages and 974 associated questions. Those conditions make the result relevant to parametric retention rather than ordinary question answering with the passage still present.

The multi-passage result nevertheless answers a bounded question. A single continued-pretraining run over a set of passages differs from an indefinitely growing stream of independent updates. The number and organisation of updates affect what can be concluded about long-term accumulation.

The sequential translation experiment probes a more direct interference question: how learning one language affects performance after exposure to another. The authors compare independent-language learning with sequential learning and report better retention with additional consolidation stages. That is evidence about the evaluated sequence, not proof that forgetting has disappeared for arbitrary future sequences.

A useful evaluation artifact is a matrix. Each row identifies a task or knowledge batch; each column identifies a point after another update. The diagonal shows how well newly introduced material is learned. Earlier rows in later columns reveal what remains after subsequent learning. An overall final average alone can conceal a severe loss in one earlier task.

Task order also matters analytically. Two sequences can expose different interference because the material overlaps differently. A convincing continual-learning evaluation should therefore identify the order and, where feasible, examine more than one ordering. That is a proposed robustness check, not an additional experiment reported here.

Finally, retention should cover behaviour the application intends to preserve. Remembering factual answers while losing instruction-following ability would be a different trade-off from remembering both. The earlier evaluation set needs to reflect that requirement instead of becoming a convenient collection of unchanged scores.

Metrics can change the apparent breakthrough

The paper’s mathematical reasoning tables report average@16. That label should travel with any quoted score. An average over multiple sampled attempts differs from the probability that at least one attempt succeeds. Treating the two as interchangeable can substantially change how a reader interprets a result.

The same care applies to the reported ARC result. The appendix describes a filtered setting with eight held-out tasks and evaluates the fraction of generated dreams yielding a correct answer. The reported 80% should not become “80% on ARC-AGI” without that protocol. The latter wording implies a broader benchmark claim that the experiment does not establish.

These qualifications do not erase the result. They identify what improved and how it was counted. A metric can be informative even when it is narrower than the headline a casual reader might infer from it.

Ablations add another layer. Removing an objective term or expansion component can help determine whether it contributes within the tested configuration. It does not establish that the same component contributes equally at every scale, or that the procedure has a unique advantage over every alternative training recipe.

For a research reader, the best comparison pairs the result with its denominator, sampling method and preparation. The model name by itself is insufficient because the same backbone can undergo different training, adaptation and evaluation. Our model-card reading guide uses this distinction to separate a model identity from the conditions supporting a capability claim.

The compute comparison needs a target

Appendix B reports that, at the same number of steps, supervised fine-tuning is about four times as efficient as the proposed method. The authors also report the opposite ordering when comparing wall-clock time to reach selected mathematical-reasoning performance targets. These are different comparisons and can both be true.

A more expensive training step can require fewer steps to achieve a chosen quality. Conversely, a procedure that reaches one target quickly may remain unattractive under a fixed training budget or a different target. “More efficient” needs the resource and stopping condition attached.

The complete cost includes synthetic generation, teacher feedback, candidate updates, evaluation and stored capacity. The amount used for serving later requests is another quantity. Moving work into an offline period changes when the cost occurs; it does not make that computation free.

A deployment-oriented comparison would measure update latency and the number of future uses over which the update pays back. If the knowledge is queried repeatedly, an upfront consolidation cost may support useful amortisation. If the information changes before it is used, durable storage may have little value. These are consequences of the workload, not new benchmark findings.

Our guide to benchmark version parity explains why a comparison must hold the relevant conditions explicit. For Sleep, those conditions include the update schedule, task preparation, sampling and target quality. The research offers a concrete mechanism for making temporary information more durable. Its next test is how that mechanism balances acquisition, retention and cost across longer and more varied learning histories.

The technical explanation above attributes the Sleep procedure and reported experiments to Behrouz, Hashemi, Javanmard and Mirrokni’s version 2 paper, available under CC BY 4.0. It paraphrases the research and adds analysis; no new experiments were performed for this article.