AI for biology has several evidence levels. Keep them separate

An AI agent can find a useful biological lead without performing a new experiment. The important question is which part of the lead comes from a physical measurement, which part comes from statistical analysis, and which part is an explanation proposed by the model. Those contributions can reinforce one another. They cannot substitute for one another simply because the final report reads like a coherent discovery story.

OpenAI’s October 21 announcement describes distributed agents interpreting an existing mouse perturbation atlas with PerturbAI and Scripps researchers. Its significance is a new way to explore complex measurements and prioritise leads. The evidence for those leads still depends on the original experiment, the analysis, and any subsequent confirmation.

This survey separates those layers using the public atlas, dataset documentation, released agent repository, and established analysis resources. The October agent manuscript was inaccessible through its public manuscript endpoint during this review, so this article does not independently reproduce its benchmark or claim a full review of that paper. The accessible materials nevertheless support a concrete map of what different results can establish.

TL;DR A perturbation measurement, a statistical pattern, a literature-supported explanation, a held-out prediction and an independently replicated mechanism answer different questions. Keep their provenance and biological context attached when an agent combines them.

A map of biological evidence

A useful evidence map follows the claim rather than ranking every experiment above every computation. A precise computational result can answer a narrow question more directly than a physical experiment measuring an unrelated endpoint. The issue is whether the observation supports the sentence being written.

The distinction between a measurement and its explanation is especially important for gene-expression data. A cell can alter many RNA abundances after a perturbation. Those changes may include direct consequences, compensatory responses, changes in cell state, and downstream effects. A report connecting them to a pathway is an interpretation of the response, even when the RNA measurements themselves are genuine.

Likewise, an explanation can be compelling without uniquely identifying a mechanism. Two proposed mechanisms may predict the same expression signature. A useful agent report should expose that ambiguity and identify a measurement on which the predictions diverge. Adding citations to one explanation does not eliminate its competitors.

This distinction complements our account of agent-assisted wet labs. An agent selecting and interpreting an experiment occupies a different position in the evidence chain from an agent analysing an already completed screen. Both can contribute to research, but their outputs need different descriptions.

The experiment underneath the agents

The March atlas preprint reports an in vivo CRISPR screen of disease-relevant genes in mouse brain, with single-nucleus RNA measurements. Its authors explicitly limit the study to loss-of-function perturbations, one sampling timepoint and RNA readouts. They identify incomplete access to some populations and limited power for subtle or rare responses. Those boundaries apply to later interpretation of the atlas.

The original screen is therefore more informative than a collection of unperturbed expression correlations. Researchers intervened on genes and observed subsequent responses. But the intervention does not make every downstream sentence causal. A claim that a particular gene perturbation changes a transcript under the study conditions differs from a claim explaining all intermediate molecular events. It also differs from a claim that reversing that transcript change would restore a biological function.

These are separate experimental questions. An expression response may be a useful marker without mediating the effect of the intervention. A compensatory response may move in the opposite direction to the underlying problem. A shared response across perturbations may reveal a common downstream state while leaving the initiating mechanisms different. Those possibilities matter when a model turns a descriptive pattern into a proposed intervention.

A further boundary concerns the word “brain.” Whole-brain sampling does not mean that every cell, developmental stage and interaction is measured equally. A dataset can offer broad anatomical coverage while remaining constrained by delivery, selection and measurement. An agent cannot recover an unmeasured effect merely by writing a plausible explanation of where it might occur.

The most faithful formulation of a result therefore includes the perturbation, the assayed population and the measured response. Broader biological implications can follow, but they need their own evidence. Keeping that structure makes the analysis useful to a researcher deciding what to test, because it shows exactly where the next experiment begins.

Cell counts and experimental independence

The dataset card describes roughly 7.7 million cells and provides multiple data representations, including cell-level tables and analysis files. Its metadata distinguishes sample and processing identifiers from the mouse of origin. It also includes guide assignments and quality-related fields. These are analytical variables, not incidental bookkeeping.

The central statistical consequence is that many measured cells do not necessarily supply equally many independent experimental replicates. Cells from one animal share aspects of that animal and its handling. Measurements passing through the same processing channel can share technical influences. An analysis that treats all rows as exchangeable can answer a different question from an analysis designed around independent biological units.

This does not make single-cell resolution less valuable. Individual measurements can reveal a response confined to one population that disappears when tissue is averaged together. The two advantages are distinct: resolution helps identify where a signal occurs, while replication helps establish how reliably it occurs across the relevant experimental units.

An agent should therefore preserve both levels. A report can describe the number of measured cells supporting a contrast and separately describe how those cells are distributed across animals or samples. If the released summary does not expose the latter, the report should leave that uncertainty visible. The larger cell count cannot answer it on its own.

The same principle applies when comparing two cell populations. If one population has much greater coverage, a larger list of detected differences may reflect greater sensitivity as well as different biology. “More significant genes” is not interchangeable with “a stronger biological effect.” The contrast needs effect sizes, measurement coverage and the analysis procedure together.

For a reader, the practical artifact is a small provenance record attached to each finding: what was perturbed, which cells were compared, where they came from, what changed, and what uncertainty remains. That record is more informative than a single impressive total at the top of a dataset page.

Quality control changes the question

Before an agent explains a pattern, the analysis must establish which measurements enter it. Quality control can remove compromised measurements, but it can also change the population being studied. That makes filtering part of the scientific claim.

The official Scanpy tutorial illustrates quality checks, doublet detection, normalisation and representation of single-cell data. It also warns that apparently poor quality can sometimes reflect genuine biology and that suitable criteria depend on the sample. That tutorial is methodological background, not evidence that the mouse atlas used its example dataset or exact processing decisions.

Consider a perturbation that changes the quantity of measurable RNA. A filter designed to retain cells within a narrow count range could preferentially exclude affected cells. The surviving population might then appear more similar to the control than the original population. Conversely, retaining damaged measurements can create a misleading difference. Neither concern is resolved by always filtering more aggressively or always retaining everything.

The analytical task is to connect the decision to the biological question. A researcher examining population depletion may need to distinguish absent cells from cells removed after collection. A researcher examining expression within a population needs to consider whether filtering changes which states of that population remain. An agent explaining a summary table should not silently assume these questions were answered upstream.

Useful reporting keeps the relevant processing version and eligibility rules accessible. It should also distinguish an observed absence from an absence after selection. This is particularly important when the report discusses a rare population or a weak signal, because relatively small changes in eligibility can affect which conclusions are supportable.

An agent can assist by finding inconsistencies across reports or surfacing the original filtering documentation. It cannot establish that filtering was harmless by generating a fluent paragraph. That conclusion depends on analysis of the measurements and the experimental design.

Statistical differences and pathway stories

Large screens create many opportunities to find patterns. The Statsmodels multiple-testing documentation describes procedures for adjusting families of statistical tests, including methods controlling different error criteria. The important analytical choice is which tests belong to the family and what error rate the chosen procedure addresses.

An agent navigating many genes, perturbations and cell populations can encounter individually interesting results selected from a much larger search. Reporting only the selected result conceals the process that produced it. An adjusted value should therefore remain attached to its test family and method. It is not a probability that the proposed mechanism is true.

Effect direction also matters. A list of genes can overlap a pathway while containing a mixture of increases and decreases. Membership alone does not establish that the pathway has been activated or inhibited. Even a directionally coherent transcriptional pattern may describe a response to pathway activity rather than measure that activity directly.

The Gene Ontology introduction explains how functional terms and evidence-linked annotations make large experiments interpretable. Such resources provide a shared vocabulary for molecular activities, processes and locations. Their value is partly organisational: they let researchers connect a long list of changes to a tractable set of biological ideas.

That compression can also hide distinctions. A broad process term may cover several molecular routes. A result enriched for that term does not identify which route produced the observation. A compelling agent narrative should let the reader expand the compressed label back into the contributing genes, directions and supporting annotations.

A pathway explanation becomes stronger when it makes an additional, discriminating prediction. If two possible processes yield the same expression pattern, the next useful step is to identify an independent endpoint that separates them. Rephrasing the pathway label in more elaborate biological language does not add an independent observation.

Literature support has its own provenance

A literature reference can establish that an idea has experimental precedent. It can also establish only that someone previously proposed the idea. An agent needs to preserve that difference while synthesising sources.

The Gene Ontology evidence guide makes the distinction explicit. Its annotation categories separate experimental support, computational inference, phylogenetic inference, author statements, curatorial statements and electronic annotation. Direct assay, physical interaction and expression-pattern evidence are themselves distinguishable. These codes describe how an annotation is supported; they are not a universal numerical ranking of biological truth.

This provides a useful model for agent reports. A statement supported by a physical interaction experiment should say which interaction was measured. A statement inferred from a related sequence should retain the fact that the support is an inference. If a review cites an earlier experiment, the report should trace the claim to that experiment before treating it as directly demonstrated.

Context belongs in that trace. A result in a different species or cell system can make a hypothesis plausible without establishing it in the assayed mouse population. A study of a protein’s abundance does not automatically support a statement about its activity. A physical interaction measured under one condition may not establish that the interaction mediates the observed perturbation response.

These differences become more consequential as agents combine many sources. Several documents may repeat one original observation. Counting the documents as independent support overstates the evidence. Conversely, apparently conflicting results may come from experiments that deliberately changed a condition essential to the effect.

The useful output is a relationship between a claim and specific evidence, including the reason that evidence applies. Our survey of verification in AI reviewing examines the related distinction between finding a citation and checking what it supports. In biology, the experimental context is part of that support check.

What the released agent system exposes

The Distributed Agents repository is useful because it exposes more than a finished narrative. Its documentation describes structured findings, evidence and references, with tools for retrieving linked records. It also makes an important access distinction: the hosted corpus does not contain the raw dataset needed to recompute statistics. Browsing a report and reproducing its analysis are different operations.

The released evidence schema gives this distinction a concrete form. Its fields include a target gene, an evidence identifier, a description, a source line and hashes connecting records to source reports. The holdout claim schema separately records a statement, scope, contributing proposals and a confidence category. These are inspectable structures in the release, rather than measurements performed for this article.

A source hash can help establish which report a record came from. It does not establish that the report’s interpretation is correct. A source line makes a claim easier to locate. It does not establish that the underlying measurement supports every part of that claim. A confidence category records the system’s classification; its meaning depends on the procedure that produced and evaluated it.

This is still a substantive improvement over an untraceable answer. A researcher can follow a proposed explanation to its constituent evidence and identify where a statistical observation ends and an interpretation begins. A disagreement can then be localised to a data contrast, a reference, a biological assumption or a synthesis step.

The architecture also creates a useful distinction between reproducibility of a report and reproducibility of a biological effect. Recreating the same table from the same saved inputs tests the computational workflow. Obtaining a similar response in another experiment tests something else. Both are valuable, and neither should be reported as if it automatically includes the other.

Held-out results and expert preference

A held-out evaluation can test whether a system predicts information it was not allowed to inspect. That is a serious capability question, but its meaning depends on the withheld unit. Hiding selected measurements differs from withholding an entire perturbation, animal or biological setting. Each split permits different kinds of generalisation claims.

Information boundaries matter as much as the final score. A model might have access to related measurements, public literature or annotations that strongly constrain the answer. That can be appropriate for a research assistant intended to use those resources. It changes what the evaluation demonstrates about inference from the experimental data alone.

The comparison should also identify what a baseline received. A system with additional retrieved context, more parallel work or a broader data view is a different research workflow. A useful evaluation can compare such workflows, but the result should not be reduced to a claim about an isolated model component.

PerturbAI’s announcement reports a blinded expert preference comparison. Expert judgement can help assess usefulness, relevance and interpretability. Preference is nevertheless a different endpoint from experimentally confirming a newly proposed mechanism. An expert may reasonably prefer a report because it generates a sharper, more testable question.

That distinction protects the value of the evaluation. A useful report need not already contain a validated discovery to improve research. It may organise neglected evidence, reveal a missing control, or propose a decisive next measurement. Those contributions should be assessed directly rather than converted into an unsupported claim that the biological problem has been solved.

Novelty and selection are separate questions

A newly written explanation need not be a new biological result. It may restate a known mechanism, connect an established mechanism to a previously unexamined population, or propose a genuinely new relationship. These categories can all be useful, but they imply different follow-up work. A researcher checking novelty needs a dated search trail and the precise claim being compared with earlier literature.

An agent report can make this tractable by separating the familiar components from the proposed extension. If a mechanism is established in one population and inferred in another, the novelty belongs to the extension. The evidence supporting the original mechanism should not be presented as if it directly validates that extension. This keeps a reasonable hypothesis from becoming a stronger claim through citation alone.

Selection introduces another distinction. A team may inspect many proposed leads and take only the most promising into the laboratory. A high confirmation rate among that selected set describes the combined selection and testing process. It does not establish the accuracy of every generated lead. Conversely, a low rate among deliberately difficult or surprising candidates need not describe the utility of routine reports.

The denominator should follow the decision being evaluated. For an assistant organising an atlas, coverage and traceability may be relevant. For a system prioritising experiments, the relevant outcome includes how the selected candidates perform under the stated selection budget. For a system proposing mechanisms, the test must concern those mechanisms rather than the mere presence of an expression difference.

A prospective record prevents the goal from shifting after results arrive. Record what the agent predicted, which candidates were selected, why they were selected, and which outcomes would support or weaken the explanation. That record turns a compelling retrospective story into an assessable research contribution. It also makes negative results informative: the team can identify whether a prediction, a selection rule or an experimental assumption failed.

The next experiment earns the broader claim

The most useful mechanistic hypothesis narrows the space of possible outcomes. It explains the current observation and specifies what should happen under a further intervention or measurement. The test should distinguish the hypothesis from a serious alternative, not merely repeat the pattern that motivated it.

For example, a proposed chain connecting a perturbation to an expression response contains multiple links. Measuring the expression response again can establish its repeatability. Measuring a predicted intermediate can support the chain. Altering that intermediate and observing a specified consequence can test a stronger causal claim. These are different increments of evidence, even if an agent describes them in one paragraph.

A rescue experiment also needs a precise endpoint. Restoring an RNA pattern does not automatically demonstrate restoration of a cellular function. Restoring one cellular function does not automatically establish an organism-level benefit. A research report becomes more informative when it names the endpoint it can actually support.

Independent replication adds another dimension. Repeating a result under closely matched conditions tests reliability within that setting. Changing the organism, developmental stage or measurement method tests transportability across a stated boundary. A failed transfer can reveal a meaningful dependency rather than simply erase the original result.

The concrete handoff from an agent should therefore preserve the claim, original observation, proposed mechanism, competing explanation and discriminating next test. It should identify which parts are measured and which are inferred. A researcher can then judge whether the next experiment is feasible and whether either possible outcome would change the explanation.

Biological agents can make a large experimental resource easier to interrogate. Their reports become scientifically useful when they preserve the structure of the evidence they assemble: intervention and response, statistical contrast and uncertainty, precedent and interpretation, prediction and prospective test. The discovery claim grows when the evidence grows. The clarity of the narrative should help readers see that progression.