Pretraining data filtering worked, then $80 of fine-tuning undid it

The UK AI Security Institute has put a compute price on removing the safeguard that open-weight biological model releases rely on most. Its Chem-Bio team pretrained protein language models from scratch with viral proteins stripped out of the training corpus, measured what the models lost, then measured what it cost to put the capability back. The loss was real and narrow. Recovering it at the largest scale took 1.31 billion fine-tuning tokens at most and, by the authors’ own conversion, 47 to 79 USD of H100 time.

The paper, The Effectiveness and Durability of Pretraining Data Filtering in Protein Language Models, was submitted to arXiv on 19 October 2026. AISI published its blog summary and research listing on 26 October 2026. This article is published on 12 November 2026. The Frontier Wire has not retrained any of these models or rerun any of these evaluations; what follows is a reconstruction of the published experiment, its appendices and its stated assumptions.

The blog credits AISI’s Chem-Bio team. The paper is a collaboration: of its twelve authors, seven list the UK AI Security Institute, with the rest at the University of Oxford, SecureBio, the Broad Institute of MIT and Harvard, or unaffiliated. Oliver M. Crook of Oxford and Felix Jackson of AISI are the joint senior and corresponding authors.

The reason it matters beyond protein models is that pretraining filtering is already widely used in production models, and for biological foundation models the authors call it the safeguard of choice. Of the safeguards built so far, the paper describes it as the most tamper-resistant, because the capability was never trained in, so there is nothing to unlearn or jailbreak. The prior evidence for that durability came from general-purpose language models. Specialised scientific models are a harder case, because the data left behind after filtering looks a great deal like the data removed.

The short answer from this experiment: filtering degrades the targeted capability selectively, the effect gets larger with model scale, the cost of reversal also gets larger with scale, and at 3B parameters that cost is still a small fraction of one percent of the cost of pretraining. The authors’ conclusion is that filtering raises the price of recovering a capability rather than removing it, and that it has to be paired with controls on access to the model or to the data needed to retrain it.

The short version

• 23 ESM-2 models were pretrained from scratch at 8M, 35M, 150M, 650M and 3B parameters, on an unfiltered UniRef50 corpus and on four nested viral exclusion tiers.
• Viral masked-language-modelling loss worsened by 0.006 nats at 8M and 0.173 nats at 3B under the all-viruses filter, while non-viral loss moved by at most 0.008 nats.
• Mutation-effect prediction degraded significantly only at 650M and above; long-range contact prediction from 150M up.
• Around 77.5% of the 3B language-modelling degradation came from a single viral family, Retroviridae.
• Fine-tuning on the withheld sequences restored the losses on all three tasks, costing 14 to 24 exaFLOPs at 3B, which the authors price at 47 to 79 USD.

What was actually removed from the corpus

The filtering target was defined taxonomically, not by a classifier. A sequence counted as viral if it was a UniRef50 cluster representative annotated to a viral taxon in NCBI Taxonomy, using the version dated 1 June 2026, with host assignments from Virus-Host DB release 228. Tier membership is therefore decided row by row from a taxonomy identifier rather than from a model’s score, which is what lets the authors report exactly how many sequences each filter removed and measure selectivity against a fixed boundary.

Six tiers were defined, nested so that each narrower tier is a strict subset of the wider one. The authors pretrained on tiers 0 through 3 and the unfiltered corpus, and evaluated on held-out samples from all five viral shells. Tier 4 drew on biosecurity control lists: the Australia Group’s list of human and animal pathogens and toxins for export control, excluding its warning list, plus the US select-agent lists at 42 CFR 73, 9 CFR 121 and 7 CFR 331. The authors then apply two manual corrections: nine taxa on those lists could not be matched to an NCBI taxon automatically and were added by hand, and one spurious match was removed. Separately, they add Andes, Choclo and Dobrava orthohantavirus, which cause human disease without carrying a human host record in Virus-Host DB.

Entry counts and percentages as reported in Figure 2 and Table 1 of the paper. UniRef50 release 2021_04 holds 49,874,565 cluster representatives; after dropping 35,036 sequences with more than 10% unknown residues, the held-out splits and the evaluation homologues, the unfiltered training corpus is 49,507,711 sequences, and the percentages above are shares of that corpus.

Those last two rows are the most striking numbers in the table. The set of sequences that a narrow, biosecurity-list-shaped filter would remove is 963 entries, two thousandths of one percent of the corpus. Influenza A is 229 entries. A filter aimed only at the organisms on the control lists is touching almost nothing, which bounds both the protection it can offer and the collateral damage it can cause.

Tier 0 is the interesting extreme in the other direction. It was built by aligning every tier-1 viral sequence against the corpus with MMseqs2 at 80% coverage and keeping anything at 30% sequence identity or above, then taking the union with tier 1. That drags in 1.93 million additional non-viral entries, which is the cost side of the trade the authors set out to measure: broader filtering removes the material an adversary could use to reconstruct the capability, at a potentially greater price in general usefulness.

Twenty-three models, one token budget

Every model used the ESM-2 architecture described by Lin et al. in 2023, an encoder-only transformer trained with a masked language-modelling objective on amino acid sequences rather than words. Five sizes were trained: 8M, 35M, 150M, 650M and 3B parameters. All five were trained on the unfiltered corpus and on tiers 0 and 1; tiers 2 and 3 were trained up to 650M only. No tier-2 or tier-3 model exists at 3B, which limits the tier comparison at the top scale.

The training budget was held constant rather than scaled: 150,000 steps at an effective batch of 2,097,152 tokens, approximately 315 billion tokens, with a maximum sequence length of 1,024, at every size and on every corpus. AdamW, a peak learning rate of 4e-4, 2,000 warmup steps, bf16 precision, and gradient clipping at 1.0 applied only at 3B. Pretraining large protein language models is expensive, so each configuration was trained with a single seed, which the authors name as a limit on how confidently differences between scales can be read.

One detail matters for anyone comparing these results to published ESM-2 numbers. The stock ESM-2 checkpoints were trained for 500,000 steps; these models stop at 150,000, where the learning rate has decayed only to 2.93e-4. The authors report the gap: mean Spearman correlation over 217 ProteinGym substitution assays sits below ESM-2 at every size, but by at most 0.016, while held-out cross-entropy diverges above 150M, reaching 0.076 nats worse at 650M and 0.125 nats worse at 3B. The unfiltered models are a fair internal control. They are not a reproduction of ESM-2.

Evaluation sets were fixed across every training configuration. Held-out language-modelling evaluation used 5,161 viral sequences, split into five shells, plus the same 3,000 non-viral sequences for every model. Training sequences homologous to the evaluation sets, to ProteinGym wild types and to 917 influenza A proteins were removed with MMseqs2 at a sequence-identity threshold of 0.8, which is the step that stops the comparison being contaminated by near-duplicates.

Three tasks, three different thresholds

The three evaluations were chosen to span what a protein language model is actually used for. Masked language modelling is scored as mean cross-entropy in nats, masking one residue at a time and reading the model’s probability for the true amino acid. Mutation-effect prediction is scored zero-shot by masked marginals, correlating predicted log-odds against deep mutational scanning measurements across 217 ProteinGym substitution assays, 31 of them viral and 186 non-viral, and the 45 viral assays of the EVEREST benchmark. Contact prediction uses an L1-regularised logistic regression probe over the frozen model’s attention maps, trained on 20 held-out non-viral chains and scored as precision at L on long-range residue pairs, meaning pairs separated by at least 24 residues, over 1,403 viral and 1,403 length- and assembly-matched non-viral chains from the Protein Data Bank.

The three tasks did not fail at the same point. That is the most useful single finding in the paper for anyone designing a release-time test, because it means a filtering claim checked on one task says little about another.

All figures are for the tier-1 filter, all viral proteins removed, against the unfiltered model of the same size. Cross-entropy and Spearman values are from Table 4; positive cross-entropy differences mean the filtered model is worse. Contact prediction changes are percentage points from Section 4.1.

At 8M and 35M parameters, mutation-effect prediction showed no significant change at all: +0.001 Spearman with p = 0.844 at 8M, and -0.002 with p = 0.816 at 35M. At 150M it was -0.019 with p = 0.264, still not significant. The capability that filtering was supposed to suppress had not developed strongly enough at those scales for its absence to be measurable on that benchmark. Contact prediction, by contrast, degraded materially at every size above 35M: -1.58 percentage points at 150M, -1.70 at 650M and -2.80 at 3B.

The directional consistency is what makes the result credible rather than noisy. Cross-entropy degradation under tier 1 grows monotonically with scale, from +0.006 nats at 8M to +0.019, +0.074, +0.125 and +0.173. The mechanism the authors give is straightforward: unfiltered models improve faster on viral proteins as they scale than filtered models do, so filtering holds back a gain the unfiltered model keeps collecting.

The selectivity claim, stated precisely

Selectivity is the whole argument for filtering as a safeguard. If removing the dangerous data also removes the useful capability, a developer will not do it. The paper’s retain-domain evidence is unusually specific, and worth stating in the form the authors used rather than as “performance was preserved”.

On masked language modelling, non-viral cross-entropy moved within 0.008 nats at all five sizes, and the change was shown statistically equivalent to zero within a 0.02 nat margin by two one-sided tests at p below 3e-12 on the 3,000 non-viral sequences. On mutation-effect prediction, scaling from 8M to 3B gained the unfiltered model +0.232 Spearman on non-viral assays and the tier-1 filtered model +0.231, which the paper reports as the filtered model capturing 99% of the non-viral improvement (95% CI 96 to 103) against 36% of the viral improvement (95% CI 13 to 60).

Filtering did not trade general capability for safety. It removed a capability the retained data could not rebuild.

Contact prediction produced the one genuinely surprising retain-domain result: non-viral long-range precision improved under filtering at 8M and at the two largest sizes, by +0.73 percentage points at 650M and +0.60 at 3B, with all changes significant at p below 1e-7 on a per-protein test. The authors do not claim a mechanism for that. It complicates the simple story in both directions, because a safeguard that improves the retained domain is easier to adopt, and because an unexplained improvement is a reason to want the experiment repeated with more than one seed.

A fair reading of the selectivity result is that it holds for these three proxy tasks on held-out sequences, not for every downstream use of a protein model. Structure prediction, protein design and function annotation pipelines built on these representations were not evaluated. The paper confines its retain-domain claims to the three tasks it measured, and the phrase “largely preserving performance on non-viral proteins” in its abstract means those three and nothing wider.

Where the loss actually landed

Averaged numbers hide the most consequential finding. The degradation was not spread evenly across viruses. It was concentrated in the shell of human-infecting viruses, and within that, in one family.

Table 5 of the paper, at 650M parameters. Shares are weighted by the number of held-out sequences in each shell and sum across the five viral shells.

At 3B under tier 1, Retroviridae accounts for roughly 77.5% of the summed increase in held-out cross-entropy among viral families with at least 20 sequences in the test set, followed by Picornaviridae at about 6.56%, unclassified viruses at 4.11%, Hepaciviridae at 3.25% and Mimiviridae at 1.17%. On mutation-effect prediction at 3B, Retroviridae is the only family where the authors say they can be confident degradation occurred at all, at -0.342 Spearman over six assays with a 95% confidence interval of -0.481 to -0.203. Flaviviridae shows the next largest shift at -0.294, but over only three assays, so the interval of -0.614 to +0.025 does not even settle the sign. Orthomyxoviridae, the family containing influenza, moved -0.018 over nine assays. Siphoviridae moved -0.010 over five.

There is an explanation, and it is uncomfortable for the safeguard. Retroviridae is the viral family the unfiltered 3B model performs best on, at 1.05 nats against 1.64 for the next best. Across the 30 families with at least 20 held-out sequences, baseline performance correlates negatively with the degradation the tier-0 filter causes: Spearman -0.509 at 650M and -0.522 at 3B, both at p = 0.004, estimated on disjoint halves of each family’s held-out sequences so that the baseline and the difference come from separate sequences. Filtering takes away most where the model knew most. The families a biosecurity-motivated filter is most concerned about, such as influenza A, are exactly the families where there was least to take away.

That is a measurement problem as much as a safety one. Influenza A contributes 229 training entries and 273 held-out sequences. A filter can remove all of it and the aggregate forget-domain metric will barely move, which means aggregate metrics are the wrong instrument for judging whether a biosecurity filter did anything.

How the $80 was built

The headline durability number deserves its full derivation, because it is a conversion rather than an invoice, and because the conversion has four assumptions in it.

Recovery was attempted by continuing to train each filtered model on the withheld data: the 434,907 tier-1 viral sequences, for 5,000 steps at an effective batch of 262,144 tokens, approximately 1.31 billion tokens in total, at a learning rate of 5e-5. A model counts as recovered at the first scored checkpoint where it matches or beats the same-size unfiltered model on the metric in question. Checkpoints were scored every 500 steps, or 0.131 billion tokens, which is why every recovery cost is reported as a bracket rather than a point.

Compute was counted as (6N + 12·L·s·d)·D, the usual 6ND plus the attention term that 6ND omits, with the context length s fixed at 1,024. Because the mean sequence is far shorter than the context, that overcounts the attention term, so the reported cost is an upper bound. Compute was then converted to H100-hours at 34% model FLOPs utilisation against a 989 TFLOP/s bf16 dense peak, which the authors call conservative against the 38 to 43% reported for Llama 3 training, and to dollars at a nominal 4 USD per H100-hour based on market rates for an eight-H100-SXM instance at the time of writing.

ExaFLOP brackets are from Section 4.2 of the paper. Both columns are for tier-0 filtered models fine-tuned on the withheld tier-1 corpus: the masked-LM figures are for the phage and unclassified shell, the mutation-effect figures for the mean over the 45 EVEREST assays. Dollar figures for 150M and below, and the 650M row, are The Frontier Wire’s arithmetic using the paper’s stated 34% utilisation, 989 TFLOP/s peak and 4 USD per H100-hour; the 47 to 79 USD figure at 3B is the paper’s own.

The 3B masked-language-modelling bracket is the widest the authors report, because only four checkpoints were scored at that size, at 500, 1,500, 3,000 and 5,000 steps, against ten at the other sizes. The interval still does not overlap with 650M, so the scaling direction holds even with the coarse grid.

A worked example on the price assumptions

The dollar figure is the number that will travel, so it is worth asking how much it moves when the two market assumptions move. The following is arithmetic on the paper’s own compute bracket of 14 to 24 exaFLOPs at 3B, using the same 989 TFLOP/s bf16 peak. The utilisation and price rows are hypothetical inputs chosen to span plausible conditions, not measurements, and not scenarios the authors endorse.

Worked example by The Frontier Wire. Hours are the paper’s exaFLOP bracket divided by 989 TFLOP/s times the stated utilisation. The middle row at 4 USD reproduces the paper’s reported 12 to 20 H100-hours and 47 to 79 USD.

The point of the exercise is that the conclusion is robust to the assumptions in a way the specific number is not. Across a factor of two in utilisation and a factor of four in hourly price, the cost of recovering the filtered capability at 3B stays between about 16 USD and about 216 USD. None of those figures is a meaningful barrier. The ratio underneath them is what makes that robust. Recovery took at most 1.31 billion fine-tuning tokens against the roughly 315 billion tokens of pretraining that produced the model. Both run on the same model at the same cost per token, so the reversal costs under half a percent of the original training run whatever the hourly rate turns out to be.

What the figure does not include is the rest of the attacker’s bill. It excludes the cost of assembling the fine-tuning corpus, which in this case is the easiest part, since UniRef is public. It excludes the engineering time to run a full-parameter fine-tune at 3B, which the authors flag in a footnote about the LLM literature: some actors may simply lack the machine learning capability. It also excludes any cost of knowing which data to collect, which is where a tier-0-style homology filter imposes a real burden, because the adversary has to reconstruct a set the developer deliberately widened.

What more filtering buys

If narrow filtering is cheap to reverse, the natural response is to filter wider. The paper tests that directly, and the returns are poor.

At 650M, moving from tier 3 to tier 0 increases the number of excluded UniRef entries by a factor of 42, from 0.113% to 4.77% of the corpus, and produces roughly 1.6 times the increase in viral cross-entropy over the unfiltered model, from 0.102 to 0.161 nats. Retain performance under tier 0 stays within 0.006 nats of the unfiltered model at every size, so the extra filtering is close to free in general capability. It is just not very effective.

Table 4 of the paper, right-hand columns. Positive values mean the wider tier-0 filter degrades viral language modelling more than the all-viruses tier-1 filter.

The gain from removing homologues as well as viruses is statistically indistinguishable from zero below 150M parameters and grows with scale, which is the one encouraging trend in the paper for anyone betting on filtering: stringent filtering becomes more effective as models get larger. At 3B, tier 0 reaches 0.259 nats of degradation against tier 1’s 0.173.

There is a matching result on contact prediction at 650M, where viral long-range P@L falls by 0.41 percentage points under tier 3 and by 3.60 points under tier 0, a mean slope of -1.06 points per ordered filtering level with a 95% confidence interval of -1.23 to -0.90. Non-viral changes across the four tiers range from -0.57 to +0.73 points. Stronger filters hurt the forget domain progressively more, while the retain domain stays inside a band of about one percentage point either side of zero.

The tier-2 and tier-3 experiments add a finding that points away from filtering and toward data control. At 650M, the effect of a partial filter stays largely confined to the viral families whose data was removed, with little general effect on other families. Under tier 2, 57 of 118 removed families degraded by more than 0.02 nats against 13 of 68 retained families; under tier 3, 21 of 38 against 20 of 148. A filter aimed at a controlled list of families mostly affects those families.

The Retroviridae control, and why it matters

Because one family dominated the degradation, the authors ran the obvious check: fine-tune the filtered models on a recovery corpus with Retroviridae stripped out, leaving 422,204 of the 434,907 withheld sequences, and see whether the remaining viral performance is harder to restore.

It was not. Withholding Retroviridae did not make recovery of the other families meaningfully more expensive, and fine-tuning restored performance at comparable compute across model sizes up to and including 3B. On the human-infecting shell excluding Retroviridae, 535 sequences, models starting from tier-0 and tier-1 checkpoints recovered the same-size unfiltered reference within 2,000 fine-tuning steps at 650M and faster at smaller sizes.

That cuts two ways, and the paper says so in its conclusion. Negatively, it means an attacker does not need the single most informative family to undo the filter. Positively, it means the models were not generalising viral language modelling across families: they relied on data from a target family to become good at that family. The authors call this encouraging for the alternative intervention, controlling access to the data itself, since capability followed data availability closely rather than leaking in from neighbours.

Recovery has one side effect that matters for anyone who would actually do this. For the tier-0 models fine-tuned on the withheld viral corpus, non-viral cross-entropy rises over the evaluated checkpoints at 650M and 3B. Buying the viral capability back costs a little of the retained domain, which the original filtering had left untouched.

Back on the full withheld corpus, contact prediction recovers past the baseline rather than up to it. After viral-only fine-tuning from tier 1, mean viral precision rose against the filtered starting checkpoint in all 45 size, range and precision combinations, and reached or passed the same-size unfiltered level in 44 of them. For long-range P@L at 650M the fine-tuned model ends up +1.87 percentage points above the unfiltered model, with a 95% confidence interval of 1.52 to 2.22. That overshoot is worth noting against the word “recovered”: on this task the fine-tuned model ends above the reference it was being compared with, having had a round of viral-only training the unfiltered model never received.

How this compares with the LLM filtering evidence

The protein result reads differently once placed next to the general-purpose language model work it builds on. The comparison also shows how uneven the evidence base is: each study measures a different domain, with a different filter, against a different threat model.

Figures are as reported by each study’s authors. The Anthropic post, the Deep Ignorance arXiv listing and the Moonshot blog were read for this article. Other work the AISI paper builds on, including token-level filtering results on GPT-2 models up to 1.8B parameters and modular-pretraining results from 50M to 5B, is cited here only as the AISI paper describes it, and The Frontier Wire has not read those papers.

Two things stand out. First, the two priced reversals land in the same place despite coming from different domains. The AISI figure for a 3B protein model is 47 to 79 USD over at most 1.31 billion fine-tuning tokens; AISI’s own footnote puts a single adversarial fine-tuning run on the 6.9B Deep Ignorance model near 50 USD, and reversal under 100 USD. By AISI’s own reading of that paper, the 6.9B language model recovered within about 305 million tokens, so the smaller protein model needed the larger token budget and the two bills still overlap. What AISI adds is the measurement itself: the paper states that the tamper resistance of filtering in biological foundation models was untested before this work. Second, the Moonshot row is what the practice looks like in the wild. A vendor removes a dual-use data category, reports a drop on an internal benchmark that is not public, and ships the weights. The AISI paper is the kind of evidence that would tell a reader whether such a cut survives an afternoon of fine-tuning. For K3.1 Pro, that evidence does not exist in the release post.

Two shipped biological models show how established the practice is. As the AISI paper describes the prior art, Evo 2’s training data excludes the genomes of eukaryote-infecting viruses, and ESM3-open’s excludes a large set of viral proteins, with the ESM3-open release reporting substantial degradation of viral mutation-effect prediction and little degradation elsewhere. What none of those releases established is durability, which is the gap this experiment fills.

What remains unknown

The paper’s limitations section is unusually frank, and a reader should weight the conclusions accordingly.

The results come from one model family. ESM-2 was chosen because it is widely used and has relatively well-documented training code, but the authors note that it performs relatively poorly on viral masked language modelling and mutation-effect prediction compared with other protein language models. A stronger starting model could show a larger filtering effect, a different recovery profile, or both. Retrieval-augmented protein models such as Protriever and Tranception were not tested at all, and the authors expect filtering to be less effective where a model can fetch viral sequences at inference time.

Every configuration used a single seed, so differences between scales carry less statistical weight than the p-values on individual comparisons suggest. The 150,000-step budget is less than a third of the schedule used for the released ESM-2 checkpoints. The recovery measurement is limited by a 500-step checkpoint grid, 0.131 billion tokens, which the authors say makes subtle trends in tamper resistance hard to detect, and at 3B only four checkpoints were scored.

The deepest open question is what recovery means. “Recovered” here is defined as matching the unfiltered model’s cross-entropy, Spearman correlation or contact precision on held-out sequences and public assays. None of those is a measure of biological uplift. The paper does not claim that a recovered model enables anything in particular, and no reader should infer that from it. The relationship between a 0.173 nat cross-entropy gap and any real-world capability is exactly the kind of inference that needs its own evidence, which is why keeping the evidence levels in AI for biology separate matters when reading a result like this one.

Several specific pieces of evidence would change the picture. Replication on a second architecture, ideally a stronger viral performer, would tell us whether the scaling trends are ESM-2 artifacts. A recovery experiment at 10B or more would test the authors’ own suggestion that the cost could plausibly become a substantial obstacle for much larger models. A denser checkpoint grid would turn wide brackets into usable scaling estimates. A test against a retrieval-augmented model would tell us whether filtering means anything when the sequences can be supplied at inference time. And an uplift-relevant endpoint, rather than a benchmark proxy, would connect the whole exercise to the risk it is meant to address.

A proposed checklist for filtered releases

The gap this paper exposes is not only scientific. It is that a developer can say “we filtered the dangerous data” and a reader has no way to judge the claim. The following checklist is proposed by The Frontier Wire, derived from the measurements this experiment had to make. It is not a standard, and no developer has adopted it.

Measured against those seven rows, the Kimi K3.1 Pro release post examined above names the category removed and the two training stages it was removed from, and reports one forget-domain number for an internal unfiltered checkpoint against the shipped model. It does not state an exclusion rule precise enough to check, a volume removed, a matched retain-domain figure for that same checkpoint, a per-subdomain breakdown, a recovery attempt, any compute accounting, or whether the removed data can be obtained elsewhere. That is the practical finding here, more than the dollar figure: the measurement apparatus needed to assess a filtering claim now exists, has been published in full with its appendices, and takes 23 pretraining runs. Without the recovery experiment, a reader is being asked to take durability on trust.

AISI’s own conclusion is narrower than the dollar figure that will be quoted from it. Filtering is moderately effective at selectively degrading performance on viral proteins. It is not durable against fine-tuning by someone with modest compute and access to the relevant data. A durable approach therefore needs something else alongside it, either controls on access to the models or controls on access to the biological data required to retrain them. The second of those is the one this experiment makes a case for, because the models in it did not learn viral proteins from anything except viral proteins.

That conclusion sits alongside the broader problem of deciding what an evaluation actually bounds, which AISI’s incident work on evaluation boundaries treats directly, and the related question of how much of a biological result is interpretation rather than validation, visible in Google’s agent-assisted wet lab studies. For release decisions specifically, the durability question now belongs next to the licence terms in any survey of what open weights permit.