AI writing, AI reviewing and the verification bottleneck

An AI review is valuable when it turns a vague doubt into something a researcher can check: an unsupported citation, a counterexample, a failing test or a precisely stated missing assumption. A fluent report alone does not do that. The emerging review stack contains several distinct kinds of verification, and the evidence becomes easier to interpret when each tool is assigned the question it can actually answer.

TL;DR

• Document extraction, citation identity, technical criticism, experimental reproduction and formal proof are different operations.
• Author satisfaction and error detection measure different outcomes. Neither alone establishes that an AI should decide which papers are accepted.
• The most useful review output links a specific claim to an inspectable artifact and a proposed resolution.

Google’s October 14 Deep Review announcement describes public document feedback extending work on its Paper Assistant Tool. The earlier conference programmes offered private author assistance. This survey examines the underlying verification jobs and their limits; The Frontier Wire has not run the product or independently measured its accuracy.

The distinction matters because reviewing a manuscript is not a single task. A paper can have accurate citations and an invalid experiment. It can have working code and an unjustified interpretation. It can contain a valid theorem whose assumptions do not fit the application. The review system needs to make these differences visible rather than combine them into a general impression of rigour.

Extraction comes before criticism

The first task is to recover the document the author actually submitted. GROBID’s project documentation describes extraction of scientific-document structure, bibliographic references and citation contexts into structured representations. Such extraction can make later analysis possible. It does not establish whether the extracted content is scientifically correct.

Consider a hypothetical formula containing a minus sign in a denominator. If the parser drops that sign, a later critic may correctly identify a contradiction in the extracted formula while falsely accusing the original paper of an error. The reasoning stage can be internally coherent and still assess the wrong object. The artifact that resolves the dispute is the page region alongside the extracted representation.

Tables create a related problem. A number without its column header can be assigned the wrong meaning. A footnote might limit the population or exclude a subset. If those qualifiers disappear, an apparently precise comparison can be wrong before any advanced reasoning begins. A sensible reviewing interface preserves a route back to the original visual location.

For a research team evaluating a system, this suggests a separate extraction audit. Select examples with unusual notation, tables, references and figures; compare the representation with the original; record which later findings depend on questionable extraction. This is a proposed assessment, not a reported Deep Review result. It makes the source of an error identifiable rather than labelling every false criticism a reasoning failure.

The resulting division of work is useful even when humans do the review. The person adjudicating a disputed claim can first ask whether the reviewer saw the correct content. That inexpensive question may prevent a lengthy argument over a problem introduced by a conversion step rather than by the research itself.

A citation can exist and still be wrong

Crossref’s REST API exposes scholarly metadata deposited by publishers and trusted sources. It can help resolve a work’s identity and retrieve fields such as titles, identifiers and publication information. That is an important check against invented or mismatched references. It is not an entailment test for the sentence carrying the citation.

Imagine a hypothetical manuscript claiming that a training method improves accuracy under a particular distribution shift. The cited paper may exist, have the expected authors and concern the same method. If it tests only an in-distribution setting, the citation still does not establish the manuscript’s narrower claim. Bibliographic correctness and evidentiary support have come apart.

A useful citation checker should therefore expose at least two states. One says that a work was identified. Another says that a particular passage was examined for support. A third, stronger state might record that the conditions match: population, model size, evaluation split or other relevant setting. These are proposed categories for review records, not a claim that any specific product implements them.

Versioning adds another layer. Crossmark is designed to surface updates, corrections and retractions supplied by participating publishers. A reviewer needs to know whether it is relying on a superseded statement or a current record. The presence of a status mechanism is useful evidence access, not an automatic guarantee that every research claim remains sound.

The practical output is a citation ledger with the manuscript sentence, resolved source, relevant source passage and any mismatch. That gives an author a concrete decision: narrow the sentence, find better evidence or explain why the interpretation is justified. A general assurance that all references were checked does not offer the same accountability.

Reasoning checks produce challenges to inspect

An AI critic can examine whether a conclusion follows from a paper’s stated assumptions. Its most useful contribution may be a small counterexample or an identification of an unsupported step. This differs from writing a persuasive review paragraph. A counterexample gives the author a concrete object to evaluate.

For a hypothetical claim that every member of a set has a property, one valid member without that property disproves the universal statement. The critic should supply the object, show that it satisfies the premises and identify the failed conclusion. If any part is missing, the author has a suggestion rather than an established refutation. This distinction applies whether the suggestion comes from a model or a human.

The same discipline helps with less formal arguments. Suppose a manuscript compares two systems but changes both the algorithm and the data filtering. A critic can identify the resulting ambiguity without proving that either system is worse. The supported criticism is that the design does not isolate the claimed cause. Declaring the claimed improvement false would go beyond that evidence.

The PAT research paper describes segmentation, deeper per-segment review and synthesis, with attention to grounding and duplicate criticisms. That architecture is relevant to how a system can search for issues in a long document. It does not convert every generated objection into a checked fact or establish equal accuracy across scientific domains.

A useful review record marks each criticism by status: proposed, checked, rebutted or unresolved. The statuses should attach to issues rather than to the review as a whole. An AI can find one genuine error and several mistaken ones in the same document. Preserving that mixed record is more informative than a binary label that the entire review was helpful.

Executable checks test an explicit expectation

Code offers a different verification surface. A reviewer can ask whether a computation produces the values a manuscript reports, or whether a unit test exposes a proposed bug. The key is to state the expectation before treating a successful run as scientific confirmation.

The pytest assertion documentation provides a simple example of this structure: a test compares an actual value with an expected one and reports a failure when they differ. A test framework helps make the comparison visible. It does not choose the scientifically appropriate expectation on the researcher’s behalf.

Consider a hypothetical table whose total should equal the sum of three mutually exclusive groups. A check can recompute the sum and detect an inconsistency. That is a valuable arithmetic result. It does not establish that the groups were sampled fairly, that the labels are valid or that the observed relationship is causal. Each requires another question and another form of evidence.

Likewise, reproducing a training script can establish that a specified environment runs and yields a particular output. It cannot, by execution alone, show that the test set was independent of training data. A workflow that reproduces leakage faithfully remains a flawed experiment. The reviewer must inspect the design and data lineage as well as the terminal output.

An effective artifact bundle therefore includes the question, code revision, environment, inputs, command, outputs and interpretation. This is a proposed minimum record for a reproduction, not an assertion that one was performed here. It lets another researcher distinguish a failed computation from a disagreement about what the computation means.

Formal proofs establish a specified proposition

Formal proof systems occupy a stronger but narrower position. The Lean language reference describes a small trusted logical kernel and an extensible environment for constructing proofs. The essential distinction is between the process that searches for an argument and the machinery that checks the resulting formal object.

A hypothetical reviewer might ask an agent to formalise a small lemma on which a larger argument depends. If the lemma and assumptions are correctly represented and the proof is checked, the result addresses that proposition. It does not establish that the English paper was translated correctly, that all of the larger argument was included or that the lemma answers the empirical question in the title.

This is why a verified theorem should travel with its statement. A bare green indicator discards the very information that makes the result useful. Readers need the assumptions, the conclusion and the relationship between the formal vocabulary and the scientific objects under discussion. Otherwise the assurance can be technically valid and practically misunderstood.

Formalisation can also expose an ambiguity before a proof exists. If a phrase such as stable under perturbation has several interpretations, translating it into a proposition forces a choice. That clarification is useful work even if proof search then fails. A failed search is not itself proof that the proposition is false; it can reflect incomplete automation, a missing lemma or an unsuitable representation.

The taxonomy therefore places formal proof beside, rather than above every other kind of review. It can give unusually strong evidence about a precise statement. Empirical validity, relevance and originality remain different questions. Our examination of model cards as audit documents makes the same general point about matching an evidentiary artifact to a claim.

A map of verification jobs

The categories below organise the workflow by the question answered. They are an analytical taxonomy for this survey. They are not a ranking of products or an assertion that every stage has a fully automated implementation.

An author could use the map to route an objection. A missing reference identifier belongs in the bibliographic layer. A disagreement about a comparison group belongs in experimental design. A proposed contradiction in a theorem belongs in reasoning or formalisation. Routing avoids asking one generic reviewer to provide the same kind of assurance about every problem.

For a hypothetical paper on a new optimiser, a citation checker might identify the baseline paper, code checks might reproduce the table and a technical reviewer might challenge whether the compute budgets were matched. All three can be right. Their combined value comes from answering different questions, not from voting on whether the manuscript feels convincing.

Conference assistance is distinct from selection

The ICML programme retrospective describes private author feedback separate from official peer review and reports author-survey results. Satisfaction and reported changes are useful evidence about the experience of participants. They are not an independently adjudicated estimate of every finding’s correctness or a causal estimate of improved scientific quality.

That distinction is especially important when a tool helps an author revise before submission. The final manuscript is no longer the original document the tool assessed. A later acceptance decision reflects many factors. Without a controlled design, attributing the decision to the AI assistance would require more evidence than the programme description provides.

The NeurIPS author-assistance announcement likewise separates private feedback from the review process. It is a primary document for programme scope. It should not be treated as a retrospective measurement of the outcomes of every submission or confused with a separate experiment in AI-assisted reviewing.

The ICLR programme announcement preserves the same boundary and explicitly warns that the tool can flag correct statements and miss flaws. That warning is not a reason to dismiss assistance. It tells the user what role the output occupies: material for critical assessment, with responsibility remaining in the research process.

These programme documents support a clear conclusion about the institutions’ described use. They offered an additional feedback path. They do not establish that a model was given authority to accept papers. Keeping that distinction prevents the phrase AI peer review from obscuring the most consequential design choice: who is actually making the decision.

Review quality has several denominators

A system that produces many criticisms can increase the chance of finding a real flaw while also consuming more human attention on false alarms. Counting only the successful discoveries misses that cost. Counting only the number of reports misses the scientific value of the issues found.

Take an invented example in which a tool flags twenty issues, of which five are genuine. Another tool flags four, all genuine, but misses a fatal error the first tool found. Neither a single precision measure nor the raw finding count settles which is more useful. The answer depends on severity, review effort and the consequences of missing that particular flaw. These numbers are illustrative, not a product result.

The unit of analysis should therefore be defined carefully. Are duplicate descriptions one issue or several? Does a vague warning count as detecting an error if it fails to locate the problem? Does correcting a notation ambiguity have the same weight as refuting the main claim? A study needs explicit rules before an impressive percentage can be interpreted.

Author surveys answer another question. A user can find feedback helpful because it improves clarity even if it discovers no new technical defect. Conversely, a correct but costly objection might receive a less enthusiastic response. Helpfulness is a legitimate outcome. It should not be renamed technical accuracy simply because both are desirable.

Our guide to comparing evaluation frameworks explains why dataset design and scoring rules belong beside aggregate results. In AI reviewing, the same principle extends to adjudication: who decided an objection was correct, what evidence they saw and whether they knew which system produced it all affect how a comparison should be read.

The time saved can move elsewhere

The verification bottleneck is not resolved simply because a review report arrives quickly. The report may transfer work to authors, domain experts or programme organisers. The relevant question is whether it helps them reach better-supported conclusions with an acceptable total effort.

Imagine a hypothetical junior researcher receiving ten pages of technical criticism. If every objection includes a precise passage, a checkable example and a suggested resolution, the report can focus the next day’s work. If the same length consists of broad demands to improve rigour, it may create a second manuscript to interpret. Output volume is an incomplete measure of useful review.

A practical assessment would record time to inspect findings, time to resolve true issues and time spent rebutting incorrect ones. It would also record discoveries the author would otherwise have missed. This is a proposed measurement design, not a claim that the conference pilots collected all of these quantities. It gives the word productivity a denominator that includes the person receiving the feedback.

The same logic applies to review teams. Automated screening might remove easily resolved issues before a human sees the paper, leaving a harder average case for the human. Lower apparent reviewer throughput after automation would not necessarily mean the tool failed. The task mix may have changed. Measuring the complete workflow is more informative than timing only the model call.

A convincing efficiency result would therefore identify the beneficiary and the stage. Faster author revision, fewer unresolved technical errors and better editorial decisions are different outcomes. A tool can help one without demonstrating the others. The distinction makes positive findings more credible by keeping their practical meaning specific.

Confidentiality changes the available workflow

The privacy boundary also belongs in this taxonomy. A manuscript can contain unpublished results, embargoed information or data that the author is not free to send to an external service. The scientifically best criticism is not useful if obtaining it violates the conditions under which the work was shared. This is a workflow constraint, separate from a model’s reasoning quality.

The ICLR programme document describes its own data handling and limited-access arrangements. Those terms apply to that programme; they should not be assumed to describe the public Deep Review product or another reviewing service. An author choosing a workflow needs the terms for the actual destination, including retention, access and permitted use. Product names and underlying model families do not establish that two services handle manuscripts identically.

For a hypothetical collaboration, the appropriate first step could be an agreement that only a public appendix may leave the team’s environment. A bibliography checker could then operate on references while the sensitive experimental section stays local. That arrangement reduces what the external critic can assess, and the resulting report should acknowledge the missing context. Restricted access is neither proof of privacy failure nor permission to claim a full-document review.

A journal editor faces a different boundary: a reviewer may have received a paper under venue-specific confidentiality rules. The author-assistance programmes described above cannot be treated as blanket permission for reviewers to upload manuscripts elsewhere. The relevant institution’s policy governs the task. This is another reason to name the actor and purpose instead of treating all AI reviewing as one interchangeable activity.

A review trail that survives revision

A manuscript evolves. A criticism against one version may be fixed, become irrelevant after a rewritten section or reveal a problem that survives into the next draft. A useful review trail links each finding to a version and records its disposition. Otherwise the same concern can be repeatedly generated or prematurely assumed resolved.

For a hypothetical theorem correction, the trail would preserve the original statement, the proposed counterexample, the author’s assessment, the revised assumptions and the resulting proof obligation. That sequence helps a later reviewer understand whether the change fixed the underlying issue or merely made the wording more cautious. It is more useful than deleting every closed comment.

For an empirical correction, preserve the original experiment, changed code or data, regenerated outputs and revised interpretation. A new table alone cannot show which step changed. The point is not bureaucratic completeness. It is to make the scientific correction inspectable by someone who did not participate in the conversation.

A review system should also distinguish missing evidence from an established error. A data-access statement can be incomplete without implying misconduct. A method can be underspecified without being impossible. Those distinctions help an author respond accurately and protect the review process from turning uncertainty into accusation.

The definitive unit of progress is consequently a resolved claim with a durable evidentiary trail. AI can help create and inspect that trail at several stages. It can identify a reference, propose a test, locate a contradiction or produce a formal argument. The decision to treat the scientific claim as established still depends on which of those jobs was completed, under what conditions and with which unresolved assumptions.