The Bottleneck Was Often Execution, Not Comprehension
The hardest part of reusing a computational paper is often not understanding its claim. It is getting the code, dependencies, data paths, and execution order to work together once. Researchers must find the repository, build an environment, resolve version problems, follow the tutorials, and decide whether the outputs match the paper’s figures. None of this is a new scientific insight, but it determines whether a method can leave its authors’ machines.
Paper2Agent makes that hidden engineering work explicit. Developed by Stanford researchers Jiacheng Miao and James Zou, the system takes a paper and its codebase and produces a Model Context Protocol server. Once connected to Claude Code or another MCP-compatible agent, users can invoke the paper’s methods through natural language instead of rebuilding the workflow manually from a PDF and a repository.
The important change is not that a paper can now chat. Each generated server exposes three kinds of components: executable tools for the methods, resources containing the manuscript, datasets, and figures, and prompts that encode the order of multi-step workflows. The paper can therefore specify not only what a function does, but also when it should be called, how it should be combined with other functions, and which outputs count as references.
The Core Design Turns Tutorials into Executable Contracts
Paper2Agent is not primarily a language model browsing a repository without constraints. Its central orchestrator dispatches specialized sub-agents through a fixed pipeline. One locates and downloads the codebase, another builds an isolated environment, a tutorial scanner identifies usable workflows, and a tutorial executor runs them end to end. A tool extractor then turns those tutorials into parameterized MCP tools, while a verifier checks whether they reproduce the reference outputs.
The validation gate operates at the level of files, numbers, and images. Expected files must appear, numerical outputs must remain within a 3% tolerance, and figures are compared with perceptual hashes whose Hamming distance must stay below 20. Each function receives up to six repair attempts. Tools that continue to fail are excluded rather than being presented as reliable interfaces. This sacrifices some coverage, but prevents functions that merely appear callable from entering the final server.
From an engineering perspective, this is the important redesign: a tutorial becomes a reference implementation for both tool generation and acceptance testing. In the AlphaGenome case, the system produced 22 tools in about 45 minutes at a cost of $14, and all passed validation. Across 300 questions, the agent reached 91.2% accuracy, compared with 80.3% for Claude Sonnet 4 alone. Average cost and runtime per question were $0.20 and 1.6 minutes, versus $0.38 and 4.3 minutes for the comparison setup.
Passing the Tests Does Not Settle the Biology
Paper2Agent is most likely to be overestimated precisely where it is most persuasive: it can reproduce a tutorial consistently. Across 100 bioinformatics papers, 74 were successfully agentified, and 593 of 599 candidate tools passed validation. This suggests that the workflow is approaching batch engineering, but it also shows that conversion is not universal. The available material does not explain whether the remaining 26 papers failed because of code, environments, data, or tutorials, so the result cannot be generalized to all research software.
The sharper boundary appears on new questions. When Paper2Agent used AlphaGenome to re-examine an LDL-cholesterol-associated variant, it ranked SORT1 as the more likely causal gene, while the original paper emphasized CELSR2 and PSRC1. The agent could invoke the right tools, complete the computation, and produce a ranking, but procedural reproduction alone cannot resolve conflicts among new data, model assumptions, and biological interpretation. Executability makes disagreement easier to expose; it does not remove the disagreement.
Technical leaders should therefore treat Paper2Agent as a verifiable adaptation layer for research software, not as an autonomous corresponding author. The best initial candidates are workflows with public code, explicit tutorials, stable outputs, and comparable reference results. In deployment, the original paper’s claims, the agent’s outputs on new data, and expert judgment should be recorded separately. Passing regression tests should never be allowed to masquerade as proof of a new scientific dis
