Evidence at a glance

Simon Willison 2026 10 7Evidence
Boggan 24Evidence
BogganEvidence
BogganEvidence
Boggan OpenAI/math problem 180Evidence
OpenAI/math 722Evidence

Two Different Clocks Behind “Supposedly Proven”

On October 7, 2026, Simon Willison published a Hacker News comment by Jake Boggan. Boggan pointed to problem 180 in the OpenAI/math repository and said Barnette's Conjecture was “supposedly” proved. He also recalled working on it on and off for 24 years, spending thousands of hours on it, and even thinking for a few days last summer that he had solved it himself. The comment records what he had heard and what the problem meant to him. It does not confirm that the proof is valid.

The tension worth examining is between two clocks. A research project can appear in a public repository quickly, while one researcher may have lived with the same question for decades. If the proof holds, mathematical knowledge advances, but Boggan's sense of loss is not thereby a misunderstanding. He compared it to hearing that a former partner had died suddenly, a striking way of saying that an open problem can become more than a task to close. It can be an ongoing relationship with a line of inquiry.

Finite Checks Cannot Establish the General Claim

Proposed in 1969, Barnette's Conjecture asserts that every finite, simple, planar, bipartite, cubic, 3-vertex-connected graph contains a Hamiltonian cycle, a cycle that visits each vertex exactly once. The crucial word is “every.” The claim is not that all graphs already examined satisfy the property, nor that many examples have failed to produce a counterexample. It requires the conclusion to hold for all graphs meeting the stated conditions. A proof must support that universal claim.

Existing computational work is useful context, but it does not cross that logical gap. A 1985 study verified cases with at most 64 vertices. That establishes a bounded range, not a result for graphs of arbitrary size. For technical readers assessing mathematical output from a model, this distinction can be obscured by fast dissemination: checking many cases, producing a persuasive reasoning summary, and proving a universally quantified statement are different kinds of evidence.

The Repository's Scale Measures Output, Not Correctness

OpenAI/math expands model evaluation on open mathematical problems and gathers results generated by an internal model into manuscript families and papers. The supplied material describes a collection of 722 manuscripts grouped into 372 families, with the model asked about 4,000 questions during evaluation. Each result used an average of about three hours of ChatGPT Pro thinking compute, and the great majority came from a single unreleased internal model. These figures describe the scale of generating and organizing research leads, not a count of theorems confirmed by the mathematical community.

The value of this arrangement is that it turns otherwise scattered model outputs into an entry point that readers can browse and trace. The repository also publishes reasoning summaries for ten results, giving readers a view into how some of the work proceeded. But a larger collection makes it more important to distinguish “included as a manuscript,” “reasoning is available to inspect,” and “the conclusion has been proved.” Reading the number of entries as a success rate would create a level of certainty the evidence does not support.

Formalization Is Part of the Evidence, Not Decoration

The repository includes supporting material, some Lean formalizations, and reasoning summaries. The supplied account also says that not every manuscript has been formalized in Lean, and that results without formalization may contain problems. The entries therefore sit at different stages of verification. Their presence in the same repository does not give them equal credibility. For problem 180 in particular, the available material only records Boggan hearing that it was “supposedly” proved. It provides neither the proof text nor a verdict on its validity.

The role of formalization also needs to be stated precisely. Encoding a proof in Lean allows a formal system to check the reasoning steps in that encoding. It does not, by itself, establish that the formalization fully captures the original claim, that the proof has received independent mathematical review, or that the overall result has no remaining gap. To assess a result, readers need to know where the proof is, what its formalization covers, which parts still rely on informal argument, and who has checked it. A summary alone cannot provide that information.

Treat Model Output as a Research Lead, Not a Theorem Library

For research teams and technical leaders, a collection like this is best used for discovery and triage: it can help researchers screen mathematical leads produced by a model and find results worth checking or formalizing. It should not be treated as a trusted theorem library whose entries can be cited without further work. For a claim that affects theoretical conclusions, the process should begin with the original proof, examine its reasoning and assumptions, and then check the extent of formalization and independent review. A title or an entry count cannot establish that the claim is true.

Boggan's comment is a reminder that research tools change more than the speed of producing results. They also affect the pace at which findings are announced, encountered, and absorbed by people who have spent years on a problem. But the material here does not provide the proof of Barnette's Conjecture or say whether it has passed peer review. It therefore cannot establish that the conjecture is solved. The prudent judgment is to treat problem 180 as a research lead until the original proof and its verification status support a stronger conclusion. In mathematics, publishing a candidate answer begins the evidential process. It does not finish it.