Evidence at a glance
One Profile, Two Model Tasks
Simon Willison released web-edit 0.1a1, a pre-release of a web app for editing files in a local folder through a browser. The release includes performance work: Willison gave Claude Opus 5.5 a Python 3.15 profiling report generated by GPT-6 Astra, and Claude used it to guide code changes.
The important feature of this workflow is not that a model was simply asked to make the program faster. The work was divided into inspectable steps: use runtime data to locate a hotspot, change the implementation around that hotspot, then check the result with tests and benchmarks. A person supplied the context, selected the report, and carried the change into a release. The models contributed analysis and implementation, not an autonomous end-to-end maintenance process.
The Profile Pointed to a Specific Cost
Before the change, the profiler recorded 15,296 stack samples. The preserve_endings path appeared in 64.8% of them, and SequenceMatcher.find_longest_match appeared in 38.3%. Those figures overlap and should not be added together. They point to a specific operation: when saving a file, the program tries to retain the original line endings of unchanged lines, and general-purpose matching was consuming substantial time on some inputs.
This was not a comprehensive performance diagnosis across all usage patterns. The report used a deliberately constructed workload, focusing on saving a large file with mixed line endings. It identified a promising place to work, but hotspot percentages alone cannot establish that typical users encounter the same bottleneck most often. The profile also did not measure browser-side performance.
The Speedup Came from Recognizing File Structure
The change did not simply swap one general-purpose matcher for another. It first distinguishes the file's line-ending structure. If the original file uses only LF or only CRLF, the program skips line-by-line matching, normalizes the edited text, and converts it back to the original line ending. A simple structure gets a short path without searching for a correspondence for every line.
Mixed line endings require different treatment because unchanged lines may need to retain different endings. The new logic uses identical lines that appear only once on each side as anchors, uses bisect to find anchors in the same order, and limits SequenceMatcher to the gaps between them. The optimization narrows the expensive matching work rather than removing all matching.
What the Large Benchmark Gains Do Show
The pull request benchmark ran on Linux x86_64 with four CPUs and Python 3.13.16. It used a 12,000-line file and a directory containing 800 files, and reported medians across repeated runs. For a single-LF file, preserve_endings fell from 61.1 milliseconds to 1.1 milliseconds. For a single-CRLF file, it fell from 63.9 milliseconds to 1.7 milliseconds. For a mixed-ending file with one edited line in every five, it dropped from 3,963 milliseconds to 29 milliseconds. With one edited line in every twenty, it fell from 1,117 milliseconds to 25 milliseconds.
The improvements were not limited to saving. Under the same workload, the Uvicorn loopback HTTP edit-cycle throughput rose from 6.5 to 12.7 operations per second, while browsing 800 files fell from 28.5 milliseconds to 8.6 milliseconds. Directory traversal now uses os.scandir and string paths, binary-file detection uses bytes.translate, and file writes calculate a hash while copying instead of rereading the destination afterward. These results show substantial reductions in the measured costs of this benchmark. They do not establish that the entire application is faster by the same proportion for every user.
Performance Is Not the Only Result to Check
The line-ending optimization has a behavioral tradeoff: a few lines in a mixed-ending file may not retain their original endings. The pull request compared 600 sample files containing 266,444 unchanged lines. The new method preserved the correct endings overall 99.93% of the time, compared with 99.80% for the old method. That does not mean there were no regressions: in 25 cases, the new method preserved fewer correct endings than the old one, affecting one to six lines each time.
The scope of validation also needs to be stated precisely. The pull request reports 130 tests passing on Python 3.11 through 3.15 and 20,000 randomized cases used to compare the line-matching logic. However, the profile came from Python 3.15 while the benchmark ran on Python 3.13.16. A sound reading for technical leads is that this is a targeted optimization with substantial measured gains and a known behavioral difference. Teams applying the workflow should benchmark against workloads that reflect their real file distributions and treat content correctness and performance as separate acceptance criteria.