Research Can a coding agent keep an app working as the user keeps asking for more? EvoWeb gives agents 32 web apps to build and then extend, ten requests each. After every request, a browser agent tests the running app.
TL;DR
Building an app with an AI agent rarely ends at the first prompt. People come back to add features, and each new request lands on code the agent wrote earlier. Existing benchmarks score a single build. EvoWeb scores the whole journey.
- No agent keeps an app fully working. The best, Fable 5 and Kimi K3, reach a Grade of about 0.69 out of 1.
- Connecting features is the hard part. For every model, the tests that combine a new feature with an earlier one score lowest.
- Apps keep building but stop running. Build failures stay rare, while runtime errors climb from 2% to 12% over ten requests.
Leaderboard
| Model | Grade ↑ | Survival ↑ | Turns / req. | $ / app |
|---|
Watch an app evolve
This is one real run: Fable 5 builds a lead-generation site for a credit repair company, then grows it over nine follow-ups with a score estimator, success stories, a dispute letter builder, a case tracker, a dashboard, a restyle, sign-up and booking. Each feature is adapted from a real product, Lexington Law or Credit Repair Cloud. After each request the judge runs five checks: three on the new feature, one that combines it with an earlier feature, and one re-run of an earlier check. Step through the requests and click any check.
The first three requests pass every check. Request 4 adds a dispute letter that should be signed with the name from the home page's lead form. Fable 5 wired the signature to a different form, so the letter keeps signing off as "[Your Name]". Each feature works on its own. The bug is in the connection between them. Later, the restyle at request 7 misses the tracker's badges, and request 8 loses an add-on. The rest of the app keeps working, so how much it is worth depends on how patient the user is.
Scored for real users
Most sequential benchmarks stop at the first failure, as if every user quits right away. Others score every request, as if no one ever quits. EvoWeb instead averages over every level of patience. Drag the slider to see how much of an app a user with a given tolerance actually gets.
Not everyone asks in English
People describe what they want in their own language, even when the app itself is in English. We translated every request into French, German, Chinese, Hindi and Hebrew and kept everything else the same: the apps, the tests and the judge. Only Sonnet 4.6 holds steady, staying within 0.03 of its English Grade in every language. Other models swing in both directions, even in widely spoken languages. Agents should be tested in the language their users write in.
| Request language | Sonnet 4.6 | Haiku 4.5 | GPT 5.6 Terra | GLM 5.2 | GPT 5.6 Luna |
|---|
Try it
Run any model, whether through an API or served locally with vLLM, on all 32 apps:
git clone https://github.com/two-delta/EvoWeb.git && cd EvoWeb
docker build -f harness/Dockerfile -t evoweb-harness:latest .
pip install -r requirements.txt && pip install -e .
evoweb run --name haiku --model anthropic/claude-haiku-4-5
python3 -m evoweb.score_run runs/haikuCitation
@article{nukrai2026evoweb,
title = {EvoWeb: A Benchmark for Coding Agents on Evolving Web Applications},
author = {Nukrai, David and Alon, Bar and Levy, Nativ and Atad, Ido Andrew and Cohen, Ido
and Zimerman, Itamar and Wolf, Lior and David, Eli and Naiman, Ilan},
journal = {arXiv preprint arXiv:2610.16402},
year = {2026},
url = {https://arxiv.org/abs/2610.16402}
}