A new benchmark called Imaging-101 is revealing the gap between LLMs’ performance on general coding tasks and their ability to handle the specialized demands of scientific computational imaging. The benchmark, published on arXiv by a team led by Siyi Chen, evaluates seven frontier LLM coding agents across 57 expert-verified tasks drawn from six scientific domains.
Each task in Imaging-101 is grounded in a peer-reviewed paper and canonicalized into a standardized four-stage pipeline: preprocessing, forward physics modeling and inverse solver, and visualization. The benchmark tests agents across three tracks, planning, function-level unit tests and end-to-end reconstruction, to pinpoint where they succeed and where they break down.
The results reveal systematic weaknesses that general coding benchmarks do not capture. LLMs struggle with algorithm selection in scientific contexts, failing to choose the appropriate reconstruction method for a given imaging modality. They mishandle physical conventions such as signs, units and normalization factors. Most critically, they falter at pipeline integration, connecting the preprocessing, physics modeling, inverse solving and visualization stages into a coherent, working end-to-end system.
These are not minor bugs. Computational imaging, which recovers hidden signals from indirect and noisy measurements, underpins quantitative discovery across microscopy, MRI, CT, astronomy and optics. A coding agent that can pass standard LeetCode-style challenges but cannot correctly implement a forward physics model is not yet useful to a domain scientist.
The authors argue that the findings point toward skill-augmented, domain-specialized agents as the practical path forward, rather than expecting general-purpose LLMs to develop scientific coding competence through scale alone. For now, the benchmark provides a structured map of where the gaps are, which is a prerequisite for closing them.
Sources: Imaging-101: Benchmarking LLM Coding Agents on Scientific Computational Imaging (arXiv, July 2026)