VISION MODEL BENCHMARK

Can LLMs count?

Same images. Different models. Explore how closely vision models count the objects they see.

Loading results…

Absolute percentage deviation · lower is better

0% · exact50%100%+

— Missing · N/A when actual is zero

Percentage deviation = |predicted − actual| ÷ actual × 100. Normalized error is the mean deviation across images, with each image weighted equally. Missing predictions and zero actual counts are excluded; coverage appears beneath the score. Colors use the same continuous scale in both tables, capped at 100%; displayed values are not capped. “Predicted / actual (%)” shows the predicted fraction of the true count: 100% is exact, above 100% is an overcount.