Vals AI Benchmark Keeps Frontier Models Under 20 Points

Vals AI recently introduced CUA-Bench, a benchmark that tests AI agents on six commercial PC games using only visual screen inputs, a keyboard, and a mouse. The evaluation features three public titles—Minecraft, SUPERHOT, and eFootball—alongside three undisclosed games from the same genres to detect training contamination. Across all tested frontier models, the highest average score was just 19.2 points out of 100. GPT-6 Astra led the pack with a mean score of 26.7 on public games and 11.7 on the held-out set.

A major bottleneck for the agents was latency, as the game clocks continued to run while the models processed decisions. The systems took between 15 and 59 seconds per decision cycle, with a median of 34 seconds, producing only 2 to 19 input events per minute. In Minecraft, no model managed to find iron. For instance, Claude Opus 5 took 50 minutes to collect cobblestone and 97 minutes to craft a stone pickaxe. Meanwhile, GPT-6 Astra achieved the best result in SUPERHOT by clearing five of its 25 levels. In eFootball, Astra recorded 743 direction-key holds, winning twice at Beginner difficulty but failing at Regular.

The evaluation ran vendor-specific command-line agents, including OpenAI's Codex, Anthropic's Claude Code, and Google's computer-use agent for Gemini, using the Model Context Protocol to control a GPU desktop. Performance gaps between public and hidden titles highlighted generalization issues. Claude Fable 5.1 scored 18.3 points higher on public games, and GPT-5.6 Sol had a 2.0-point gap. Conversely, Gemini 3.8 Flash reversed this trend, scoring 5.0 points higher on the held-out games.

For AI developers, these results demonstrate that current frontier models struggle with continuous, real-time feedback loops. Unlike static text or coding tasks, environments like CUA-Bench require rapid visual processing, spatial reasoning, and immediate adaptation. This benchmark serves as a crucial reality check for practitioners building autonomous desktop agents or robotics systems, proving that today's models still lack the low-latency control and long-horizon planning needed to operate effectively in dynamic, real-world environments.