TryAI put twelve models head-to-head on four coding tasks: a Doom-style 3D raycaster, an animated Rubik’s cube, a calculator, and Conway’s Game of Life. Each model got five attempts per task, with costs and latency recorded.

Why it matters to you. When choosing a tool for real work, abstract benchmarks aren’t enough—you need concrete examples on realistic tasks. This test shows that GPT-5.6 Sol is most consistent on the raycaster (5/5 attempts), Claude Fable 5 never fails on the Rubik’s cube (5/5), and Grok 4.5 is a solid pick at a quarter of Sol’s price. Muse Spark 1.1 impresses when it works, but fails three of five attempts. Open-weight models (Qwen, DeepSeek, Kimi, GLM) cost less but are less reliable: good for experimenting, risky for production.

If you want to try it. The site publishes every attempt (twenty per model): you can run them in your browser and see where they fail. It’s the most direct way to know whether a model handles your kind of task before paying for an API key.

In detail

TryAI’s previous tests drew criticism on Hacker News for giving each model only one attempt. This time they answered with rigor: twelve models, four apps, five attempts each, with costs and latency measured on every run.

The models tested

The lineup includes three variants of GPT-5.6 (Sol, Terra, Luna), Grok 4.5, GPT-5.5, Claude Opus 4.8 and Claude Fable 5, Meta’s Muse Spark 1.1, and four open-weight models: Qwen 3.7 Plus, DeepSeek V4 Pro, Kimi K2.6, and GLM-5.2, all served via Fireworks.

Task 1: 3D raycaster

A first-person maze with WASD movement, wall depth, floor, and ceiling. GPT-5.6 Sol delivers five for five, with more detail in results than GPT-5.5. Grok 4.5 goes 5/5 at 27 cents for five runs—a quarter of Sol’s cost. Claude Opus 4.8 manages only 4/5, less consistent than expected. Muse Spark 1.1 surprises: only two successes, but those were competitive with Fable and Sol.

Task 2: Rubik’s cube

A 3D colored cube with Scramble and Solve buttons that animate rotations. Claude Fable 5 is flawless: 5/5, no glitches, smooth animations. GPT-5.6 Sol does 4/5 (one attempt produced a completely black cube—an inspired choice). GPT-5.6 Luna crashes: 0/5, cube breaks on the first scramble. Claude Opus 4.8 fails to produce even one perfect attempt, an unexpected result given Fable 5’s five for five.

Tasks 3 and 4: calculator and Game of Life

Full details are on TryAI’s site, with every attempt runnable in-page. The calculator required correct operator precedence; Game of Life needed to animate generations with play/pause controls.

Practical implications

Results vary significantly between attempts by the same model on the same task—why a single attempt isn’t enough to judge. GPT-5.6 Sol costs more but fails less; Grok 4.5 is a solid budget choice; Claude Fable 5 excels where visual precision matters. Open-weight models (Qwen, DeepSeek) cost little but variance between attempts is high: good for experimenting, risky where reliability counts.

Test limitations

Four tasks aren’t a complete benchmark, and evaluation (“playable,” “clean solve”) is subjective. TryAI says so openly: this isn’t a scientific verdict, it’s a real sample published in full. The value lies in the links to actual attempts: you can run them and form your own opinion.

As we discussed on July 5th, generic benchmarks don’t tell you if a model works for you—you need a testbed with your own cases. This test gives you four concrete cases, five attempts per model, and links to run them: it’s the starting point for deciding before you pay.