Ploy, the platform that builds marketing sites with an agent, migrated its system from Claude Opus 4.8 to GPT-5.6 Sol. The numbers on real workloads: 2.2x faster to complete a build, 27% cheaper, half the output tokens. It wasn’t just changing a name in the config.

Ploy’s team documents the three pitfalls of a real migration: tool calls with 25 parameters sent every time even when unnecessary (Claude sends 2 or 3, GPT-5.6 fills them all inventing values), prompt caching that behaves differently between providers, and reasoning replay between turns that shifts strategy. Each difference required changes to their eval suite before touching the agent itself.

Why it matters to you. If you have an agent in production on Claude and are evaluating GPT-5.6, expect different behaviors hidden by the universal SDK. The post is a migration guide written by people who actually did it, with full traces and side-by-side comparisons on real builds. It’s not a synthetic benchmark: it’s the same task (building a homepage from a reference) run by two models, measured by the same criteria, with differences documented file by file.

In detail

Context: four months with Opus as default

Since March 2026, Ploy has used Claude Opus as the default model for its site-building agent. Opus 4.7 first, then 4.8. Each frontier release was tested against it, and none held up on their workload: building React components, generating images, taking screenshots, deciding when work is done. GPT-5.6 was the first model to pass that test.

What changed in the numbers

The table Ploy publishes compares 11 complete builds with Opus against 10 with GPT-5.6, same test suite. Average cost per build: from $3.06 to $2.22. Time: from 8 minutes to 3 minutes 42 seconds. Output tokens: from 33,000 to 17,100. Visual score (design evaluation on 10 binary criteria) rises slightly: 0.936 vs 0.970.

GPT-5.6 writes more compact code: in one compared case, Opus produced a CSS file of 17,957 characters with 174 CSS variables (complete color ramps, mostly unused), GPT-5.6 wrote 2,508 characters and 45 variables for a comparable rendered page.

The three hidden pitfalls

Tool calls with all parameters. Ploy’s code tool has 25 top-level parameters, one required (action) and the rest optional. Claude sends only the ones it uses. GPT-5.6 sends them all, every time, inventing plausible values for the ones it doesn’t need: offset: 0, timeout: 120000, siteId: 00000000-0000-0000-0000-000000000000. Three days of production traces: 6,635 code(read) calls, all with 25 properties. This pattern forced Ploy to revise tool argument validation.

Prompt caching differs between providers. Ploy uses caching to avoid repassing the entire codebase on each turn. Anthropic and OpenAI implement this feature with different rules about what gets cached and how long it lasts. The migration required redoing tests on when cache hits actually occur, because assumptions valid for Claude didn’t hold with GPT-5.6.

Reasoning replay. Between one turn and the next, the agent must see its previous work to decide the next step. Claude tends toward a sequential strategy (one file at a time), GPT-5.6 emits parallel calls. Their eval executor didn’t support batches of file reads, something Opus used rarely and GPT-5.6 uses constantly. About a third of failures in the first cross-model run traced to test harness limits, not the model.

Design: clean, but uniform

Ploy’s team notes a stylistic convergence: GPT-5.6 produces very clean, modern, tight-grid layouts, but trends toward that look without explicit steering. With their old harness designed for Opus 4.8, GPT-5.6 tended to ignore existing design systems and produce visually generic output, even if technically correct. Ploy writes that the solution deserves a dedicated post, and that with their design team they managed to achieve “world-class brand adherence that you don’t get out of the box”.

Implications for those evaluating a migration

If you’re using Claude in production and looking at GPT-5.6, Ploy’s takeaway is clear: your eval suite is tuned to the incumbent model more than you think. Tool call budgets sized for one model’s style, executors that assume specific behaviors, invisible thresholds inherited from defaults. Before trusting a cross-model pass rate, triage the traces one by one. Otherwise you’re measuring how well the new model imitates the old one, not how well it solves the task.

The migration was completed July 9, 2026, and GPT-5.6 Sol is now the default for every Ploy workspace. The original post includes side-by-side screenshots of real output and complete comparison tables.