Investor Matt Turck, whose fantastic podcast has hosted the people who built ARC-AGI, summed up Astra’s blockbuster benchmarks with three words: “This is wild.” Then he added four more in parentheses: “w/ its native harness.”ARC Prize ran GPT-6 Astra through its own standard harness, and the model scored 62.7%. When run through OpenAI’s Provider Adapter, the same model in the same reasoning setting scored 98.6%. The model didn’t change, but the software around it did, adding 36 points to the score. And the better-performing system cost less: $17,332 with OpenAI’s adapter versus $26,098 with ARC Prize’s.That’s why harness engineering is becoming as important as model selection. The software around the model can change both what it accomplishes and what it costs.The benchmark measures the system, not the modelARC-AGI was built to resist the brute-force scaling that eats so many other benchmarks. ARC-AGI-3 raised the bar again this year by dropping a model into interactive environments with no instructions, no stated goal, and no stated rules, then scoring how efficiently it learns to operate. When ARC Prize launched it this year, humans scored 100%. Frontier AI scored 0.51%.On Thursday, Amanda Caswell covered OpenAI’s improving score on the ARC-AGI-3 benchmark. As she noted, Astra ran under different settings from competing models. ARC Prize is specific about what those settings do. Its standard harness lets a model carry forward notes it chooses to keep. OpenAI’s adapter preserves the opaque reasoning state between requests and compresses longer conversations, so the model can resume its own thinking instead of reconstructing it. ARC Prize published every reasoning level.div.tnsht{--ink:#1B2A4A;--teal:#0D8F8F;--muted:#5D6D7E;--band:#F4F6F8;--line:#DDE3E9; font-family:Arial,Helvetica,"Helvetica Neue",sans-serif;color:var(--ink); background:#FFFFFF;margin:28px 0;max-width:760px;box-sizing:border-box;}div.tnsht *{box-sizing:border-box;}div.tnsht p.tnsht-hed{font-size:17px!important;font-weight:700!important;line-height:1.3!important; margin:0 0 4px!important;padding:0!important;color:var(--ink)!important;}div.tnsht p.tnsht-dek{font-size:13px!important;line-height:1.5!important;margin:0 0 12px!important; padding:0!important;color:var(--muted)!important;}div.tnsht div.tnsht-wrap{overflow-x:auto;-webkit-overflow-scrolling:touch; border:1px solid var(--line);border-radius:5px;background:#FFFFFF;}div.tnsht table{border-collapse:collapse!important;border-spacing:0!important;width:100%!important; margin:0!important;padding:0!important;border:0!important;font-size:14px!important; background:#FFFFFF!important;display:table!important;table-layout:auto!important;}div.tnsht thead{background:transparent!important;}div.tnsht tr{background:transparent!important;border:0!important;}div.tnsht thead tr th{background-color:#1B2A4A!important;background-image:none!important; color:#FFFFFF!important;font-size:11.5px!important;font-weight:700!important; text-transform:uppercase!important;letter-spacing:.05em!important;line-height:1.35!important; text-align:left!important;vertical-align:bottom!important;padding:10px 14px!important; border:0!important;white-space:normal!important;}div.tnsht thead tr th:first-child{width:32%;}div.tnsht tbody tr th,div.tnsht tbody tr td{padding:11px 14px!important; border:0!important;border-bottom:1px solid var(--line)!important; color:var(--ink)!important;background-color:#FFFFFF!important;background-image:none!important; font-size:14px!important;line-height:1.4!important;vertical-align:top!important; white-space:nowrap!important;}div.tnsht tbody tr th{text-align:left!important;font-weight:700!important;}div.tnsht tbody tr td{text-align:left!important;font-weight:400!important;}div.tnsht tbody tr:nth-child(even) th,div.tnsht tbody tr:nth-child(even) td{background-color:#F4F6F8!important;}div.tnsht tbody tr:last-child th,div.tnsht tbody tr:last-child td{border-bottom:0!important;}div.tnsht span.tnsht-score{font-weight:700!important;color:var(--ink)!important;}div.tnsht span.tnsht-cost{color:var(--muted)!important;font-size:13px!important;font-weight:400!important;}div.tnsht p.tnsht-foot{margin:10px 0 0!important;padding:0!important;font-size:11.5px!important; line-height:1.5!important;color:var(--muted)!important;}div.tnsht p.tnsht-foot a{color:var(--teal)!important;text-decoration:underline;}@media (max-width:520px){ div.tnsht table{font-size:13px!important;} div.tnsht tbody tr th,div.tnsht tbody tr td{padding:9px 10px!important;font-size:13px!important;} div.tnsht thead tr th{padding:9px 10px!important;} div.tnsht span.tnsht-cost{font-size:12px!important;}}Same model, two harnessesGPT-6 Astra on ARC-AGI-3, by reasoning effort. At every setting, the runinside OpenAI’s Provider Adapter scored higher and cost less than the same model insideARC Prize’s standard harness. Reasoning effort ARC Prize standard harness OpenAI Provider Adapter Max 62.7% for $26,098 98.6% for $17,332 XHigh 59.3% for $37,317 98.4% for $18,147 High 54.8% for $40,705 99.9% for $18,817 Medium 38.6% for $48,090 98.4% for $19,285 Low 17.5% for $38,166 98.0% for $21,298 None 35.2% for $49,791 96.7% for $23,457 Source: ARC Prize.Astra inside OpenAI’s harness with no reasoning effort at all scored 96.7% for $23,457. The same model at max reasoning within ARC Prize’s standard harness scored 62.7% and cost $26,098. The harness beat the reasoning dial outright. I’ve been arguing the harness matters for months. I didn’t expect the result to be this lopsided.The score wasn’t the only gap. Across the 167 game-reasoning pairs both harnesses solved, ARC Prize clocked the Provider Adapter runs at 49% fewer tokens and roughly 3.66x faster.OpenAI isn’t hiding the details: The adapter runs on documented Responses API capabilities anyone can call. What you can’t buy is the assembled system that scored 98.6%.That’s also why OpenAI President Greg Brockman’s claim during a press briefing this week — “I think it’s not unreasonable to feel that we are now in the AGI era” — lands harder than it should. Brockman is describing a benchmark result produced by a particular system, not establishing that the underlying model is AGI. Frederic Lardinois’ launch coverage for The New Stack gets the distinction right: The framing goes well beyond what the evidence establishes. Maybe we’re in the AGI era. But this benchmark doesn’t prove it.The harness is becoming the productOn coding, the frontier models now cluster inside a few points. Artificial Analysis scores its Coding Agent Index by running each model inside a harness rather than on its own: Astra in Codex at 67, Opus 5 and Fable 5 in Claude Code at roughly the same, Muse Spark 1.3 in Muse Code alongside them, and Fable 5.1 in Claude Code leading at 70. The unit being measured is already the pair. The labs already figured this out. Back in April, Janakiram MSV documented the four-way split: Anthropic, OpenAI, Google, and Microsoft all treat the harness as a product to sell, and disagree only on how to charge for it. Anthropic meters Managed Agents at $0.08 per session hour, in addition to token rates. OpenAI gave its Agents SDK away with no runtime fee at all. Google and Microsoft bill sessions, memory, code execution, and observability as separate line items.Nobody is treating the harness as a free accessory to the model.Other companies are moving the same way. Stripe paid a reported $8 billion for OpenRouter in August, as Paul Sawers reported for TNS, acquiring a gateway that routes 10 trillion tokens a day across more than 400 models for 10 million developers. Patrick Collison’s framing was that tokens are the central currency for companies building with AI. Stripe bought the routing layer that sits in front of the models. Nvidia built a harness of its own.We covered the proof two weeks ago. Adrian Bridgwater reported that Claude Opus 5 scores 30.2% on ARC-AGI-3’s public set on its own. Wrapped in Nvidia’s AVO, which gives it persistent memory and programmatic supervision that steps in when progress stalls, it cleared all 183 levels across 25 environments. That one isn’t the clean A/B that ARC Prize ran on Astra. Nvidia changed the memory, supervision, and context management at once. More than one variable moved.Nvidia said it beautifully here: “Model capability matters enormously, but the surrounding system determines how effectively that capability can be converted into sustained autonomous progress.”Harness engineering is the jobJanakiram MSV found token usage varying 70-fold across Aider, Claude Code, and OpenClaw running an identical model. Cache hit rates swung from about 70% down to 1.5% depending on the serving path. No model choice explains a spread like that.The work itself is pretty ordinary. Deciding what an agent remembers and what it forgets. What it’s allowed to touch, and when it has to stop and check with a person. Jeremy Daly’s piece on our site is the version with the engineering, and it’s the one to read if you’re the one building.I argued in June that model triage was the skill worth hiring for. I’d revise that. Picking the model is the easy half, and it gets easier every quarter as the frontier converges. A year from now, I think the people running agents will spend less of the week picking models and more of it building what goes around them. The post OpenAI will sell you Astra, but not the system that scored 98.6% on ARC-AGI-3 appeared first on The New Stack.