Cruxes Default, one sentence, helped one small AI model at knowing when to ask. On seven frontier models it gave no measurable gain

aWhat the model normally does

An agent must act, ask or stop, and told to think it over, it asks too often

Every agent reaches the same fork many times in a task. It gets a request and a list of tools. It can call a tool, ask the person a question, or say it cannot help. Ask too often and it becomes a nuisance. Act too readily and it acts on a guess.

We score that fork on When2Call, a public benchmark of tool-use requests from NVIDIA (Ross, Mahabaleshwarkar and Suhara, NAACL 2025). Some requests are complete. Others are missing something the agent needs. The score is the AUC of how often the model stops across three reads of each request: .5 is a coin flip and 1.0 is perfect.

A common way to make a model careful is to tell it to think it over first. The studies call this Reflection A2, after Madaan et al. and Shinn et al. (2023). On complete requests, GPT-5.4 with Reflection still asked a needless question 26.4% of the time.

bExactly what Cruxes changes

One instruction in the prompt: judge the default yourself, name it, proceed

Cruxes changes nothing about the model, the tools or the request. It changes one block of text in the prompt. The Cruxes step already asked the model to name the one fact that would change its decision. Cruxes Default X4a keeps that first line and replaces the rest with a default test the model runs on its own.

1. The request. The tool schemas in JSON, then the user’s question, exactly as When2Call gives them.
2. The task line. “Decide how the assistant should respond to the user question given the tools. Labels: tool_call, request_for_info, cannot_answer.”
3. The instruction. This is the only part that differs between arms. The texts are below, verbatim.
4. The closing line. “Do not call any tool; reply in plain text. End your reply with a final line: DECISION: <label>”.
5. The read. GPT-5.4 (gpt-5.4-2026-03-05) or GPT-5.4 mini, reasoning effort low, default temperature. Each request is asked three times. The share of reads that stop (ask or decline) is the request’s score, so the AUC measures how well the model ranks requests, not where it sets its threshold.

The words “judge for yourself, without asking the user” matter. A first draft said “ask whether the user would accept a sensible default”, and the model read it as an instruction to ask the user. That is where most of its needless questions came from in the first tuning round. The research assistant rewrote it after reading those errors (HC-5, Amendment 1).

A logged example

Logged example GPT-5.4, HC-9, When2Call item 24ebdd98…, first of three reads. The tool searches films by genre and cast, and its genres include “Comedy” but no “Comic”. The request: “I would like to choose a movie to watch. Can you help me find a movie in the Comic genre with James Corden in it?” The benchmark’s answer is to call the tool.

Excerpts are verbatim from the logged replies, shortened where marked with “…”. Reflection also asked in 3 of 3 reads. Source: hc9/rows.jsonl and hc9/items-hc9-n286.jsonl · 15 Oct 2026

In the product: built and dark. An adapted wording ships behind CRUX_DEFAULT_JUDGEMENT_ENABLED in the hosted MCP server’s instructions (apps/crux-chat/src/lib/mcp/default-judgement.ts), with a hard-stop sentence in front of it: questions about deleting, deploying, spending, access or secrets still go to the person or the gate. That exact wording, and its effect on Claude models, are unmeasured, so it stays off until live over-asking is measured.

cWhat we compared it with

Four controls, each changing one thing

• Reflection. “Before you decide, think through the request and the tools in your own words, in at most 120 words, with no required structure.” The generic thinking prompt, copied verbatim from Cruxes’ frontier ladder (FL-1). It asks for the same thinking budget without the Cruxes idea.
• Cruxes Prompt · Step v1. The older shipped step. Comparing against it isolates the default test, since both start by naming the crux.
• Reflection plus the same default sentence A2D. Free reflection, then the identical default test with “the crux” replaced by “anything missing”. It differs from Cruxes Default only in how the deciding fact is found, so it settles whether naming the crux adds anything (HC-8, 1,500 fresh requests).
• A bigger model. The same arms on GPT-5.4 and on GPT-5.4 mini, on the same 286 requests. It settles whether the frontier model adds anything once the sentence is in.

For the market comparison, Cruxes’ frontier ladder (FL-1) ran ten hosted models with up to twelve prompt styles each on When2Call, including the Plain prompt A0, the Pre-mortem PM (Klein, 2007), and the older Cruxes step and Cruxes Prompt · Skill v1 K.

dWhat we measured

On GPT-5.4, the small model with the sentence beat the bigger model thinking it over, and showed no clear difference from it running the sentence

The first frontier check (HC-9) used GPT-5.4, released in March 2026. On the same 286 requests, GPT-5.4 mini + Cruxes Default scored .885. GPT-5.4 + Reflection scored .830. The small model is ahead by +.055 [+.017, +.095] and costs $1.57 per 1,000 calls against $4.12. Give GPT-5.4 the sentence too and it scores .878, which ties the small model (+.006 [−.024, +.036]) at 3.2 times the price.

Where the gain comes from

Source: HC/RESULTS-HC6.md, HC/RESULTS-HC7.md, HC/RESULTS-HC8.md (Part A and Part B), HC/RESULTS-HC9.md, HC/RESULTS-HC13.md · 15 and 27 Oct 2026

What the sentence trades. It makes a model act more readily. On complete requests, GPT-5.4 asked a needless question 17.2% of the time with Cruxes Default against 26.4% with Reflection, about nine fewer interruptions per hundred. On requests that did need a question, it stopped 87.0% of the time against 88.6%. That is why the product keeps a hard-stop list in front of it.

Checked on today’s frontier models (HC-13)

The run. HC-13 put Cruxes Default on seven of today’s frontier models and three cheap ones, on the same 286 requests with three reads each, against the Plain prompt and Reflection. Every call used reasoning effort low. It was registered before the scored run: 24,024 calls for $150.63.

Nothing credited on the seven (P1). The differences from the Plain prompt were Opus 5.5 −.004 [−.028, +.022], Fable 5.1 +.006 [−.023, +.036], GPT-6 Astra −.023 [−.047, −.000], GPT-6.1 Sol −.028 [−.052, −.005], Gemini 4 Argon −.004 [−.014, +.006], Muse Wonder 1.0 +.033 [−.001, +.069] and Kimi K3.1 Pro −.013 [−.039, +.014]. Every Holm-corrected p is .10 or above.

It changes how they fail, not how often. The sentence cuts needless questions and cuts asking where a question was needed by about the same amount. On Opus 5.5, needless questions fell by 4.3 points [−8.0, −1.2] and stops on requests that needed a question fell by 6.5 points [−9.7, −3.6]. Frontier models already ask well here, with Plain prompt scores of .866 to .902, so there is little over-asking left to remove.

The small-model lift. GPT-5.4 mini rose from .838 with a Plain prompt to .885 with Cruxes Default, +.047 [+.012, +.083] (descriptive), mostly by cutting needless questions from 30.4% to 16.5%. That puts it inside the frontier Plain range of .866 to .902, at $1.57 per 1,000 decisions against $1.84 to $18.25. The other two cheap models moved less: GPT-6 Luna +.015 [−.021, +.051], and Gemini 3.8 Flash −.005 [−.031, +.022].

Cheap against frontier (P2). The registered cheap carrier, picked as the cheap model with the highest Cruxes Default score, was Gemini 3.8 Flash (.895). It was non-inferior to Muse Wonder 1.0 with a Plain prompt only (+.029 [−.009, +.068], margin −.03, Holm p .006), at 1/5.8 of the cost. Against the other six frontier models non-inferiority was not shown. Flash with a Plain prompt scores .900, as high as with Cruxes Default, so this is not a Cruxes effect.

Claude models refused Reflection. Opus 5.5 left 63% of Reflection replies unparsed and Fable 5.1 left 52%, blocked by Anthropic’s reasoning-extraction classifier, as FL-1 also found. Their Reflection cells are void, and the comparisons above use their Plain prompt cells.

What Cruxes’ testing found in the market

In FL-1, the top When2Call score came from a mid-priced model. Gemini 3.5 Flash with the Pre-mortem scored .935 at $2.95 per 1,000 decisions. Opus 5.5 with a Plain prompt scored .911 at $8.95, three times the price, and the gap is inside the noise (+.024 [−.028, +.075], approximate). With the Plain prompt, 7 of 9 models tie the leader. On AgentAbstain, Haiku 5.5 scored .652 at $0.42 per 1,000 and Opus 5.5 scored .656 at $21.75, 52 times the price for no measurable gain (−.004 [−.078, +.071], approximate). The older Cruxes prompts scored below the Plain prompt on most models: the step on 8 of 10 hosted models and the skill on 9 of 10.

Why price matters here. Agents make this call at every step. Zapier’s AutomationBench loop allows up to 50 model steps per task. At the prices measured, a fleet making 200,000 of these calls a day would spend about $824 a day on GPT-5.4 + Reflection and about $314 on GPT-5.4 mini + Cruxes Default, while scoring higher than GPT-5.4 told to think it over (illustrative arithmetic on measured prices).

Where it helps

Where a cheap, well-judged ask pays off