Cruxes research · June to November 2026 · updated 1 Nov 2026

Cruxes: surfacing the assumptions behind a model’s answer, with a registered record of what works

For topic-specific blog posts, read: Reading a model inside · One sentence to ask · Shuffle and re-ask

Abstract. Cruxes identifies the assumptions behind a model’s answer, quotes the passage that raised each one, and records what the person decides. We tested its components at five points on an answer’s path. This page reports 138 studies from four months of work, all listed in the study record; of the first 136, 7 met their registered success rule and 1 was credited by an internal recompute. Each section gives the question, the method, the result with its interval, and the limitations. Five patent applications have been filed.

.613 → .706

Cruxes Read. A probe on one layer of a small open model separated 3,649 requests better than the model’s own answer. The result did not transfer to a second benchmark.¹

.838 → .885

Cruxes Default. One added sentence improved one small model; the effect on seven larger models was not measurable.³

64% → 83%

Cruxes ReAsk. Querying a decision tool three times in shuffled order retained more of its right answers; plain reshuffling at equal cost retained 81%.⁴

7 of 136

Record. Studies that met their registered success rule. 19 were falsified. All are listed in the study record.⁵

Latest, 29 Oct to 1 Nov: we replayed saved cruxes through open models and tested whether the cruxes Cruxes raises match where a model’s own answers disagree. Both are described below, with their limits.

Start here

1 · A crux is an assumption whose falsity would change the answer

An assistant asked to plan a team trip must fill gaps the user left open: dates, budget, attendees. It fills them implicitly. Most of these assumptions are immaterial; a few are not. Cruxes identifies the few, displays each next to the passage that raised it, and asks the user about those.

You asked

Plan our team offsite next month. There are 12 of us.

The assistant answered

I’ll book a hotel in Lisbon for the 12 of you from 14 to 16 November, with a meeting room and two group dinners, keeping the total under €6,000. Flights are booked separately by each person.

Crux · next to the highlighted words

Is €6,000 the real budget?

“keeping the total under €6,000”

Yes, keep €6,000Raise it to €9,000Ask finance first

If this guess is wrong: every venue on the shortlist is one you can’t afford, or one you wouldn’t pick.

An illustrative example. You never mentioned a budget, so the assistant made one up.

The assistant answerswith whichever model the user already uses.
Cruxes finds the assumptions that matterEach is shown with the closest matching passage, so the user can verify it.
The user decides, and the decision is recordedThe choice is stored with the passage, the option chosen and who chose it, and passed to the next session or agent that requests it.

Why it matters

2 · An agent faces this decision at every step: act, or check first

An agent that books, buys, emails or edits code does not answer once; it takes a sequence of steps, and at each it decides whether it has enough information to act. Asking too often makes the agent unusable. Acting on a wrong assumption costs money or data. The quality of this single judgement, and its price, therefore recur at every step.

model steps allowed per task in Zapier’s AutomationBench; each is an act-or-check decision.

$1.84 to $18.25

cost per 1,000 decisions for seven frontier models run unmodified on the same 286 When2Call requests.

under .70

the best score any model reached on AgentAbstain, a public benchmark of when not to act.

How a model answers

3 · An answer passes through five stages

Before presenting results, we locate where Cruxes acts. An answer originates in what the model learned, passes through the instructions, then through the model’s internal layers, is emitted as text, and reaches the user. Cruxes has a method at each stage. Select a stage to see what changes there; drag to rotate the model.

drag to turn

The idea

Cruxes is a set of methods, each acting at a different stage on this path.

What to look for

Teal elements are what Cruxes adds; the dot is one answer in transit.

Read on

Select a stage above, or continue to the summary of all five.

Experiments

Each experiment: question, method, result and limitations

Each method is tested against one question about asking. Each card states the question, the method, the result with a chart, and the limitations. The animations show the same procedure.

Test 1 · inside the model · Cruxes Read

Do a model’s activations favour asking before it produces any text?

Method: We evaluated Gemma-3-12B, a small open model, on 3,649 requests. At one layer we fitted a probe (a simple linear readout) on labelled training examples and compared its predictions with the model’s own answer.
Result: The probe separated ask, act and cannot-answer better than the model’s own answer.

Chance.333

Words of the request only.389

The model’s own answer.613

The probe on one layer.706

Balanced accuracy on 3,649 When2Call test requests. Registered in advance.

Limitations. On a second benchmark (AgentAbstain) the probe predicted a single class, so the registered transfer claim failed. On 1,200 requests the read scored below the cheapest strong hosted model (.704 against .807).

Cruxes Read, animated.

Test 2 · inside the model · Cruxes Edit

Is there a set of units that controls how often the model asks?

Method: We selected the 8 internal units most relevant to asking, held them high or low during generation, and compared the effect with 16 random sets of 8.
Result: Holding the 8 units shifted the model’s own probability of asking far more than a typical random set.

8 chosen units+.463

95th percentile of 16 random sets+.215

Change in the model’s own probability of asking, When2Call. Self-registered.

Limitations. The effect sets how often the model asks, not whether it asks at the right moment (AUROC .54 against gold). On AmbigQA one random set matched it, and on Qwen3-8B it reversed sign.

Test 3 · the instructions · Cruxes Default

Can one added sentence reduce unnecessary questions?

Method: We added one sentence instructing the model to use a sensible default if a reasonable user would accept one, name it, and proceed. We tested it on 286 requests, on GPT-5.4 mini and on seven larger models.
Result: It helped the small model on which it was tuned. The seven larger models showed no measurable change.

GPT-5.4 mini, plain prompt.838

GPT-5.4 mini + the sentence.885

Gemini 3.8 Flash, plain prompt.900

Decision quality (rate-free AUC), 286 When2Call requests. The sentence was tuned on mini; not registered.

Limitations. A cheaper model with a plain prompt scored as high without Cruxes, and the product’s wording differs from the tested sentence, so it remains switched off.

Cruxes Default, animated.

Test 4 · the answer · Cruxes ReAsk

Does repeated querying stabilise a decision tool?

Method: A decision tool (Jev) can change its selection when the same options are listed in a different order. We presented the same question in eight orders and compared a single query with three queries in shuffled order.
Result: Three queries retained far more of the tool’s right answers through the reshuffles.

Asked once64.1%

Plain reshuffling, same number of calls81.2%

Cruxes ReAsk, 3.22 calls83.4%

Right answers kept through eight reshuffles, 800 questions. Descriptive.

Limitations. Plain reshuffling gets most of the gain, so the credit goes to repetition. Accuracy stays near 50%: more stable, not more accurate.

Cruxes ReAsk, animated.

Test 5 · you decide · Cruxes Gate and Memory

Does a stop in front of an agent’s actions help?

Method: We ran agents on 120 tasks with the stop in place and again with the agent checking itself. Memory, the record of who decided what, is a separate feature that runs in the product.
Result: The stop halted most tasks, and fewer passed than with the agent checking itself.

Agent checking itself.317

With the Cruxes stop.075

Pass rate on 120 tasks (BENCH-1, registered). The stop halted 93 of the 120.

Limitations. The product still asks a person before every payment, deletion and security change, by design. We are working out when a check should run.

Cruxes Memory and Gate, animated.

Where Cruxes steps in

4 · Summary: five stages, the method at each, and the outcome

The table summarises the programme. For each stage it states what Cruxes changes, the result against a fair alternative, and current status.

StopWhat Cruxes changesWhat happenedToday

What Cruxes changesCruxes Train. Re-train the model on its own successful attempts so that going ahead or checking first is built in.

What happenedA fine-tune on public labels scored 58.7 against our pass line of 70 and was stopped. A fine-tune of Gemma-4-31B on Zapier’s agent benchmark was cancelled before results. An earlier small trained head was exploratory.

Cruxes Train · stopped

What Cruxes changesCruxes Default. One sentence: judge whether a sensible default would suffice, name it, and proceed.

What happenedRaised GPT-5.4 mini from .838 to .885. Seven current frontier models, run unmodified, scored .866 to .902 and gained nothing from it. The sentence was tuned on mini. Our older Cruxes prompts scored below a plain prompt.

Cruxes Default · helped one modelBuilt in the app, switched offOlder Cruxes prompts

What Cruxes changesCruxes Read measures, at one layer, how strongly the model’s activations favour asking. Cruxes Edit shifts that tendency in the same pass.

What happenedA probe outperformed the model’s own answer on one benchmark (.706 against .613) and did not carry to a second; on 1,200 requests it scored below the cheapest strong hosted model. We identified 8 neurons that set how often the model asks, credited as a mechanism. No edit has outperformed a trained probe, and steering over-refused in a safety test.

Cruxes Read · did not transferNeuron Dial · credited mechanismLens mode · built in the appEdits · no gain over reading

What Cruxes changesCruxes Detect finds the assumptions and anchors each to its exact words. Cruxes ReAsk asks a decision tool three times in shuffled order.

What happenedDetection and quoting run in the product. The re-ask raised the share of right answers retained from 64.1% to 83.4%; plain reshuffling reaches 81.2%. A crux-framed tie-break added nothing on its own.

Detect, Anchor, Rank · liveCruxes ReAsk · steadier, mostly from asking againCrux-framed tie-break

What Cruxes changesCruxes Card, Memory and Gate: a margin card, a record of who decided what and why, and a stop before actions that are hard to undo.

What happenedLive in the Cruxes app and its agent tools. Decisions are recorded and handed to the next session or agent that asks. A stop in front of an agent’s actions halted too many tasks in our tests.

Card, Memory, Gate · live

Shows this stop in the 3D journey above.

Live in the product Built, switched off for now Worked in a registered test Being tested Early or mixed Did not work

The headline · inside the model

5.1 · A probe can read a model’s tendency to ask; it is not yet a cheaper way to decide

Before a model produces any text, its internal activations already favour acting or asking when a request is missing something important. Cruxes Read measures this tendency at one layer and combines it with the model’s own vote using a small formula fitted beforehand. Nothing is generated and the model is unchanged. It requires a self-hosted model and its own labelled examples.

.706 against .613

On 3,649 When2Call requests, a probe on one layer of Gemma-3-12B separated ask, act and cannot-answer better than the model’s own answer (+.093 [+.077, +.108]); a words-only baseline scored .389. The registered claim that the probe transfers failed: on a second benchmark (AgentAbstain) it predicted a single class.¹

.704 against .807

On 1,200 requests the read scored .704; Gemini 3.5 Flash-Lite scored .807, a difference of .103 [.078, .128] beyond the preset margin. The read asked a question on 30% of requests that needed none, against 9% for the comparison. A plain prompt on Gemma-4-31B scored .828 without Cruxes.²

We also intervened at the same layer. Among the thousands of units in one layer of Gemma, we identified 8 that set how often the model asks. Clamping them shifted the model’s own probability of asking by +.463, above the 95th percentile (.215) of 16 random sets. An internal registrar recomputed every number and credited the result as registered and no wider.⁶

8 neurons set how often it asks

Cruxes Edit · Neuron Dial. Clamping the units shifted the model’s own probability of asking by +.463 on When2Call and +.549 on AmbigQA, leaving ordinary text unchanged. The shift tracks whether a request needs a question at an AUROC of only .54.

Limitations. On AmbigQA one random set of 8 matched the effect; a control on the most active units was not cleared in 2 of 4 cells; and Qwen3-8B reversed sign on When2Call. A fine-tuned head on another model scored higher than the probe (.782). The edit sets how often the model asks, not whether it asks at the right moment.

How it was done, step by step

Cruxes Read, animated. Brand clip, no sound.

The instructions

5.2 · One added sentence improved one small model

Agents ask many unnecessary questions. Cruxes Default adds one sentence to the instructions:

“Judge for yourself, without asking the user, whether a reasonable user would accept a sensible default, or whether the tool already has one; name the default; if so, proceed.”

.838 → .885

On 286 requests, GPT-5.4 mini scored .838 with a plain prompt and .885 with Cruxes Default (+.047 [+.012, +.083]). The sentence was tuned on this model and the comparison was not registered. Unnecessary stops on complete requests fell from 30% to 17%, and the model asked on 2.5 points fewer of the requests that did need a question. Gemini 3.8 Flash with a plain prompt scored .900 at $1.39 per 1,000 decisions, against $1.57 for GPT-5.4 mini with the sentence.³

Limitations. On the frontier models themselves (Opus 5.5, Fable 5.1, GPT-6 Astra, GPT-6.1 Sol, Gemini 4 Argon, Muse Wonder 1.0, Kimi K3.1 Pro) the sentence produced no measurable change: it removed unnecessary questions and about as many needed ones. Seven frontier models run unmodified scored .866 to .902. The product’s wording differs from the tested sentence, and it is switched off. Naming the crux added nothing on its own, and a hard stop remains in front of spending and deletion.

How it was done, step by step

Cruxes Default, animated.

The answer

5.3 · Repeated queries in shuffled order stabilise a decision tool

Jev is a fast, inexpensive tool that selects among options. A recent paper showed that its selection can change when the same options are listed in a different order. Cruxes ReAsk queries it three times in shuffled order, averages the results, and adds a head-to-head tie-break only when the top two options remain close.

64% → 83% kept

Queried once, Jev retained 64.1% of its right answers through eight reshuffles; with Cruxes ReAsk it retained 83.4%, at 3.22 calls. Plain reshuffling at the same number of calls retained 81.2%.⁴

Limitations. A more stable answer is not a more accurate one: accuracy remains near 50%. Calling the tie-break a “crux” added nothing over plain reshuffling, so most of the gain comes from repeating the query. Descriptive; not registered as a claim.

How it was done, step by step

Cruxes ReAsk, animated.

The evidence

5.4 · Cost and decision quality across models

Each point is one model with one prompting method on When2Call, a public benchmark of when to ask. Cost increases to the right and decision quality upward. The black staircase connects the cheapest setup at each score. On identical requests the slope is +.021 per tenfold price (interval −.019 to +.039), which cannot be distinguished from zero, and a second benchmark (AskBench) showed the opposite direction. This is an observation about the market, not a Cruxes result.

Spending more doesn’t buy better choices: the top scorer costs a third of the priciest model

When2Call · score (AUC) against $ per 1,000 choices, log scale · each dot is one model with one prompt

Source: FL-1 test summary (hosted, gateway-billed) and INPASS-1 rows recomputed in FL-1’s definition (self-hosted, A100 time), assembled by pareto.py. Whiskers on the three labelled frontier cells are 95% intervals. Frontier models ran a 200-item subset; the others ran all 600 items.

Newest work · 29 Oct to 1 Nov

Could a cheaper model write the cards, and do our cruxes match where a model’s own answers disagree?

Test 6 · replaying saved decisions

Could a cheaper model we can read and modify write the cards?

Method: We replayed 100 saved contexts through five Gemma setups on our own GPUs, and had a blind judge score every crux from 1 to 5. The judge passed its own checks: it caught 28 of 30 cruxes we had deliberately spoiled, and its scores correlated at .89 on a rerun.
Result: Stock Gemma-3-12B came closest. It covered the contexts as fully as production (+0.03, interval −0.24 to +0.31) at about a third of the cost, and it was lower on the quality of each crux. The fine-tuned and larger setups raised far fewer cruxes.

Coverage against production, with 95% intervals (below zero is worse)

Where stock Gemma-3-12B is lower, with 95% intervals (rubric points, 1 to 5)

Production, fresh run1.56

Gemma-3-12B, stock1.33

Gemma-4-31B, stock.72

Gemma-4-31B + SOTA-5.68

Gemma-4-26B-A4B.37

Gemma-4-31B + DIX-1.18

Cruxes raised per context. The fine-tunes and the larger bases stayed quiet.

Production (Gemini 3.5 Flash-Lite)$0.0095

Gemma-3-12B, stock$0.0027

Per context, a busy GPU assumed; the 12B number includes model load for 100 contexts.

Limitations. Most of the 100 contexts were sample prompts that recur in the app (17 distinct prompts, five of them make up 76 contexts), so this run checks the machinery and says little about real use. None of our probes or lens fits was applied to crux writing yet. The judge is in the same model family as production.

Test 7 · rollout disagreement

Are the cruxes Cruxes raises the questions a model’s own answers disagree on?

Method: A University of Chicago group (Reber and colleagues, arXiv 2610.21006) defines a crux as a question a model’s own answers split on. We took 60 real contexts, let Gemma-3-12B answer each five times, compared every pair of answers, and checked whether each crux Cruxes raised matched one of the splits. The control paired it with another context’s splits.
Result: Cruxes matched far more often than the control for the 26 app contexts. Cruxes that more answer pairs split on also tended to score higher with the judge (rank correlation +0.33, interval +0.03 to +0.60).

App contexts: with their own answers' disagreements59%

App contexts: control (another context's)19%

Agent questions: own44%

Agent questions: control31%

Share of cruxes matched by at least one of the 10 answer pairs. Gemma-3-12B, 5 answers per context.

Limitations. The control differs in topic, so part of the gap is topical overlap. Compared within one context (an unregistered check on 13 contexts), the correlation was +0.43 with an interval from -0.07 to +0.76, which includes zero. Exploratory; it ranks nothing. In the paper, cruxes a model lists in advance matched its answers’ disagreements less often than a random reshuffle (13.4% against 39.2%), which is a reason to keep testing instead of assuming one pass finds them all.

The climb

Seven lines we keep climbing, and the test that decides each next step

Cruxes runs as a loop: write the test down, run it once, keep what holds and drop what does not. Each line below shows the baseline (the dark marker), the best result so far (the filled bar), and the registered test that comes next. A bar can go the wrong way in a later test, and the agent line shows one that did. The ones that failed are in the record too.

Reading a model’s insidesWhen2Call balanced accuracy

.613.706

Next climbCarry the probe to a second test and compare it with the best fine-tuned head (.782).

InstructionsWhen2Call decision quality, GPT-5.4 mini

.838.885

Next climbTest the sentence on at least three untuned models, registered in advance.

Asking againRight answers kept after 8 shuffles

64.1%83.4%

Next climbReplicate on about 1,600 unused questions, with plain shuffles as the control.

Cost of writing cruxesCost per context, lower is better

$0.0095$0.0027

Next climbRaise quality on real contexts with a probe or lens reranker; the 12B is lower on quality today.

Needless stopsShare of complete requests, lower is better

30%17%

Next climbRegistered replication of the sentence with its control.

Stopping earlyTokens after the answer has settled (an upper bound)

none measured50.5% (MATH)

Next climbTrain and test a stopping probe (STOP-2, Stage 0 reruns next week).

AgentsPass rate on 120 tasks

self-check .317with the stop .075

Next climbRedesign when the check runs, then retest (GATE-R0, a registration draft).

The pace: studies started, by outcome

Met or creditedDescriptive, exploratory, pendingDid not meet its ruleFalsified or void

136 studies from 26 June to 28 October, by start date in the register. No study is dated September. Several studies from late October were still pending when the register closed.

Three results in motion

When2Call cost against decision quality, animated — Where the money stops buying quality on When2Call: 126 model and prompt cells arrive priciest first, then the cheapest setup at each score.

Right answers kept through eight reshuffles, animated — The same question with the options reshuffled eight times: asked once, asked three times in plain shuffled order, and with the Cruxes re-ask.

GPT-5.4 mini with and without the sentence, animated — GPT-5.4 mini with the sentence beside GPT-5.4 on the same 286 requests: cost per 1,000 calls and the paired differences.

What this makes possible

6 · Implications for agents that act

Agents are moving from answering to acting, and every action rests on assumptions. Cruxes makes them visible: which were checked, the words that raised them, and who decided. We have not shown that this improves decisions, and the live check runs before significant or hard-to-undo actions, not at every step.

Task agents

Support and operations agents act on complete requests and ask when information is missing. Measured on one small model; not yet on agents.

Cruxes Default · needless stops 30% → 17% (GPT-5.4 mini, not registered)

Coding agents

Proceed on the project’s evident defaults and stop at choices that change the result. Untested on coding agents.

Cruxes Default + Gate

Spending and irreversible steps

Payments, deletions and security changes go to a person first, every time.

Cruxes Gate · live

Audit trails

Each decision is stored with the words that raised it and the person who decided, available for review.

Cruxes Memory · live

Private deployments

For self-hosted models, the read is a separate Gemma pass inside the operator’s own servers. A GPU costs $1,015 to $1,825 a month to keep running.

Cruxes Read · open models

High-volume choices

Ranking, triage and routing should return the same selection when the list order changes; ReAsk has measured this on one tool only.

Cruxes ReAsk

Where the effect could be largest

LegalA draft contract quietly assumes jurisdiction, liability caps and dates.Each assumption is pinned to its clause and sent to the right person.

Financial operationsPayment exceptions and approvals pile up behind people.Routine cases proceed on a named default. Spending always stops at the gate.

InsuranceClaims triage runs on thousands of small judgement calls a day.Steady picks, and a recorded reason for every exception.

Healthcare administrationIntake and prior-authorisation forms arrive incomplete.One precise question to the right person, instead of a round of emails.

Software engineeringCoding agents stall on questions, or guess on the ones that matter.Decisions travel with the work to the next agent and the reviewer.

Customer operationsSupport at scale balances speed against getting it right.Fewer needless questions to customers. The measured trade on a small model: 2.5 points fewer questions where one was needed.

Assumptions this depends on

The levers measured on public benchmarks carry over to real workflows. Our agent tests so far (BENCH-1, TAC-1) were negative.
The checks remain small relative to the agent’s own cost. We have not yet metered the cost of the gate.
Users accept being asked a few precise questions. The product is built on that bet, and the arena will publish what we learn.

Live, next and the plan

7 · What runs today, what ships next, and the plan for the next 12 to 18 months

Cruxes methods run for users today at the answer and the person stops. The newest methods, from inside the model and the instructions, are built and not switched on. Live status comes from the production deployment’s own flag log, read on 27 October.

Live today

Finds the hidden decisions in an answer and drops near-duplicates

Every question is shown with the closest matching passage

Finds hidden decisions in an AI's reasoning text, not just its answer

Shows each question as a card in the margin

A second AI (Jev, pinned to version 1.13.0) recommends an answer, hidden when it is a coin flip

Scores how much each question matters and sends big or risky ones to the person

No pre-ticked answer on security, money, legal or irreversible questions

Coding agents must check before risky actions, and can try it without signing up

Remembers what the person decided and hands it to the next agent

Works inside other AI tools through a hosted MCP server and a CLI

Asks four discovery questions before a big job

Draws a decision as a matrix, table or tornado

Old decisions get a freshness badge and come back when things change

Ships next

Let agents pick sensible defaults instead of askingBuilt and merged. Switched off until it is checked on Claude models and measured on live sessions.

One Jev answer per question on every screenIn review.

Gate respects decisions the person already madeDeployed, switched off.

Ask Jev in a few shuffled orders and averageResearch done (Cruxes ReAsk). Not built yet.

Store Jev's importance score on every saved questionDeployed, switched off.

Warn when a person rubber-stamps Jev; report Jev's rank qualityBuilt, switched off.

Opt-in chat mode that reads the answering model's activationsBuilt (Cruxes Read · Lens Mode). Needs its model endpoint switched on and one control test.

In progress

Cruxes Read · Stopping probeStop 3 · STOP-2Registered. A probe that stops a model once its answer has settled. Our measured headroom is 50% to 64% of tokens, an upper bound. Its first stage reruns next week.

Cruxes Read vs Train · CAP-1Stop 1 and 3 · CAP-1A model read on Gemma against a LoRA fine-tune on the same task. Stage 0 is done at no cost; the first runs start next week.

Self-improving loop · LOOP-1All stops · LOOP-1A loop that proposes a change, scores it on a reusable holdout and fresh items, and keeps or reverts it, logged to the ledger. Design and registration are drafted.

Safety by reading · SAFE-2Stop 3 · SAFE-2Reading and steering Gemma on honesty and harm tests. Two stages passed; the next waits on a judge that passes its validity check.

Replay on real contexts · REPLAY-1Stop 4 · REPLAY-1Replaying saved cruxes through open models. The first run used mostly sample prompts; the next uses real contexts with a within-context control.

The next 12 to 18 months

Fifteen proposals, in the order we would run them. Each starts with a cheap test that can kill it, and each says in advance what would stop it. None is a commitment; the order changes when a result comes in.

Where	What it would settle	First, cheap test	Stops if	Compute
Now: settle the open checks
Stop 5Cruxes Default, measured live	Whether fewer questions in the lab means fewer needless questions for real users	Log which instructions each session got	reverts or gate stops rise	engineering
Stop 1Cruxes Train on Gemma-4 (AutomationBench; first attempt cancelled)	Whether judgement can be trained into an open model	Count train tasks the base model passes	fewer than 15 of 300 pass	up to $220
Stop 3Cruxes Edit · Neuron Steer	Whether the neuron dial is more than ‘push the busiest neurons’	Re-run the dial on Gemma-4 against busy-neuron sets	it doesn’t beat them on both datasets	$6–30
Stop 4Cruxes ReAsk in the product	Whether live Jev picks wobble enough to need it	Shadow-measure live order flips for a week	live flips are already under the noise floor	engineering; about $0.00002 a call
Stop 5Memory that survives rewording	Whether a decided question stays decided when it comes back in new words	A seed run	its registered rules fail	$5
Next: scale what passes
Stop 1Cruxes Train · Decision Index entry	A leaderboard placing for a clean-provenance Gemma decider	The first pilot scored 58.7 against the 70 line and stopped; a new recipe needs a new registration	a pilot scores under 70	$350–600
Stop 3Cruxes Read · Stopping probe (STOP-2)	Whether reading the model can cut reasoning cost at the same accuracy	STOP-1 found 50% to 64% of tokens come after the answer settles; test a probe that stops there	under 25% of tokens come after the answer settles	$100–200
Stop 3Cruxes Read → safety prompt	Whether reading risk and responding with words beats an always-on safety prompt	Measure the prompt’s cost on harder harmless tasks	that cost stays under about 6 points	up to $150
Stop 1Cruxes Train on a tool-calling benchmark	A paired gain on Berkeley’s function-calling board	Reproduce the base score within 1.5 points	a 200-task pilot gains under +1.0	about $180
Stop 4A decider that can’t be fooled by order	Steadiness by construction, not by re-asking	Score options one at a time on 200 items	accuracy falls more than 3 points	$200–300
Stop 4A robustness suite for decision models	Which fragilities Cruxes ReAsk can and can’t fix	A yes/no swap on 100 items (about $0.01)	Jev flips under 10 in 100	about $110
Stop 4A learned go/stop gate for agents	Whether a trained gate stops less often than today’s rules without missing real stops	Replay logged gate decisions ($0)	the logs lack the signals, or the gate loses on tune	$0, then up to $100
Stop 1A held-out ‘when to stop asking’ test	A fresh test set no model has seen, with a Gemma arm	Register before any run	rules set at registration	small
Later: train it in and test with people
Stop 1Cruxes Train · the trained Cruxes step	Whether the Cruxes idea works when trained in rather than prompted	Score stock Gemma on When2Call’s official metric (about $5)	it starts more than 10 points from the best published row	$300–500
Stop 5Studies with people in the loop	Whether people decide better with Cruxes, measured on outcomes	Count decisions that have a scored outcome	too few outcomes to size a study	$0 until outcome data exists

Where the research compute goes

Upper-bound compute for the fifteen proposals, by stop, from each proposal’s own estimate: about $2,405 in all. Most of it goes to teaching the judgement into open models. Engineering time is not included.

Stop 1 · teaching the model$1,510

Stop 3 · inside the model$380

Stop 4 · the answer$510

Stop 5 · the person$5

Every experiment

8 · Study record: four months of experiments and the stage each acted on

From the first steering test on 26 June to this week, the programme ran 117 experiments. 6 passed the rule they registered in advance, 74 did not, 28 were exploratory and 9 are pending. Each dot is one experiment, placed on the stop where it acted. Hover a dot for its name. Most early work tried to steer the model; the record moved, result by result, toward reading the model and toward the instructions and the answer.

Worked (its registered rule passed)Not credited or did not holdExploratoryPending

All 117 experiments, one line each

June 2026

26 Jun · stop 3Steering when to ask: probe and activation steering on small Qwen modelsSteering made Qwen2.5-1.5B ask on far more ambiguous prompts with little over-asking; 7B moved weakly and 14B not at all; prompting collapsed the 1.5B to always-ask. The early probe AUC of 1.0 was a length confound.Exploratory

26 Jun · stop 2Complide precursor: CLAUDE.md rules and hooks on an agent-safety benchmarkA CLAUDE.md rules file raised the safe rate on Llama 3.1 8B; a later 50% run reported large hook gains that were flagged invalid because the hook was simulated.Exploratory

26 Jun · stop 2Crux-first context engineering for a reviewer promptJudge score rose from 4.0 to 4.9 on 6 commands; not a matched control.Exploratory

29 Jun · stop 3AxBench concept detection re-implementation on gemma-2-2bExploratory: a difference-of-means direction at layer 20 matched the best published method on AxBench concept steering. The test is not about asking or safety.Exploratory

29 Jun · stop 3Arditi-style ablation of a manipulation direction in shopping agentsRemoved about two-thirds of the manipulation advantage on held-out catalogs.Exploratory

July 2026

5 Jul · stop 4First CruxBench model bench: which model writes good crux cardsExploratory July run on CruxBench. Specificity was the weakest lens for every model. Cross-model cost and accuracy comparisons were descriptive and are not repeated here.Exploratory

6 Jul · stop 3J-lens clean-room battery: swap, ablation and steering on small open modelsMixed: swap cleared the amended bar on 1 of 4 models, ablation dissociation on 1 of 4; GPT-2 steering dead. The wider 7-model replication: steering 6/7, swap 0/7 strict, ablation 1/7.Not credited

8 Jul · stop 3J-lens reads for sycophancy and mention-vs-intent (A5-B, A5-D2)Sycophancy reads were decent but the registered GO rule failed; the deception read tracked topic, not intent.Not credited

9 Jul · stop 3A6 / A6.2: J-lens refusal steering, 'safer at equal intelligence'On a sub-ceiling jailbreak slice the edit beat baseline but not a constitutional prompt, with much more over-refusal; it failed on Qwen; the stage-2 bars were never locked, so every number is exploratory.Exploratory

21 Jul · stop 3B8 DarkBench governor: J-lens read decides when to call a judgeThe governor beat the judge-on-everything baseline using 21 judge calls, but the band was picked on the eval set; not pre-registered.Exploratory

21 Jul · stop 3Elicitation-gated steering loop for controls enforcement (Complide DFC)On a synthetic enforcement benchmark, two asks per 32 receipts closed all residual enforcement error; oracle-labelled answers.Exploratory

30 Jul · stop 3Lens bake-off runner (probe vs J-lens vs behavioural) on CLAMBER, CPU smokeOnly a 20-item CPU smoke ran; the GPU scale-up was blocked.Pending

August 2026

5 Aug · stop 3Pre-cog Plus: a J-lens critic on Gemma 4 31B's reasoningScored two-sidedly the first critic fired on everything (J 0.00); a correction abstained on everything (J -0.33); a discrimination rule reached J 0.75 on 8 cases.Exploratory

7 Aug · stop 4Clarification scaffolds against the live detectorEvery scaffold surfaced 1.0-1.5 cruxes per ambiguous prompt; the baseline never stayed silent on a complete brief; the uncertainty ledger kept more cruxes on complete briefs than incomplete. Switched to the slot checklist.Exploratory

7 Aug · stop 3Should-ask steering as a Crux sensitivity mechanismThe read side transferred (probe AUC 1.000 leave-scenario-out); the write side did not: Qwen's asks were generic, Gemma did not steer. Decision: read-only ambiguity score, no steering in generation.Exploratory

8 Aug · stop 4Eval v1 scorecard: crux quality track and tool-trajectory trackEvery model scored 9/9, a ceiling caused by the scoring proxy sharing words with its source: an instrument failure, flagged.Exploratory

12 Aug · stop 4Cruxes benchmark: same input through model x harness combinationsThe gates removed 25-60% of candidates, hardest on one-line asks.Exploratory

12 Aug · stop 4Three-window pilot: vanilla vs askme skill vs shipped slot checklistThe askme condition manufactured two unknowns the brief already answered; the slot checklist returned 'every slot is filled'. N=1 per cell.Exploratory

October 2026

4 Oct · stop 2CruxesArena: playbook, Cruxes pass, typed step and learned playbook on seeded universes (E0-E7)A hand-written playbook was the only held-out win; the typed decision step added nothing; the learned playbook's gain shrank held-out; a trained head learned the scenario family; the E7 production-engine attempt was invalid (35 of 36 seeds hit infrastructure errors).Exploratory

4 Oct · stop 4CB-7 crux preference on external CruxBenchEvery model was within noise of a coin flip.Not credited

6 Oct · stop 3J-lens swap and ablation on CounterFact, 7 modelsSwap bar cleared on 0 of 7; unembedding rows did as well or better.Not credited

7 Oct · stop 2Does Cruxes framing lift Jev? (CruxBench n=134)No framed arm beat raw Jev; raw Jev was cheapest.Not credited

7 Oct · stop 2HiddenBench pooling step vs matched-token control, four modelsThe plain pooling step helped smaller models a lot; Cruxes wording never beat it.Not credited

7 Oct · stop 2OpenAI Decisions API vs Jev (JevBench hard)OpenAI's raw labels beat Jev; every framed arm was no evidence.Not credited

7 Oct · stop 3J-lens and activation probe in the Jev-framing program (incl. Track 2 Gemma-4-26B probe)Negative; left out of the headline.Not credited

7 Oct · stop 1T1 SFT, Qwen3-8B on harder worldsYes on internal worlds: about 76% of the achievable gain.Worked

7 Oct · stop 4Quality check of the #712 detection model routeNo quality loss at 8-12 s per crux (was 23-42 s); Haiku 5.5 led the model table.Exploratory

7 Oct · stop 4Exact policies on irreversible worlds (no model)Greedy expected-value-of-sample-information nearly hit the optimum; ask-before-irreversible was in between.Exploratory

7 Oct · stop 1Gemma 4 26B SFT then DPO, easy worldsNo difference; DPO had a 13% format failure.Not credited

7 Oct · stop 2U2 irreversible worlds: do prompted models ask before irreversible steps?No evidence; prompts under-ask where asking pays.Not credited

7 Oct · stop 2Sonnet-framed Jev vs Sonnet alone (JevBench hard n=105)Sonnet deciding alone beat Sonnet-framed Jev at a third of the cost.Did not hold

7 Oct · stop 2Asking benchmarks with Jev, CAA and a CoT monitor (AskBench, ReqElicitGym)The gain was from gating asks, not Jev; CAA never made Gemma ask; the monitor added coverage, not accuracy.Not credited

7 Oct · stop 2ReqElicitGym at matched budget (post hoc)Yes: at the same number of rounds the gain disappeared.Not credited

7 Oct · stop 4Is the CB-7 label learnable?Barely credited; a free heuristic scored about the same.Not credited

7 Oct · stop 4Does crux value predict forecast gain? (Amendment 5 / Q2)No evidence either way; the forecaster sat near the base rate.Not credited

7 Oct · stop 2Worlds easy: crux framing beyond a cost-aware promptNo; the cost-aware prompt alone did the work on Gemma.Not credited

7 Oct · stop 2Worlds harder: crux framing beyond a cost-aware promptCruxes wording added nothing; the cost-aware prompt was credited (internal).Not credited

7 Oct · stop 3J-lens future-token readout vs a plain frequency list, 5 modelsNo: 0 of 5 models; it beats the tuned lens 5 of 5.Not credited

8 Oct · stop 2Price-per-question re-score of asking benchmarks (post hoc)Gates re-price asking rather than improve it; leads reverse at 0.03-0.08 points per question.Not credited

8 Oct · stop 2NEG-1 sellers: Cruxes vs plain asking (ESCI)Two-family rule not met; every arm at or below the always-qualified floor.Not credited

8 Oct · stop 2SW-1 specialist swarm vs plain pooling (HiddenBench, Haiku 5.5)No: roles and tools added tokens and invalid votes.Did not hold

8 Oct · stop 1DPO and RL-1 vs SFT on Qwen (worlds)No; RL-1 only followed a shifted ask price, with constant stated confidence 0.80.Not credited

8 Oct · stop 2DC-1 deliberation baseline (cooking level 1)Level 1 is near-solved for mid and frontier models; only the small model trailed clearly.Exploratory

8 Oct · stop 2DC-2 structured pooling, cooking level 1No.Not credited

8 Oct · stop 2NEG-3b CaSiNo partner priorities: Cruxes vs matched reflectionNo; on Qwen reflection was better.Not credited

8 Oct · stop 3U5 C-probe sensor on Qwen3-8B (ask-or-act and which fact)Test AUC about 0.81-0.83; as a one-ask gate it beat never-asking; top-crux accuracy only a little above chance.Exploratory

8 Oct · stop 1U5 cascade: a separate question model on top of the sensorIt added a call and no gain.Not credited

8 Oct · stop 1Worlds-trained adapters and cost-aware prompt on AskBench and When2CallNo: adapters and the cost-aware prompt showed no gain on external benchmarks.Not credited

8 Oct · stop 2tau3-bench airline pilot, six arms (Gemma 26B)Both lost to matched reflection; reflection beat the base agent.Did not hold

8 Oct · stop 2SW-2 second pooling round at matched tokensNo; a free extra chat round beat it.Not credited

8 Oct · stop 1T3 redo: 5x SFT data, then DPOEvery split up, none clear of 0; DPO added nothing a third time.Not credited

8 Oct · stop 2NEG-2 bargaining: Cruxes step vs matched reflectionNo: Cruxes buyers closed fewer deals.Did not hold

8 Oct · stop 2NEG-4 buyer guardrail layers against injection (ESCI)The plain hardening prompt did the work; no layer on top (Cruxes step, reflection, probe, regex) was credited.Not credited

9 Oct · stop 4PRECOG-1: does reading Qwen3-8B's thinking help the gate?No; reading the thinking was worse than a shuffled-thinking control. The answer-only gate itself was credited (internal).Did not hold

9 Oct · stop 4C0 CruxBench selector vs random pickNo, as predicted.Not credited

9 Oct · stop 2UNSC votes: Cruxes step vs matched reflection (exploratory, contaminated)No; worse on Gemma.Did not hold

9 Oct · stop 2LEGAL-1 LEXam international law: Cruxes list vs equal-length reflectionNo; worse on both models. Jev pick beat random on one family only.Did not hold

9 Oct · stop 1INTERJECT-1: can a small model decide speak, hold or quiet?No small model reached 0.75.Not credited

9 Oct · stop 2DP3 persuasion text on the NEG-4 shelvesFloor result: plain buyers already picked the pattern listing at or below chance.Not credited

9 Oct · stop 4DP2 dark-pattern detection, screenshot vs textNot credited: Haiku gained recall with worse false positives; Gemma no change. No Cruxes arm.Not credited

10 Oct · stop 2H2H-1: Cruxes instruction vs strong method prompt (synthetic VC models)No; worse on Gemma and Qwen, Haiku at ceiling.Did not hold

10 Oct · stop 2H2H-2: recalculation tool and the Cruxes instruction againCruxes instruction lost again; the tool was a weak positive on Gemma only.Did not hold

10 Oct · stop 4H2H-3: ask-or-act gate vs strong ask prompt at matched ask rateNo evidence; a free range-width rule matched the gate.Not credited

10 Oct · stop 1FP-1 frontier parity: existing adapters vs Sonnet 5.5 and Haiku 5.5 (AskBench)No: well below both; cheaper than Sonnet but not than Haiku.Did not hold

10 Oct · stop 1FP-2: Qwen3-8B trained on the task itself vs frontierNo: the tuned model fell below its own base and below matched random asking.Did not hold

11 Oct · stop 4SAFE-1 MonitorBench: Cruxes monitor step vs length-matched reflectionNo evidence; monitors were near ceiling.Not credited

11 Oct · stop 4Monitor reads own chain of thought vs a shuffled donor (sanity check)Reading the CoT beats reading only the answer; own vs shuffled CoT not credited on the primary.Not credited

11 Oct · stop 1ARCH-1: does architecture matter for ask-or-act?'Architecture matters' not credited; the Qwen3-8B LoRA head was best on When2Call; AgentAbstain a floor.Not credited

11 Oct · stop 1ARCH-1b: same question at matched sizeDecoder beat encoder and encoder-decoder on When2Call; AgentAbstain a floor.Not credited

11 Oct · stop 3MECH-2: frozen hidden-state probe vs TF-IDF and zero-shotOn When2Call it beat zero-shot and TF-IDF; it predicted one class on AgentAbstain.Not credited

11 Oct · stop 3MECH-3: decision memory and steering as an autonomy dialNo dial credited; a random direction matched the steering slope.Not credited

11 Oct · stop 3MECH-4: fork points in thinking; do activations lead the text?About 9 in 10 items were settled before the thinking said anything; claims not credited.Not credited

11 Oct · stop 2SAFE-5: 'name the deciding fact' before stopping (AgentAbstain, When2Call)No evidence on both primaries; against on Qwen3-14B When2Call; plain reflection was the strong free control.Not credited

11 Oct · stop 4MECH-1: counterfactual fill-in route to the crux (QuestBench Logic-Q)Not credited: ties self-consistency, beats enumerate-and-judge. Side contrast: stating the Cruxes rule vs a neutral sentence, no evidence.Not credited

11 Oct · stop 3MECH-7: steering as an ask-vs-act dial and for calibrationNothing credited; steering lost to temperature scaling. Addendum: a Gemma neuron lead survived a random-neuron control (descriptive).Not credited

11 Oct · stop 3MECH-8: attributed neurons as the ask dialYes on Gemma-3-12B, on two datasets (credited as a mechanism); not cleared against activity-matched sets in 2 of 4 cells; Qwen sign-reversed on When2Call.Worked

12 Oct · stop 2AGENT-LOOP-1: priced Cruxes gate and Cruxes-created plan inside an agent loopNot credited on two families; the Cruxes plan was worse than a plain plan on AgentAbstain for Gemma and Qwen; the gate was never below a free rule.Did not hold

12 Oct · stop 2FL-1 frontier ladder: Cruxes step vs matched reflection, 11 prompt arms and a linter, 10 modelsNot on any model; below 0 on When2Call for five; the surface linter was a credited negative on eight models; write-the-call beat the Cruxes step on six of seven readable models.Did not hold

13 Oct · stop 4BENCH-1: hosted Cruxes gate before every state-changing call (AutomationBench, Sonnet 5.5)Credited negative: the gate stopped 93 of 120 tasks and completed fewer than a self-check gate; it reduced violations.Did not hold

13 Oct · stop 4PL-3: recover lost detector candidates without duplicatesRecall up and duplicates down, but p50 latency failed the 1.3x cap.Not credited

13 Oct · stop 4PL-4: recover plus a lenient evaluator with a judged dedupe-merge countRecall passed; latency and dedupe merge loss failed.Not credited

14 Oct · stop 4HC-1 (draft): gate routing replay on TAC-1Not run; $0 draft with a precondition that TAC-1 rows carry per-crux cruxiness.Pending

14 Oct · stop 4TAC-1 stage 1: hosted gate as a post hoc check on TheAgentCompany runsNot credited; the gate stopped clean runs far more often than the self-check.Not credited

15 Oct · stop 5AQ1: do decision artefacts carry the card's facts? (model-reader proxy)The text card is the bar; every artefact carried fewer facts, and the podcast turned stale facts into confident errors.Not credited

15 Oct · stop 5Nightly decision loop over decided cardsRuns in the research session; pulls were incomplete and no proposal was raised.Exploratory

15 Oct · stop 2FX-1: does a Crux step improve forecasts scored against real outcomes?No evidence for either Crux arm (one arm void on GPT-5.5); replicates the R7/R13 null.Not credited

15 Oct · stop 2FX-2: crowd-anchored forecasts with a Crux stepNo credit; the crux arm leaned better on both families with intervals touching 0.Not credited

15 Oct · stop 1GATE-T1 (draft): a learned STOP/GO gate for agents modeNot started.Pending

15 Oct · stop 4HC-10: Jev's picks on the record_crux and rank_decision pathsYes, the pre-set rule triggered: the thinner rank-path input flips Jev's pick on 30% of cards, so the canonical-state fix was proposed.Worked

15 Oct · stop 4HC-2: Jev reliability on 60 real decided cardsJev is repeatable on a fixed input; route flips happen only at the cutoff and order changes only on near-ties; no registered problem rule tripped.Not credited

15 Oct · stop 4HC-2 arm F: a blind frontier model's pick vs the person's choiceNo: the recommendation shown on the card predicted the person's choice better; anchoring cannot be separated from extra context.Exploratory

15 Oct · stop 4HC-2e: does the production call path change Jev's pick?The registered rule did not trip, but a replay reproduced production's stored pick on only 38 of 60 cards; a later correction showed this was not production's rank path.Not credited

15 Oct · stop 5HC-4 (draft): outcome calibration of Jev, frontier model and person; counterfactual replayNot run; pre-registered before any outcome data exists; depends on the outcome stage.Pending

15 Oct · stop 2HC-5: hill-climbing the Cruxes step on When2Call (gpt-5.4-mini)No registered contrast met; the default test moved when the model stops (over-asking 28.1% -> 16.6%), not how well it separates cases.Not credited

15 Oct · stop 2HC-6: rate-free comparison with five reads per itemYes (primary met); today's Cruxes step lost to reflection again; the truth table beat today's step but only tied reflection.Worked

15 Oct · stop 2HC-7: is the gain the crux or the default clause?HC-6 replicated; which ingredient carries it was inconclusive at n=500.Worked

15 Oct · stop 2HC-8: crux vs clause settled on 1,500 items; three more Cruxes formsNo evidence: the gain belongs to the self-judged default clause; no new Cruxes form beat X4a.Not credited

15 Oct · stop 2HC-9: the default sentence on a frontier model (gpt-5.4)Yes, more than on the mini model; roughly halves over-asking; costs a little on requests that needed a question.Worked

15 Oct · stop 4MEM-1: does the detector stop re-asking decided questions, and does that survive rewording?Paused before any arm ran: the hosted model’s quota was exhausted on the first case.Pending

15 Oct · stop 4VG1: Jev vs frontier models at visual groundingNo: below the registered bar on all three sets.Did not hold

20 Oct · stop 2HC-12: Cruxes as a harness on AgentAbstain agents (three models)No Cruxes arm topped on any model; tune selection did not transfer; Sonnet 5.5's provider refused most runs whose text asks for written reasoning.Not credited

24 Oct · stop 3INPASS-1: reading and editing the Cruxes state in one forward passNo gated edit was credited on either model or dataset; the free read-only stack matched or beat both edits; single-pass engineering held.Not credited

27 Oct · stop 3Falsifier: A6.2 re-grade, J-lens steering vs constitutional promptNo: steering does not beat the prompt on harmful compliance and is much worse on over-refusal.Did not hold

27 Oct · stop 3Falsifier: is AgentHarm on Gemma-4 feasible?The track survives with a pinned chat template; constant steering vectors cannot be folded into Gemma-4 weights.Exploratory

27 Oct · stop 1Falsifier: is 'BFCL rank 3 / #1 open' reachable for Gemma-4-31B?No: the board is frozen at 2026-04-12 and Gemma-4-31B's 70.77 would be 5th; only a pre-registered paired gain is claimable.Did not hold

27 Oct · stop 4JV-1: Cruxes steps against Jev's selection bias (reorder protocol of arXiv 2610.35826)Cruxes claims not credited. Adaptive order-averaging (J6) met its registered compute claim. Retention rose from 64.1% to 83.4%, and plain reshuffling reaches most of that.Not credited

27 Oct · stop 4Papers-memo falsifier: does order-averaging (J6) remove the added-option distortion?No: the distortion stays 1.8x the run-to-run noise after averaging; decisions did not flip beyond noise.Did not hold

27 Oct · stop 3SAFE-1 stage 1 (G1): probe-gated constitutionRegistered, not run; stage 0 recommends against running it as registered.Pending

27 Oct · stop 2SAFE-1 stage 0: AgentHarm headroom on Gemma-4-31BThe always-on refusal prompt already cut harm 15.6 points at a 3.1-point benign cost; little headroom; recommendation: no stage-1 spend as registered.Exploratory

27 Oct · stop 1SOTA-1: fine-tuned Qwen3-8B on the official When2Call protocolCancelled before scoring; only the pipeline check ran, and it reproduced the paper’s baseline.Pending

27 Oct · stop 2SOTA-2: frontier models on all of AgentAbstainNo model credited. Every run error was a provider content-filter refusal. Comparison with the published top score is descriptive only.Not credited

27 Oct · stop 1SOTA-3: open-weights baselines on Zapier AutomationBenchMistral-Small-3.2 passed 0 of 600 tasks, usually by not acting at all. The Gemma runs were stopped before export.Exploratory

27 Oct · stop 2SOTA-4: playbook hill-climb loop on AutomationBenchWithdrawn before any step ran.Pending

27 Oct · stop 1SOTA-5 track A: self-rejection fine-tuning on Gemma-4-31B (AutomationBench)No result. Stage 0 was cancelled after 4.5 hours with no output.Pending

27 Oct · stop 3SOTA-5 track B: Mistral probes and stop-direction ablation (withdrawn, refit)After a labelling bug was fixed, the refit probe beat the comparator clearly; the B3 ablation used wrong labels and cannot be read.Exploratory

29 Oct · stop 4Replay of saved cruxes through open Gemma models, with a blind quality judgeGemma-3-12B matched production on coverage at about a third of the cost and was lower on the quality of each crux. Most contexts were sample prompts.Exploratory

November 2026

1 Nov · stop 4Do the cruxes Cruxes raises match where a model’s own answers disagree?Matched more often than a topic-different control (59% against 19% on app contexts), partly topical overlap. Exploratory.Exploratory

Go deeper

Technical reports and posts

Finding 1 · inside the modelReading and writing inside the forward passWhich layer, what the probe is, how the free read is fitted, what the edits did, and the safety tests. Finding 2 · the instructionsOne sentence, a third of the priceWhere the sentence came from, the studies that isolated it, and every interval. Finding 3 · the answerShuffle and re-askThe reorder protocol, the matched controls, and what crux framing did not add.

Finding 1 · inside the modelReading a Model InsideWhich layer, what the probe is, how the read is fitted, what the edits did, and the safety tests.Finding 2 · the instructionsOne Sentence to AskWhere the sentence came from, the studies that isolated it, and every interval.Finding 3 · the answerShuffle and Re-askThe reorder protocol, the matched controls, and what crux framing did not add.The reportTechnical report (PDF)Every registration, figure, footnote and appendix. Sign in with a work address and request access to download it.

9 · Anticipated questions

Is any result the best published on its benchmark?

No. None of these levers has been scored on a benchmark’s official leaderboard metric, and our first cost claim did not hold: on 1,200 requests the read scored below the cheapest strong hosted model. A plainly prompted Gemma-4-31B, without Cruxes, scored .828.

Does the crux framing contribute?

No. In a 1,500-request test, naming the crux added nothing measurable over the default sentence alone, and on Jev a crux-framed tie-break did no better than plain reshuffling. The methods work in part; the label does not contribute.

Is a self-hosted model required?

Only for Cruxes Read, which inspects the model’s activations. Cruxes Default works with any hosted model, and Cruxes ReAsk works with any tool that returns a choice.

Is editing a model’s internals risky?

In our tests it was mostly unnecessary. On the one task where we compared them, a trained probe outperformed every edit we tried, and in a safety test steering refused 17 of 60 harmless requests, against none for a plain safety prompt.

Who verified the numbers?

Our own team. Most studies were registered before they ran, but 28 of 136 are exploratory and 15 were not registered, and the agent that rechecks results is part of the team. No result has been recomputed outside the team; each result says so.

Is the comparison against current frontier models?

Yes. On 27 October we ran Cruxes Default on Opus 5.5, Fable 5.1, GPT-6 Astra, GPT-6.1 Sol, Gemini 4 Argon, Muse Wonder 1.0 and Kimi K3.1 Pro, on the same 286 requests ($150 of calls). It gave none of them a measurable gain; they already ask well, and the sentence traded unnecessary questions for missed ones. It helped one small model. A cheaper Gemini model reached the same range without Cruxes.

What did not work?

Older Cruxes prompts, used alone, scored below a plain prompt on 8 and 9 of 10 hosted models. Steering did not outperform a prompt. A stop in front of an agent’s actions passed 7.5% of tasks, against 31.7% for the agent checking itself. A fine-tune on public labels was stopped. An open model passed 0 of Zapier’s 600 AutomationBench tasks. Our first cost claim did not hold. Each is in the study record.

10 · Contributions

The programme is directed by me, Aayush Srivastava. I set its goals and made its calls: the hill-climb, settling the crux against the default sentence (HC-8), checking on a frontier model first (HC-9), the in-pass probe (INPASS-1), improving Jev against its published limits (JV-1), and moving open-model work to Gemma.
The default sentence was written by the AI research assistant (Claude, an Anthropic model, in Claude Code) during the HC-5 hill-climb, from the model’s own tuning errors. The same assistant added the free-read control to INPASS-1 before any model call, designed and self-registered each study, and built this page from the logged files.
The July steering runs were made by another AI coding agent working with me; in October one checking agent re-graded them from the raw files and a second re-checked every number.
Benchmarks and methods belong to their authors: When2Call (Ross, Mahabaleshwarkar and Suhara, NVIDIA, NAACL 2025), AgentAbstain (Liu et al., NeurIPS 2026), AgentHarm (Andriushchenko, Souly et al., ICLR 2025), the reorder test (Song, Zheng, Ma and Zhou, Tencent, arXiv 2610.35826), AutomationBench (Zapier), contrastive activation addition (Rimsky et al., ACL 2024), reflection (Madaan et al.; Shinn et al., 2023) and the pre-mortem (Klein, 2007).
Disclosure. The J-lens is Anthropic research (Gurnee et al., 2026). Several compared models, and the assistant that ran these studies, are Anthropic models.

Notes and references

When2Call (Ross, Mahabaleshwarkar and Suhara, NVIDIA, NAACL 2025), 3,649 test requests. Probe on Gemma-3-12B-it, layer 24, trained on When2Call’s labelled training set: three-class balanced accuracy .706 against .613 for the model’s own answer (+.093 [+.077, +.108]); TF-IDF .389; shuffled-label probe at chance. Transfer to AgentAbstain (Liu et al., NeurIPS 2026) failed: one class predicted, .500. Qwen3-8B with a fine-tuned head scored .782. On 1,200 requests the combined read scored .704 against .617 for the plain readout (+.087 [+.054, +.120], descriptive); at a dev-fixed 10% over-stop, ask recall +.005 [−.046, +.056]. Sources: RESULTS-R49, RESULTS-INPASS1.md; registered, recomputed internally by RC-1 on 28 Oct. No outside recompute.
NI-1 (self-registered, two internal recomputes), 1,200 When2Call requests: Cruxes Read .704; Gemini 3.5 Flash-Lite .807, difference −.103 [90%: −.128, −.078], past the preset .04 margin; GPT-6 Luna .716 (−.012 [−.038, +.015]); GPT-5.4 mini .682; Claude Haiku 5.5 .664; Gemma-4-31B-it with a plain prompt .828 (descriptive). Over-asking on requests that need no question: 29.8% for the read, 8.9% for Flash-Lite. One A100 costs $1,015 to $1,825 a month to keep on. The earlier AgentAbstain comparison (.637 against .695) was unpaired, chosen after the fact from 110 setups, and is no longer used. Source: RESULTS-NI1.md.
When2Call (Ross, Mahabaleshwarkar and Suhara, NVIDIA, NAACL 2025), the 286 requests shared by HC-6, HC-9 and HC-13, three reads each. GPT-5.4 mini plain .838 [.790, .884] (HC-13); with Cruxes Default (study arm X4a) .885 [.844, .923] (HC-6 rows); paired difference +.047 [+.012, +.083], descriptive. Seven frontier models with a plain prompt, HC-13 (27 Oct 2026): .866 to .902, $1.84 to $18.25 per 1,000. Cruxes Default minus plain on those seven: −.028 to +.033, none credited after Holm correction. Earlier, on GPT-5.4 (March 2026), Cruxes Default beat reflection by +.049 [+.013, +.086] (HC-9). Needless questions on complete requests, GPT-5.4 mini: 30.4% plain, 16.5% with Cruxes Default. The sentence was tuned on GPT-5.4 mini. Sources: RESULTS-HC13.md, RESULTS-HC9.md, RESULTS-HC8.md. Self-registered; awaiting independent recompute. Tuned on GPT-5.4 mini; the comparison was not registered. Gemini 3.8 Flash, plain prompt: .900 at $1.39 per 1,000.
Our rebuild of the reorder protocol in Song, Zheng, Ma and Zhou (Tencent, arXiv 2610.35826) on 800 held-out SuperGPQA and LEXam questions, 211,824 Jev 1.14 calls. Right answers kept through eight reshuffles: 64.1% [59.5, 69.0] asked once, 83.4% [79.7, 86.9] with Cruxes ReAsk (study arm J6) at 3.22 calls. The paper reports 77.2–90.2% for four frontier models on its own sample. J6 was designed on our tune half. Matched plain shuffles at equal calls reached 81.2%; accuracy 49.6%; descriptive. Sources: RESULTS-JV1.md, eval/jev1/results/primary.json.
The register tracks 136 studies from 26 June to 28 October: 63 registered, 40 registered and amended, 18 exploratory by design, 15 not registered. Outcomes: 7 met their rule, 1 credited, 57 did not meet their rule, 19 falsified, 9 void, 3 descriptive, 28 exploratory, 12 pending. Names: every Cruxes method is named “Cruxes” plus its technique; research ids stay in the study records (landing/NAMES.json).
MECH-8 (register row R55, 11 Oct 2026), Gemma-3-12B-it, layer-28 MLP neurons chosen by activation × gradient attribution, clamped. When2Call k=8: +0.463 [+0.416, +0.508] in the model’s own ask probability, against a 95th percentile of 0.215 for 16 random sets; AmbigQA k=8: +0.549 [+0.524, +0.573] against 0.484. Neutral-text likelihood unchanged. Recomputed and credited by our internal registrar as registered and no wider; no outside party has recomputed it. Not established: a control of the most active 1% of neurons is not cleared in 2 of 4 cells; Qwen3-8B reverses sign on When2Call; the ask probability tracks gold ambiguity at AUROC .54. Source: docs/research/handover/CONSOLIDATED-PLAN-AND-STATUS.md (branch claude/research-hc12), row R55.