Gearshift: Can One Model Think and Another Finish?

A note from Impossible Computing

This is the first research note on the Impossible Computing site.

I want to share work while it is still work. Some experiments will work. Some will fail. Some will expose a bad assumption or a bug in the experiment. I would rather show that process than wait until every result can be wrapped in a clean success story.

Gearshift is a good first example. The original idea still has not worked. Along the way I found a training change that held up on a fresh confirmation set and a second training seed. A follow-up sparse-state experiment showed that a small amount of accurate receiver state can repair much of the damage in my mapped state. The selector I expected to matter did no better than random repair on this small screen. Then an economic calculation showed that the specific optimization I started with has almost no payoff on this workload.

That is where the project is today.

The question

I started Gearshift with a simple question. Suppose a larger model works through a hard problem. Can it hand that work to a smaller model, which then produces the answer more cheaply?

The obvious implementation is text handoff. The large model writes its reasoning. The smaller model reads the problem and that reasoning, then answers. It works, but the smaller model has to process the whole history again.

Gearshift tries to skip that reread. Instead of handing over text, I translate the larger model’s KV cache into state the smaller model can continue from.

A KV cache is the attention state a transformer keeps for tokens it has already processed. A software analogy is moving a paused computation between two runtimes with different internal representations. memcpy is not enough. The state needs a conversion.

The Gearshift mapper is that conversion. Both language models stay frozen. What I train is a relatively small cross-model cache translator: source and receiver layers are paired by normalized depth, source keys have their rotary position encoding removed before the affine mapping, receiver positional encoding is applied afterward, and keys and values are mapped separately. Once those mapper weights are trained, the conversion itself is deterministic.

The small-model experiments started on my laptop, then moved to my Mac Studio. When I moved to the Qwen3-32B to Qwen3-8B coding experiments, I moved the model work to NVIDIA H200s in BF16, rented from RunPod. The confirmation and sparse-state work also ran on H200s. The Mac Studio became the orchestration and analysis machine for the later runs; I did not treat Apple Silicon measurements as interchangeable with the CUDA results.

Prior work

Cross-model reasoning handoff and cache communication are active research areas. CoT-X studies stronger-to-cheaper reasoning transfer through text. Cache-to-Cache studies learned communication through model caches. Other work explores linear cache translation, cache bridging, and shared cache representations.

Gearshift sits inside that body of work. My question is mostly a systems one: when one model has already produced a long reasoning history, can another model reuse that work accurately enough and cheaply enough to improve the end-to-end system?

Native text replay is the clean control. I give the receiver the same history and let it process the text normally. If replay works and translated state fails, the receiver is capable of using the information. The conversion is losing something.

The first encouraging result was too encouraging

I started with Qwen3-1.7B as the source and Qwen3-0.6B as the receiver. On one 100-question arithmetic follow-up, a functionally trained cache mapper produced 78 correct extracted answers. Native text replay produced 76.

I was excited by that result. Then I widened the test.

On 400 fresh arithmetic questions, native replay scored 344 and mapped transfer scored 316. On 100 HumanEval+ coding tasks, native replay passed 57 and the frozen mapper passed 9. The receiver could use the history when it read the history normally. My translated state was losing useful information.

A larger receiver did not magically fix it

I moved to Qwen3-32B handing off to Qwen3-8B on coding tasks. On the first 40 development tasks, the large model passed 34. The 8B model passed 27 when reasoning independently and 34 when it read the large model’s reasoning as text. The mapped handoff passed 3.

The failures were useful. Some outputs lost the requested function interface. One mapped answer turned a minimum weighted-distance problem into a maximum unweighted one. The receiver was sometimes starting from a damaged representation of the assignment.

I changed the handoff so the receiver processed the original prompt normally and I translated only the large model’s reasoning suffix. That hybrid moved the score from 3 to 18. Prompt-only answering without a separate reasoning stage already passed 17. Preserving the assignment fixed a real problem, but the transferred reasoning still was not doing much useful work.

Then I noticed what I was actually training

The original training recipe applied direct loss at a few fixed answer positions: early tokens, then small windows around positions 32, 128, and 512.

An audit showed that some of those windows landed mostly in introductory prose. Two inspected histories had no code tokens in the sampled windows. That gave me a clean experiment.

I trained two copies of the same starting mapper. The models stayed frozen; the only trainable parameters were the cache-translation weights. Both runs got the same number of optimizer updates and the same number of supervised answer positions.

FIXED reused the same few answer windows on every update:

FIXEDSame budget · 32 positions per update

Early answerLater answer →

Update 1

Update 2

Update 3

ROTATING kept the same supervision budget but moved the later windows through the answer over training:

ROTATINGSame budget · 32 positions per update

Early answerLater answer →

Update 1

Update 2

Update 3

So ROTATING did not get more supervised positions per update. It got broader coverage of the answer over time. That meant the training signal was more likely to see the actual code, function signatures, returns, and later parts of long answers.

ROTATING looked much better on the 21-task validation set I had already examined. I froze the endpoint and moved to a separate 200-task confirmation set.

The 200-task confirmation

The confirmation run used 200 task clusters kept out of Gearshift’s training, development, and validation loop. Each condition got three fixed answer draws per task. The task is the independent unit in the statistical analysis.

Two labels matter in the table. “Pure mapping” means the receiver gets translated cache state for the whole prior history, including the original prompt. “Hybrid” means the receiver processes the original prompt itself, in its own native representation, and I translate only the large model’s reasoning suffix.

The prespecified primary comparison was ROTATING versus FIXED for pure cache mapping. ROTATING improved success by 10.17 percentage points. The paired 95% interval was +6.33 to +14.33 points.

Confirmation · 200 task clusters

Coverage helps. Text handoff still leads.

Three fixed draws per task: 600 answers per condition. Pass rate averages all draws; tasks are the independent units.

• FIXED pure mapping19.0%114/600 draws
• ROTATING pure mapping29.2%175/600 draws
• FIXED hybrid31.8%191/600 draws
• ROTATING hybrid38.2%229/600 draws
• Smaller model reasoning55.0%330/600 draws
• Larger model alone64.3%386/600 draws
• Native text handoff65.0%390/600 draws

That result held up under a second training seed. In the second run, FIXED pure mapping scored 15.8% and ROTATING scored 32.8%, a +17.0 point difference with a 95% paired interval of +12.67 to +21.50. The hybrid comparison moved from 32.7% for FIXED to 41.7% for ROTATING, a +9.0 point difference.

I am comfortable making one specific claim from this: spreading the same amount of training supervision across more of the answer improved this mapper on this model pair and coding setup. Two training seeds showed the same direction on the same frozen 200-task population.

The full system still loses

The best primary-seed cache handoff scored 38.2%. The smaller model scored 55.0% when it reasoned independently. Native text handoff scored 65.0%.

The second seed improved the best hybrid to 41.7%. It still trailed the independent smaller model by 13.3 points and native text replay by 23.3 points.

I found a training method that improves the translator. The resulting system is still worse than the alternatives I care about.

I also broke my own evaluator

Before the fresh confirmation run, an audit found that my coding sandbox capped output at 2 MiB even though one benchmark task could legally require roughly twice that much output. I also found timeout sensitivity in a few scoring runs.

I repaired the scoring policy, froze a new scorer, and rescored the same saved programs without regenerating or editing them. Thirteen of 1,512 validation outputs changed from failure to pass. The ROTATING versus FIXED pure-mapping difference stayed at +20.6 points on that validation set.

The harness is part of the experiment. Fixing it changed some scores and left the central training result intact.

The sparse-state experiment

Earlier this year I spent a bunch of time experimenting with sparse attention: whether a model really needs to attend to its entire history at every step. In some of those experiments, a small query-dependent subset of the history preserved surprisingly good results. The systems wins were much less automatic; selecting and moving the sparse state could eat the theoretical savings.

That made me wonder whether Gearshift was solving too much of the cache. Maybe the receiver did not need every part of the historical state at every answer step. I ran a bounded diagnostic on 12 development tasks to find out.

This was an oracle experiment. I paid for the full native receiver cache, then let the receiver use only a fraction of the historical reasoning state at each step. I also tried repairing the mapped cache by replacing a fraction of it with accurate native state. This is a quality diagnostic. The implementation still built the full native state and used dense masked attention, so it does not demonstrate a speedup.

The native sparse result surprised me. At 5%, 10%, and 25% per step, the receiver passed 33 of 36 draws, exactly the observed count for dense native replay.

Mapped sparse state stayed much worse. At 10%, native sparse state passed 33 of 36 and mapped sparse state passed 21 of 36. Matched-query probes showed the same pattern in attention-output error. Translation fidelity still looks like the local problem.

Repair was even more interesting. Replacing 5% of the mapped historical state with accurate native state brought the observed score from 21 of 36 to 33 of 36. Ten and 25 percent did the same.

Then the control ruined the clever story (in a useful way!) Random 10% repair also scored 33 of 36 and matched attention-targeted 10% repair on every pass/fail outcome in these 12 tasks. Recent-token repair scored 29 of 36. A recent paper called Random Attention reports a related result for within-model KV eviction: random retention can match stronger selectors on reasoning tasks when the prompt is preserved. Their setting is different from mine, but it makes the random control here especially worth paying attention to.

Exploratory only · 12 previously inspected tasks

A small development screen

36 draws per condition = 12 development tasks × 3 draws. Oracle repair pays for the full native cache. These are not confirmation results or a demonstrated speedup.

• Dense native replay33/3691.7% of draws
• Mapped hybrid21/3658.3% of draws
• Native sparse, 5%33/3691.7% of draws
• Mapped sparse, 5%21/3658.3% of draws
• Repair, 5%33/3691.7% of draws
• Native sparse, 10%33/3691.7% of draws
• Mapped sparse, 10%21/3658.3% of draws
• Repair, 10%33/3691.7% of draws
• Random repair, 10%33/3691.7% of draws
• Recent-token repair, 10%29/3680.6% of draws

This screen is small and uses development tasks I had already inspected. I do not know yet whether 5% repair generalizes, and I have no evidence that my attention-based selector beats random repair.

There was another catch. Ten percent per step touched 77.7% of the historical layer/head/position pairs over a complete answer on average. Five percent per step still touched 65.1%. A small instantaneous working set can move around enough to require access to most of the backing state over time.

The economics changed my mind about the original target

I also ran the calculation I should have run earlier: assume a perfect converter. It costs zero, transfers instantly, and reproduces native text replay exactly. How much time could it save by eliminating receiver prefill?

On the 200-task confirmation population, the answer is 1.468 seconds per answer. The recorded text-handoff stage sum drops from 726.96 seconds to 725.49 seconds. That is 0.202%.

The timing breakdown explains it. Large-model reasoning averaged 696.64 seconds. Receiver prefill averaged 1.47 seconds. Receiver answer generation averaged 28.83 seconds.

Records-only economic calculation · same 200 tasks

1.468 seconds out of 726.96

Mean seconds per answer, charging the full source history. A perfect, free converter removes 0.202% of this recorded stage sum; it is not a measured serving speedup.

• Large-model reasoning696.64 s
• Receiver answer generation28.83 s
• Receiver prefill1.468 s
• Cache clone0.010 s

I had been trying to optimize a tiny term.

The economic result is narrower than “heterogeneous inference doesn’t work.” What it says is that naive reason-then-handoff, where Gearshift’s main win is avoiding the receiver’s reread, is not useful on this workload. That reread is only 0.202% of the measured stage sum. There just isn’t much to save.

The larger idea is still interesting to me. A different system could hand off earlier, reduce the expensive source-reasoning stage, make receiver decoding cheaper, or target workloads where prefill is actually expensive. Those are different experiments. This one tells me to stop optimizing the 1.47-second part of a roughly 727-second path.

Where Gearshift stands today

I started with a cache-conversion idea. The experiments gave me a more useful map of the problem.

Broader training coverage improves the mapper, and that result repeated under a second training seed. Accurate receiver state can rescue the mapped-state failure on a small exploratory screen. Sparse native access also looks surprisingly tolerant on that screen. The current selector doesn’t seem all that special yet, and the complete-answer working set is much larger than the per-step percentage suggests.

For the systems goal, the biggest result may be the economic one. Perfect prefill replacement barely changes the cost of the workload I tested.

I am going to stop treating receiver-prefill elimination as the main objective for Gearshift on this workload. The next question should attack the expensive work.

So that’s where Gearshift is right now. I started out trying to avoid replaying a reasoning trace and ended up discovering that replaying it was barely costing anything.

I’m not done with the larger question. I still think there’s something interesting in having expensive models do the work they’re uniquely good at and handing the rest to cheaper models. I just don’t think this version of Gearshift is the way to do it.

The code, configs, numeric records, and reports are available on GitHub. If you think I’m drawing the wrong conclusion, see an experiment I missed, or have a better idea for where to attack the problem next, I’d love to hear from you.

In the meantime, I’m going after the expensive part.

Related work and evidence

The closest references I am using to position Gearshift include CoT-X, Cache-to-Cache, cross-model KV cache transfer, CacheBridge, and latent-space communication via KV-cache alignment. Gearshift is an empirical systems investigation inside that research area.

CoT-X: An Adaptive Framework for Cross-Model Chain-of-Thought Transfer and Optimization

Cache-to-Cache: Direct Semantic Communication Between Large Language Models

Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse

CacheBridge: Efficient Cross-Model KV Cache Transfer

Latent Space Communication via K-V Cache Alignment

Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning

Evidence for this note is frozen in gearshift-progress-02. The release README maps the code, configs, numeric records, and reports. The release page includes the audit receipts.

• Primary confirmation
• Second training seed
• FIXED-versus-ROTATING training
• Scorer repair
• Sparse-state and selective-repair screen
• Economic ceiling
• Earlier small-model results