Nvidia researchers have found that KV cache transfer between LLMs is a linear problem, not a deep learning one. Their cross-model KV cache transfer technique maps the prefilled KV cache from a source model directly into a target model using ridge regression, no gradient-based training required. On compatible model pairs, including the 8.8x parameter jump from Llama 3.1 8B to 70B, the method runs 2.7 to 25 times faster than full prefill recomputation while retaining 73% to 98% of the target model's standalone accuracy.
The mechanics are worth reading closely. The mapper operates in three steps: per-head ridge regression fit on just 500 calibration sequences, cross-layer source selection that identifies which source layers best predict each target layer, and RoPE-stripping before translation so the mapper generalizes beyond its training context length. On a Qwen3 14B-to-32B transfer, combining multiple source layers recovered 79% of key variance and 65% of value variance. The researchers tested across Qwen3, Llama 3.1, and Ministral 3 families, ranging from 3 billion to 70 billion parameters, on benchmarks including MMLU, GSM8K, ARC-Challenge, and the multi-turn CoQA task.
The current scope is intentionally narrow: within-family transfers only, where models share tokenizers and KV head counts. Two of the six tested pairs showed weaker results, and even a backpropagation-trained neural network failed to close that gap, pointing to a structural incompatibility the linear method alone cannot fix. The paper's framing of cross-family transfers, mismatched KV head counts, and hybrid attention architectures as open problems is where the real long-term stakes sit.
[READ ORIGINAL →]