ALTK-Evolve tops the τ²-bench leaderboard on customer service tasks using a non-frontier model, beating systems built on GPT-4 and Claude by relying on learned memory rather than raw model scale.
The core mechanism is a memory system that evolves through task experience, storing and retrieving structured knowledge to guide future decisions. This is not retrieval-augmented generation bolted onto a large model. It is a lightweight agent architecture that gets smarter by doing, accumulating procedural knowledge across customer service interactions without retraining the base model.
The result challenges the default assumption that leaderboard performance requires frontier model access. The full paper details the memory architecture, the benchmark methodology on τ²-bench, and the ablation results that isolate exactly what the learned memory contributes. That breakdown is worth reading carefully.
[READ ORIGINAL →]