The Prompt That Beats the Fine-Tune

IBM Research's ALTK-Evolve tops a customer-service AI leaderboard using GPT-4.1 and a list of rules—no weight updates required.

By The ai-compute Desk

A customer-service agent built on GPT-4.1—an older, non-reasoning model—has taken the top spot on the τ²-bench retail leaderboard, beating every frontier model entry in the October 12, 2026 snapshot. [1] It didn't get there through fine-tuning or a bigger model. It got there through a markdown list appended to its system prompt. [1]

The core idea is disarmingly simple: run the agent on training tasks, let the same model analyze what went right and wrong, and distill those lessons into short behavioral rules—how to authenticate a customer, confirm a transaction, or handle a return. Those rules get added to the prompt. The same GPT-4.1 model that powers the agent also does the distillation. No human annotation. No weight updates. [1]

The result surfaces a tension that has been building quietly in applied AI research: the difference between a model that is capable and one that is dependable. Most leaderboard comparisons collapse those two things into a single score. This work pulls them apart.

τ²-bench, developed by Sierra, is the proving ground here. It measures how well an agent handles customer-service conversations under real constraints. Each task pairs the agent with an LLM-simulated customer, a set of domain tools—look up an order, issue a refund, rebook a flight—and a written policy document the agent must follow. [1] A task counts as solved only when the final database state matches the expected outcome, a strict binary reward. [1]

The benchmark covers two domains—retail and airline—and every task runs four times. [1] That repetition is where ALTK-Evolve's results become interesting. The leaderboard ranks submissions on Pass^4: the percentage of tasks the agent solves on all four independent attempts. A support agent that resolves a refund three times out of four still fails a quarter of real customers. Pass^4 asks whether you can trust the agent every time, not just on average. [1]

On that metric, the gains are significant. With learned guidelines appended, the GPT-4.1 agent's mean accuracy climbs in both domains—retail +4.4 points (78.1 to 82.5) and airline +7.5 points (57.5 to 65.0). [1] Reliability moves further: the strict Pass^4 rises +10 points in retail (57.5 to 67.5) and +10 points in airline (35.0 to 45.0). [1] The agent reaches Pass^4 = 67.5% on retail, ahead of every frontier model entry and ahead of RAFT-30B-A3B (61.4%), a model fine-tuned on synthetic τ²-bench retail data. [1]

Outperforming a fine-tuned system on its own benchmark data—by +6.1 percentage points—using only a prompt-appended playbook is the kind of result that invites scrutiny. [1] IBM Research's setup is transparent: it uses τ²-bench's own ReAct scaffold with a two-phase THINK → ACT loop, GPT-4.1 as the agent model, and gpt-5.5 as the customer-simulator model; the only change is the learned guidelines block. [1] The guidelines are extracted from training trajectories, then applied unchanged on the held-out test set. [1]

This is where the method's elegance becomes most legible. The agent is not learning in the weight-update sense. It is learning in the way a new employee learns from an annotated case file—absorbing institutional knowledge encoded as plain text, then carrying it into every subsequent conversation.

The benchmark's airline domain adds nuance to the story. Here, more guidelines are not always better. The lossless configuration—every guideline retained after consolidation—leads on mean accuracy at 65%, but the compact sup3 core, which keeps only guidelines with support across three or more distinct training tasks, takes Pass^4 to 45% against lossless's 40%. [1] The sup3 core for airline contains 14 guidelines. [1] IBM Research frames it plainly: support count is a knob worth tuning per domain, not a value to hard-code. [1]

The cost structure is equally instructive. Guidelines push up prompt input tokens—retail's 292-guideline prompt increases cost from roughly $0.084 to $0.139 per conversation—but they do not make the agent more verbose. [1] On matched task runs solved under both baseline and guideline conditions, the agent with guidelines actually takes slightly fewer turns: lossless trims roughly 0.3 to 0.4 turns in both domains (retail 10.59 to 10.16, airline 9.15 to 8.82). [1] Fewer turns means lower end-to-end latency. And in production, IBM Research notes, the fixed guideline block is identical across every call, making it largely reclaimable with prompt caching. [1]

The team then tested whether the approach only patches gaps in a weaker model. They reran the airline benchmark with GPT-5.5 as both the agent and the guideline extractor. The results: accuracy rises +10 points (72.5 to 82.5) and Pass^4 by +15 points (55.0 to 70.0)—a larger reliability gain than GPT-4.1 saw on the same tasks, from a much higher floor. [1] On the full base split, the GPT-5.5 agent with guidelines reaches Pass^4 = 82.0% on airline, topping the leaderboard outright and clearing every frontier entry in the 66–72% range. [1]

τ²-bench itself has continued to expand in scope. The τ-bench 2.0 release adds τ-i18n: the same customer-service tasks in six languages—English, Spanish, Portuguese, Hindi, Chinese, and Korean—in text and in full-duplex voice. [2] The benchmark is becoming the standard proving ground for production-grade service agents.

The broader competitive context matters here. Fine-tuning a model on benchmark data is expensive, often brittle, and rarely transfers cleanly to adjacent tasks or domains. ALTK-Evolve matches or beats a fine-tuned system on retail using only prompt-level guidelines extracted from the agent's own runs, automatically, without human annotation, and without test-set leakage. [1]

The method is also open. ALTK-Evolve is available at github.com/AgentToolkit/altk-evolve, and the τ²-bench leaderboard submissions are linked directly to pull requests—PR #542 for retail and PR #551 for airline—so the numbers can be inspected and reproduced. [1] The benchmark framework itself, described across its own GitHub repository, is designed for contribution: custom domains, new languages, and new agent types can all be added. [2]

The implication is worth sitting with: a model that has already shipped can be made substantially more reliable by teaching it, in plain text, what it has already learned to do. [1] The frontier model is not always the answer. Sometimes the answer is a better-annotated prompt.

Pass^4 asks whether you can trust the agent every time, not just on average.IBM Research