Ai2 Trades GPU Priorities for Budgets

Allenai's infrastructure team scrapped priority-based scheduling and replaced it with time budgets—cutting on-call toil by 74%.

By The ai-compute Desk

At any given moment, Ai2's GPU clusters field requests for 2–3x more compute than they can supply. [1] The old priority-based scheduler was supposed to sort the queue. Instead, it created three durable pathologies.

GPU "squatting"—parking no-op workloads to hold a slot—emerged because researchers couldn't launch debug jobs fast enough to catch live failures. [1] Priority inflation followed: eventually 100% of scheduled workloads declared HIGH priority, starving lower tiers entirely. [1] On-call engineers spent a majority of their ticket response time negotiating the organized shutdown of non-preemptable workloads running on hosts with known maintenance problems. [1] Ai2's infrastructure team recognized the pattern: a textbook tragedy of the commons.

The fix was structural. [1] Managers allocate portions of GPU time the way investors size positions—before workloads exist—rather than adjudicating priority case by case. [1] A sliding 7-day lookback window tracks actual occupancy against each team's budget and re-sorts the queue accordingly. [1]

[1]

The lesson travels beyond any single lab. Scarcity makes gaming rational; changing the rules makes honesty cheaper than cheating. Budget-based scheduling doesn't eliminate the zero-sum nature of compute allocation—it just moves the negotiation to a place where tradeoffs are legible.

[1] The scheduler solved the commons problem; the workspace problem is next.

The new scheduler makes it feel like we have an extra 30% compute. In the old scheduler, if we had moments when we didn't need our full slot limit, that compute was basically lost.Chris Clark