PREreview de Pretrain Once, Route Anywhere: Towards a Foundation Model for LLM Routing

## Summary

Most LLM routers are "locally fitted": trained on one workload and one pool, then retrained whenever the domain, pool, or modality changes. RouteFM asks whether routing can instead be a reusable, foundation-model-style capability. It frames routing as target-conditioned ranking over anonymous candidates: each candidate is described only by behavioral observations — historical query embeddings (frozen 4096-dim Qwen3-VL), observed quality scores, normalized relative costs — encoded into a compact capability profile (24 learnable tokens) plus the raw observation sequence. The target query retrieves evidence from both branches, candidates are compared jointly with a permutation-equivariant Transformer, and heads predict target-specific quality and relative cost. Pretraining is episodic: episodes vary the task, candidate composition (6–24 anonymous, randomly permuted candidates), and observation budget (K=8 to 1024). Supervision combines point-wise regression, pairwise ranking, and a routing-regret objective over 212,470 query–task instances and 1.73M query–model observations from four corpora. On RouterEval held-out queries, the frozen router beats the strongest baseline by 0.84 points at K=8; on MMR-Bench — multimodal, excluded from pretraining — it wins by 2.23 points at K=8 (0.7323 vs 0.7100), shrinking to 0.07 in the large-observation regime. It incorporates new models from 8 observations and matches baselines' full-context quality using 1–26% of their observations.

## Strengths

1. **The foundation-model framing is the right question, and episodic pretraining is a genuine advance.** Pretraining a reusable routing procedure — anonymous candidates, permuted slots, varying budgets, a regret loss that optimizes the decision rather than just the prediction — goes beyond "stuff observations into context." The protocol is the contribution, and it is well specified.

2. **Honest ablations, including one against their own premise.** Table 2(b) tests whether pretraining was even necessary: training from scratch on 20% labeled target-domain data scores 0.7415 vs the frozen pretrained router's 0.7523, and fine-tuning the pretrained router barely moves the needle. Reporting marginal fine-tuning gains is the kind of result most papers bury.

3. **The low-observation regime is where this matters.** Figure 3c — matching full-context Weighted kNN with 1% of observations, MLP with 3%, EmbedLLM with 26% — is operationally meaningful: in production, the router's cost is dominated by collecting graded observations per new model, so needing 8 instead of 256 is a maintenance-cost argument, not just an accuracy argument.

## Major concerns

1. **"Cost" is never denominated in anything a practitioner can budget.** The cost head predicts relative cost normalized within each episode — decisions maximize predicted quality minus λ times a dimensionless relative cost. There is no dollar-denominated cost–accuracy comparison: no answer to "spend X% less, lose Y points." For a field whose purpose is the quality–efficiency trade-off, a router evaluated without currency is half-evaluated; the primary metric is realized quality alone.

2. **Serving-time latency is waved away.** Figure 3(b) concedes simple non-parametric methods "remain faster in absolute latency" — per query, RouteFM still runs a capability profiler plus a candidate-comparison Transformer over every candidate. No ms/token numbers, no latency-at-matched-quality curve. Pretraining amortizes fitting cost; it does nothing for per-query serving latency, which is what a deployment pays every request.

3. **The 8-observation cold start assumes something expensive: graded observations.** Every observation carries a quality label — a judged outcome, not just a query. The celebrated K=8 regime needs 8 graded query–model pairs per new candidate, and the paper never costs that labeling: who grades them, with what judge, at what price? With monthly model churn, the labeling pipeline is the router's real operating cost, and it is off the books.

## Practitioner perspective

The durable idea is pretraining the routing procedure rather than fitting a routing policy — the first routing paper I've read that treats router maintenance as the first-class problem. But I would not deploy on quality numbers alone: I need the latency curve, the labeling cost behind the cold-start claims, and a dollar-denominated cost comparison before this becomes a build-vs-buy decision. The in-domain margins are modest (0.84 points at K=8, shrinking with evidence), so the bet is the cross-environment story — exactly where latency and labeling costs bite hardest.

## Overall assessment

A well-executed paper with the right ambition and honest ablations — the episodic pretraining protocol is a real contribution to routing as infrastructure. Recommend with revisions: report latency at matched quality, cost the observation-labeling pipeline behind the cold-start claims, and give at least one dollar-denominated cost–accuracy comparison.

Competing interests

The author declares that they have no competing interests.

Use of Artificial Intelligence (AI)

The author declares that they used generative AI to come up with new ideas for their review.