Skip to preprint detailsSkip to PREreviews

PREreviews of GAUGE: When Not to Trust LLM-as-a-Judge in User-Simulated Evaluation of Task-Oriented Agents

1 PREreview

  1. PREreview by Jiayu Long

    This paper studies whether an LLM judge, used with simulated users, can choose the better task-oriented agent. Across 25 agent settings, the judge ranks agents well when their performance differs greatly, but is less reliable when they perform similarly. The paper also shows that a conversation…

    Read the PREreview by Jiayu Long