PREreviews de Towards Evaluating the Diagnostic Ability of LLMs

Saltar a detalles del preprint Saltar a PREreviews

Towards Evaluating the Diagnostic Ability of LLMs

Publicado: 12 de octubre de 2024
Servidor: Preprints.org
DOI: 10.20944/preprints202409.0688.v3

Resumen

On average, one in ten patients die because of a diagnostic error and medical errors are the third largest cause of death in the US. While LLMs have been proposed to help doctors with diagnoses, no research results have been published on comparing the diagnostic ability of many popular LLMs on an openly accessible real-patient cohort. In thus study, we compare LLMs from Google, OpenAI, Meta, Mistral, Cohere and Anthropic using our previously published evaluation methodology and explore improving their accuracy with RAG.

Leer el preprint

1 PREreview

Redactar una PREreview Solicitar una PREreview

PREreview de Daniela Saderi y otros/as en JMIR Publications

Redactada por Daniela Saderi, Randa Salah Gomaa Mahmoud, Goktug Bender, Olajumoke Ope Oladoyin, Paul Hassan Ilegbusi, Arya Rahgozar, Ph.D., Manikant Roy, Maria J C Machado, Benjamin Senst, CLARA AMAKA NKPOIKANKE AKPAN, Nour Shaballout, Sylvester Sakilay, Sandeep Vohra, Mitchell Collier, Raimi Morufu Olalekan (BSc. M.Sc. Ph.D. MNES, REHO, LEHO, FAIWMES)), Uday Kumar Chalwadi y Toba Olatoye en JMIR Publications

This review is the result of a virtual, collaborative live review discussion organized and hosted by PREreview and JMIR Publications on November 14, 2024. The discussion was joined by 29 people: 2 facilitators, 2 members of the JMIR Publications team, 2 preprint authors, and 23 live review…

Leer la PREreview de Daniela Saderi et al en JMIR Publications