PREreviews de Towards Evaluating the Diagnostic Ability of LLMs

Aller directement aux détails du preprint Aller directement aux PREreviews

Towards Evaluating the Diagnostic Ability of LLMs

Publié: 12 octobre 2024
Serveur de preprints: Preprints.org
DOI: 10.20944/preprints202409.0688.v3

Résumé

On average, one in ten patients die because of a diagnostic error and medical errors are the third largest cause of death in the US. While LLMs have been proposed to help doctors with diagnoses, no research results have been published on comparing the diagnostic ability of many popular LLMs on an openly accessible real-patient cohort. In thus study, we compare LLMs from Google, OpenAI, Meta, Mistral, Cohere and Anthropic using our previously published evaluation methodology and explore improving their accuracy with RAG.

Lire le preprint

1 PREreview

Rédigez un PREreview Demander une PREreview

PREreview Daniela Saderi et al. du club JMIR Publications

Rédigé par Daniela Saderi, Randa Salah Gomaa Mahmoud, Goktug Bender, Olajumoke Ope Oladoyin, Paul Hassan Ilegbusi, Arya Rahgozar, Ph.D., Manikant Roy, Maria J C Machado, Benjamin Senst, CLARA AMAKA NKPOIKANKE AKPAN, Nour Shaballout, Sylvester Sakilay, Sandeep Vohra, Mitchell Collier, Raimi Morufu Olalekan (BSc. M.Sc. Ph.D. MNES, REHO, LEHO, FAIWMES)), Uday Kumar Chalwadi et Toba Olatoye du JMIR Publications

This review is the result of a virtual, collaborative live review discussion organized and hosted by PREreview and JMIR Publications on November 14, 2024. The discussion was joined by 29 people: 2 facilitators, 2 members of the JMIR Publications team, 2 preprint authors, and 23 live review…

Lire la PREreview de Daniela Saderi et d’autres membres du club JMIR Publications