Avaliações PREreview de Towards Evaluating the Diagnostic Ability of LLMs

Ir para detalhes do preprint Ir para avaliações PREreview

Towards Evaluating the Diagnostic Ability of LLMs

Publicado: 12 de outubro de 2024
Servidor: Preprints.org
DOI: 10.20944/preprints202409.0688.v3

Resumo

On average, one in ten patients die because of a diagnostic error and medical errors are the third largest cause of death in the US. While LLMs have been proposed to help doctors with diagnoses, no research results have been published on comparing the diagnostic ability of many popular LLMs on an openly accessible real-patient cohort. In thus study, we compare LLMs from Google, OpenAI, Meta, Mistral, Cohere and Anthropic using our previously published evaluation methodology and explore improving their accuracy with RAG.

Ler o preprint

1 PREreview

Escrever uma Avaliação PREreview Solicitar uma Avaliação PREreview

Avaliação PREreview feita por Daniela Saderi et al. do/a JMIR Publications

De autoria de Daniela Saderi, Randa Salah Gomaa Mahmoud, Goktug Bender, Olajumoke Ope Oladoyin, Paul Hassan Ilegbusi, Arya Rahgozar, Ph.D., Manikant Roy, Maria J C Machado, Benjamin Senst, CLARA AMAKA NKPOIKANKE AKPAN, Nour Shaballout, Sylvester Sakilay, Sandeep Vohra, Mitchell Collier, Raimi Morufu Olalekan (BSc. M.Sc. Ph.D. MNES, REHO, LEHO, FAIWMES)), Uday Kumar Chalwadi e Toba Olatoye do/a JMIR Publications

This review is the result of a virtual, collaborative live review discussion organized and hosted by PREreview and JMIR Publications on November 14, 2024. The discussion was joined by 29 people: 2 facilitators, 2 members of the JMIR Publications team, 2 preprint authors, and 23 live review…

Ler a avaliação PREreview de Daniela Saderi et al. do/a JMIR Publications