Challenging ChatGPT 3.5 in Senology—An Assessment of Concordance with Breast Cancer Tumor Board Decision Making
With the recent diffusion of access to publicly available large language models (LLMs), common interest in generative artificial-intelligence-based applications for medical purposes has skyrocketed. The increased use of these models by tech-savvy patients for personal health issues calls for a sc...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Philipps-Universität Marburg
2023
|
| Subjects: | |
| Online Access: | PDF Full Text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | With the recent diffusion of access to publicly available large language models (LLMs),
common interest in generative artificial-intelligence-based applications for medical purposes has
skyrocketed. The increased use of these models by tech-savvy patients for personal health issues
calls for a scientific evaluation of whether LLMs provide a satisfactory level of accuracy for treatment
decisions. This observational study compares the concordance of treatment recommendations from
the popular LLM ChatGPT 3.5 with those of a multidisciplinary tumor board for breast cancer (MTB).
The study design builds on previous findings by combining an extended input model with patient
profiles reflecting patho- and immunomorphological diversity of primary breast cancer, including
primary metastasis and precancerous tumor stages. Overall concordance between the LLM and
MTB is reached for half of the patient profiles, including precancerous lesions. In the assessment of
invasive breast cancer profiles, the concordance amounts to 58.8%. Nevertheless, as the LLM makes
considerably fraudulent decisions at times, we do not identify the current development status of
publicly available LLMs to be adequate as a support tool for tumor boards. Gynecological oncologists
should familiarize themselves with the capabilities of LLMs in order to understand and utilize their
potential while keeping in mind potential risks and limitations. |
|---|---|
| Item Description: | Gefördert durch den Open-Access-Publikationsfonds der UB Marburg. |
| DOI: | 10.3390/jpm13101502 |