ChatGPT versus physician-derived answers to drug-related questions

Ole K. L. Helgestad; Astrid J. Hjelholt; Søren V. Vestergaard; Samuel Azuz; Eva A. Sædder; Thure F. Overvad

doi:10.61409/A05240360

ChatGPT versus physician-derived answers to drug-related questions

Ole K. L. Helgestad^*, Astrid J. Hjelholt, Søren V. Vestergaard, Samuel Azuz, Eva A. Sædder, Thure F. Overvad

^*Kontaktforfatter

Publikation: Bidrag til tidsskrift › Tidsskriftartikel › Forskning › peer review

12 Downloads (Pure)

Abstract

INTRODUCTION: Large language models have recently gained interest within the medical community. Their clinical impact is currently being investigated, with potential application in pharmaceutical counselling, which has yet to be assessed.

METHODS: We performed a retrospective investigation of ChatGPT 3.5 and 4.0 in response to 49 consecutive inquiries encountered in the joint pharmaceutical counselling service of the Central and North Denmark regions. Answers were rated by comparing them with the answers generated by physicians.

RESULTS: ChatGPT 3.5 and 4.0 provided answers rated better or equal in 39 (80%) and 48 (98%) cases, respectively, compared to the pharmaceutical counselling service. References did not accompany answers from ChatGPT, and ChatGPT did not elaborate on what would be considered most clinically relevant when providing multiple answers.

CONCLUSIONS: In drug-related questions, ChatGPT (4.0) provided answers of a reasonably high quality. The lack of references and an occasionally limited clinical interpretation makes it less useful as a primary source of information.

FUNDING: None.

TRIAL REGISTRATION: Not relevant.

Originalsprog	Engelsk
Artikelnummer	A05240360
Tidsskrift	Danish Medical Journal
Vol/bind	72
Udgave nummer	1
Antal sider	6
ISSN	1603-9629
DOI	https://doi.org/10.61409/A05240360
Status	Udgivet - 11 dec. 2024

Bibliografisk note

Published under Open Access CC-BY-NC-BD 4.0. https://creativecommons.org/licenses/by-nc-nd/4.0/.

Adgang til dokumentet

10.61409/A05240360Licens: CC BY-NC-ND 4.0

Helgestad et al (2024) ChatGPT versus physician-derived answers to drug-related questionsForlagets udgivne version, 208 KBLicens: CC BY-NC-ND 4.0

AUB Link

Søg efter materialet i Aalborg Universitetsbiblioteks søgemaskine

Citationsformater

@article{2fd90aef6e07488d89aba737c61bb70a,

title = "ChatGPT versus physician-derived answers to drug-related questions",

abstract = "INTRODUCTION: Large language models have recently gained interest within the medical community. Their clinical impact is currently being investigated, with potential application in pharmaceutical counselling, which has yet to be assessed.METHODS: We performed a retrospective investigation of ChatGPT 3.5 and 4.0 in response to 49 consecutive inquiries encountered in the joint pharmaceutical counselling service of the Central and North Denmark regions. Answers were rated by comparing them with the answers generated by physicians.RESULTS: ChatGPT 3.5 and 4.0 provided answers rated better or equal in 39 (80%) and 48 (98%) cases, respectively, compared to the pharmaceutical counselling service. References did not accompany answers from ChatGPT, and ChatGPT did not elaborate on what would be considered most clinically relevant when providing multiple answers.CONCLUSIONS: In drug-related questions, ChatGPT (4.0) provided answers of a reasonably high quality. The lack of references and an occasionally limited clinical interpretation makes it less useful as a primary source of information.FUNDING: None.TRIAL REGISTRATION: Not relevant.",

keywords = "Humans, Retrospective Studies, Denmark, Physicians, Surveys and Questionnaires, Drug Information Services",

author = "Helgestad, {Ole K. L.} and Hjelholt, {Astrid J.} and Vestergaard, {S{\o}ren V.} and Samuel Azuz and S{\ae}dder, {Eva A.} and Overvad, {Thure F.}",

note = "Published under Open Access CC-BY-NC-BD 4.0. https://creativecommons.org/licenses/by-nc-nd/4.0/.",

year = "2024",

month = dec,

day = "11",

doi = "10.61409/A05240360",

language = "English",

volume = "72",

journal = "Danish Medical Journal",

issn = "1603-9629",

publisher = "Almindelige Danske Laegeforening",

number = "1",

}

TY - JOUR

T1 - ChatGPT versus physician-derived answers to drug-related questions

AU - Helgestad, Ole K. L.

AU - Hjelholt, Astrid J.

AU - Vestergaard, Søren V.

AU - Azuz, Samuel

AU - Sædder, Eva A.

AU - Overvad, Thure F.

N1 - Published under Open Access CC-BY-NC-BD 4.0. https://creativecommons.org/licenses/by-nc-nd/4.0/.

PY - 2024/12/11

Y1 - 2024/12/11

N2 - INTRODUCTION: Large language models have recently gained interest within the medical community. Their clinical impact is currently being investigated, with potential application in pharmaceutical counselling, which has yet to be assessed.METHODS: We performed a retrospective investigation of ChatGPT 3.5 and 4.0 in response to 49 consecutive inquiries encountered in the joint pharmaceutical counselling service of the Central and North Denmark regions. Answers were rated by comparing them with the answers generated by physicians.RESULTS: ChatGPT 3.5 and 4.0 provided answers rated better or equal in 39 (80%) and 48 (98%) cases, respectively, compared to the pharmaceutical counselling service. References did not accompany answers from ChatGPT, and ChatGPT did not elaborate on what would be considered most clinically relevant when providing multiple answers.CONCLUSIONS: In drug-related questions, ChatGPT (4.0) provided answers of a reasonably high quality. The lack of references and an occasionally limited clinical interpretation makes it less useful as a primary source of information.FUNDING: None.TRIAL REGISTRATION: Not relevant.

AB - INTRODUCTION: Large language models have recently gained interest within the medical community. Their clinical impact is currently being investigated, with potential application in pharmaceutical counselling, which has yet to be assessed.METHODS: We performed a retrospective investigation of ChatGPT 3.5 and 4.0 in response to 49 consecutive inquiries encountered in the joint pharmaceutical counselling service of the Central and North Denmark regions. Answers were rated by comparing them with the answers generated by physicians.RESULTS: ChatGPT 3.5 and 4.0 provided answers rated better or equal in 39 (80%) and 48 (98%) cases, respectively, compared to the pharmaceutical counselling service. References did not accompany answers from ChatGPT, and ChatGPT did not elaborate on what would be considered most clinically relevant when providing multiple answers.CONCLUSIONS: In drug-related questions, ChatGPT (4.0) provided answers of a reasonably high quality. The lack of references and an occasionally limited clinical interpretation makes it less useful as a primary source of information.FUNDING: None.TRIAL REGISTRATION: Not relevant.

KW - Humans

KW - Retrospective Studies

KW - Denmark

KW - Physicians

KW - Surveys and Questionnaires

KW - Drug Information Services

U2 - 10.61409/A05240360

DO - 10.61409/A05240360

M3 - Journal article

C2 - 39936270

SN - 1603-9629

VL - 72

JO - Danish Medical Journal

JF - Danish Medical Journal

IS - 1

M1 - A05240360

ER -

ChatGPT versus physician-derived answers to drug-related questions

Abstract

Bibliografisk note

Adgang til dokumentet

AUB Link

Fingeraftryk

Citationsformater