Original scientific paper
https://doi.org/10.3325/cmj.2025.66.399
ChatGPT and Gemini in warfarin counseling
Muhammet Hüseyin Erkan
; Department of Cardiovascular Surgery, Balıkesir University, Balıkesir, Turkey
*
Ömer Faruk Rahman
; Department of Cardiovascular Surgery, Faculty of Medicine, Izmir Bakircay University, Izmir, Turkey
Abdullah Güner
; Department of Cardiovascular Surgery, Konya City Hospital, Konya, Turkey
Fevzi Ayyıldız
; Department of Cardiovascular Surgery, Afyon State Hospital, Turkish Ministry of Health, Afyonkarahisar, Turkey
Emin Barbarus
; Department of Cardiovascular Surgery, Balıkesir University, Balıkesir, Turkey
* Corresponding author.
Abstract
Aim To compare the accuracy, scientific adequacy, and
clarity of responses provided by ChatGPT-4o and Gemini
to frequently asked patients’ questions about warfarin use.
Methods Forty patients’ questions were posed to ChatGPT
and Gemini using the zero-shot method. Four cardiovascu
lar surgeons evaluated the responses for adequacy, scien
tific accuracy, and clarity on a 5-point Likert scale. The first
and second set of data evaluations were separated by sev
en days to minimize any memory effect. The experts were
blinded to the type of LLM that created the response.
Results ChatGPT responses were significantly shorter
(187.3 ± 47.6 vs 291.4 ± 98.1 words; P < 0.001) but scientifi
cally more adequate (4.38 ± 0.30 vs 4.17 ± 0.35; P = 0.004).
Gemini scored higher in terms of clarity (4.69 ± 0.24 vs
4.48 ± 0.33; P < 0.001). The two models did not significantly
differ in terms of accuracy (P = 0.606).
Conclusion Both LLMs provide reliable and understand
able information for warfarin counseling. While ChatGPT
provides dense and scientifically detailed responses, Gem
ini excels in clarity and user-friendly communication. How
ever, expert supervision and appropriate clinical guidance
are critical for safe and comprehensive guidance.
Keywords
Hrčak ID:
352559
URI
Publication date:
21.12.2025.
Visits: 0 *