Testing the Ability of ChatGPT to Categorise Urgent and Non-Urgent Patient Conditions: Who ya gonna call?

Authors

  • Dorian Fildor University of Zagreb, Faculty of Economics and Business, Croatia
  • Mirjana Pejić Bach University of Zagreb, Faculty of Economics and Business, Croatia https://orcid.org/0000-0003-3899-6707

DOI:

https://doi.org/10.54820/entrenova-2023-0010

Keywords:

artificial intelligence, healthcare, triage, patient conditions, urgency categorization, digitaliziation, automation, medical decision-making, Chatgpt, gpt-based language models, healthcare optimization, triage optimization

Abstract

This research explores the feasibility of utilising ChatGPT to categorise patient conditions as urgent and non-urgent. The primary objective is to assess the ChatGPT model's capacity to aid in the automation and digitalisation of healthcare processes, thereby alleviating the workload on healthcare professionals. The study employed a unique approach by presenting patient cases to the GPT and categorising the conditions based on urgency. In collaboration with an experienced hospital doctor, a set of questions was prepared and presented to a medical expert, along with the GPT model. Subsequently, the medical expert was consulted to assign urgency modalities for the same cases. The generated categorisations and the expert-assigned modalities were compared to evaluate the model's accuracy. The outcomes of this research have significant implications for healthcare management. Implementing AI to support triage processes and decisions could streamline patient care, ensuring appropriate and timely treatment allocation. By delegating specific tasks to AI, healthcare employees could focus on providing direct medical attention, leading to enhanced efficiency and improved patient outcomes. However, the results indicate that there is still uncertainty in using ChatGPT to provide medical advice. Ultimately, this study contributes to the broader exploration of AI's potential in healthcare decision-making, promoting the integration of advanced technologies to optimise medical services and enhance patient experiences.

Author Biographies

Dorian Fildor, University of Zagreb, Faculty of Economics and Business, Croatia

Dorian Fildor is a master's degree student who works as a project manager for AI implementation and operations manager in a company that organises scientific conferences worldwide. Dorian is deeply passionate about the integration of Artificial Intelligence in various industries. Dorian continues pursuing his academic and professional endeavours, aiming to create a positive impact in AI and beyond. The author can be contacted at dfildor@net.efzg.hr

Mirjana Pejić Bach, University of Zagreb, Faculty of Economics and Business, Croatia

Mirjana Pejić Bach is a full professor at the Department of Informatics, Faculty of Economics in Zagreb. She holds a PhD in system dynamics modelling from the Faculty of Economics, University of Zagreb. Mirjana is the leader and collaborator of numerous projects in which she cooperates with Croatian companies and international organisations, mainly through European Union projects and the bilateral research framework. Her research areas are the strategic application of information technology in business, data science, simulation modelling, research methodology, qualitative and quantitative, especially multivariate statistics and modelling structural equations. The author can be contacted at mpejic@net.efzg.hr

 

References

Ahuja A. S. (2019). The impact of artificial intelligence in medicine on the future role of the physician. PeerJ, 7, e7702. https://doi.org/10.7717/peerj.7702

Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023). Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustworthy Artificial Intelligence. Information Fusion, 99, 101805. https://doi.org/10.1016/j.inffus.2023.101805.

Alzubaidi, L., Bai, J., Al-Sabaawi, A. et al. A survey on deep learning tools dealing with data scarcity: definitions, challenges, solutions, tips, and applications. J Big Data 10, 46 (2023). https://doi.org/10.1186/s40537-023-00727-2

Asan, O., Bayrak, A. E., & Choudhury, A. (2020). Monitoring Editor: Gunther Eysenbach and Qing Zeng. Reviewed by Erin Chiou, Daniel Walker, and Karen Fortuna. J Med Internet Res, 22(6), e15154. https://doi.org/10.2196/15154

Bartlett, L., & Vavrus, F. (2017). Comparative Case Studies: An Innovative Approach. Nordic Journal of Comparative and International Education (NJCIE), 1(1). https://doi.org/10.7577/njcie.1929

Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., ... & Liang, P. (2021). On the opportunities and risks of foundation models. arXiv preprint https://arxiv.org/abs/2108.07258

Coiera, E., Ash, J., & Berg, M. (2016). The Unintended Consequences of Health Information Technology Revisited. Yearbook of medical informatics, (1), 163–169. https://doi.org/10.15265/IY-2016-014

Davenport, T., & Kalakota, R. (2019). The potential for artificial intelligence in healthcare. Future healthcare journal, 6(2), 94–98. https://doi.org/10.7861/futurehosp.6-2-94

Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., Albanna, H., Albashrawi, M. A., Al-Busaidi, A. S., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., Brooks, L., Buhalis, D., Carter, L., ... Wright, R. (2023). Opinion Paper: "So what if ChatGPT wrote it?" Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice, and policy. International Journal of Information Management, 71, 102642. https://doi.org/10.1016/j.ijinfomgt.2023.102642.

Filip, R., Puscaselu, R. G., Anchidin-Norocel, L., Dimian, M., & Savage, W. K. (2022). Global Challenges to Public Health Care Systems during the COVID-19 Pandemic: A Review of Pandemic Measures and Problems. J Pers Med, 12(8), 1295. https://doi.org/10.3390/jpm12081295

Frosolini, A., Gennaro, P., Cascino, F., & Gabriele, G. (2023). In Reference to "Role of Chat GPT in Public Health", to Highlight the AI's Incorrect Reference Generation. Annals of Biomedical Engineering. Advance online publication. https://doi.org/10.1007/s10439-023-03248-4.

Guo, B., Zhang, X., Wang, Z., Jiang, M., Nie, J., Ding, Y., Yue, J., & Wu, Y. (2023, January 18). How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection. arXiv:2301.07597 [cs.CL]. https://doi.org/10.48550/arXiv.2301.07597

Harris E. Large Language Models Answer Medical Questions Accurately, but Can't Match Clinicians' Knowledge. JAMA. Published online August 07, 2023. https://doi.org/10.1001/jama.2023.14311

Hendy, A., Abdelrehim, M., Sharaf, A., Raunak, V., Gabr, M., Matsushita, H., Kim, Y. J., Afify, M., & Awadalla, H. H. (2023). How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation. arXiv:2302.09210 [cs.CL]. https://doi.org/10.48550/arXiv.2302.09210).

Imamguluyeva, R. (2023). The Rise of GPT-3: Implications for Natural Language Processing and Beyond. International Journal of Research Publication and Reviews, 4(3), 4893-4903. https://doi.org/10.55248/gengpi.2023.4.33987.

Javaid, M., Haleem, A., & Singh, R. P. (2023). ChatGPT for healthcare services: An emerging stage for an innovative perspective. BenchCouncil Transactions on Benchmarks, Standards and Evaluations, 3(1), 100105. ISSN 2772-4859. https://doi.org/10.1016/j.tbench.2023.100105.

Jeyaraman, M., Balaji, S., Jeyaraman, N., et al. (August 10, 2023). Unraveling the Ethical Enigma: Artificial Intelligence in Healthcare. Cureus, 15(8), e43262. https://doi.org/10.7759/cureus.43262

Johnson, D., Goodman, R., Patrinely, J., Stone, C., Zimmerman, E., Donald, R., Chang, S., Berkowitz, S., Finn, A., Jahangir, E., Scoville, E., Reese, T., Friedman, D., Bastarache, J., van der Heijden, Y., Wright, J., Carter, N., Alexander, M., Choe, J., Chastain, C., … Wheless, L. (2023). Assessing the Accuracy and Reliability of AI-Generated Medical Responses: An Evaluation of the Chat-GPT Model. Research square, rs.3.rs-2566942. https://doi.org/10.21203/rs.3.rs-2566942/v1

Khan, M., Mehran, M. T., Haq, Z. U., Ullah, Z., Naqvi, S. R., Ihsan, M., & Abbass, H. (2021). Applications of artificial intelligence in COVID-19 pandemic: A comprehensive review. Expert systems with applications, 185, 115695. https://doi.org/10.1016/j.eswa.2021.115695

Kossen, J., Rainforth, T., & Gal, Y. (2023). In-Context Learning in Large Language Models Learns Label Relationships but Is Not Conventional Learning. arXiv:2307.12375 [cs.CL]. https://doi.org/10.48550/arXiv.2307.12375).

Lomis, K., Jeffries, P., Palatta, A., Sage, M., Sheikh, J., Sheperis, C., & Whelan, A. (2021). Artificial Intelligence for Health Professions Educators. NAM perspectives, 2021, 10.31478/202109a. https://doi.org/10.31478/202109a

Popović, M., Poncelas, A., Brkic, M., & Way, A. (2020). Neural Machine Translation for translating into Croatian and Serbian. In Proceedings of the 7th Workshop on NLP for Similar Languages, Varieties and Dialects (pp. 102–113). Barcelona, Spain (Online): International Committee on Computational Linguistics (ICCL). https://aclanthology.org/2020.vardial-1.10/.

Portoghese, I., Galletta, M., Coppola, R. C., Finco, G., & Campagna, M. (2014). Burnout and Workload Among Health Care Workers: The Moderating Role of Job Control. Saf Health Work, 5(3), 152–157. https://doi.org/10.1016/j.shaw.2014.05.004

Siala, H., & Wang, Y. (2022). SHIFTing artificial intelligence to be responsible in healthcare: A systematic review. Social Science & Medicine, 296, https://doi.org/10.1016/j.socscimed.2022.114782

Thirunavukarasu, A.J., Ting, D.S.J., Elangovan, K. et al. Large language models in medicine. Nat Med 29, 1930–1940 (2023). https://doi.org/10.1038/s41591-023-02448-8

Törnberg, P. (2023, April 13). ChatGPT-4 Outperforms Experts and Crowd Workers in Annotating Political Twitter Messages with Zero-Shot Learning. arXiv:2304.06588 [cs.CL]. https://doi.org/10.48550/arXiv.2304.06588

Van, H. (2023). Mitigating Data Scarcity for Large Language Models. (2023). arXiv:2302.01806 [cs.CL]. https://doi.org/10.48550/arXiv.2302.01806

Yang, W., Wei, Y., Wei, H. et al. Survey on Explainable AI: From Approaches, Limitations and Applications Aspects. Hum-Cent Intell Syst (2023). https://doi.org/10.1007/s44230-023-00038-y

Downloads

Published

2024-08-26

How to Cite

Fildor, D., & Pejić Bach, M. (2024). Testing the Ability of ChatGPT to Categorise Urgent and Non-Urgent Patient Conditions: Who ya gonna call?. ENTRENOVA - ENTerprise REsearch InNOVAtion, 9(1), 101–112. https://doi.org/10.54820/entrenova-2023-0010

Issue

Vol. 9 No. 1 (2023)

Section

Microeconomics

License

Copyright (c) 2024 Mirjana Pejić Bach, Dorian Fildor

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.