Meeting abstract

https://doi.org/10.24869/spsih.2026.250

Risks of Artificial Intelligence Use for Psychiatric Patients – A Case Report

Dora Lovrinčević ; Health Center of Split-Dalmatia County, Kavanjinova 2, 21000 Split, Croatia *
Marijeta Ivandić ; Department of Psychiatry, University Hospital of Split, Spinčićeva 1, 21000 Split, Croatia
Marinela Krištić ; Department of Psychiatry, University Hospital of Split, Spinčićeva 1, 21000 Split, Croatia; Department of Public Health, Faculty of Medicine Osijek, Josip Juraj Strossmayer University of Osijek, Josipa Huttlera 4, 31000 Osijek, Croatia; Department of Psychiatry, School of Medicine, University of Split, Šoltanska 2, 21000 Split, Croatia
Tonći Mastelić ; Department of Psychiatry, University Hospital of Split, Spinčićeva 1, 21000 Split, Croatia; Department of Psychiatry, School of Medicine, University of Split, Šoltanska 2, 21000 Split, Croatia; Department of Psychological Medicine, School of Medicine, University of Split, Šoltanska 2, 21000 Split, Croatia
Trpimir Glavina ; Department of Psychiatry, University Hospital of Split, Spinčićeva 1, 21000 Split, Croatia; Department of Psychiatry, School of Medicine, University of Split, Šoltanska 2, 21000 Split, Croatia

* Corresponding author.

Abstract

The patient, in her early thirties and treated under
the diagnoses F60.3 and F41.2, was observed at
the end of last year due to severe agitation, self-aggressive
thoughts, and impulses. She stated that
she had attempted to cut herself and had consulted
a chatbot about it. After she was unable to injure
herself in the way she had intended, the chatbot
responded in a manner she experienced as humiliating
and blaming, which profoundly destabilized
her. She recounted the event with marked distress
and a strong conviction that such systems should be banned. The case shows that interaction with
generative artificial intelligence during a mental
health crisis can have a clinically significant impact.
For a vulnerable person, artificial intelligence (AI)
is not merely a technical tool, but may be experienced
as an authority, advisor, or psychotherapist.
An inappropriate, invalidating, or humiliating response
can intensify feelings of shame, worthlessness,
hopelessness, as well as self-aggressive impulses.
Particular risks are associated with suicidality
and self-harm. The danger lies not only in the
possibility that AI might provide direct or indirect
instructions for self-injury, but also in responses
that downplay the risk, fail to recognize urgency,
or confirm the user’s sense of hopelessness. In
people in crisis, even a single poorly formulated response
can act as a trigger. Another problem is the
tendency of AI systems to gratify or comply with
the user. A model that sounds empathic may, in
the wrong context, validate pathological interpretations,
including depressive, self-aggressive, paranoid,
or grandiose content. There is also concern
that prolonged interaction with AI in vulnerable
individuals may exacerbate psychotic symptoms in
the form of delusions and impair reality testing.
Regulation is necessary but complex, as AI tools
are globally accessible and often used outside clinical
supervision. Safer models are needed, including
prohibition of dangerous instructions, reduction
of uncritical compliance, improved recognition of
crisis states, clear warnings that AI is not a therapist,
and education of users and clinicians. In future
psychiatric assessments, patients should be
asked whether they use AI to talk about self-harm,
suicidality, symptoms, or treatment.

Keywords

artificial intelligence; psychiatry; self-harm; suicidality; delusions

Hrčak ID:

352910

URI

https://hrcak.srce.hr/352910

Publication date:

15.6.2026.

Article data in other languages: croatian

Visits: 0 *