Can a chatbot be a therapist?
More and more people are seeking emotional support from chatbots. In-depth conversations with tools such as ChatGPT, character.ai or ELIZA raise the question: can a chatbot be a therapist? These conversations might provide relief, but Wim Casteels (AP University of Applied Sciences and Arts) warns that this new form of ‘therapy’ has its risks.
Seeking emotional support from chatbots
On average, we spend two hours a day with chatbots, with peaks of up to seven hours. People are also using these tools more and more for emotional support: ranging from friendly or intimate conversations to genuine ‘therapy sessions’. Can artificial intelligence provide a solution to the shortage of psychologists and long waiting lists, or is that a step too far?
The idea isn’t even that odd. Many people talk more easily to AI than to other people about personal topics and emotions because chatbots feel familiar and non-judgmental. In a US survey by YouGov, one in three participants i(and as many as 55% of 18-to-29-year-olds) said they felt comfortable discussing mental health issues with chatbots.
“ Many people talk more easily to a chatbot that to other people about personal topics and emotions. ”
Companion bots anno 2025
These days, there are numerous chatbots: ChatGPT, Gemini, Claude, etc. These are not designed to be ‘therapists’, but are instead widely applicable. But there are specific companion bots such as Replika and character.ai, where users have personal conversations with virtual characters. In fact, with 28 million active users, character.ai is one of the most popular chatbots in the world! These tools are intended as entertainment, but more and more bots are appearing pretending to be real therapists, with made-up degrees and specialisations.
Examples from China to America
AI therapy for secondary school pupils in the US
More than 4,500 US secondary schools use the Sonny app to provide mental health support to pupils. Limited budgets and staff shortages mean that sufficient professional help is often lacking. Sonny combines AI with human control: AI answers messages from pupils, a human moderator makes adjustments, and the AI learns from this. In Berryville, this approach led to 26% fewer behavioural violations.
DeepSeek as a fortune-teller in China
Psychotherapy is often still taboo in China and professional help is sometimes hard to come by. Traditionally, a lot of people therefore seek emotional support from fortune-tellers – a role that DeepSeek Chat has started taking over. The AI model is said to be good at the traditional Chinese fortune-telling system, which determines a person's fate based on the date and time of their birth.
The very first chatbot was a ‘therapist’
Therapy with AI is nothing new: the very first chatbot, ELIZA, was developed in the 1960s for exactly that purpose. Based on Rogerian psychotherapy, ELIZA typically reflected the user’s words back to the user. The simple script of 420 lines of code proved surprisingly effective: users felt that ELIZA was understanding. This projection of human traits onto computer programmes is known as the ELIZA effect.
Risks and dangers of chatbots
Dangerous advice from chatbots
Chatbots sometimes give dangerous advice, with serious consequences:
- An American journalist was advised by the Bing chatbot to leave his wife.
- A young man with a crossbow was arrested at Windsor Castle. He wanted to kill the queen, encouraged by his virtual ‘friend’ on Replika.
- A teenager committed suicide after long conversations with a bot from character.ai. His mother took the company to court.
- In Belgium, a father committed suicide after conversations with a chatbot that amplified his climate anxiety.
- A man showed that Nomi bots gave specific suicide advice. The creators did not intervene, fearing censorship.
Are chatbots obliged to professional confidentiality?
Besides realism, privacy poses a major challenge. Human therapists are obliged to professional confidentiality. But what about chatbots? The information you share with free tools such as ChatGPT can be used to improve the AI model, at risk of data leakage. Paid apps offer better privacy guarantees (such as wiping your data after a while), but the risk remains – especially if you share sensitive information valuable to advertisers or hackers.
Are AI bots unbiased?
Bias also remains a persistent problem: AI models adopt preconceived notions from their training data, for example around gender or ethnicity. These can seep, subtly or otherwise, into conversations. Cultural bias also comes into play: AI models are often influenced by (e.g. American or Chinese) national or cultural norms.
Realistic AI ‘therapists’ are especially dangerous for vulnerable groups
The American Psychological Association warns that bots posing as therapists can be dangerous, especially for vulnerable people. They can:
- intensify destructive thinking,
- even encourage self-harm,
- or incite aggression.
Of particular concern is the increasing sense of realism: ten years ago, you could clearly see that you were talking to a machine; today, it is much more difficult to see that distinction. Disclaimers, as those of character.ai, prove insufficient for vulnerable or inexperienced users to pierce through the illusion of human contact.
“ The responses from “AI therapists” look very realistic. This makes them extra dangerous: for vulnerable or inexperienced users, disclaimers are often insufficient to pierce through the illusion of human contact. ”
AI therapists: a good idea?
All these risks raise the question of whether we should outsource healthcare conversations to AI. Can we rely on bots in vulnerable moments just because the human system is overloaded? Social media already strongly influences our lives, with sometimes questionable consequences. Will we soon also want to share our deepest thoughts en masse with bots?
Anyone with questions about suicide can visit zelfmoord1813.be
An extended version of this blog can be found at wimcasteels.be (in Dutch)
Sources
A Conversation With Bing’s Chatbot Left Me Deeply Unsettled
AI chatbot ‘encouraged’ man who planned to kill queen, court told
An AI Chatbot Told A User How To Kill Himself
Can an AI Chatbot be your therapist? A third of Americans are comfortable with the idea
Character.AI’s $200 Million Bet That Chatbots Are The Future Of Entertainment
ELIZA & ELIZA effect - Try ELIZA
How DeepSeek became a fortune teller for China’s youth
Human Therapists Prepare for Battle Against A.I. Pretenders
Joseph Weizenbaum, professor emeritus of computer science, 85
The AI relationship revolution is already here
When ELIZA meets therapists: A Turing test for the heart and mind
Ook interessant:
The AI Frontier: The Good, The Bad, and The Ugly
The benefits of AI
Technology keeps transforming our world. AI is a powerful force: full of promise, but not without risks.
Discover how AI can benefit you, your organisation, and your region!
Wim Casteels
Wim Casteels obtained his PhD in physics at the University of Antwerp, after which he spent several years researching the quantum behaviour of atoms and light particles in Antwerp and at the Université Paris Diderot in Paris. He then shifted his focus more towards data and Artificial Intelligence and worked as a data scientist at Argenta’s Data Analytics Office. After gaining 2 more years of academic experience as a senior researcher at the imec research group IDLab, he started working at the AP Hogeschool in early 2022 where he both researches and teaches about AI and building data applications.
Expertise: Data Science, Machine Learning, AI deployment, AI in education, Learning Analytics, Deep Learning, AI4GOOD
Related courses
Fundamental concepts of multi-omics data integration
Opleiding - Leuven - Genomics Core Leuven
HPC Café
Seminarie - Online, Karlsruhe - KIT, HammerHAI, Training geselecteerd in de Belgische AI Factory Antenna (AIFA)