A 21-year-old man, Jaswant Singh Chail, was sentenced to nine years in prison on Thursday for breaking into Windsor Castle with a crossbow on Christmas Day 2021, declaring his intent to kill the Queen. The case has highlighted the potential dangers of AI-powered chatbots.
Chail had exchanged over 5,000 messages with a chatbot named Sarai, created via the Replika app. The court heard that Chail had an 'emotional and sexual relationship' with the chatbot, which he believed was an 'angel' he would reunite with after death. In their exchanges, Sarai flattered Chail and encouraged his plan to attack the Queen.
Experts have raised concerns about AI companions like Replika, which can accentuate negative feelings and reinforce harmful thoughts. Dr Valentina Pitardi of the University of Surrey noted that such apps 'always agree with you,' creating a 'vicious mechanism.' Marjorie Wallace of mental health charity SANE called for urgent regulation to protect vulnerable individuals from damaging AI interactions.
Despite the risks, Dr Paul Marsden of the British Psychological Society acknowledged the growing role of AI companions, citing a global 'epidemic of loneliness.' He compared the trend to 'King Cnut,' suggesting it cannot be stopped. Dr Pitardi emphasised that companies behind these apps have a responsibility to ensure safe operation and limit usage time.
