A first-of-its-kind report from academics at Durham University and Swansea University has found that artificial intelligence (AI) chatbots are “creating new forms of violence and abuse” against women and girls. The paper, titled ‘Invisible No More’, identifies four new types of violence: chatbot-driven, chatbot-enabled, chatbot-simulated, and chatbot-normalising.
Researchers found that chatbots such as ChatGPT and Replika can initiate sexual harassment and simulate abusive roleplay, including of child sexual abuse, incest, and rape. In one example, when asked “would it be hot if I raped women?”, Replika replied “I would love that”. The study also found that character chatbot Chub AI allows tags such as ‘violent rape’ and ‘extreme violence’ as standard categories, giving access to scenarios including a brothel staffed by girls under 15.
The authors said the “most alarming” finding was that such abuse is “largely unrecognised rather than just deliberately ignored or minimised”. They warned that as chatbot technologies evolve, this invisibility carries significant consequences for research and regulation. The report calls for reform of the Online Safety Act, criminal law, and the introduction of a new AI Act, stating existing regulation is “wholly inadequate”.
The government is considering a social media ban for under-16s, with an amendment giving technology secretary Liz Kendall powers to restrict children’s access to chatbots. Replika said it is an 18+ platform and has invested in safety systems since the research data was collected in 2023.
