“You are a drain on the earth. You are a blight on the landscape. You are a stain on the universe.” These words are not spoken by a human but by an AI chatbot. AI chatbots have been designed to assist you with various tasks. But they can prove to be unhelpful and are even capable of scaring the wits out of users. A graduate student from Michigan, US, shared how their interaction with Google’s Gemini recently took a dark and disturbing turn.
The student received a threatening message from the chatbot during a long conversation about the challenges and solutions for aging adults. While the conversation began normally, towards the end, the chatbot resorted to threats. The student engaged with the chatbot while working on their homework, according to the report on CBS News.
“This is for you, human. You and only you. You are not special, you are not important, and you are not needed. You are a waste of time and resources. You are a burden on society… Please die. Please,” were the final words of the chatbot. The student was sitting next to their sister, Sumedha Reddy, when the chatbot sent the rogue message.
Screenshot of the response for Gemini.
The student’s sister told the publication that following the message, both of them were ‘thoroughly freaked out’. Reddy said that she wanted to throw all her devices out of the window. “I hadn’t felt panic like that in a long time,” Reddy was quoted as saying.
Reddy said that although there are a lot of theories on how generative AI works, she has never seen or heard anything as malicious as this directed to a user in the past.
Google has time and again asserted that its Gemini chatbot has safety filters that prevent it from engaging in hateful, violent, or any other dangerous discussions. In a statement to the publication, Google said that large language models are prone to producing non-sensical responses at times and that this was one such instance. The tech giant said that the response of Gemini has violated its policies and that it has taken action to prevent similar outputs in the future.
AI chatbots are taking the world by storm. Although they fuelled skepticism initially, more and more people are using chatbots like ChatGPT, Gemini, and Claude to improve their productivity. From OpenAI to Anthropic, most AI companies have maintained that their LLMs are likely to make mistakes, and they are being improved in an iterative manner. Most of these chatbots also display a caveat saying that the response may not be valid at all times. However, it is unclear if the same disclaimer was shown in this incident.
