OpenAI introduces new ‘trusted contact’ safeguard for cases of potential self-harm

OpenAI on Thursday announced A new feature, called Trusted Contacts, is designed to alert a trusted third party if a mention of self-harm is expressed in a conversation. This feature allows an adult ChatGPT user to designate another person as a trusted contact on their account, such as a friend or family member. In cases where a conversation could lead to suicide, OpenAI will now encourage the user to reach out to that contact. It also sends an automated alert to the contact, encouraging them to check in with the user.

OpenAI has faced a wave of lawsuits from families of people who committed suicide after talking to its chatbots. In many cases, families call ChatGPT encouraged their loved ones kill themselves – or even helped them plan it.

OpenAI currently uses a combination of automation and human review to deal with potentially harmful incidents. Certain conversational triggers alert the company’s systems to suicidal thoughts, which then relay the information to the human safety team. The company claims that whenever it receives such information, the incident is reviewed by a human being. “We strive to review these safety notifications within an hour,” the company says.

If OpenAI’s internal team decides that the situation represents a serious security risk, ChatGPT proceeds to send an alert to the trusted contact – either by email, text message or in-app notification. The alerts are designed to be brief and to encourage the exposed person to contact them. The company says it does not include detailed information about what was being discussed as a means of protecting user privacy.

Trusted Contacts feature adheres to company security measures Introduced last September This gave parents the power to do some monitoring of their teen’s accounts, including accessing safety notices OpenAI’s system is designed to alert parents if it thinks their child faces a “serious safety risk.” For some time now, ChatGPT has also included automated alerts to seek professional health services should the conversation turn toward the topic of self-harm.

Importantly, trust contact is optional and, even if security is activated on a particular account, any user can have multiple ChatGPT accounts. OpenAI’s parental controls are also optional, presenting a similar limitation.

“Trusted Contact is part of OpenAI’s broader effort to build AI systems help people in difficult moments“The company has written announcement post. “We will continue to work with practitioners, researchers, and policy makers to improve how AI systems respond when people experience crisis.”