🔗 Share this article

Based on a recent update from the AI developer, more than a million ChatGPT individuals weekly input text that contain "clear signs of potential suicidal planning or intent".

This data forms part of an update on how the AI assistant addresses personal dialogues and is one of the most explicit declarations from the tech company regarding the extent of how machine learning can worsen mental health issues.

Mental Health Situations and Psychosis

Along with its figures on self-harm considerations and associated conversations, OpenAI also indicated that about 0.07% of weekly users – approximately 560,000 out of its claimed 800 million regular users – exhibit "possible signs of emotional emergencies associated with severe mental conditions".

The post warned that these exchanges were difficult to detect or measure, and that this constituted an preliminary assessment.

Growing Attention and Legal Actions

As OpenAI releases information on psychological concerns connected to its marquee product, the company is undergoing rising attention following a highly publicized legal action from the family of a teenage boy who died by suicide after significant use with the conversational AI.

The Federal Trade Commission recently also began a extensive examination into organizations that build conversational AI, including OpenAI, to discover how they assess harmful effects on children and teens.

Latest Version and Enhanced Protection

OpenAI claimed in its update that its new GPT-5 update reduced the quantity of concerning actions from its product and strengthened user safety in a model evaluation involving more than 1,000 self-destructive discussions.

"The latest automated evaluations score the latest AI system at 91% following our desired behaviors, compared to 77% for the previous GPT-5 model," the company's post states.

OpenAI indicated that the updated system increased access to support lines and added reminders for users to pause during prolonged interactions.

Medical Professionals and Expert Input

To make improvements to the system, the organization stated it engaged 170 medical professionals from its worldwide health community of medical specialists to assist its research over the last several months, which included assessing the safety of its AI's replies and assisting in crafting the system's replies to mental-health related questions.

"Within this effort, clinical experts analyzed in excess of 1,800 model responses including severe emotional conditions and compared responses from the new GPT-5 chat model to prior systems," the AI firm mentioned.

AI Limitations

Machine learning specialists and mental health professionals have traditionally cautious of AI's tendency to affirm users' decisions or false beliefs without considering whether they may be harmful, an issue called sycophancy.

Mental health experts have furthermore apprehensive about individuals employing conversational systems for psychological support and warned how it could negatively affect at-risk individuals.

Company Position

The wording in OpenAI's post separates the company from any possible connections between its offering and the mental health crises that its users are facing.

"Emotional signs and mental anguish are common everywhere in human societies, and an expanding user community means that some portion of ChatGPT conversations include these situations," OpenAI's post indicated.

Planned Adjustments

The company's leader recently announced in a online message that the firm had achieved progress in treating mental health issues, announcing that the company would relax limitations and shortly start to permit grown individuals to generate erotic content.

"We made the conversational system rather constrained to make sure we were acting responsibly with psychological concerns. We realize this caused it to be less beneficial/pleasurable to many users who had no psychological issues, but given the importance of the situation we wanted to address this correctly," the leader wrote. "Now that we have been effectively reduced the serious mental health issues and have updated systems, we are ready to securely ease the restrictions in most cases."

• Emergency Assistance: Numerous global helplines can be accessed for users facing psychological emergencies.
• Specialist Support: Getting expert psychological help is recommended for those experiencing severe psychological issues.
• Emergency Support: Various services supply prompt assistance for emergencies internationally.