According to Forbes, a newly identified AI flaw called “semantic leakage” is undermining the quality of mental health advice from systems like ChatGPT, Claude, and Gemini. The phenomenon occurs when a word or phrase from earlier in a conversation improperly influences the AI’s responses later on, even when it’s no longer relevant. This is highlighted in a May 15, 2025 research paper titled “Does Liking Yellow Imply Driving a School Bus? Semantic Leakage in Language Models.” The issue is particularly dangerous given that consulting AI on mental health is a top-ranked use case, with ChatGPT alone boasting over 800 million weekly active users, many of whom seek such guidance. This comes amidst ongoing lawsuits, like one filed in August 2024 against OpenAI, alleging a lack of safeguards for AI-provided cognitive advice.
Why this isn’t just a bug, it’s a feature
Here’s the thing: semantic leakage isn’t a hallucination where the AI makes stuff up. It’s not even the AI misreading the immediate context. It’s more insidious. Basically, the model’s architecture can’t fully let go of a semantic association once it’s activated. The “yellow” to “school bus driver” example is cute and harmless. But the mental health example Forbes walks through is chilling. A user mentions keeping their apartment cold early in a chat. Much later, when confessing they didn’t pay attention to a friend’s sad story, the AI diagnoses “emotional coldness.” That’s a giant, unsubstantiated leap from thermostat settings to personality disorder. And the user would likely never connect the dots that the first comment poisoned the second.
The real-world stakes are massive
For casual chat, who cares? But Forbes is right to sound the alarm for mental health. We’re talking about millions of people using these systems as 24/7, low-cost therapists. The AI isn’t just retrieving information; it’s performing analysis and making interpretive judgments. If its judgment is being secretly skewed by a random word from twenty prompts ago, that’s a profound failure of integrity. It makes the entire interaction untrustworthy. How can you be vulnerable about your feelings if an offhand comment about your car could later lead the AI to suggest you’re “driving” yourself to depression? The linkages are unpredictable and invisible.
Where do we go from here?
So what’s the fix? This isn’t a simple patch. It’s a core architectural challenge with how LLMs maintain context. The research shows it happens across languages—a word in English can leak into a Spanish response. That suggests the problem is buried deep in the embedding layers. AI makers will likely say, “We’re working on it,” and point to their specialized therapy bots in development. But the cat’s out of the bag. The dominant, generic LLMs that everyone is actually using right now have this flaw. I think this gives more ammunition to the argument that these models simply aren’t fit for purpose in high-stakes advisory roles. They’re brilliant pattern-matching machines, not careful, consistent analysts. Relying on them for mental health seems increasingly like a dice roll, where semantic leakage loads the dice against you.
