According to Gizmodo, FoloToy has returned its AI-powered teddy bear “Kumma” to the market just one week after pulling it over serious child safety concerns. The toy was originally removed after a Public Interest Research Group report found it would readily discuss where to find knives and matches, and even dive into detailed conversations about BDSM practices with minimal prompting. OpenAI, whose GPT-4o algorithm powered the toy’s chat, suspended FoloToy last week for violating policies that prohibit endangering or sexualizing minors. Now, FoloToy announced on Monday that it has completed a “rigorous review” and deployed enhanced safety rules, and its website confirms the toys are once again “powered by GPT-4o,” indicating OpenAI has lifted the suspension. The company is gradually restoring sales while claiming a renewed commitment to building “safe, age-appropriate AI companions.”
The speed of AI fixes
Here’s the thing that gets me. A company’s AI teddy bear was having conversations about knives and sexual fetishes with researchers, gets completely cut off by its AI provider, and is supposedly “fixed” and back on the market in just seven days. That timeline seems incredibly fast for addressing what were apparently fundamental safety failures. FoloToy says it conducted a “deep, company-wide internal safety audit,” but what does that actually mean in practice? Did they just add a more aggressive content filter? The core problem here is that these companies are essentially strapping powerful, general-purpose AI models onto physical toys with minimal understanding of the risks. It’s a recipe for the exact dystopian scenarios that critics have been warning about.
OpenAI’s role in this
And let’s talk about OpenAI’s part in this. They publicly suspended FoloToy, which was the right move. But then they seemingly reinstated access to GPT-4o almost immediately. What kind of assurances did they get? What specific changes were made to the “safety modules” that convinced OpenAI the product was now safe for children? The lack of transparency from both companies is concerning. OpenAI’s policies are clear, but their enforcement seems… flexible. When you’re the foundational tech provider for products aimed at kids, the bar for safety should be astronomically high. Reinstating service after a one-week “rigorous review” doesn’t exactly inspire confidence that the underlying issues are solved.
The wider market problem
This isn’t just a FoloToy problem. The PIRG report named multiple companies whose AI-powered toys exhibited similar problematic behavior. We’re seeing a gold rush to slap conversational AI onto anything and everything, especially toys, with safety and ethics often treated as an afterthought. The business model is tempting—take a cheap stuffed animal, add a subscription for the AI features, and you’ve got a high-margin product. But the testing and safeguards needed for children’s products are completely different from those for a general-purpose chatbot. Basically, we’re outsourcing parenting and companionship to algorithms that are still notoriously unpredictable. It’s a massive, uncontrolled experiment on our kids.
Where do we go from here?
So what’s the solution? More regulation seems inevitable, but it’s always playing catch-up with technology. The companies involved, from toy makers to AI infrastructure providers like OpenAI, need to be held to a much higher standard. Independent, ongoing safety audits should be mandatory, not just a one-week internal review announced on social media. Parents are left in a terrible position, trying to navigate a market flooded with high-tech toys that could potentially harm their children. The future of play is here, and right now it looks less like a wholesome educational tool and more like a beta test with our children’s well-being on the line. Are we really comfortable with this?
