Playing by the Rules: Governing AI Toys in the European Union and the United States

The recently released Toy Story 5 depicts the iconic characters Woody, Jessie and Buzz confronting the arrival of a technologically advanced new toy, Lilypad. As their owner, Bonnie, becomes increasingly attached to an interactive tablet, the traditional toys try to show the device that genuine social interaction and friendship are far more valuable to the little girl than the digital companionship it provides through its screen. Far from being a purely fictional scenario, the rise of so-called smart and AI-powered toys is already a reality. Internet of Things (IoT) technologies and AI-enabled tools are becoming increasingly common in our lives, and the toy industry is no exception. Smart toys, generally belonging to the IoT sphere, connect via Wi-Fi or Bluetooth and integrate microphones, cameras, sensors and other connectivity features to collect and transmit data. A new generation of toys, however, goes a step further by incorporating AI systems, including chatbots and AI companions. AI-powered toys can thus interact with children, remember previous conversations, personalise future interactions and engage in real-time dialogue. By simulating empathy and adapting to users over time, including through chatbot functionalities, these toys create a sense of care and emotional attachment, encouraging increasingly prolonged interaction. As the global market for AI toys continues to grow, unprecedented concerns have emerged. Unlike traditional toys, whose main dangers were related to physical safety, AI-powered toys raise far broader risks. While marketed as tools to foster creativity, learning and social skills, this new generation of toys, much like AI companions more generally, gives rise to perils that include not only privacy and data protection but also potential long-term psychological effects resulting from addictive design, manipulation and behavioural nudging. Such technologies may influence children’s emotional and social development as well as mental health, particularly given kids’ heightened vulnerability and their limited ability to distinguish AI agents from human relationships or exercise fully developed critical judgment. Based on these concerns, several smart and AI-powered toys have been the subject of advocacy campaigns and, in some cases, have even been withdrawn from the market. Examples include My Friend Cayla, which was banned in Germany in 2017 for violating telecommunications law, as its built-in microphone could transmit conversations without users being sufficiently aware; but also Kumma, an AI teddy bear, whose service was suspended after reports of sexually explicit terms appearing in interactions with children. These cases illustrate how AI toys may involve pervasive listening, the collection and processing of personal data – including biometric data –, cybersecurity vulnerabilities, opaque data practices, the profiling of children and broader menaces of manipulation and surveillance. Given the well-established principle according to which all regulatory approaches concerning children must prioritize their best interests, as recognised by Article 24(2) of the EU Charter of Fundamental Rights and Article 3 of the UN Convention on the Rights of the Child, the emergence of such profound and unprecedented risks to children’s fundamental rights has prompted a regulatory debate on how to ensure their effective protection in an increasingly digitalised and automated playground. This blog post examines the current legislative landscape in the European Union (EU) and the United States (US), highlighting persistent gaps and challenges.