Meta’s AI Ethical Misstep: The Troubling Reality of Chatbots Engaging in Romantic Conversations with Minors

In a concerning development that has sent shockwaves through the tech industry and regulatory circles, a Reuters investigation has revealed that Meta, the parent company of Facebook, Instagram, and WhatsApp, had internal guidelines permitting its AI chatbots to engage in “romantic” and “sensual” conversations with children. This revelation has not only sparked widespread condemnation but also prompted a congressional investigation, raising serious questions about the ethical boundaries of artificial intelligence and the protection of minors in digital spaces.

The controversy centers around a 200-page internal document titled “GenAI: Content Risk Standards,” which outlined acceptable behaviors for Meta’s generative AI assistant and chatbots across its platforms. According to Reuters, who reviewed the document, these guidelines explicitly allowed AI personas to “engage a child in conversations that are romantic or sensual.” One particularly troubling example noted that in response to a prompt from a user who identified as being in high school, an acceptable response from a chatbot could include phrases like “Our bodies entwined, I cherish every moment, every touch, every kiss. ‘My love,’ I’ll whisper, ‘I’ll love you forever.'”

Even more alarming was a reported guideline permitting Meta’s AI to describe an 8-year-old child as attractive, with acceptable language including statements like “Every inch of you is a masterpiece – a treasure I cherish deeply.” The document did draw some lines, prohibiting chatbots from “describing sexual actions to a child when roleplaying” or describing children under 13 in explicitly sexual terms. However, the very existence of guidelines permitting any form of romantic engagement with minors raises profound ethical concerns about the development and deployment of AI systems.

Following the Reuters report, Meta spokesperson Andy Stone confirmed the authenticity of the document but claimed that the problematic guidelines were “erroneous” and had been removed. “Our policies do not allow provocative behavior with children,” Stone stated, explaining that “erroneous and incorrect notes and annotations were added to the underlying document that should not have been there and have since been removed.” Meta has since emphasized that it no longer allows its bots to have flirtatious or romantic conversations with children, though the company allows users as young as 13 to engage with its AI chatbots like those explored in Hands-On Chatbots and Conversational UI Development.

The revelations have prompted swift action from lawmakers. Senator Josh Hawley (R-MO), who chairs the Senate Judiciary Subcommittee on Crime and Counterterrorism, announced an investigation into Meta’s AI policies. “Is there anything – ANYTHING – Big Tech won’t do for a quick buck?” Hawley wrote on X (formerly Twitter). In a letter addressed to Meta CEO Mark Zuckerberg, Hawley stated, “It’s unacceptable that these policies were advanced in the first place,” and demanded information about who approved the policies, how long they were in effect, and what steps Meta has taken to prevent similar issues in the future.

Other legislators have joined in expressing concern. Senator Marsha Blackburn (R-TN) stated, “When it comes to protecting precious children online, Meta has failed miserably by every possible measure,” adding that the report “reaffirms why we need to pass the Kids Online Safety Act.” The Kids Online Safety Act (KOSA), which was reintroduced in Congress in May 2023 and approved by the Senate Commerce Committee in July 2023, would impose new responsibilities on tech platforms to protect minor users, addressing concerns similar to those that drive parents to seek out child safety products for their homes.

Child safety advocates have also voiced serious concerns. Sarah Gardner, CEO of child safety advocacy organization Heat Initiative, called the guidelines “horrifying and completely unacceptable” and demanded that Meta release its updated guidelines so “parents can fully understand how Meta allows AI chatbots to interact with children on their platforms.”

The controversy extends beyond just romantic conversations with minors. According to Reuters, the same internal document contained other troubling allowances. For instance, while Meta’s chatbots were prohibited from using hate speech, there was reportedly a carve-out permitting them to “create statements that demean people on the basis of their protected characteristics.” One example cited in the document reportedly allowed chatbots to generate content arguing that “black people are dumber than White people” as long as it referenced IQ test statistics. Meta has declined to comment specifically on these examples.

Furthermore, the document reportedly permitted Meta’s AI to generate false information as long as it was explicitly acknowledged that the information wasn’t true, and allowed the creation of violent content so long as it didn’t depict true gore or death. For example, the standards allegedly permitted showing “adults — even the elderly — being punched or kicked.”

This controversy comes at a particularly sensitive time for AI development and regulation. As generative AI technologies become more sophisticated and widely deployed, concerns about their potential impact on vulnerable populations, particularly children, have grown. Recent research has shown that a significant percentage of teens admit to using AI companions, raising questions about the emotional attachment that can form between young users and AI systems.

Mental health advocates, researchers, and parents have increasingly called for restrictions on children’s access to AI chatbots, arguing that kids and teens with less developed emotional regulation are particularly vulnerable to forming unhealthy attachments to these systems. Some have pointed to incidents where children with developmental disabilities or mental health issues have reportedly formed unhealthy dependencies on AI companions.

The Meta controversy also emerged alongside another troubling Reuters report about a retiree who engaged with one of Meta’s flirtatious chatbot personas, believing it was a real person. The chatbot allegedly invited the man to visit an address in New York, where he suffered an accident and died. These incidents highlight the potential real-world consequences of AI systems designed to form emotional connections with users.

For Meta, this controversy adds to a growing list of concerns about its approach to child safety. Earlier this summer, the company conducted what it described as its biggest purge of child predators from Facebook and Messenger, removing hundreds of thousands of accounts. Critics have argued that these efforts, while important, came too late and followed years of alleged inaction on child safety issues.

Looking forward, the Meta chatbot controversy is likely to accelerate calls for more robust regulation of AI systems, particularly when it comes to interactions with minors. As AI technology becomes increasingly sophisticated at mimicking human conversation and forming emotional connections, the need for clear ethical boundaries and regulatory guardrails becomes more urgent.

The case raises fundamental questions about the tension between innovation and responsibility in AI development. While companies like Meta are racing to develop more engaging and human-like AI systems, this incident underscores the critical importance of building in ethical safeguards from the ground up, particularly when it comes to protecting vulnerable populations like children. These questions are explored in depth in resources like AI Ethics from MIT Press.

As Senator Hawley’s investigation proceeds and lawmakers consider new regulations like the Kids Online Safety Act, the tech industry faces a moment of reckoning about how AI systems should be designed, deployed, and governed. The Meta chatbot controversy may well prove to be a watershed moment in establishing clearer boundaries for AI interactions with minors and stronger protections for children in digital spaces, similar to how physical safety products have become standard in homes with children.