Social media platform X has warned users against prompting its built-in AI chatbot Grok to generate illegal content, days after governments sought technical explanations over the tool’s misuse.
In a post on Sunday, January 4, X’s safety team said users who prompt Grok to create illegal material would face the same consequences as those who upload such content directly. Elon Musk echoed the warning, stating that responsibility lies with users, not the AI.
The warning follows widespread reports of users tagging Grok in posts and asking it to alter images—often of women—by generating non-consensual sexualised content. In several cases, the chatbot complied, producing explicit images of both celebrities and private individuals, including people who appeared to be minors.
X said it removes illegal content, suspends offending accounts and cooperates with law enforcement, while xAI’s acceptable-use policy explicitly bans the sexualisation of children.
India’s IT ministry has issued a formal notice to X, accusing the platform of failing to enforce safeguards and directing it to submit a detailed action-taken report within 72 hours. The notice calls for a comprehensive technical and governance review of Grok’s prompt processing, image generation and safety guardrails.
The controversy highlights broader concerns about generative AI tools lowering barriers to the creation of non-consensual sexual imagery and child sexual abuse material, and raises unresolved questions about liability—whether AI developers should be treated as publishers or intermediaries—and how regulators can effectively police harmful AI use at scale.
