In unprecedented provisional ruling, judge holds platform liable for allegedly false information generated by an artificial intelligence chatbot.
A court in São Paulo has ordered X, formerly Twitter, to remove posts generated by its artificial intelligence chatbot that contained information deemed false about a lawyer. The provisional ruling is believed to be the first court decision in Brazil to hold a platform provider liable for content produced by an AI system.
In the interlocutory relief, Judge Jorge Passerini of the 5th Civil Court of São Carlos, in São Paulo state, stated that there is a “rebuttable presumption of fault” on the part of X because the content was published by an “artificial dissemination mechanism” maintained by the company. According to the judge, the mechanism, known as Grok, presented “as facts, rather than mere opinion, dishonorable conduct attributed to an identifiable person, with references to alleged sources that would confirm those allegations, combined with internal inconsistencies in the content itself.”
The posts involved lawyer João de Senzi, who specializes in digital law and the defense of social media influencers. He is seeking R$20,000 in damages for emotional distress, although the merits of the case have not yet been decided. According to the lawsuit, X is now attempting to reach a settlement with the lawyer, allegedly to avoid the creation of unfavorable legal precedent. Sought for comment, the social media platform did not immediately reply to a request for comment.
According to the case file, a user asked Grok to discuss Senzi’s “court defeats.” The chatbot then claimed the lawyer had lost a case because of a “weak strategy, generating criticism of his competence.” It also cited a case in which he allegedly filed a petition containing 43 fabricated court precedents, supposedly invented by ChatGPT, a competitor of Grok.
The posts further stated that, as a result, the lawyer had been fined for bad-faith litigation and reported to the Brazilian Bar Association (OAB), which Senzi says never occurred.
Grok listed as sources articles from websites specializing in legal matters. However, Senzi argues in the lawsuit that none of those articles referred to him. The AI system also cited a case involving actor José de Abreu, which he says had no connection to the lawyer.
In the lawsuit, filed in late March, Senzi argues that Grok’s posts directly and severely harmed his professional reputation. He claims that X “makes available, promotes and economically benefits from the artificial intelligence tool without exercising adequate control over the content it generates.” By doing so, he argues, X “assumes the risk of the activity and must be held liable for the damages caused.”
Andressa Barros, a digital law specialist and CEO of Fragata e Antunes Advogados, said the novelty of the case lies in holding the provider responsible for content produced by one of its own tools.
“For the first time, a judge has understood that content generated by an AI system, even when disseminated through a platform that is not itself a content producer, removes the applicability of the third-party liability theory discussed by the Supreme Federal Court in the case involving Article 19 of the Brazilian Civil Rights Framework for the Internet,” she said.
Barros was referring to the precedent established by the Supreme Federal Court, under which social media platforms must remove false and offensive content from their platforms even before a court order is issued. The exception is for crimes against honor, which still require a judicial decision. “But here the judge is saying: ‘You are responsible for the content produced by the platform itself; this is not external content, it is internal,’” she said.
She added that the risk is that this reasoning could be extended to other platforms without sufficient caution. She cited the example of a lawyer who relies on court precedents fabricated by an AI system and is later fined by the judiciary. “Could I sue ChatGPT, for example, in those situations?” she asked.
Grok has already drawn the attention of Brazilian authorities because of the creation of sexualized images of children, adolescents and adults without consent. In February, the Federal Prosecution Service (MPF), the National Data Protection Authority (ANPD) and the National Consumer Secretariat (Senacon) ordered X to provide monthly reports detailing the measures it is taking to prevent the spread of sexualized images.
Brazil’s Congress is currently debating a bill that would regulate the development and provision of artificial intelligence in the country, known as the AI Framework. Among its main proposals are the risk classification of AI-based products and a ban on the development of AI applications that could significantly and irreparably compromise people’s physical and moral integrity.
The bill was approved by the Senate in December 2024 but is still under discussion in the Lower House. In March, its Communications Committee approved the text, which was then sent to the Science, Technology and Innovation Committee, where it is awaiting a report from the bill’s rapporteur, Congressman Rodrigo da Zaeli.
In the United States, California, Utah, Nevada and Illinois have enacted laws restricting the use of AI tools as psychological counselors. Those laws require AI systems to remind users that they are interacting with a machine, implement protocols against self-harm and refer users expressing suicidal thoughts to appropriate health services. The immediate trigger was the case of a 16-year-old teenager in California who died by suicide after months of what were described as “consultations” with ChatGPT.
Last month, OpenAI, the developer of ChatGPT, was subpoenaed in a lawsuit brought by several U.S. states led by New York. According to The Wall Street Journal, the case alleges practices that may have caused harm to users. The lawsuit seeks information about the company’s advertising policies, user-retention strategies, policies for minors and elderly users, and the management of customers’ health data.
Source: Valor International
