Laurence Dutton/Getty Images
A federal district judge issued a harsh rebuke and tossed out the testimony of a Stanford misinformation expert who submitted a court document, under penalty of perjury, containing misinformation in a Minnesota election law case.
Jeff Hancock, who specializes in “research on how people use deception with technology,” was retained by the office of Attorney General Keith Ellison to submit expert testimony defending Minnesota’s new law banning election deepfakes, which was signed in 2023 and updated the following year.
After Hancock filed written testimony last November, attorneys for plaintiffs Rep. Mary Franson, R-Alexandria, and YouTuber Christopher Kohls noticed that the document contained several citations to academic articles that do not exist.
The plaintiffs moved to have the testimony thrown out, and Hancock subsequently filed a document admitting he used a version of ChatGPT to draft the testimony, which included the non-existent citations, known among AI researchers as “AI hallucinations.” The Attorney General’s Office argued Hancock should be allowed to file an amended declaration containing correct, non-hallucinated citations.
But in a ruling dated Jan. 10, U.S. District Judge Laura Provinzino strongly disagreed.
Hancock’s citation of fake sources “shatters his credibility with this Court,” Provinzino wrote. While acknowledging that artificial intelligence software may have valid uses in legal settings, she concluded that “when attorneys and experts abdicate their independent judgment and critical thinking skills in favor of ready-made, AI-generated answers, the quality of our legal profession and the Court’s decisional process suffer.”
The judge makes repeated note of the fact that Hancock submitted his original document under penalty of perjury. “Signing a declaration under penalty of perjury is not a mere formality,” she wrote, but is rather an acknowledgement of the “gravity of the undertaking” and a mechanism for ensuring “truthtelling and reliability” as well as trust.
“That trust was broken here,” she added. “Given that the Hancock Declaration’s errors undermine its competence and credibility, the Court will exclude consideration of Professor Hancock’s expert testimony in deciding Plaintiffs’ preliminary-injunction motion.”
Provinzino also reminded the Attorney General’s Office they have a responsibility to “validate the truth and legal reasonableness of the papers filed,” and suggested that in the future they should ask witnesses whether or not they used AI to produce any of their material.
Hancock is billing the Attorney General’s office $600 an hour for his services, according to a copy of the contract obtained by the Reformer under a Data Practices Act request, with billing capped at $49,000.
The Attorney General’s Office did not provide information on how much had been paid out under that contract so far, or whether the office knew in advance that Hancock would be using AI to draft his declaration.
“Professor Hancock, a credentialed expert on the dangers of AI and misinformation, has fallen victim to the siren call of relying too heavily on AI — in a case that revolves around the dangers of AI, no less,” Provinzino wrote. “The irony.”