A lawyer used ChatGPT to generate briefs that included completely made-up witnesses and testimony while representing a man accused of shooting his wife. When he was caught by the judges, the lawyer blamed it on his own “stupidity.”

“It's little comfort to know that my stupidity is what brings us together this afternoon,” New Mexico attorney Stephen D. Aarons said in a hearing last month before five judges. He “assumed” ChatGPT would generate a “bulletproof summary of proceedings.” Instead, ChatGPT generated false testimony from “wholly fabricated witnesses,” according to an order of direct contempt from the New Mexico Supreme Court. Aarons was representing a man who was found guilty of killing his wife earlier this year; Aarons said he hadn’t told his client directly that he was caught using ChatGPT on the case because he’s currently serving life in prison.

The false testimony included a nonexistent person named Danny Stanton saying he received threats, and another made-up person, Linda Stanton, saying her husband received threats. Other false testimony from more AI-generated people in Aarons’ brief included statements “regarding the shooter’s clothing and appearance,” according to the order.

Reuters reported on the case earlier this month. "I am remorseful but hopeful that the disciplinary board takes into account it was an honest mistake," Aarons told Reuters. "It is a lesson learned for all professionals who rely upon this powerful but sometimes unstable technology."

you are viewing a single comment's thread
view the rest of the comments
[–] 5 points 4 days ago

Aarons’ reasoning for trusting ChatGPT, he said in the August hearing, was that he’d heard of doctors using AI for medical research. 

“So, Counsel, do you watch the news? Do you listen to the radio? Do you read anything about what's going on in the world?” one of the judges asked Aarons. “Because the problem with lawyers relying on AI hallucinations is an above-the-fold story every single day.”

…

“So, either you buried your head in the sand. And that's a choice to do that. An intentional choice to be uninformed,” the judge continued to Aarons. “Or, you took a gamble. And neither of those are consistent with the code of conduct. So I'm really struggling with your response, saying, ‘Well, I didn't understand. I didn't know about hallucinations.’ My 13-year-old nephew knows about hallucinations. My 75-year-old stepmother knows about hallucinations. So, either you made a decision not to be aware of what's happening in the world in using this tool appropriately, which is a willful choice. Or you decided to roll the dice.”

“So you didn't say, ‘I relied on ChatGPT, and it submitted lies to the court, and the court is mad at me, and now there's a hearing.’ I'm guessing you didn't tell your client’s family members that,” a judge asked Aarons.

“I didn't talk about ChatGPT. I just said there was a problem with a brief that I filed,” Aarons said.

  • source