Generative artificial intelligence tools have quickly become part of everyday legal practice. Lawyers are using these tools to draft briefs, summarize cases, and streamline research. But as recent cases make clear, those efficiencies come with a serious risk: sometimes, AI simply makes things up.

Courts across the country are now confronting filings with fabricated case law, inaccurate quotations, and non-existent citations. In one widely reported instance, a law firm submitted a brief with “pervasive inaccurate, misleading, and fabricated…representations of legal authority.” In another, a lawyer was sanctioned after submitting a filing containing hallucinated material and failing to verify its accuracy. And in yet another case, a lawyer included multiple made-up cases in a filing and later admitted he had not confirmed whether the authorities actually existed. The consequences can be severe.  In a recent civil lawsuit, a federal judge sanctioned all four lawyers involved after determining that filings on both sides contained AI-generated hallucinated citations, canceled their trial, imposed monetary sanctions, and barred two attorneys from appearing before the U.S. District Court for the Northern District of Mississippi for two years.

These are not isolated incidents. They reflect a broader problem: AI-generated content can appear plausible, even persuasive, while being entirely wrong. And when that content is filed in court, the consequences are real.

Courts Are Losing Patience

Judges are increasingly treating AI hallucinations not as a novelty or a one-off mistake, but a recurring problem that “shows no sign of abating.” The message from courts is straightforward: lawyers are responsible for what they file, regardless of how it was generated. Whether a brief is drafted by a junior associate or an AI tool, the obligation remains the same: lawyers must ensure that the law cited is real and accurately represented.

This is not a new principle. Under Rule 3.3 of the Model Rules of Professional Conduct, lawyers have always owed a duty of candor to the court, which includes the obligation not to present “a false statement of fact or law to a tribunal.”  The frequency of these errors, and the accompanying judicial response has prompted courts to directly address the use of AI and its implications.  New York courts are now part of that broader regulatory response.

New York Courts Respond

In response, courts are beginning to formalize rules governing the use of AI in litigation. In the First Judicial Department (New York County Supreme Court, Civil Branch), a growing number of judges have adopted rules addressing AI use in court submissions.

While these rules vary in approach, they reflect a consistent goal of preventing the submission of inaccurate or fabricated legal authority.

Broadly, the rules fall into three categories.

• Strict or Prohibitive Approaches. Some judges have taken a hardline stance. For example, one court has prohibited the use of AI during virtual conferences and hearings altogether, including for note-taking, and warned that submitting false citations will result in sanctions.
• Moderate Approaches: Disclosure and Control.Other judges permit the use of AI but impose clear conditions. These rules typically require attorneys to disclose whether AI was used, identify the specific tool, and specify which portions of a submission were generated by AI. They also often prohibit the use of AI during court appearances, whether for recording or photographing, and explicitly warn that violations may result in sanctions.
• Baseline Approaches: Certification and Verification.  The most common approach requires a certification in connection with motion practice. Under these rules, attorneys must affirm either that no AI was used or that any AI-generated content was reviewed for accuracy and approved.

Many of these rules go further, requiring attorneys to identify the AI program used and specify which portions of the submission were generated by it. Related rules emphasize that attorneys must independently verify that their filings contain no fabricated or fictitious cases, statutes, or other material. These rules reinforce a basic principle – one that predates AI: lawyers are responsible for ensuring the accuracy of their work.

Statewide Response

The trend toward regulating AI in litigation has now expanded beyond individual judges.  Effective June 1, 2026, the New York State Unified Court System adopted Part 161, a statewide model rule governing the use of AI in court filings. While the rule does not prohibit AI use or require its disclosure, it reinforces attorneys’ existing obligations to review and verify AI-assisted work and ensuring that filings do not contain fabricated or fictitious authority.

What This Means for Practitioners

For practitioners, the takeaway is straightforward: AI can assist in drafting and research, but it cannot replace verification.

As a best practice, attorneys should (1) review the assigned judge’s individual rules to determine practices concerning use of AI; (2) independently verify every cited authority and quotation to ensure accuracy; and (3) consider maintaining a record of that verification if AI was used in drafting or research.

Lawyers remain responsible for every citation, quotation, and legal argument they submit to the court because when a filing contains hallucinated law, “Citation Not Found” is not a technical error – it’s a professional one.