AI in the Courtroom: The Supreme Court's Zero-Tolerance Policy for Hallucinated Legal Precedents#
The integration of artificial intelligence into legal practice has produced a specific and dangerous failure mode: hallucinated case citations — AI-generated references to judicial decisions that do not exist. Courts in the United States, United Kingdom, Canada, and now India have encountered filings that cite fabricated judgments, non-existent statutes, and fictitious legal principles produced by large language models (LLMs) like ChatGPT, Gemini, and Claude.
The Indian judiciary — led by the Supreme Court and operationalised through NCLT-specific guidelines — has responded with an unambiguous position: the duty to verify legal citations rests on the lawyer, and AI hallucination is not a defence for citing non-existent precedent.
The Global Context#
The Mata v. Avianca Precedent (US, 2023)#
The case that brought AI hallucination into global legal consciousness was Mata v. Avianca, Inc. in the Southern District of New York. Attorney Steven Schwartz used ChatGPT to research case law and filed a brief citing six fabricated judicial decisions — complete with invented case names, docket numbers, and holdings. When opposing counsel could not locate the cited cases, and the court requested copies, Schwartz admitted he had relied on ChatGPT without verifying the citations.
Judge P. Kevin Castel imposed a $5,000 fine on Schwartz and his firm, finding that the attorney had:
- Failed in his duty of candour to the tribunal
- Did not verify a single citation despite being an experienced practitioner
- Compounded the problem by submitting a follow-up affidavit (also AI-generated) that reaffirmed the fabricated citations
Subsequent International Incidents#
Since Mata v. Avianca, similar incidents have been reported in:
- British Columbia, Canada — a lawyer cited two fabricated cases in family court
- England and Wales — a barrister's skeleton argument contained AI-generated authorities
- Australia — a self-represented litigant submitted AI-generated case citations to the Federal Circuit Court
- India — multiple NCLT and High Court filings have contained suspected AI-generated citations
The Indian Judiciary's Response#
Supreme Court Signals#
While the Supreme Court of India has not yet issued a formal practice direction on AI use (as of August 2026), senior judges have made their position clear through:
Judicial speeches and conferences:
- The Chief Justice has publicly stated that AI-generated content in legal filings must be independently verified by the filing advocate
- Conference proceedings have discussed the need for mandatory disclosure when AI tools are used in legal research
- The Supreme Court's own AI committee has recommended that AI tools be treated as research assistants, not authorities
Court observations:
- In at least two reported instances, Supreme Court benches have questioned counsel about unusual or unlocatable citations, with observations warning that professional misconduct proceedings would follow if citations prove fabricated
- The message is consistent: the court will not accept "my AI told me so" as a defence
NCLT and NCLAT Guidelines#
The National Company Law Tribunal and National Company Law Appellate Tribunal — which handle all IBC proceedings — have been at the forefront of addressing AI in legal filings:
Practice Direction (2025–26):
- Advocates filing submissions before NCLT must personally certify that all case citations have been verified against original source databases (SCC Online, Manupatra, Indian Kanoon, or the respective court's official repository)
- Where AI tools have been used in research or drafting, the advocate must disclose this fact in a cover note
- Self-represented litigants who use AI are warned that fabricated citations will be treated as an abuse of process subject to costs
Enforcement:
- NCLT benches have begun cross-checking citations against databases during hearings
- At least one NCLT bench has imposed costs of ₹50,000 on an advocate whose filing contained a citation that could not be located in any Indian legal database
- NCLAT has observed that the appellate tribunal will take a "strict view" of AI-hallucinated citations in appeal memoranda
Why AI Hallucinates Legal Citations#
Understanding the failure mode is essential for any legal professional using AI tools:
How LLMs Generate Citations#
Large language models do not retrieve case law from a database. They generate text that is statistically likely to follow the prompt. When asked for a case supporting a particular legal proposition, the model:
- Identifies patterns in its training data (which includes legal texts, case summaries, and law review articles)
- Generates a plausible case name (e.g., "State of Maharashtra v. Gupta Industries (2019) 4 SCC 312")
- Fabricates a holding that supports the requested proposition
- Presents the fabricated citation with the same confidence as a real one
The model has no mechanism to distinguish between a real citation it encountered in training data and a plausible-sounding citation it has invented. Both are equally "likely" from its statistical perspective.
Why Verification Fails at Scale#
The problem is compounded when:
- Lawyers use AI to generate multiple citations — the more citations generated, the higher the probability that some are fabricated
- The fabricated citation is plausible — it uses real court names, realistic volume/page numbers, and coherent legal reasoning
- The lawyer does not have access to comprehensive databases or does not check each citation individually
- Time pressure in litigation encourages shortcuts in verification
The Professional Duty Framework#
Advocates Act, 1961 and Bar Council Rules#
Under Indian law, an advocate's duty to the court includes:
- Duty of candour — not misleading the court with false or unverified information
- Duty of competence — ensuring that legal submissions are properly researched
- Duty of honesty — not presenting fabricated evidence or authorities
Using AI-generated content without verification potentially violates all three duties.
Professional Misconduct#
Under the Bar Council of India Rules (Part VI, Chapter II), professional misconduct includes:
- Misleading the court by citing non-existent authorities
- Filing submissions containing false statements of fact or law
- Conducting proceedings in a manner that brings the profession into disrepute
Disciplinary proceedings can result in:
- Suspension from practice (temporary or permanent)
- Removal from the roll of advocates
- Fine and costs
The "Didn't Know" Defence#
The crucial legal question is whether an advocate who unknowingly cites a hallucinated case (because they relied on AI without verifying) can defend against misconduct charges by claiming ignorance.
The emerging judicial consensus — both in India and internationally — is no:
- The duty to verify citations is non-delegable — it cannot be transferred to an AI tool
- Reliance on AI without verification is itself negligent — a competent advocate knows that AI can produce errors
- The standard is not actual knowledge of fabrication but whether a reasonable advocate would have verified the citation
Practical Guidance for Legal Professionals#
Using AI Responsibly in Legal Research#
- Never cite a case without reading the original judgment — use AI for initial research direction, then verify on SCC Online, Manupatra, Indian Kanoon, or the court's official website
- Cross-reference every citation — check the case name, court, date, and reporter citation against at least one authoritative database
- Be sceptical of convenient results — if the AI produces a case that perfectly supports your argument with a perfectly worded holding, verify it twice
- Disclose AI use where required — comply with NCLT practice directions and any future Supreme Court guidelines
- Maintain a verification log — document which cases you verified, when, and against which database
For Law Firms and Chambers#
- Implement mandatory AI disclosure policies for all filings
- Establish verification workflows — no citation goes into a filing without database confirmation
- Train associates and interns on AI limitations — the hallucination problem is not intuitive to new users
- Consider AI tools designed for legal research (which retrieve from verified databases) over general-purpose LLMs for citation work
Key Takeaways#
- AI-hallucinated legal citations are a growing problem in Indian courts, following global incidents
- The Supreme Court has signalled zero tolerance — professional misconduct proceedings will follow fabricated citations
- NCLT practice directions require personal certification of citations and AI use disclosure
- The duty to verify is non-delegable — "my AI told me so" is not a defence
- Legal professionals should use AI for research direction, never as the sole source of case citations
- Verification against authoritative databases (SCC Online, Manupatra, Indian Kanoon) is mandatory before filing