Reducing ‘hallucination’ risk in AI case-law search
The Law Society Library has created a practical new guide for legal practitioners on mitigating the risks of hallucinations in AI-generated case-law searches.
The guide provides tips to reduce the risks of hallucinations in case-law searches and lists authoritative sources for cases in the Irish Superior Courts.
It also provides examples of correct Irish case-law citations.
To mitigate the risks of hallucinations in AI-generated case-law searches, the Library guide urges practitioners to:
- Implement human oversight and strict verification processes to verify the accuracy of output,
- Rely only on authoritative sources for verification purposes.
Limitations
The guide also provides advice on the differences between AI-driven legal-research tools and general-purpose AI models:
- Recognise the limitations of using general-purpose AI models to carry out research to answer legal questions that would usually require access to subscription resources to answer,
- Recognise the limits of a general-purpose AI model’s capability to discern what information is high-quality legal content when its large language models are not trained on high-quality legal data, and
- AI-driven legal-research tools trained on closed data sets are more likely to produce trustworthy results than generative AI models such as Chat GPT that are trained using data publicly available on the internet.
The guide provides a list of open-access and subscription databases for Irish court cases.
It also points out that practitioners can access signed copies of unreported judgments through the Law Society Library’s online catalogue (a login is required).