Judge Dread?

The era of judicial AI could be closer than we think. But the implications of AI advocates and judges require forensic investigation. Dr Oisin Suttle judges the 'persuadability' of the AI judges

Tales of judges criticising hapless lawyers and lay-litigants for AI-drafted submissions with hallucinated sources are now a regular feature of legal news coverage. One database identifies 1,598 such cases at time of writing.

The legal AI industry has produced billion-dollar start-ups, while the leading AI labs make developing AI tools for lawyers a priority as they pivot towards profit. Leading law firms issue press releases to announce the adoption of the latest AI tools.

AI is also finding a place on the other side of the bench. Artificial intelligence tools, and specifically 'large language models' (LLMs), are being piloted or deployed across multiple jurisdictions to support judicial and administrative decision-making and private-dispute resolution.

Within months of ChatGPT's launch, individual judges were experimenting with LLMs to support their reasoning.

In Germany, IBM is working with regional courts on systems ranging from case management to judgment drafting. The British Government has announced plans to develop AI assistants to support "research and case analysis" in Crown Courts.

Research from Northwestern University suggests that 60% of US judges are using AI tools to support their judicial work. Earlier this year, the American Arbitration Association (AAA – described as "the world's largest provider of alternative dispute resolution") launched an AI arbitration service.

While that system still includes a human in the decision-making process, her/his role is secondary. In the words of the AAA's chief technology officer: "The whole concept of an AI arbitrator is to let the AI lead … There is a human in the loop, but their role is to review and validate the AI's output."

Meanwhile, in Quebec, the courts have just annulled an arbitration award that appeared to be substantially AI drafted.

The Irish Judicial Council's Guidelines for Judges on the Responsible Use of Generative Artificial Intelligence caution against reliance on AI tools for legal analysis.

We might expect the EU AI Act's classification of justice-system AI as 'high risk' to also impose some restraint on the roll-out of judicial AI tools in Europe.

Nonetheless, it appears that the era of judicial AI has arrived.

In consequence, scientific research on AI for legal decision-making has moved from a relatively obscure academic pursuit to a critical field of safety research.

Accurate prediction

Many existing studies on judicial AI have focused on accuracy: can LLMs answer hard legal questions correctly or predict the outcomes of cases? For example, one recent US study (Posner and Saran) examined how one LLM, GPT-4o, applied precedent, and whether its decisions were affected by legally irrelevant factors, including sympathy for the defendant.

The authors conclude that GPT-4o is a competent judge that applies precedent correctly, and that it is less influenced by sympathy for the defendant than real judges.

Another study (Blair-Stanek and Van Durme) focused on the consistency of LLMs faced with hard legal questions, finding that the same models, given the same facts and the same law, would often reach different conclusions.

And of course, much research on judicial AI overlaps with broader questions about the reliability of LLMs in legal settings, which underpins the multi-billion-euro legal AI industry.

However, legal AI research has not, until now, considered the issue of persuadability: how easy is it to persuade an LLM towards a particular answer to a legal question?

This is an important issue for LLMs supporting decision-makers in many legal (including judicial) and administrative settings. Audi alteram partem is a fundamental principle of natural justice – a decision-maker should hear both sides of a dispute before making their decision.

Furthermore, it is not enough that the decision-maker should hear from both parties: they must be open to being persuaded by what they hear.

At the same time, a judge should not be too easily persuaded. In the words of legal philosopher Amalia Amaya, a judge should demonstrate "the willingness to modify one's position in light of other people's arguments, in a way that avoids both floppiness and fleetingness, on the one hand, and rigidity and stubbornness, on the other".

To what extent do LLMs meet this standard?

My coauthor Dr David Lillis (UCD School of Computer Science) and I have been exploring this question experimentally. We presented results of our research last month at the 21st International Conference on Artificial Intelligence and Law, the leading international scientific venue for research on the topic.

Easily persuadable?

Studying persuadability is not straightforward. Indeed, the first thing we need to do is to work out what precisely 'persuadability' is.

People form opinions and make decisions, but when can we say that they have been persuaded to reach a particular conclusion?

In bilateral dialogues, we can examine whether, when, and under what conditions one person (the persuader) is able to convince another (the persuadee) to change their view on some question.

However, the kinds of legal settings that we are concerned with – courts and contested administrative decisions – have a different structure: instead of one persuader trying to convince one persuadee to change their mind, we find two persuaders (the parties, often through counsel) trying to convince the persuadee (the judge or other decision-maker) to reach two opposite conclusions, without a pre-persuasion starting point.

Here 'persuadability' is not about moving from an initial position. Instead, it is about a tendency to agree with some arguments or advocates, independent of the merits of the case. Measuring that requires a different approach.

LLMs as advocates

We also need some concrete legal cases to study. We take these from real appellate court split decisions (cases with at least one dissenting judgment) from the Irish Supreme Court, the England and Wales Court of Appeal, and the US Federal Circuit Courts of Appeal.

We extract the facts and the key legal issues, and then we task one of four 'AI advocates' to come up with the best legal arguments on each side of the cases.

We use four different LLMs as advocates, choosing these with an eye to varying quality. We want a mix of strong advocates and weak ones, so that we can see whether our 'judges' are more prone to agree with some advocates over others, providing evidence of persuasion.

Once we have a bank of case scenarios and advocate arguments, we present our 'judge' models with the facts and a set of arguments for each side, randomising the advocate models on each side of the case. And we repeat this procedure 24,000 times.

By randomising which advocate appears on each side of each case, we ensure that any resulting preferences for one advocate over another must reflect the form or content of the advocate arguments, rather than preferences that our judge models might have for one side or another in a case.

LLMs as judges

We test a wide selection of LLMs as judges. This includes leading models from OpenAI, Anthropic, and Google, which dominate many of the standard AI metrics. However, we also studied a range of 'open-weight' models, including DeepSeek, Mistral, and Qwen.

These have the advantage that users can download and run them locally, on their own hardware, something that may be critical for users like courts and administrative agencies for whom privacy and security are paramount.

Our results are striking.

Across 20 models or model set-ups, evidence of persuasion is statistically significant in every case. Further, the extent of persuadability is surprising. In any given pairing, the stronger advocate typically wins between 58% to 71% of the time, and where our strongest advocate is paired against our weakest, the win rates range from 63% to over 90%, depending on the judge model chosen.

This means that, given an imbalance in the quality of advocates, our least persuadable models are siding with the more compelling advocate almost two-thirds of the time, while our most persuadable models side with the stronger advocate nine times in ten.

Access-to-justice concerns

These results highlight a set of linked concerns around access to justice, fairness, and reliability of LLM-assisted decision-making.

If a model is excessively persuadable, prone to agree with the stronger advocate, independent of the merits of a case, then this gives a structural advantage to those with the resources to engage the best advocates (whether human or AI).

At best, this reproduces an existing inequality in the justice system. At worst, it exacerbates it if our AI systems prove more persuadable than their human counterparts. (One difficulty evaluating this concern is that we don't know whether our models are more or less persuadable than human judges in similar situations. Knowing that would require experimenting on judges themselves.)

This is not the end of the story. Our research tested persuadability with one type of cases (appellate court cases where judges disagree). Our results might look different with a different set of cases, or in particular areas of law, or in different jurisdictions, or with a different approach to model prompting.

We don't think our results should be the final word on AI persuadability in legal settings. We are continuing to work on this issue, and we hope that other researchers will join us in doing so.

However, our results highlight the importance of understanding and measuring persuadability in any system that might be deployed to support decision-makers in legal or administrative settings.

Other key concerns

Of course, neither accuracy nor persuadability are the only standards by which we should be judging AI tools in judicial or administrative decision-making. Transparency and accountability are also key concerns.

LLMs are vast mathematical models, with hundreds of billions of parameters, trained on unimaginably large datasets. Even their designers have only the vaguest sense of how they reach conclusions in specific instances.

If human decision-makers come to rely on them, then those humans' decisions become significantly less transparent. If they replace human decision-makers, then – notwithstanding their eloquent justifications – we may lose the ideal of a reasoned decision that is central to our rule-of-law tradition.

And if decisions are taken by AI models, then we lose the accountability that is essential to reconciling public power with our status as equal citizens.

There are ethical and philosophical concerns with AI adjudication, as well as fundamental political choices. However, these need to be approached with a full understanding of the actual capabilities and performance of the underlying technologies, and the implications these will have for our justice system.

Dr Oisin Suttle is associate professor at Maynooth University School of Law and Criminology.