Would You Trust a Robot Judge?

The Algorithm That Sent People to Prison

In 2013, a Wisconsin man named Eric Loomis was sentenced to six years in prison. The judge cited, among other evidence, a risk score generated by a software system called COMPAS. Loomis appealed, arguing he had a constitutional right to know how the score was calculated. The Wisconsin Supreme Court disagreed. The algorithm's methodology was a trade secret. He had no way to challenge a number that helped determine years of his life.

This was not science fiction. It was not a dystopian thought experiment. It was American jurisprudence in 2013, and variants of this story have played out in courtrooms across the country ever since. The question of whether we should trust machines with justice is no longer hypothetical — it is administrative procedure.

What an AI Judge Actually Is (and Isn't)

The term "AI judge" conjures a robot in robes, pronouncing sentences with metallic authority. Reality is both more mundane and, in some ways, more troubling. The tools in use today are risk-assessment instruments: statistical models that ingest data about a defendant — age, criminal history, employment status, answers to a questionnaire — and output a score predicting the likelihood of reoffending. Judges then use that score when deciding bail amounts, sentencing ranges, and parole eligibility.

COMPAS is the best-known example, but it is not alone. Arnold Foundation's Public Safety Assessment is used in dozens of U.S. jurisdictions. The Netherlands, the UK, and several Australian states have piloted similar tools. None of these systems make the final call — that remains, legally, a human judge's prerogative. But when a number sits in a case file and a judge is overloaded with hundreds of decisions a week, the distinction between "recommendation" and "decision" can blur in practice.

The Bias Problem: When Data Inherits History's Sins

In 2016, ProPublica published a landmark investigation into COMPAS. Journalists analysed risk scores for more than 7,000 people arrested in Broward County, Florida, and compared those scores against whether defendants actually reoffended over the following two years. The results were damning. Black defendants who did not reoffend were incorrectly flagged as high-risk at nearly twice the rate of white defendants. White defendants who went on to commit new crimes were more often labelled low-risk.

Equivant, the company behind COMPAS, disputed the methodology. Other researchers — notably Northpointe itself and a team at MIT — argued that the algorithm was actually calibrated fairly, meaning its score meant the same thing regardless of race. Both sides were, in a specific technical sense, correct. The problem is that fairness in machine learning has multiple incompatible mathematical definitions, and you cannot satisfy all of them simultaneously. This is not a bug that can be patched. It is a fundamental tension baked into what it means to be fair.

• –Black defendants were flagged as high-risk at roughly twice the rate of white defendants, according to ProPublica's analysis
• –The algorithm does not use race as an input — yet racial disparities emerged from correlated variables like neighbourhood and criminal history
• –Multiple peer-reviewed papers confirmed that no risk-scoring tool can simultaneously satisfy all formal definitions of statistical fairness
• –Defendants are rarely told their score or given meaningful opportunity to contest it
• –The same accuracy rate can mean very different things depending on which group bears the cost of the errors

Are Algorithms Actually More Accurate Than Human Judges?

One of the strongest arguments for algorithmic sentencing is that human judges are not particularly consistent. Research has shown that parole boards are more lenient right after lunch. Judges give harsher sentences on Mondays after their sports team lost. The same crime gets wildly different sentences depending on which courtroom you walk into. Proponents argue that a consistent algorithm, even an imperfect one, is preferable to capricious human intuition.

The empirical record is more complicated. A 2018 study in Science found that COMPAS was no more accurate than predictions made by random people recruited online with no legal training — a finding that received considerable attention. But subsequent meta-analyses have been more mixed. Some structured risk tools do outperform unaided clinical judgment in specific contexts. The honest answer is that accuracy depends heavily on what you are predicting, for whom, and over what time horizon. The category of "accurate" conceals enormous variation.

When the Algorithm Is Wrong, Who Pays?

Legal accountability for algorithmic errors is, at present, mostly undefined. If a human judge makes a demonstrably unjust ruling, there are mechanisms — appeals, professional sanctions, public accountability — however imperfect. If an algorithm contributes to an unjust ruling, the trail goes cold fast. The vendor claims the judge had final authority. The judge says the score was just one factor. The company says its model is proprietary. The defendant serves the sentence.

This diffusion of responsibility is not accidental — it is one of the structural features that makes algorithmic systems attractive to institutions. When no individual is clearly culpable, accountability becomes everyone's problem in theory and no one's problem in practice. Legal scholars have begun calling this "the accountability gap," and closing it requires either new law, new transparency requirements, or both.

Black Boxes in the Courtroom: The Right to Know Why

Due process, in most democratic legal systems, includes the right to confront evidence used against you. The Loomis case tested whether that principle extends to algorithmic scores, and the Wisconsin Supreme Court said it did not — at least not in a way that required the vendor to open its code. Critics see this as a profound rupture in legal tradition. For centuries, defendants could scrutinise witness testimony, challenge forensic evidence, and cross-examine experts. A proprietary risk score is, by design, immune to all of that.

The European Union has taken a different position. The GDPR includes a right to "meaningful information about the logic involved" when automated decisions have significant legal effects. The forthcoming AI Act goes further, classifying AI systems used in criminal justice as high-risk, requiring transparency, human oversight, and robust documentation. Whether these requirements will be enforced with meaningful teeth remains to be seen — but the regulatory philosophy represents a genuine departure from the American approach.

Can an Algorithm Understand Justice?

Beyond the practical problems of bias and accountability lies a deeper philosophical question: is justice the kind of thing a statistical model can compute? Philosophers in the consequentialist tradition might say yes — if we can measure outcomes accurately enough, we can optimise for them. But most legal traditions embed something else: mercy, context, the recognition that two cases with identical statistics can carry entirely different moral weight because of circumstances no dataset fully captures.

A judge who hears that a defendant stole food to feed their children can weigh that differently from a defendant who stole for profit, even if both score identically on a risk instrument. Some theorists argue this kind of contextual judgment is not a deviation from justice — it is what justice is. An algorithm that cannot be moved by a compelling story is not neutral; it is inflexible in a way that the concept of equity was specifically designed to correct.

The Same Debate, Smaller Stakes: Hiring, Credit, and Insurance

The courtroom is the most visible arena for this debate, but the same logic plays out daily in decisions with life-altering consequences that fall short of incarceration. Credit-scoring models determine who can buy a house. Hiring algorithms screen resumes before any human eye sees them. Insurance pricing tools adjust premiums based on behavioural proxies. In each case, the structural questions are identical: who audits the model, what recourse do you have when it's wrong, and whose historical patterns did it learn from?

The difference in criminal justice is the severity of the stakes and the coercive power of the state. When a loan algorithm rejects you, you can apply elsewhere. When a sentencing algorithm contributes to a five-year term, there is no elsewhere. This asymmetry is why courts have received the most scrutiny — and why the lessons learned there will shape how we govern algorithmic decision-making across every institution that uses it.

What SplitVote Users Actually Chose

SplitVote puts this question to a global audience without softening it: should a robot judge determine your sentence? The split in responses is sharp and illuminating. A significant plurality choose "no" — citing fairness, transparency, and the irreducible importance of human judgment. But a meaningful minority choose "yes," and their reasoning tends to focus on the failures of the human alternative: unconscious bias, inconsistency, overburdened courts where the quality of your lawyer matters more than the quality of your case.

What the data reveals is not a clean culture war. Younger users are not uniformly more trusting of AI. Legal professionals who encounter the dilemma often vote against algorithmic authority — but so do many people with prior experience of the criminal justice system, for reasons that have nothing to do with technological scepticism. The distribution suggests people are weighing two competing fears: bias in algorithms versus bias in humans. Neither option feels safe.

The Regulatory Moment: From Executive Orders to the EU AI Act

Governments are beginning to catch up, though unevenly. The Biden administration's 2023 Executive Order on AI included provisions on equity in criminal justice algorithms and called for auditing federal use of risk-assessment tools. The Trump administration subsequently rolled back parts of that order, reflecting deep political disagreement about whether regulation or market discipline should govern AI deployment. In Congress, the Algorithmic Accountability Act — requiring impact assessments for automated decision systems — has been introduced in multiple sessions without passing.

The EU AI Act, which entered into force in 2024, represents the most comprehensive binding framework to date. It designates AI systems used for risk assessment in criminal justice as high-risk, mandating human oversight, technical documentation, and the ability to contest decisions. Whether the Act's requirements will translate into meaningful change in courtrooms — or remain compliance theatre — depends on enforcement capacity that most regulators are still building.

The Question Isn't Whether AI Enters the Courtroom. It Already Has.

The frame of "should we allow AI in justice systems" is already outdated. COMPAS has been in use for over two decades. Hundreds of thousands of sentencing and parole decisions have been touched by algorithmic scoring. The real question — the one that carries moral weight going forward — is on whose terms AI continues to operate in these institutions, and who gets to set those terms.

That question is not primarily technical. It is political and philosophical. It asks what we think justice is for, who counts as a legitimate decision-maker, and what it means to hold power accountable when that power is distributed across code, corporations, and courts. The algorithm does not answer those questions. It inherits the answers we have already given — or failed to give — and scales them. Which is precisely why the conversation happening on platforms like SplitVote, wherever it is happening, is not a game. It is a rehearsal for governance decisions that are already being made.

This article discusses ongoing research and regulatory developments in algorithmic criminal justice. SplitVote dilemma response data is aggregate and anonymised. Individual legal situations vary by jurisdiction — this content is for informational purposes only and does not constitute legal advice.