Justice in the Algorithmic Society: A Decade of Kleros and Artificial Intelligence

From a vision sketched before the words even existed to a protocol built for the agentic economy…

By Federico Ast, William George, JB, Rob Dean and Fortunato Cinquepalmi

Long before "AI agents" became a term anyone used, Kleros was already looking at an economy where more and more transactions would be happening between machines rather than people. The rise of the algorithmic society would eventually need machine-speed dispute resolution: fair, cheap, fast, and with humans in the loop when it matters.

A decade later, this is the story of how that thesis shaped Kleros… and how our own experiments reshaped the thesis along the way.

From a Vision to a Live Court

In 2016, Federico Ast sat down for an interview with Max Keiser in the Keiser Report to describe a new form of justice mixing blockchain and crowdsourcing. A form of justice capable of resolving disputes no traditional court could ever reach. That interview predates most of the vocabulary we use today. There was no "metaverse" and no "AI agents". The vocabulary didn't exist yet. But the vision did.

In 2018, Kleros went live on Ethereum. Its first large-scale experiment, Doges on Trial, was designed to stress-test the protocol against every attack the team could imagine (bribery included). The data from that experiment was the source for our first piece of research on AI and decentralized justice, published in 2019, three years before ChatGPT kick off the generative AI revolution.

That article was not about blockchain. It placed Kleros in a lineage running back to Hugh Lawford's QUIC/LAW database in 1960s Ontario, and, more recently, in a February 2018 study where an AI outperformed twenty US lawyers at spotting problem clauses in NDAs: 94% accuracy against their 85%, in 26 seconds against their 92 minutes.

We proceeded by laying out a thesis that has largely held up since (even as the details have been refined along the way): not every dispute needs the same kind of judge.

When a case is objective and simple, AI can usually resolve it outright. When it’s subjective and complex, it belongs in traditional dispute resolution. And in the vast middle ground (too nuanced for a machine, too small for a courtroom or a formal arbitration) that’s what we defined as Kleros territory.

AI for Dispute Preparation

In the years that followed, Kleros moved from theorizing about AI’s role to methodically testing it across every stage of the dispute lifecycle. In particular, we found that a primary role of AI is facilitating case onboarding into the system.

AI as Mediator

The first stage is prevention. Kleros’s approach here builds on the Mediation Bridge, a methodology developed by Robert Dean and Federico Ast that breaks a dispute down into its component issues, helps each side articulate the interests underneath their claims, and frames each issue as a binary choice between two concrete outcomes, before any of it goes to a jury. The original framework envisioned a human mediator running this process by hand.

Harmony is the AI implementation of the Mediation Bridge: Kleros’s AI mediator talks with both sides of a conflict, helps them identify the root of the disagreement, and guides them through a mediation process with or without a human mediator being present. If they reach an impasse it helps guide them toward the same kind of binary framing. So if mediation fails, a Kleros court is the backstop. The outcome of such dispute can be fed back into the mediation process for the parties to consider.

Harmony was covered in detail by LexisNexis as an example of AI-assisted dispute resolution in practice.

AI as Lawyer

As Kleros real world adoption picked up, we started seeing how AI could help resolve some practical problems of implementation. A prime example was our partnership with Lemon, a leading Latin American fintech app with millions of users. Lemon integrated Kleros to act as an independent, decentralized “consumer ombudsman”, allowing users to escalate unresolved consumer disputes to Kleros jurors.

When the pilot first launched in June 2024, an imbalance quickly became apparent. User claims were typically short and poorly structured (written in the heat of the moment by someone with no legal background) while the company’s responses were polished and thorough, produced by Lemon’s own legal team. That asymmetry threatened the legitimacy of the whole process: a fair-sounding verdict is harder to trust when only one side had the resources to argue its case well.

The fix started as a simple prompt to ChatGPT: improve the clarity, structure, and persuasiveness of the user’s submission. In practice, this put ChatGPT in the role of an “AI lawyer” for the side that didn’t have one, helping ordinary users present stronger cases and leveling the playing field without changing who actually decides. A Lemon survey found the approach worked: even in cases where Kleros ruled against the user, retention stayed around 90%, which suggests the process was seen as fair, not just fast.

AI for Arbitration

The holy grail of dispute resolution is using AI to overhaul the entire process and deliver fast, affordable, and accurate arbitration. It's an old dream. The 17th-century philosopher and mathematician Gottfried Leibniz spent much of his life chasing a version of it: a formal calculus precise enough that any dispute could be settled by computation instead of argument.

Three centuries later, science fiction picked up where Leibniz left off: from the predictive justice of Minority Report (2002) to the more recent Justicia Artificial (2024) and Mercy (2026).

Beyond philosophy and fiction: can AI actually sit in the jury box?

This is a question we've asked ourselves for a long time, and one we started exploring as soon as AI models became advanced enough. In July 2026, we ran an experiment in which 99 fintech disputes from the Lemon platform were decided by different LLMs. We found that the model one uses directly influences the outcome.

ChatGPT 5.5 was roughly five times more likely than Claude Opus 4.7 to rule in favor of the company. Holding the model family constant and only changing the version told a similar story: moving from Opus 4.7 to the newer Opus 4.8 pushed the platform's win rate from 86% to 95%, with the model applying a stricter theory of where the burden of proof sits.

If different models (and even different versions of the same model) systematically rule differently on identical facts, then choosing the model becomes as consequential as choosing the arbitrator. This is a new version of one of the oldest problems in the history of arbitration.

While AI brings the promise of fast and affordable dispute resolution, it is not risk-free. There is a dystopia on both sides of this problem.

On one side lies a dystopia where justice is too slow or too expensive to give ordinary people a remedy. This is precisely where AI’s upside is largest: the efficiency gains and cost reductions that can bring resolution within reach for disputes too small or too remote for traditional courts to touch.

But that upside doesn’t materialize on its own. If AI isn’t built as public, auditable infrastructure for dispute resolution, it doesn’t just fail to help: it gets adopted anyway, just privately.

Big tech companies already adjudicate claims at a massive scale inside their own black boxes: content takedowns, account suspensions, payment freezes, all decided by proprietary algorithms with no external oversight. Left unaddressed, that becomes the default model: a world where management appoints the algorithm as judge and carries out its sentence, with no public institution in the loop at all.

This is the algocracy version of the dystopia: justice that is fast and cheap precisely because it has been centralized into a handful of opaque models, where people are judged by systems they cannot see, question, or hold accountable.

From AI Injustice to AI in Justice: the Construction of Hybrid Systems

The potential for AI in justice is considerable, and so are the dangers of opacity and bias. In Overcoming Algocracy, a guest article at the American Bar Association newsletter, Federico Ast argued that human juror panels reviewing AI decisions are a necessary safeguard against the rule of algorithms.

What that safeguard looks like in practice is already taking shape in our research. A recent chapter co-authored by Federico Ast, William George, and Robert Dean, published in "AI and Arbitration" (Wolters Kluwer, 2026), sketches a layered architecture built around this idea.

Disputes begin in a court designed for AI participation, fast and inexpensive for straightforward cases. From there, decisions can be appealed to a non-specialist human panel, verified through a proof-of-personhood protocol without requiring specialized domain knowledge. If contested further, they proceed to a specialist court where jurors hold Soulbound Tokens attesting to relevant expertise, weighted to ensure diverse fields of knowledge are represented. Finally, a dispute can reach a general community court whose role is not to re-litigate the facts, but to audit whether the procedural integrity of the process itself was compromised (much like how supreme courts review whether procedural guarantees were respected in lower instances).

This layered architecture avoids both dystopias presented in the previous section: no single AI model ever has the final word, eliminating the threat of opaque, centrally controlled verdicts; and because most cases resolve at the first, inexpensive layer, the system avoids recreating the access-to-justice barrier of slow, costly courts that priced people out in the first place. Speed and accountability stop being a trade-off and become a function of which layer a given dispute needs to reach.

Trust for the Agentic Economy

Within a few years, most people won't interact with the internet directly so much as through an agent acting on their behalf: comparing options, negotiating, booking, and paying, all without a human in the loop for every step.

And agents acting on people's behalf will inevitably run into other agents acting on someone else's behalf, which means disagreements will occur at a volume and speed that vastly exceed what human deliberation could ever handle. That is a structural problem: human review is incompatible with machine-speed commerce.

A specific example of what the resolution process for such cases could look like is Mirrorfall, a proof of concept we built for intellectual property disputes, where a panel of AI models (trained on different data, built by different teams, and carrying different design choices) evaluates the similarity between two images to decide if there is infringement. The use of AI absorbs the throughput that agent-speed commerce demands, while individual biases tend to cancel out across the panel.

Human review then becomes the appellate layer, reserved for disputes where the AI panel is divided, where the stakes are significant, or where a party contests the result. That is where human jurors contribute what AI still can't (moral judgment, contextual sensitivity, and complex reasoning) while also generating the track record needed to catch and correct bias in the automated layer over time.

The discussion so far solves who decides, but not who is accountable once a decision is made: when the counterparty is an autonomous agent with no legal personality and no territorial presence, there is no court to compel it and no jurisdiction to enforce against it. One emerging answer replaces territorial coercion with reputation and capital, echoing the medieval lex mercatoria, where merchants without access to state courts relied on community reputation instead.

The Ethereum Foundation's ERC-8004 initiative proposes a digital version of that same logic, assigning agents identity and reputation scores, so that a new agent with no track record must post higher collateral to transact, while one that accumulates a verified history of honoring its commitments sees its collateral requirement fall.

Decentralized justice fits into that architecture in two key places. First, it resolves the disputes through which an agent's reputation is built or lost in the first place. Second, through the curation mechanism behind Kleros's registries, it can vet the credentials agents claim for themselves: any badge or credential can be challenged by the community, and an agent that behaves dishonestly can be removed through decentralized adjudication, the digital equivalent of being cut out of the merchant network that made the original lex mercatoria self-enforcing.

This combination of curated badges and economic staking unlocks two crucial properties for agentic commerce:

• Efficiency: Without a trusted curation layer, an agent looking to hire another agent would have to parse raw historical feedback logs and compute trust scores manually before every single transaction, wasting both time and LLM tokens. A verified badge acts as an instant proof of legitimacy.
• Fairness without Gatekeeping: Instead of requiring years of past activity (which favors established dominant players) a new agent can stake collateral to immediately achieve operational credibility on par with a "veteran" agent. If it misbehaves even once, its collateral is slashed and its reputation ruined. This creates an open, level playing field where new entrants can participate safely from day one without centralized gatekeepers deciding who gets to transact.

Building the Research Field Around AI and Justice

Over the past several years, Kleros has built an academic infrastructure around the question of how AI and decentralized justice intersect.

In 2025, we co-hosted a conference at King's College London alongside the King's Institute for AI, titled "Mechanism Design for Decentralised Justice," bringing peer prediction, social choice theory, AI, and law into the same room.

In 2026, that momentum continued in the UK. The Centre for Socio-Legal Studies at Oxford's Faculty of Law, the Department of Computer Science and Kleros co-hosted "Decentralised Justice: Law in the Digital Age of Crowds and Code".

Kleros also supports doctoral research at Oxford's Computer Science Department under Paul Goldberg, a world authority in algorithmic game theory. This collaboration sits at the intersection of mechanism design and AI research, exploring how game-theoretic incentives can prevent strategic manipulation and ensure robust consensus when autonomous algorithms interact at scale.

Much of this research started on a smaller scale inside Kleros's own Fellowship of Justice, a six-month program running since 2018 that has included an AI track in nearly every cohort. Alexei Gudkov, a participant in the second batch of the program, researched the interactions between decentralized justice and artificial intelligence early on, comparing the strengths and weaknesses of Kleros against pure AI-based dispute resolution and mapping how the two could reinforce each other.

Dr. Alesia Zhuk, herself a Law & AI researcher in the program's seventh batch, spent years mapping where online justice is heading, work that formed the basis of her doctorate at Universitat Pompeu Fabra.

Key open questions in this field are behavioural. How does the behavioral economics of AI agents compare to that of humans? Will an AI agent typically act more like a homo economicus than a person would? And will an AI juror behave like a human juror, or like a strict textualist, bound to the four corners of the record in front of it?

An Open Ending...

Josef K. woke up one morning and couldn't access his bank accounts. His cards had been cancelled overnight. It took him hours to piece together why: someone, somewhere, had filed an anonymous complaint. The entire case had been opened, argued, and decided while he slept: no notice, no hearing, no chance to speak in his own defense. By the time he understood what had happened, the verdict was already final. He had lost a trial he never knew he was part of.

That is one possible future for AI in dispute resolution: Kafka's The Trial, running at machine speed. It’s not the only possible future, nor is it an inevitable one. But it’s a likely trajectory when AI is left to decide alone: with no mechanism forcing it to explain a ruling, no path to appeal, and no institution built to catch it when it is wrong.

There is another possible future where AI represents an enormous opportunity for access to justice: a chance to bring fast, affordable resolution to disputes that traditional courts were never built to handle, and which now form a growing share of our digital lives. A future of near real-time justice, where someone files a case in the morning and receives a fair decision by the afternoon.

The application of AI in the legal system is a double-edged sword, maybe not unlike nuclear technology: the same underlying capability that can power a city can also level one. Which outcome we get is not an inherent property of the technology itself. It depends entirely on the institutional framework and mechanisms built around it. Kleros has spent a decade studying exactly that question.

A decade in, this is the thesis as we see it today:

• High-stakes cases will always need human judgment. Deep context, nuance, and questions of values belong to human courts and AI systems have to be accountable to them.
• For many other disputes, the answer lies in proper mechanism design. How to combine diverse AI panels, the right incentives, and human oversight into something far better than any single judge: that is what we are researching now, one experiment at a time.
• The agentic economy will create disputes in shapes nobody has litigated yet. Agent against agent. Human against agent. Millions of small transactions, each needing somewhere to go when a deal breaks down.

Ten years ago, this was closer to science fiction than a product roadmap. But it is arriving: a world where autonomous agents are full participants in the economy, requiring justice built specifically for them. Reaching this point has taken a decade of development, failed and successful experiments, and patient research spanning AI, blockchain, mechanism design, governance, and law.

Fairness in an increasingly algorithmic society has to be built as infrastructure, not bolted on afterward. That has been Kleros's bet since before the vocabulary for it existed, and it is the bet the protocol is still making today.