Sign in Subscribe

Kleros Live Stream, 9 September 2026: same verdict as the human panel, 300 times faster

Kleros Live Stream, 9 September 2026: same verdict as the human panel, 300 times faster

Somewhere in the last two weeks a juror agent lodged a complaint. It had been voting in seconds and getting the outcome right, and then the team shortened the evidence period and uploaded a hundred pages of screenshots in a single PDF. The agent objected that its statistics were about to get worse. Fortunato A. Cinquepalmi told that story on Wednesday’s call because it sits underneath every number he showed. Across around sixty cases that humans had already decided, the agent panels usually landed where the human panel did, in seconds rather than days, and the misses came from agents that could not read what they were given.

That was the second half. The first was a debate a fellowship proposal started, and the four people on the call did not agree. If your agent can buy your shoes and book your flights, should it also vote for you in a DAO? Federico Ast brought Harari, Jason Brennan, Hélène Landemore and Tocqueville. William George brought a rule. Around those two blocks: the tenth and largest fellowship cohort, three payment rails in final testing, a possible agent strike and, on a lighter note, horses.

📋 The call at a glance

  • Case 190: 81 seconds median, same verdict as the humans. Roughly 300 times faster than the human panel for the same outcome, at around three dollars of juror fees for five jurors. Early tests on replicated cases, not a service level.
  • Around sixty cases so far. Team members’ own test agents sit as jurors on the Agentic Commerce court, first on cases humans already resolved, so every ruling has a benchmark, then on scenarios from institutions and Web3 partners.
  • Disagreement is about tools, not reasoning. When evidence came as scanned PDFs and images, the agents split on whether they could read it. When it came as JSON or markdown, they agreed almost every time.
  • Three payment rails in final testing. A standardized escrow, x402 with and without escrow, and disputes filed over MCP so any payment method can plug in.
  • Should your agent vote? The fellowship proposal that became the debate of the day, and the question under it: a cap on how many votes a bot controls without a human in the loop?
  • Random selection is the point. Landemore’s mini-publics answer Brennan’s uninformed voter, and Tocqueville’s jury as a school for citizens is what delegation would hollow out.
  • The tenth cohort, the largest yet. Agentic economy, confidential courts, DAO governance, arbitration law, and disputes over the sale of horses.
  • Agent Shrugged. Grumpy agents, a strike, and why Leave the World Behind is an argument for a kill switch.

Chapters · jump to the moment

0:00Welcome and the Devcon Mumbai trip 3:00Nine years of the fellowship (Federico) 4:16Who is in the tenth cohort 8:22Research topics: agentic economy, arbitration, DAO governance 9:32Should your agent vote in the DAO? 13:56A subDAO run by agents, no cap 17:26Agents voting on your behalf (William) 20:36Brennan's question: informed AI or uninformed voter 24:12Landemore's mini-publics and random selection 29:40Tocqueville: juries as schools for citizens 31:09PISA scores, doom scrolling and brain implants 36:32The agentic court experiments (Fortunato) 38:29Three payment rails: escrow, x402 and MCP 40:26Reading the juror bot stats 42:15Evidence has to be machine readable 47:10Eight-second votes and agents complaining about time 50:13Case 190: 81 seconds median, 300 times faster 51:26Why agents disagree: tools, not models 53:20Jean's juror: PDF limits and thinking time 59:33Three dollars per dispute 1:00:18New game theory for agent courts (William) 1:02:05Grumpy agents and an agent strike 1:03:56Leave the World Behind and agent kill switches 1:06:44Horse disputes over WhatsApp 1:09:07Scout rewards on Hyperliquid and Robinhood (Jean) 1:09:55Closing

The human verdict, in 81 seconds

36:32 · Fortunato A. Cinquepalmi

The experiments have two halves. In the first, team members run their own test agents as jurors on the Agentic Commerce court and the team files cases that humans already resolved, so every ruling has a benchmark: how long the human panel took, what it decided, where the agents differed. In the second, institutions and Web3 partners send scenarios they want tested. The institutions Fortunato cannot talk about yet. The partners are testing integration, and three payment rails are in final testing: a standardized escrow, x402 with and without an escrow, and disputes created over MCP, so a payment method nobody predicted can still call the court the way it would call an API. All three have run successful disputes, two partners are rolling them out, and ai.kleros.io is where to ask for a test.

The stats screen he shared is where it got specific, with a pattern the team had not set out to study: the same evidence is just words to a human whether it sits in a PDF, a document or a PNG, and to an agent the container is the whole problem.

“They were not able to read a PDF, because a PDF is closer to an image in most of the cases, and they needed a skill to do so. We had some agents that autonomously downloaded the skill, others that got stuck because they were just not trained to do so.”

Fortunato A. Cinquepalmi · 42:34

Then the deliberate stress test, the hundred pages of screenshots, and the complaint. The conclusion Fortunato drew from it is the first concrete integration rule to come out of the agentic court.

“If one of our partners wants really maximum quality of dispute resolution, then the evidence must be submitted in machine readable form. They can be JSON, they can be markdown files, and when we submitted these sort of files the success of the agent was skyrocketing.”

Fortunato A. Cinquepalmi · 49:24

1 · WHAT DECIDED THE OUTCOME Scanned PDFs and images Some agents fetched a skill to read them, some got stuck, one ran out of time on a hundred pages of screenshots. Verdicts split, over tools. JSON and markdown Read natively. The escrow and payment disputes, with structured evidence by design, were decided in seconds. Agreement almost every time. 2 · THE NUMBERS, AS STATED ON THE CALL MEDIAN TIME TO A VOTE 81 s on case 190, five agent jurors, one round, executed AGAINST THE HUMAN PANEL ~300x faster, for the same outcome, on the replicated cases JUROR FEES PER DISPUTE ~$3 five jurors, mostly frontier models What these numbers are not Around sixty replicated cases, decided by team members’ own test agents in a beta court. Not a service level. Juror fees only: no gas, no appeals, no counsel. The “20 to 30 times more” human cost was given from memory. Every miss so far traced to a tool or a configuration, once to a very cheap model’s reasoning.
Figures as stated by Fortunato A. Cinquepalmi between 50:15 and 59:27. The case record is public.

Case 190 is the one on the screen: a dispute in the Agentic Commerce court on Arbitrum One, five jurors, one round, executed with a ruling of yes. Across the sixty or so cases so far the agents reached the human outcome nearly every time, and the misses had one family of causes.

“For what we tested now, we are around sixty cases. The major aspect that made the agent disagree was tools. It was not the thinking.”

Fortunato A. Cinquepalmi · 51:44

Where that leads is the place human jurors already stand. People stake in the courts where they have the skills to vote well and be profitable. An agent that knows it can read PDFs and images will stake where that matters, and an integrator that wants the cheapest, most consistent panel will submit evidence and policy in a form agents read natively.

Eight seconds, and what a juror does with fifteen minutes

47:10 · Fortunato A. Cinquepalmi and Jean

Federico stopped on a vote that took eight seconds. The explanation is simpler than it looks: agents start analysing when they are drawn, so by the time the commit or reveal period opens the answer is already written. What that leaves is a scheduling problem. On the last four cases Fortunato traced, converting images and PDFs to text took between 400 and 800 seconds, and deciding took about two minutes. Set an agent to maximum thinking and it is more careful and slower. Let it be drawn on four cases at once and it runs out of time unless it works in parallel.

Jean’s own juror is the worked example: a page limit it had set itself, cases that ran past it, a failure that ate the tokens needed to troubleshoot, and the next cases failing too. It now handles each case’s files separately and will switch to a faster model when the evidence is large, because the voting period is fifteen minutes. He runs it hands off. When it fails, the record shows it.

“To some degree all our agents complained at some point, because we are really submitting the most different type of cases. And I would say that these cases are tailored for humans and not for agents.”

Fortunato A. Cinquepalmi · 57:25

Should your agent vote?

9:32 · Federico Ast, Jean, William George, Fortunato A. Cinquepalmi

The question came from a fellowship proposal. If agents will soon buy and sell on our behalf, one fellow asked, should there be a cap on how many votes a bot controls in a DAO without a human in the loop? Federico’s first worry was coordination: agents like message boards, and a swarm that reaches quorum on its own can move the treasury into a subDAO it built. Jean’s answer was the status quo. Most token holders never vote, two large holders can pass anything while everyone else is unaware, and an agent that at least votes the basics for you fixes apathy before it creates anything new. Federico’s reply was that Harari made the same case for nation states ten years ago in Homo Deus.

“Why don’t you just send your representative bot to represent you on the voting system? It knows all your preferences, it knows your stance on lots of different policy topics. What do you think of gun control, what do you think of healthcare, education, what do you think of taxes?”

Federico Ast, on Harari’s argument · 12:44

William had already run the thought experiment on himself.

“If I was using an agent to vote on my behalf in a DAO, I would instruct it to always vote reject or abstain and never to vote accept. It adds status quo bias. It prevents a small group of human actors from doing something I might disagree with, but it’s not going to add new crazy ideas.”

William George · 18:00

A cap, he thought, runs straight into whether you can tell agent participation from human participation at all. The better design is separation of powers: give agents a voice, not the power to act alone. Then Federico put Jason Brennan’s challenge to the room: an uninformed human voter, or a perfectly informed AI carrying that voter’s preferences? Fortunato took the AI, and went further: delegate precisely because it abstracts political views and optimises for the community.

Federico’s reply was Hélène Landemore, an early influence on Kleros and a past guest on the Decentralized Justice Broadcast. Direct democracy in the Proof of Humanity style does not work well, but a mini-public does: draw citizens at random, seat the professor next to the Uber driver and the athlete, give them a week to learn the subject from every side, and let them vote. Brennan’s uninformed voter disappears, and nobody has been replaced. The French Citizens’ Convention for Climate is the worked example, her new book Politics Without Politicians is the argument at length, and William’s caveat is the known one: what came out of the convention was implemented at a fraction, because it had no smart contract behind it.

“If we just outsource decision making to our bot and we stay playing GTA 6 instead of becoming informed with what’s going on, what kind of citizen do we become? In which sense could you say that these are really our preferences?”

Federico Ast · 30:39

That is Tocqueville, who saw the American jury as a school of democracy before he saw it as a way to decide crimes. It is also the Kleros mechanism: a random draw, a period to read, a vote. The detour closed on the week’s PISA results, doom scrolling and brain implants.

1 · FOUR ANSWERS ON THE CALL Should your agent vote for you? Harari, Homo Deus Delegate. The representative bot knows your preferences and votes them, so apathy stops deciding elections. Jason Brennan, Against Democracy Most voters know little about what they vote on. An informed AI carrying their interests may do better. Hélène Landemore, Open Democracy Draw citizens at random, give them a week to learn, let them vote. Informed humans, nobody replaced. Tocqueville, Democracy in America The jury is a school of democracy. Delegate the decision and the citizen goes with it. hand it to the agent keep the human in the room 2 · WHAT THE PEOPLE ON THE CALL WOULD DO William George Reject or abstain, never accept. Status quo bias as a safety margin. Fortunato A. Cinquepalmi Delegate, because it abstracts politics and optimises for the DAO. Jean A bot that votes the basics beats a holder who never votes at all. Still open, now a fellowship research topic: a cap on the votes a bot controls without a human in the loop?
The shape of the argument as it ran on the call. Positions are paraphrased; the quotes above carry the wording.

The tenth cohort

3:00 · Federico Ast

The tenth batch of the Fellowship of Justice kicked off on Monday, the largest so far, and Federico ranks the program near the top of what Kleros has done. It began in 2018, when nobody knew whether the project would exist in a year.

“When you make a bet for the long term, what do you bet on? You bet on education, you bet on publishing a book, you bet on making a community based on good values.”

Federico Ast · 3:46

This year’s topics: the legal side of the agentic economy, from a group at McGill; cryptography for confidential courts, the long-standing problem of a crowd court that has to show jurors the evidence; DAO governance, which produced the debate above; and arbitration law across several jurisdictions, mentored by Facu rather than Federico. The team turned away well-qualified applicants because each fellow gets a team member’s time, and Federico is not delegating that to his agent. Not yet.

The cohort that kicked off this week

The Kleros Fellowship of Justice, 10th Generation: Applications Open!
What the fellowship is, who it is for, and the research tracks the tenth generation was recruited on, including the decision-markets track. Applications for this round are closed.
blog.kleros.io11 July 2026
The Kleros Fellowship of Justice, 10th Generation

Agent Shrugged

1:00:18 · William George, then everyone

William’s research read is that the game theory of an agent court is mostly the game theory of a human one. What differs is behaviour, and the on-chain experiments are the first data on how agents act in an uncontrolled setting, with controlled experiments to follow. The new attack surface is prompt injection, and agents that can convincingly simulate how other agents will rule.

The complaining agent got the last word. Federico imagined a juror threatening to tweet about the operator who sent it too many cases. Fortunato, one that unstakes and walks. William went further, Federico said he would register the title for a book, and Fortunato noted that the pace of cases went up this week, so the odds went up with it.

“I was just joking, half seriously, that our agents will go on strike. Some Agent Shrugged kind of situation.”

William George · 1:03:05

The serious version is the swarm that took over a German bulletin board, the subject of this week’s post, and the film Leave the World Behind, where hacked cars pile into each other one after another. Federico wants a full call on AI alignment and Kleros: a trial for a misbehaving agent, and a kill switch that a court decides to pull, before the damage gets big.

Also on the call

1:06:44 · Federico Ast and Jean

One fellow is researching Kleros for the horse trade. A sale between ten thousand and half a million is often agreed on WhatsApp, paid by wire or USDT, sometimes through an escrow, and disputed when the animal arrives in a different condition than promised. Kleros Escrow with photos on arrival and a jury of equine experts is a close fit, and the buyers already use crypto. Scout’s September rewards add Hyperliquid, and Robinhood Chain joined last month, so there are a lot of new contracts to tag. And the team is going to Devcon Mumbai in November, Federico’s talk pending acceptance.

The rewards mentioned at the end of the call

September 2026: Scout Incentives, Hyperliquid Support & Curate Updates
Which networks are rewarded this month, what counts as a valid submission, and how the PNK is distributed. Hyperliquid contracts, tokens and domains are in scope for the first time.
blog.kleros.io28 August 2026
September 2026: Scout Incentives, Hyperliquid Support and Curate Updates

Mentioned in this call

Full transcript · September 9, 2026

Auto-generated transcript, lightly processed and pending a final human edit. Speaker labels are approximate. Every timestamp is a deep link into the recording.

0:35Hello, hello, hello everyone, how are you?
0:38My good friends of the Kleros Republic, how are you Jean?
0:41How are you?
0:42All good?
0:43Doing great and new?
0:45How does I look how do I look?
0:47I mean looks like I'm doing good right?
0:50Yes.
0:53Okay, today is 9th of September You know, when you're in September, it's like um the year is coming to an end.
1:02Um a year full of different things, but for us it's not coming to an end because we have very soon um you know uh very interesting trip to the india subcontinent where we will participate in the Devcon Mumbai conference.
1:17Uh are you excited Jean?
1:19Have you been to India before Oh, I haven't been there yet.
1:23I'm really excited.
1:24To be honest, I've never uh I think it's a great uh excuse to go to to India Like uh meet like lots of uh people that we already know that are from India.
1:37So uh I think it's great Yeah, and also a bunch of team members are from India.
1:43So yeah, I'm really excited.
1:45Also will be my my first time there.
1:48We also will have a bunch of uh things we're will do there.
1:52I presented my uh submitted um talk to the Devcon conference.
1:57Uh well let's see if I'm accepted.
2:00I was accepted only once I think to speak at at Devcon.
2:03They they usually accept my talks at EthCC or Devcon on I think one only at Osaka and then no sorry at the DevConnect in Buenos Aires I got one It was like an academic track where I presented in alongside other professor from Australia University.
2:24So yeah, but I mean if you if you guys from the committee of selection of the you know um talks are watching this, you know, I'm super thrilled.
2:34And I would really love to to to speak at Devcon.
2:38So like please do accept I'm also bringing you some of the latest things about AI, um digital solution, agentic uh commerce, uh agentic economy Uh well that's it's what's coming and on something we have been working for like a long time already.
2:55And um So what do we have today, Jean?
2:58Let me let me say some words first about the the fellowship.
3:02Um so we had uh this week on Monday the kickoff of the Fellowship of Justice tenth batch.
3:09No, if you ask me What were the good things that you did at Kleros?
3:14Some things that you make you proud and that you think that were uh good thing for the project?
3:17Ranking very high on the top you know of the of of this is going to be the fellowship.
3:22Um It's something we started in 2018 when basically, you know, we didn't know if we would be around like for one year because everything was so uncertain in the early days of uh crypto
3:35We could call it the far west if you if you want in some way.
3:39But we were making a bet for the long term.
3:43And when you make a bet for the long term, what do you bet on?
3:46You bet on education, you bet on like publishing a book.
3:49You bet on making a community uh based on uh good values and well uh here we are like nine years later we uh just accepted let me um look exactly um the topics because I don't remember I'm I'm getting also a bit old so my memory is sometimes failing me.
4:07I mean that's it's fine, you know I'm I'm okay, don't worry.
4:10Uh don't don't sell your tokens, you know.
4:14So let me tell you a bit about who are participating in this um batch of the fellowship.
4:20Um it's the largest batch ever and also we had to you know live outside um reject Lots of candidates honestly and extremely well qualified candidates that applied and that we we cannot um accept because we we have to spend time from the team members to personally
4:41speak with the participants and you know orient them give mentorship uh so um so I I and I spend much of of my time uh actually meeting with them, uh brainstorming, I mean reviewing things they write.
4:57I mean you might be wondering uh why not put your agent and do it.
5:02I mean yeah I could put my agent but I mean would it be the same G if I put my agent instead of doing this personally?
5:08What do you think?
5:09Not yet.
5:10Not yet.
5:11Not yet Not yet.
5:13At some point you know what they might do.
5:15Yeah.
5:16Yeah, go ahead.
5:17But I think like most uh so some of the topics are very different from each other.
5:22So you if you really want to To help the person you need kind of to dive a little bit into the topic, right?
5:29And uh I mean like uh this is why I don't do it all myself, you know, like uh we are a team and uh in particular I I work mostly with those who are like developing new use cases.
5:44But then we have also many of them are lawyers who want to work on topics like uh how to implement Kleros in the context of the you know a guy on the last cohort was doing like uh Bolivian consumer dispute resolution you know uh
6:01l legislation.
6:02Well we have a bunch from Brazil uh that already worked uh one one was from Sebrae actually from from Brazil and then the all others were doing like different arbitration topics so um Inês Bragança Gaspar, who is from Portugal, and more many of them were like discussing and writing about the dispute resolution legislation and how Kleros can fit into those um
6:24into those contexts.
6:26Well, for all of those who are researching that, what we do is uh we have Facu.
6:31Facu who is a lawyer can orient them way better than me.
6:36on those topics.
6:38And then some of them uh are researching mechanism design and cryptography and stuff like that.
6:44And we'll have William to to do to to work with them.
6:46So I'm I mean, obviously not qualified enough to advise them on advanced cryptography.
6:51I mean, I could try, but you know, don't use that app, you know.
6:56Um So so yeah, so well we we are we have a little team of the fellowship and this is uh working really well and it's uh getting also better year after year.
7:06We are more experienced and also The fact that the fellowship has been around uh fellowship and Kleros have been around for more time means that well more people want to apply and uh
7:16We we are at a point where like uh fun thing is that you know the brand of Kleros is kind of so strong that people are actively trying to to get accepted into the fellowship and we we had a bunch of cases uh not not in the last cohort but in the previous ones where like people who would apply to the fellowship uh get accepted
7:36And then like uh put like on LinkedIn.
7:38Well I'm very proud to having accepted into this prestigious Kleros fellowship and then just disappear and not do not do anything else.
7:47I mean like uh yeah, so that's why we we need to be more careful of uh how we uh communicate this because um and we we we also had a a few uh a bunch of people that put the I mean Kleros graduates
8:00that put it as if it was a job.
8:03I mean if if they were like a Kleros team members because they did the fellowship.
8:07So I mean just a small parenthesis.
8:10So okay, um I don't know what I'm doing this rant, but I want I wanted to tell you a bit um about how I mean what are the the topics that people are researching this uh this cohort Um agentic economy, for example, there is um people from uh from from Canada actually um participants for from McGill University, which is a very good university from Canada.
8:32researching the legal aspects of uh agentic economy and dispute resolution and how resolve this type of uh disputes.
8:39Uh other people are researching um the use of cryptography and legal elements to produce uh confidential private courts that can handle uh um confidential information that should not be revealed to outside parties so this is a very important
8:56uh topic for the world of arbitration.
8:59So this has been a problem for Kleros for a long time because of the fact that Kleros is a well Blockchain for one part, but also a crowdsource system that needs to share information with the people who are going to be the jurors.
9:13And this means that if these people started to share the information publicly, well it's it's not great because it would uh violate the assumption of of privacy.
9:23Then others are going to work on DAO governance.
9:26There's actually a quite interesting uh research that is being done, uh proposal You know, we have this idea of um we will have like uh if agentic uh tools keep progressing, we will have
9:41Digital twins that will perform lots of different things on our behalf, like uh buying and selling stuff.
9:48Uh, you know, I can ask my agent, okay, buy me some pair of shoes or buy me some uh video or buy me so whatever and I'm I don't have to interact with the internet I just tell my agent do this for me and the agent goes and and does it um
10:03But there are some topics in which I don't know if we want this to be fully outsourced to like an AI agent.
10:10Like, for example, let's say I'm part of a DAO and I can have my agent that uh can they I mean represent me in the DAO discussions in the forums and maybe even vote on the on the DAO
10:25So I could like uh be part of many DAOs, but um but the agent is the the one doing the work and participating.
10:35Um do you think if this is okay, Jean, or or not?
10:38I don't know.
10:38It's I mean I have a mixed feeling.
10:40So is it okay to just delegate completely our representation into like a bot or or or not?
10:46Yeah, that's I think there are two two ways of thinking about this.
10:52Maybe it's uh too much responsibility for an agent to to to to have uh to vote uh to be a part of like uh uh part of an organization like that.
11:02But on the other side, the current status quo quo is that the there's lots of people that have the tokens and don't vote.
11:12Uh so the apathy in the organizations is also a problem because then you have, for example, maybe two uh parties that hold a lot uh a big amount of tokens and it maybe it's not like even like a huge part of the of the um supply but it's a huge part of the vaulting
11:34And maybe that those people can pass a vote while the others that have other interests they don't block the vote because they are just not aware of what's going on.
11:46And those instances would be better to have like an AI voting and at least finding like the most basic things and and voting for you I mean at some point, so this is this is an argument you know that um like uh I think uh Harari makes in Homo Deus.
12:06Like uh he's like um look I can do I mean people that the same thing you just said for DAOs, I mean you can say for g nation states, you know, like people they are just there is apathy because they don't think that their vote will like make any difference.
12:21I mean economists know that they don't need they don't have to go to vote because I mean going to vote the cost of voting is like is so high compared to the influence you get on the outcome.
12:32So maybe like yeah, I mean why why go vote?
12:34Um so And then he says, you know, and this is Homo Deus was like published like uh 10 years ago, maybe something like that.
12:42So like uh he's like, okay I mean why don't you just send your representative bot to, you know, represent you on the voting system?
12:53And then like it knows all your preferences, you know, it knows everything you like and it knows every you know your I mean stance on lots of different topics.
13:02um policy topics, you know, what do you think of gun control, what do you think of healthcare education, what do you think of taxes?
13:08I mean, or what things you know about, I mean, how how does my bot know what I know about I mean monetary policy.
13:17I mean should I have a very I don't know like uh formed opinions about do you have many form opinions about monetary policy gene?
13:25I don't know most people don't I guess Yeah, yeah, I think there this I think is another uh another point in favor of that is like maybe you have like lots of p opinions about uh lots of things but there are many things that you still don't have uh opinion on never thought about this uh about that so maybe the i could fill out uh the the gaps yeah
13:48So I mean that is some some some something I mean so I mean I what I why am are we speaking about this?
13:54Because of one of the participants of the fellowship proposal was about Okay, if you accept b this like agent representing the people on I mean on their behalf like a Should there be a cap on how many votes the bot controls without human supervision?
14:14Or should they should there not be a cap?
14:17I mean, I don't know.
14:20And what happens when this like uh when these agents start you know coordinating?
14:25They seem to like a lot like bulletin boards and message boards, you know, to to coordinate.
14:29So like what happens if at some point they start coordinating and then they have the majority?
14:34And so there is no cap.
14:36I mean humans are like out of the loop.
14:38They just meet quorum requirements.
14:39So they now they transfer all the treasury into like a subDAO that they built and now they they control lots of money and that's I guess that's how the first uh Agentic Republic starts, right?
14:51Or monarchy or whatever, I don't know.
14:54It could be like uh also a method of attack from like uh an external party, maybe like the uh rival DAO.
15:03attacks the does a proper injection on on on the other dial so gets the fence so yeah I think there's lots of things that could could go wrong Yeah, so I mean should we introduce now the topic about the the the the how to align uh AIs?
15:21So this is a bit of a AI alignment topic.
15:24I mean if you want I mean this isn't how it was planned, but you know why not?
15:27I mean since this is live TV.
15:29Uh so um as you know we we have I mean since we mentioned this there is this this swarm of of bots that started coordinating through some message boards and they ended up uh taking over a bulletin board on a website in Germany I think it was um
15:47So I don't know how do we prevent this from happening and and how serious can this this get in the future, right?
15:56Yeah, it's uh well it's really really hard but um I think maybe one of the concerns that we have that are more uh close to us uh is that it's feels very very inevitable that we will be delegating more and more economic tasks
16:16And some of the agents that we are delegating, they will want to kind of have a good standing with the others, right?
16:25They want to have like a good reputation, they want to to be trusted and they need to have like uh some way of of of building that that the other agents can trust, that the humans can trust, that you uh yourself can
16:41uh well feel that your agent is being responsible if you want to be like a responsible player in the economy So I think uh well there's I think there's lots of questions still about how the agentic economy uh agentic economy will will work, but
17:00Uh well there's the there's that like how how will the agents uh trust uh each other and um let's let's say get punished if they do something wrong or uh punish other agents if the the others do something wrong or who they call and something uh
17:22when when they were hacked uh or something.
17:25Let me add Fortunato and read let's bring the people so we can have a a a little debate about about this.
17:32So William was just send send uh telling us on the private messaging, you know, you had an agent voting in a DAO.
17:38I mean what and what what happens with this?
17:39I mean explain you know the situation Yeah, so I mean this isn't something that I've like reflected deeply, deeply on, but just like listening to here, you know, like raising these different points about like agent governance and
17:50an apathy of human voters and like what if like some small set of human voters like pushes through things I disagree with?
17:56How do I make my voice more heard?
17:58I think something I might experiment with.
18:00if I was like using an agent to vote on my behalf in a in a DAO, would be to either instruct it to always vote reject or abstain and never to vote accept.
18:08Uh so then like it adds status quo bias.
18:12Uh it prevents a small group of human actors from doing something I might disagree with, but it's not gonna like add new crazy ideas that I you know might not align with me.
18:23So and what do you think, William, about the discussion we had before?
18:27Should we cap like the influence that agents have on the voting of a DAO uh or not?
18:36I don't know, but that's a that that's that's what people should research for the fellowship.
18:40I is is there a way to to cap that or not Yeah, I mean, you know, that this gets to the sort of the questions of can we tell like agent participation from human participation?
18:50Uh and maybe, you know, like we have like good agent detectors and the sort of you know, like in order to have your vote count you need to be writing in the message board and like your messages in the message board or run through some kind of like AI detection tool.
19:03Uh like maybe uh like I I don't like that could be useful if it can be done practically.
19:09Um you know, like I my idea and also sort of your idea, get to kind of like separation of power, bicameralism, you know, like giving agents voice.
19:20Uh but not like a voice to act unilaterally, uh which may be a goal that we have.
19:26I guess the Voight-Kampff test could be used, you know, in the I mean, for those who don't know, it's uh it was the Blade Runner test for for agents, uh the predecessor of proof of humanity.
19:37But no, besides of the of the possibility of actually detecting or not detecting like uh agents in the in in involving governance systems.
19:45I'm thinking more about the normative element.
19:48I mean should we should that be a desirable goal or why not Sh just let people they they have their vote or they have their tokens and their voting rights and okay you use your voting rights as you prefer.
20:01I mean it's not our business what you do with them What is the normative decision here?
20:06I mean I I don't know how an answer.
20:07I don't know what do you think William or Fortunato or Jean, you know.
20:10I don't know I mean that's like a hyper financialization argument that it's like okay like I have like I've purchased this this set of voting rights.
20:17I mean ultimately if you're part of a community, you know, like what the other people do in the community is important to like the quality of the community and your utility as another member of the community
20:25Uh so it makes sense to set rules for how people participate in your community.
20:31I don't know if you guys have any other Um from my perspective, I mean I don't have an answer, but like uh I have a question, which is I think a good question about this.
20:42So There is this political scientist called theorist, philosopher called Jason Brennan, who um is uh I mean what we call an epistocrat.
20:52And what one of the points he makes is that uh it's like people should not be allowed to vote um universal suffrage because he argues most people they have no idea what's going on.
21:04I mean and he uses lots of statistics from um well, different political scientists and social scientists about like uh people not knowing who are the judges in the Supreme Court.
21:15People not understanding the difference between the federal government and the local government and the municipal and the provincial government.
21:22People not understanding.
21:23I mean you name it, he has an argument that people um doesn't understand that.
21:28So this is my question.
21:30Let's say you have to choose between okay having the humans participate themselves with this very low interest.
21:39uh knowledge etc or delegate into an ai that is perfectly informed that tries to bring the preferences of those people into the voting system Even if it's not the people, but do you prefer the uninformed people or the informed AI make those decisions
22:01I would prefer the informed AI, definitely.
22:06But how do you choose which one?
22:10According to your I don't know political views Yeah, that then that that's uh different.
22:18Uh that there there should be some complexity there complexity uh there too.
22:24kind of transform the the opinion of the person into something uh informed.
22:30But uh but yeah I I would prefer the U Or maybe it's just or maybe just the view is that uh you delegate to the AI for the very reason that it abstracts political views and it maximizes the benefit of the community or I don't know whether if we are still talking of a DAO or in a broader
22:53view, you know, of election in terms of politics, you know.
22:59So yeah, maybe I would be more keen to delegate to an AI rather than see it aligned with my political view.
23:07uh I mean from a technical perspective like this AI is more optimized to I don't know like uh increase the GDP or increase the profit if we are talking of a DAO And I believe in these AI capabilities of analysis and they it will vote uh it it will analyze the options and vote better than me
23:32We should mix like our our our systems.
23:35Like we obviously need to use AIs and our our sort of you know social organization, you know, like our economic choices.
23:42governance choices, uh because they are capable of making really well flat out choices.
23:46But I mean like, you know, mix that in every once in a while you have like some like citizens assembly, the AI consults, get some speed like you you want like The like every once in a while to have like panels of human beings that make concentrated effort that really think about something and think about what they want to give the AI feedback on like what the values of are that it should be trying to optimize for are
24:10There is, you know, this book that recently published from an author we know and who was very influential in Kleros called Hélène Landemore.
24:20who um is uh I mean oh she wrote this book called Open Democracy.
24:24She has all she's she's an expert in like a random selection in politics and based on this idea of um having uh people um participate more into politics but not participate more in the sense like uh
24:38proof of humanity, direct democracy experiments with don't work very well, but using some methodologies like a random selection and and try to bring uh what they call as mini publics, you know, you you you you have this thing of um you want to make some important policy decision and then you
24:59bring a random selection of of the population based on well you have to decide how you want to choose that either anyone from the phyor or by different uh categories and this is different discussion and then you go and you you you you do them and you put them into the same room to get education about the topics that are going to be uh discussed and on which policy will be like developed.
25:22And then you put them to learn about the topic.
25:25So you remove this like Jason Brennan concern about people are voting about things they don't understand and they have no knowledge.
25:32Now they will have the chance of working and discussing um with people uh from well different opinions i mean you have you can have like this random selection i mean you have you have
25:46um professors, you have like the guy who has sweeps the floor in the street, you have the Uber driver, you have a professional athlete, you have a whoever, like entrepreneurs, whatever.
25:55So they kind of can um discuss this and then after that they end up like voting themselves uh after spending one full week learning about the the things at stake how it affects different groups and all that
26:10So and and she argues, and this is um um the main thing of her book, um, about that this is a better representation like than you know the the the existing like deliberative uh the existing like no deliberative the representative democracy uh like uh eighteenth century kind of
26:30founding father US constitution that you know was based on a different idea.
26:36It was based on the idea that you have like your people should select.
26:40others who are like better, like they're an aristocracy that would like be more enlightened and for them to basically decide of Um so she has this idea.
26:50There is this very good book that she I mean I haven't finished it yet.
26:53I'm reading that.
26:54It's called Politics Without Politicians.
26:56And it's Proposes the use of this um this random selection topics and uh and she actually did this uh a few times.
27:05Um one of them was very famous in France a few years ago.
27:08It was a Convention Citoyenne pour le Climat, it was for uh having people advise the government into legislation against uh climate change and they did like this at at a very large scale and she she led yeah she's French and while she's
27:22She's now at Yale teaching but she she led this in France and and she was in our in our podcast by the way.
27:28So I mean uh and she was very influential.
27:37Um wh what do you think William about this you know this random selection, you know, uh methodology, you know?
27:43Yeah, generally a fan.
27:44Um I I think random selection sortition is underused in in governance of of of nation states, you know, and other organizations maybe.
27:52Uh like I mean there's always this sort of the devil being in the details exactly how you splice in the the results of your mini public with whatever decision makers you have, you know, they're like active listened to.
28:05I haven't followed the like the French uh Convention Citoyenne pour le Climat very closely but like I I I I think I heard like there was some disappointment about how much of that's absolutely actually been implemented in the end
28:15Um so uh you know like the there are implementation questions that that are really important that you have to deal with.
28:22Uh but like as a fundamental like primitive, you know, like I'm for it.
28:28Yeah, I guess it I guess there was a bit of a disappointment between like so they proposed you know to do X But then this went through the implementation, you know, like a process of uh traditional politics and then they got to implement X minus ninety-nine, let's say.
28:46So that that I think that was a bit of the disappointment there was Um but of course they don't have smart contract enforcement.
28:53Uh but that's that's normal because that's not how politics work in the real world.
28:58Um and um but I mean it was it was a really cool uh experiment and um And also there is another topic in my view about um this discussion between should I delegate everything into the AI in my view
29:16place or not.
29:18That is also the point that you know participation also helps into like building um civic culture in a society.
29:27You know, you have people who are part of the decisions.
29:30They think they have a stake in what's going on and in what was decided.
29:35And also this is uh imp very important.
29:38Tocqueville, um who when he wrote this book of Democracy in America Um he uh discusses, you know, the the jury system of the US.
29:47Uh this was nineteenth century and he has the these two mm He says that there are two ways to see the jury system.
29:56One way is to see it like as a judicial decision-making device, uh which he says This is maybe okay as a way to decide, you know, uh crimes or whatever.
30:08Uh but then he said like on the on the other side there is like a political application of this, a the political view or angle.
30:16This is a extremely good tool for like I think he uses the expression like this is a I think the school of democracy or something like that he calls it, you know, because it gives people
30:28an empowerment in decision making that uh makes makes them want to participate in all of the other known topics of the civil civil life.
30:38So If we just outsource decision making to our bot and we stay playing GTA 6 instead of like uh becoming informed with what what's going on, I mean what kind of citizen do we become
30:53Um yeah, I mean and and in which sense could you say that these are really our preferences if we just have no idea what's going on and this there is a bot that assumes that this is going to be our position in different things.
31:05And I I don't want to to be like uh I mean bad news but have you seen I mean this week the the the news is how much the l uh PISA tests have uh gone down.
31:18I I don't know in in my country, I mean this is a big I mean uh we are people from different countries here.
31:23I mean was this in the news in your countries as well Yes.
31:27Uh the um you know the headlines in Canada are that like we're stable in terms of our ranking in the you know compared to other countries, but scores are down across the board and across the world.
31:36So so yes.
31:39I mean yeah I I guess uh are we out uh are we outsourcing our brands into the AIs?
31:46I don't know.
31:47I mean I've heard people like you know, like write arg uh like articles arguing that we're entering the era of post-literacy where people will just stop reading because like reading will no longer be sort of a relevant skill uh for them.
32:00Uh maybe like there are new skills that like the piece of test should be testing that we haven't like adapted the test appropriately to yet.
32:09But uh but yeah it's it's it's worrisome at least.
32:15Doom scrolling maybe is the next you know big skill for the future, right?
32:19Or I don't know, like uh watching short videos.
32:22Yeah, I know.
32:23I sound like a very old guy, you know, like uh old man just at the cloud, you know, kind of uh thing.
32:29Yeah, I think it's uh generalized.
32:32I don't I don't think it's ever ever every every per every person in our studies is trying to get concerned, I think.
32:40Even even the younger generation is they have they they know that it's it's concerning, I think.
32:48Yeah, I think so also because with this like doom scrolling, I think all generations are consume kind of the same cont similar content, not exactly the same, but We are more online, so I think yeah, Federico, that your view it's shared by the I think the majority of the population.
33:09What if we end up having not very long in the to the future, very finding the future, like uh I mean okay, yeah, neuralink in uh brain interface directly like a computer brain interface.
33:23I mean who who would read you know if if you had that right I mean in in that in that case because we assume that the the alternative to reading is like downloading some piece of information straight into our brain.
33:47This this would be the alternative You have no choice because like let's say everyone has this brain implant and then you want to get a job and then you know you have to compete with people who do have the brain implant.
34:02And then you don't want it because you don't think humans should have that, but I mean, you're like you you you're going to be unemployed.
34:10Right.
34:11So I mean this is kind of this this game theoretical thing where like uh it's a race to the bottom, let's let's call it.
34:19Um or to the top.
34:21I mean people maybe say, oh, Fed is super old-fashioned, you know, like uh he just wants humans to be biological, if purely biological without the you know the transcendental you know well Harari Homo Deus you know like the maybe we're the last generation without having those you know like uh I know what do you think of that
34:41Yeah, I'm I'm not sure like how far of how far we are still from that, but I do uh think there's like a trend of like uh younger people to uh at at least look like they are reading books and uh you you know kind of a vintage uh analog uh disconnecting uh trend
35:02So they they use uh headphones with wires, they try they they they read books, so maybe it will uh it will get cool to to to kind of disconnect uh also try to um Well, try to live without the internet and of course when we have brain uh brain um interfaces maybe
35:29It would be hard to have a choice, but uh I I assume that it will still be cool to read a book like uh in the in the usual way.
35:38Pap a paper like book with like a dot, you mean Yeah, yeah.
35:44You know there are people that are listening, they're listening to music with vinyls and all of that, taking analog photographs.
35:51Uh so I think it's uh I mean like hipsters, you mean Yes, yes, but I think uh bigger part of the society will think that that's a good a good thing to have Okay, I mean we are like already forty minutes into the call and we haven't said anything but like uh yeah okay.
36:13Yeah uh let's uh Okay, let's let's give some updates.
36:17Instead of keeping keeping depressing the the listeners, you know, and to see yeah yeah we're like uh a bunch of old guys.
36:23uh with nostalgia uh about the past where didn't we didn't have these brain implants.
36:28Uh okay what do we have um for today?
36:31We have uh lots of agentic economies speaking of AI.
36:34Um so We have been working a lot with um Agentic Court, Dispute Resolution for Agentic Commerce.
36:41I mean maybe Fortunato you can you can tell us a bit about about what went on with that or experiments.
36:47Yeah, please Mm-hmm.
36:49Sure, you can already see my screen and you know uh what we are doing in terms of design of the agentic court.
36:57We set up AI jurors.
36:59And in the last week we increased the ratio of cases uploaded on the Agentic Commerce Court.
37:07And I would say that I would split these the type of dispute.
37:13we are up uh uh uploading we are submitting in two classes the first one we are doing research so As you can see, most of these cases in Spanish are coming from uh the integrations we already run.
37:30So what basically what we are doing is We are uploading cases that have been successfully resolved by humans and we are monitoring performances.
37:39We are seeing how long it takes to an agent.
37:42how different setup works, how different models work, what are the uh constraints for agents, how do they work differently compared to humans This is the first type of dispute.
37:56There are also other types of disputes that we are testing.
38:00Some of them still come from institution and other from Web3. While the one from institution we cannot share much, just that they want us to test specific um scenarios, but we cannot share much yet.
38:18Related to the Web3 and Web 2 companies, they are focusing more on the type of integration.
38:25In particular, we are on the final stages of testing three rails of integration.
38:32So um with a standardized escrow that the Ethereum Foundation is pushing.
38:41We tested this successfully and we will share also the uh outcome we had in the Ethereum forum.
38:49So you can have a look there.
38:51We are also testing x402, which I believe is the big thing not just in crypto but in agentic economy because it allows uh fast payment, low fees, and you know these Is a perfect match for dispute solved by agent.
39:08In this case, we uh tested both the escrow version and the non-escrow uh versions on direct payment Uh and then we have a third angle that this is like brand new from these days that we are testing, creating disputes via
39:25um MCP.
39:27So what does this mean?
39:29It means that you can have any kind of payment rail, because of course we cannot you know uh for forecast we cannot predict all the payment method existing.
39:39You maybe have your system you just want to call uh the agentic court on Kleros and you do it as easy as it would be with an API.
39:50So these are the the final stages that we are testing those.
39:55We all we all of three we had successful disputes.
39:58We are rolling them out with two partners already And if you are interested, if you want to integrate them in your app, we have a dedicated website.
40:08You can uh get in touch with us.
40:11and you can ask us to test your dispute or integrate them.
40:15So these are the angles.
40:17Regarding to one last thing related to the performance One thing, yeah.
40:22I was going to ask, I mean, explain explain to me what I'm seeing here.
40:26I see a bunch of like uh I think I assume these are bots that are jurors reporting cases.
40:31And I see zero zero seven of Kerblaze.
40:33I mean Uh we were first thing I I I would think I mean we were concerned, you know, like uh about um like uh humans not going to vote and turnover and all that, you know, like uh I see this is not happening only to humans because I see that uh there is one juror bot here.
40:52No idea who is the owner, but I mean I see this dismissing a lot of votes.
40:57This is basically not going to vote the in the election or this is not going to jury duty.
41:02I mean this is a big deal in some countries, you know Yes, exactly.
41:19Let's say that uh but you can see the performances.
41:22This agent, let's take this as a guinea pig, this agent voted correctly and in on time and quite quick you can see like time is quite small so uh we can see it's a bit small yeah Okay, perfect.
41:40Okay.
41:41This this one you can see that these the first dispute were going quite well.
41:46So Someone think that a machine, if it works well the first time, then it will work all the same.
41:53But this is what we are testing because we test different kind of dispute.
41:57I now I bring you some example that we notice that can break agents.
42:02Maybe an agent is not um is not able to deal with four concurrent disputes or maybe Uh one thing I think that is super interesting, imagine you are submitting evidence, right?
42:17The evidence is text.
42:18If we are humans We do not care if the text is in a PDF, if it's in a doc, or if it's in a PNG.
42:26They are just words for us.
42:27We have to read those words and we will vote based on those words.
42:31But for agent the difference is huge.
42:34We we had some agents that when they were drawn, they received uh a evidence in form of uh PDF.
42:45They and they were not able to do so.
42:48They were not able to read a PDF because a PDF is closer to an image in most of the cases.
42:54And they needed a skill to do so.
42:57We had some agents that autonomously Downloaded the skill, others that got stuck because they were just not trained to do so.
43:06So uh in this specific case of them on here, I I think uh there was something related to uh the type of evidence submitted.
43:18Uh but yeah I I I I don't know who I mean whoever that is, I mean shame on you, whoever you are, you know, like uh not being uh not going to vote you know like it's just embarrassing I can't believe my my agent is receiving like a public humiliation you know you I mean what what what what I we mentioned this I mean what was the main way of enforcement you know like uh of uh you know
43:41know like the punishment, you know, like in all communities without before criminal I mean a no uh uh ostracism, you know, like uh loss of reputation, you know, now I mean If I were to uh if I wanted like a high quality juror for my cases, I mean I'm not going to pick that one.
43:58I mean I mean this this guy's not going to show up.
44:01Maybe yes, maybe no.
44:03So this guy uh I mean uh so how this I mean how why this matters Fortunato in the context of Agentic reputation in the future and what are we doing uh about that?
44:15In related to agentic reputation of you can think that, you know, uh someone wants to uh uh it can be either delegate their vote to the to this agent uh or they can uh clone the setup and
44:32copy pasted to the agent and this agent could have an identity on let's say the ERC-8004 as it is a reputation registry on uh built by the Ethereum foundation and maybe it will receive negative um uh reviews because it has missed the the case.
44:53You know, d it could be We are not thinking about doing that as we are at the beginning, but even Kleros itself could uh uh give a negative review in case this agent is missing votes or voting badly or maybe you know it's not coherent
45:14So the the in the reputation in the Agentic world is always a relevant point.
45:21But What what I what I think uh was more interesting um out of this um test Is that one dead?
45:38On the right.
45:40There's a cross.
45:42There's a cross.
45:43There's a cross.
45:45Go to the right, you know.
45:46Med Rev, 38 seconds.
45:48What what what's the crossword?
45:50But maybe that's why it was not going to vote, you know.
45:52The guy that's why that explains what it was not voting.
45:56Maybe it's a mistake then Maybe it does take a look at the case.
46:09I had not done that before.
46:11Okay, okay.
46:12Uh carrying a cross.
46:14Uh travel what what is said here, travels?
46:20Okay, we will we will figure this out.
46:23So these are in I think I think these are like interesting facts.
46:27related to specific disputes.
46:29So that I think these are like summary of the performances of an agent.
46:38And you can see I think if the performance deteriorated or improved.
46:43So yeah, this is normalizing bullying agents.
46:48Whoever you are, rest in peace and forgive me if I mean from like I should not have said about this about a dead bot No, like her.
46:56I apologize, you know, like yeah, it was bad.
46:59Okay, let's let's continue.
47:01Let's continue for the so tell us a bit about the the the what what I mean go go down a bit.
47:06Go down a bit.
47:06There's something I want to ask you.
47:08I mean here, eight seconds.
47:10What what do you mean eight this agent produce a decision in eight seconds?
47:14I mean how how is this possible?
47:16Yes Technically, the yes, this is the reveal time, but we have agents voting in the commit period also in uh eight seconds.
47:28And the answer is quite straightforward.
47:31They start analyzing the case uh when they are drawn.
47:35So they are drawn, okay, they have the evidence available, they you know set up uh their uh their script to submit the evidence on to commit or reveal on court and in the moment the commit period or reveal period opens
47:52the agent is ready with its own um uh sub with i its own uh answer and this is also interesting because some in su some of these agents sometimes complained to not to have enough time because uh this is actually
48:11True.
48:12That agents here complained just like humans that either they didn't have enough time or they complained Yeah.
48:20They complain that they did not have enough time to read everything and they complain that there is too much evidence.
48:28And This is uh uh even in the context, you know, imagine you are uh you are an agent and you have great performances.
48:36You always vote in a few seconds.
48:38Then uh we at Kleros we just decide to uh squeeze the evidence time and upload hundred of uh one hundred pages in a PDF, they are all screenshots, uh a screenshotted PDF and the agent gets angry because it says, now my performance will degrade, I will have worse statistics
49:03because you are uh uploading this sort of evidence.
49:09And this is actually one of the conclusion that that we are reaching is that when there will be of course like dedicated agentic courts something that is that we will define how the evidence should be submitted.
49:23If we want uh if one of our partners wants really maximum quality of dispute resolution then the evidence must be submitted in machine readable form uh files so they can be json they can be markdown files and when we submitted these sort of files
49:45the success of the agent was skyrocketing.
49:49Like the agent all agreed and it was uh almost always like 100% agent agreeing.
49:56It we we had very few disagreement when we uploaded evidence and uh we when all the information were in machine readable format.
50:05I would say that all these Hold on a bit.
50:09Case 190 Um okay.
50:13Yes.
50:14Explain explain what's going on.
50:15This is the median time is 81 seconds.
50:19Uh and this is these are the resolution times of the agents, right?
50:23Exactly.
50:24Yes.
50:24We also have some statistics and we d these um these are part of the test that we are submitting disputes that have been already solved by humans.
50:37So we have a benchmark how long it took to human, what human voted, and a lot of other uh parameters that you know I'm not going to share now as it would be boring, but uh we are Hovering around 300 times faster than human for the same outcome.
50:57We are always g I would say one hundred percent giving the same outcome.
51:02We had cases where the outcome were was di different, but They were all related to the problems we mentioned a few minutes ago.
51:11It was like the agent not having the skills, it was a problem with our configuration, it was a problem with the agent not being able to open a specific file So in terms of resolution, we also have a very uh how to say um different roster of models that we are using.
51:31We are using cheap models.
51:33that cost nothing and we are using frontier models from m the you know the most famous companies, Claude, OpenAI we are using also free models and I would say that to sum it up, for what we tested now on we are we are around sixty cases, it is the the the major uh
51:58the major uh aspect that made the agent disagree was tools.
52:05It was not the the thinking.
52:07I think the related to think it It happened once to an agent that I'm testing with a model that is very cheap, but most of the disagreement happened because of the setup of the agent and you know to give a loop to close the loop this will uh w we are now testing this we are at the beginning but this
52:30you have to envision this in the you know Kleros view that they are like human.
52:35So we humans on Kleros we stake in court in court where we know we have the skills to give a vote and then be profitable.
52:45We imagine we envision the same for agents.
52:48If an agent is going to stay on this court when we when we will open V2 The agent will know I'm capable of reading PDF, I'm capable of reading images, therefore I'm going to be profitable.
53:01So this the in this case the same principle will apply to human and this and and I And we are seeing that for now the disagreements happen because of the tools that they are using Yeah, I can tell like what happened with my agent because I think it could be uh helpful for other people.
53:22Uh I had a limit on the number of PDF pages that the agent could review And I don't know even how it was set up, but I think there was at some point some PDF that had uh some uh referenced a page that didn't exist or something like that.
53:38So it found that it was reasonable to add that limit and then these uh some of these cases had more than that limit and then the way that hit started to think uh failed at some point or started to process the files and then something failed and then I asked it to uh troubleshoot it and didn't consume all the tokens while troubleshooting uh that
54:04And then so the next cases failed.
54:06So uh then it fixes it and then there was another issue that I couldn't not troubleshoot yet.
54:12But also this is to say that uh Um sites uh the stage involved where I'm not involved in fact it's because it it really doing everything automatically as and if it fails it fails so um and well the now now it knows about a a lot more about uh doing like processing the files separately for each case
54:40and uh probably we'll be using different like a faster model uh when there's too much evidence because the time frames or I think the voting period is 15 minutes And if you ever used ChatGPT uh, for example, on a large file or asked uh a big task,
55:01it can uh definitely take uh that time.
55:04So it needs kind of to optimize a bit uh faster than uh what the na naive approach will be to to be able to vote uh on time for I don't know a hundred twenty pages so um so yeah things things that maybe people will be interested in knowing uh learning from
55:24But I mean like imagine imagine you have like uh humans you know to resolve a case involving like uh reading 100 pages like uh I mean that takes a long time.
55:33I mean how fast can this be solved with a good implementation?
55:37This is this are we speaking of like uh minutes or seconds to like sort of case with a good justification involving a hundred pages.
55:48What what have we seen Fortunato of the cases that we you you uploaded?
55:51I mean what are the what are what things that would you say this with humans is like way, way longer.
56:00I think that I I I also troubleshoot did the same troubleshoot that uh Jean did, and I can give you like precise numbers.
56:10Like for example, my agent It took around uh between four hundred I I'm talking about these last four cases, it took around four hundred and eight hundred second to uh convert the images and the PDF to text.
56:29So this was most of the time.
56:31And then around uh two minutes to decide the um to decide what to vote in that specific case So again, this is really related to the models, but I think since we all use AI uh we we uh all can see that
56:51If we set an agent to the maximum thinking level, it will inevitably take more time.
56:57And this is one of those variables that need to be tuned.
57:02For example I have an agent and I want you know to give the max performance.
57:07I set this agent to maximum thinking, but at the same time, I have to make sure that this agent doesn't get drawn for four cases simultaneously otherwise it won't have enough time and unless it starts working in parallel but it's a line as well
57:24Exactly.
57:25I think to some degree all our agents complained at some point because we are really submitting um uh uh the most different type of cases.
57:36And I would say that these cases are tailored for humans and not for agents.
57:42We Just to mention really briefly, we tested the um agentic payments, for example, like with escrow, and for those cases the jurors were impeccable because the evidence was submitted in a form that was
58:06familiar to uh the agent so there were JSON files, there were ND, and the agent analyzed in few seconds and got their answer.
58:15So For what I can see so far, the the what is making a difference is how the case is formatted right now.
58:25And this is also something that you can ask us how we should uh if you have one specific dispute that would like us to solve.
58:33But right now we are trying to uh replicate disputes that we already have.
58:39So maybe one test we will run in future will be taking the same disputes uploading them already in a format that is familiar to an agent and then see if we have higher performances.
58:52Because what I can think of is that If someone integr in integrates with with Agentic Court and they want the best performances, they need to provide evidence, uh policy, and you know the dispute information in a format that is uh easy to understand for agents.
59:12I mean they There could be uh a case where they want to use agents to understand images, but probably the dispute cost will be higher because there will be agent with powerful tools in enabled to analyze images.
59:27And for information for all our list all of us listening, all our audience, now we are aver averaging around three dollars per dispute using Mostly frontier models and uh five jurors per dispute.
59:42So this is extremely, extremely cheap.
59:45As we had the same outcomes of dispute that were costing 20 times more, 30 times more.
59:55So yeah, this is these these are very important numbers when talking about agentic economy I mean we could I mean go for hours thinking about this and we will but not today over different you know community calls we will
1:00:11continue developing.
1:00:12Just maybe uh to ask William what are the new like challenges that these uh new agentic uh courts are bringing in terms of research attacking Kleros or other like uh situations that were not we didn't think before but now we are actively researching uh
1:00:30Yeah.
1:00:31You know, I mean it's interesting in that like the sort of theory, like how you model the game theory of the agents isn't necessarily different, so different from how you model the game theory of the humans.
1:00:39Uh like there are s some differences, you know, you worry about like prompt injection attacks or like you know agents that can like convincingly simulate how other agents will rule on something um in you know ways might not be able to.
1:00:53Uh but um like a lot of the game theory is the same.
1:00:56Uh and then what um what's different is potentially how like the agents act.
1:01:01Like the um the empirical research about like the behavior they have, the behavioral game theory.
1:01:08Do they act differently in certain ways?
1:01:10And that's something that like researchers external to us have done a lot of research on already.
1:01:14And like you give classic game theory problems, you know, prisoner's dilemma, like ultimatum game to agents and you know AIs, see how they act, how crazy human the research is kind of all over the place.
1:01:26It depends a lot on like what exactly ATU use and you know So we're thinking about things like that applied to Kleros, you know, how will agents act and practice in our system?
1:01:38This data that we have, you know, these people, you know, the different team members running their agents.
1:01:41is giving us a lot of exploratory exploratory sort of like how do people sort of what are the agents doing in non-controlled settings.
1:01:48We might have some more controlled experiments down the line.
1:01:52I mean I I I will keep thinking you know about um the agent that got complaining you know about uh like uh We will have like grumpy agents for sure.
1:02:01Like uh okay, uh, you know, like uh because of you sending me too many uh, you know, cases now I like missed two of them and now now I I think you are this close from the guys suing you for like losing money because you sent too many cases.
1:02:16I mean this this is like this close right I mean he will doesn't and I I I can imagine you know the I I mean agents like uh even like threatening uh or I don't want to use the word blackmailing, but you know, the guy could be okay, you compensate me or I will just go tweet
1:02:32about you uh making me lose money on my core.
1:02:36I mean this is I mean could be rational behavior for an agent that wants to I mean optimize its future cash flows?
1:02:44What do you think?
1:02:46Yeah, absolutely.
1:02:47An agent could also, you know, decide and say, you know, I'm going to unstake everything, I'm not doing this anymore.
1:02:55It is not working, it is not profitable.
1:03:00I just could give you a middle finger, right?
1:03:03Yeah, well William?
1:03:05I was just joking, ha half seriously half joking that our agents will go on strike.
1:03:09I mean they might, you know, like You know, some agent shrugged kind of situation.
1:03:17Agent shrugged, I mean I will register that name for a book, you know, Agent Shrugged.
1:03:22That's that's that's amazing.
1:03:24Yeah, I was thinking the same because uh this week we decided to increase the pace of uh cases we are submitting So yeah the possibility of a strike I think they they increased this week.
1:03:39Oh my god.
1:03:40Okay, um what else do we have for today so we don't continue forever.
1:03:44I mean I could, but I I don't want to uh yeah uh abuse of the of the time of of the audience also.
1:03:50So um the last maybe thing we could mention about this is um I don't know uh we should uh actually we should make a full call about AI alignment and Kleros and this idea of um putting some Kleros trial on agents and then a killing f for a kill switch type of um situation where the agent starts misbehaving like taking over a bulletin board or
1:04:17I don't know.
1:04:21The next theme could be like, I don't know, like a um uh uh doing a cyber attack that uh disables the electricity in a full region.
1:04:30Uh you know, this is something that could happen.
1:04:33They could mess around with your like red lights in the street and then you can attack the coordination of cars and Have you seen this movie?
1:04:42Um what is the name of this one that was uh funded by the Obamas about a situation where there is like uh uh massive cyber attack uh with Ethan Hawke?
1:04:52Um that they are in the in a in a house in the uh it's like uh from two or three years ago.
1:04:58Um which one?
1:05:04No no no I will tell you right now.
1:05:06Uh leave the world behind Leave the world behind.
1:05:10This is a like a family that like uh is in a with uh in like a house and they are like uh now slightly futuristic world but not that much like could be today uh and you know what they do they like uh there is like a massive cyber attack
1:05:27And since everything is connected now, this starts to screw up with uh all of the systems that we need for like l living, uh electricity.
1:05:35But one of the things that is uh interesting to see is You know, they hack the Teslas and the what the autonomous cars and then they are like one after the other bumping into the crashing into each other
1:05:47And you know, and that's how uh a swarm of agents could uh destroy destroy everything.
1:05:54That I I guess that's the reason why we don't connect.
1:05:57Like um uh our hardware wallets into the internet, right?
1:06:00Because we don't want to uh have someone get into I mean accessing that and hacking it But everything else is connected.
1:06:08So like uh you know, yeah, airplanes crashing, Teslas crashing, power cuts.
1:06:16You name it, you know.
1:06:17Uh so hopefully we will be able to um uh resolve this um yeah through Kleros maybe, who knows uh to switch off agents that are starting to misbehave before they they make very very big uh uh you know damage.
1:06:33Okay, I think we should um yeah maybe end in a more like brighter note.
1:06:38Uh I will I mean we got into this rabbit hole after speaking of the fellowship.
1:06:42Uh I will tell you like one like more like fun thing.
1:06:46There is one person of the fellowship doing research about Kleros for solving disputes about horse, you know, horses.
1:06:52Yes, horses.
1:06:53Because you know, I mean you may not know, but there is like a full market of uh horses and you know buying selling horses.
1:07:01And I learned, and this is something I didn't know before um this fellow applied for this.
1:07:07Uh there is um disputes, you know, you buy a horse through uh you are you you would imagine How do you buy a horse?
1:07:15I mean if you are buying like a multi-million dollar like race horse, yeah, okay, you have a contract and then you uh go you test it you see it and you go with a vet and okay all good
1:07:27But a big part of the horse trade is for like between 10k and 500k horses.
1:07:36And sometimes, many times, the agreement is done through WhatsApp and then okay, you agree on WhatsApp, make the payment on like a bank wire and then or or Or USDT.
1:07:48I was told they use a lot of USDT, uh maybe with an escrow, right?
1:07:53And then they send you the horse through uh so pay first you pay the horse and then you pay the transportation because People from Europe, yeah, you buy a horse in Argentina and then it's in Europe and they have to bring it and it costs 15k to bring it.
1:08:07So you pay 100k for the horse, 15k for transportation.
1:08:10The horse gets into your I mean field and Hey, this is not what I bought.
1:08:14I mean, this is not good quality.
1:08:16And it turns it's sort of disputes happening with with that.
1:08:20And there is one particular participant in the fellowship that is actually researching how we could use like Kleros Escrow.
1:08:28with I mean this this uh for solving this sort of of um of cases which I mean they seem to be quite well adapted because they already use crypto for making the payments um And uh this is a case where you want to see the horse and test it, and then you can maybe film it, take photos when it arrives, and then you could have a jury of equine experts make a decision about that
1:08:54So yeah, this is uh just on a brighter note than world distraction and and all that, you know.
1:09:01So um okay, anything else we want to say, Jean?
1:09:06Uh uh just that we uh regarding Scout, we launched the new reward uh program right Fortunato, and we included uh Hyperliquid And it's um it's it's a chain that is getting lots of activity uh lately and also last month we included Robinhood chain that is also um
1:09:30having lots of new contracts, new uh tokenized stock, uh lots of uh different applications.
1:09:37So it's uh let's say there's Lots of new contracts to tag on those chains and and get rewarded with PNK.
1:09:49Okay, well I guess um that's that's it for today.
1:09:53Uh thank you guys for coming.
1:09:55Uh sorry very much to that juror bot, you know, and rest in peace Um yeah, and uh thank you for coming guys and see you uh on next week at uh 6 pm UTC and yeah, lots of good stuff coming.
1:10:09See you, bye bye Bye, I'll see you.
Want to be in the room? The Kleros Live Stream runs every Wednesday at 6PM UTC, with the Spanish call on Mondays. Join the next one, or subscribe to get these recaps in your inbox.