Colin Rule - The Godfather of Online Dispute Resolution Speaks to Kleros

Colin Rule is probably the world's leading expert in online dispute resolution (ODR). He created eBay’s user dispute resolution in the 1990s and then the eBay Community Court, the first attempt to build a crowdsourced online dispute resolution system at scale.

FEDERICO: At Kleros, we're building software for online dispute resolution (ODR). But I always say: it's not just about computer software. It's also about updating mental software. What is your perception of your experience in eBay, almost 20 years later?

COLIN: There is a quote from Gandhi which says: “First they ignore you, then they laugh at you, then they fight you, and then you win”. That’s been my experience with ODR.  When you put out new ideas, everybody goes: “wow, these ideas are so futuristic, they’re never going to happen”.

And then slowly but surely history catches up with you. The thing about being a visionary is that people think you’re nuts at the beginning, but slowly people come around. So, I think it’s important to let people come to the realization on their own timeline.

I agree with you: the obstacle is not the technology. You can make the code do whatever you need it to do. It is often the human resistance that is the bigger challenge.

When we first introduced the concept of the Community Court at eBay, people thought: “That’s an interesting idea, but it will never work”. Once we started to scale it up to do thousands and thousands of cases, people said: “Wow, maybe it will work after all”.

There is a lot of unforeseen sand traps that you can get caught in. You gotta figure those out, and (as they say in lean startups) fail forward. Create your concept, get it out there, and then discover all the problems that you didn’t foresee in your original design.

Gameability is a problem. There are going to be a lot of smart people out there who are going to try and find ways to manipulate the process to their own ends. Those aren’t fatal errors, you just need to continually innovate and explore -- figure out seams in your design and then patch the seams.

FEDERICO: What were the main challenges you faced in the process of building the Community Court?

COLIN: When we started the Community Court at eBay, we needed to get jurors, so we put out a call for volunteers. We weren’t sure how many people we would get, but we were absolutely overwhelmed. It was amazing how many people wanted to be jurors.

We actually created a slush fund of $5000, because we thought we would need to pay people incentives, to give them gift cards to have them serve as jurors. It turned out that we never spent one dollar of that, because there were so many users that were eager to be jurors, they didn’t need incentives. We had more jurors than we knew what to do with. We actually didn’t have as many cases as we thought we were going to get, so we started putting extra people on juries, because we didn’t want them to feel we were ignoring them.

But then when we started the Community Court at PayPal, we had the opposite problem. We couldn’t get jurors, no one wanted to serve.

I think it had a lot to do with the communities. eBay was talking about the community a lot; anybody who was a part of eBay felt they were a part of something. Users wanted to give back, because they had a connection to the marketplace and they were making money in the marketplace. So they wanted to donate some service to help keep eBay safe.

Whereas in PayPal, the community of users was just a bunch of different people paying for things. So, if I paid for something with PayPal and you paid something with PayPal, there is no real implied connection between the two of us.  It’s not like we would have a reason to hang out.  There was less of a feeling of community. Whereas at eBay, we would have a reason to hang out: we are both eBay members, we love eBay.

In the cryptocurrency community there is a community there, but is it more like the PayPal community or more like the eBay community? I think, to a certain extent, your success in your initial target market is dependent on the way users see other users within that community.

That is a very interesting concept: people volunteering as jurors because they believe in the idea and want to keep the community clean...

If you get your jury summons in the mail, you feel like you have an obligation – “I live in this community, I’m gonna do this active community service, even though it is a sacrifice for me, I’m gonna do it because I’m bought in”.  John Locke wrote about the social contract: do I feel like I’m a member of something and does that membership obligate me to do something?

I guess that is one of the determinants of the success of these crowdsourced dispute resolution systems, because it defines the mindset of your users. It defines the way that they think about their membership in the community and then also defines whether they feel they have any kind of obligation to give back.

I’m curious to see with your experiments, especially in different target communities, how that works.

FEDERICO: How do you see the future of ODR, since we have these technologies, in particular AI and blockchain? What is your perspective on the future, where we are going?

COLIN: I am actually writing an article with my friend, Amy Schmitz, who is a law professor at the University of Missouri, on smart contracts, blockchain and ODR. I think these are powerful technologies and there is no question about their importance moving forward.

For a lot of the ODR work I am doing, I can see a very clear role for blockchain and for smart contracts. When we’re working with the courts, if the parties can reach an agreement with each other, it should be an easy option for them to memorialize their settlement agreement in a smart contract and drop that on a blockchain. A lot of the online (and even offline disputes we are tackling with ODR) can definitely benefit from the use of these technologies in terms of creating an automatic enforcement mechanism. So, I am very interested in that side of it.

The side of it where we build in online dispute resolution to every smart contract, I think that is very interesting too, where every smart contract created, just like in person contracts, we have an ADR clause. I can see smart contracts having an integrated ODR clause.

Now that LegalZoom and RocketLawyer are starting to craft smart contracts and put them onto the blockchain, I think it would be very interesting for those providers to automatically include a dispute resolution process. And then we can talk about how that would exactly work -- though I think it could be something like Kleros.

It could be a crowdsourced approach, or it could be an algorithmic approach, like an AI. There are a lot of people hard at work creating these “digital judges”.

ODR is just the use of technology to help resolve disputes. These are powerful new technologies and there is no question they are going to be applied to disputes at scale in the near future.

The real question is: where are we going to hit scale first? There is a lot of infrastructure that needs to be built around ODR in order for smart contracts to operate.

The reason why we are having traction in crypto initially is because most of the players in these transactions have publicly accessible key performance indicators (KPIs) that you can use for smart contracts to trigger off those data sources.

For other things, RocketLawyer or LegalZoom are talking about running smart contracts for wills or employment contracts, and a lot of the KPIs for those types of contracts are not necessarily visible yet. So, we have to build a lot of new infrastructure to get these new technologies to really be useful.

But, I think we are getting there. There is so much momentum behind these technologies, so much enthusiasm; the traction that you’re making is a great example. We are going to make the investments necessary to open up other categories of agreements to these types of technologies and that’s going to mean that their applicability is going to skyrocket.

FEDERICO: What are you working on? What are you researching these days?

COLIN: Through Tyler Technologies, I am very focused on the courts. We are launching ODR mechanisms for small claims, for landlord-tenant, for debt, for family cases,  The Pew Charitable Trusts have announced that they will be putting hundreds of millions of dollars into research into the effectiveness of ODR as a means of expanding access to justice.

So, Pew and the National Center for State Courts (NCSC) have announced they are picking 15 different court ODR projects to work with, so there is a lot of excitement – plus the A2J (access to justice) program at Harvard Law School has announced that they will be doing a big project evaluating the effectiveness of ODR as well.

International expansion is a huge frontier for us. I just wrote an article about ODR in Brazil and presented in Iguazu Falls, and then a few days later I was in Tokyo and presented an article about ODR in Japan. APEC is launching cross-border ODR system for small business disputes among APEC members. So, there is a lot of activity around how we can roll that out successfully, leveraging the work that was done on ODR within UNCITRAL.

I’ve been talking about ODR for years and now it’s actually happening, all over the world. I kind of feel like the dog that caught the car. There are big challenges in bringing it all live, but it’s exciting. It can be a little overwhelming, but I truly feel this is the biggest opportunity we’ve had to expand global access to justice in the past 100 years. But we have to do it right.

Join Kleros!

Join the community chat on Telegram.

Visit our website.

Follow us on Twitter.

Join our Slack for developer conversations.

Contribute on Github.