Kialo wants to change online debating. Does it work? A hands-on.

Some weeks ago, my Facebook feed suggested me repeatedly to visit a startup website called Kialo and join a conversation on prison sentences. A guy that I know from a few years ago had endorsed the app. I followed the link to see what it is about.

Kialo is sick of the internet shouting factory (sic) and aims to be a new platform for online debating. It’s founded with the intention to stop hateful discussions that lead to nothing but ideological manifestation and segregation. That’s a great ambition. It’s at the very least an interesting approach. Here’s a quick video overview.

How it works

Basically, Kialo introduces a clear hierarchical structure. Each debate starts with a proposition. Arguments can be brought forward to support or reject the proposition. Each argument in itself constitutes another proposition that can receive support or rejection one level lower.

On the surface, any identifying information is removed from a proposition. Users need to click on a small icon to see who “suggested” a proposition, and who “accepted” it (that’s usually the moderator). There, other users can also comment on the phrasing, and these comments document any changes that are made to the phrasing of the proposition. I see a risk that this leads to micro-debates within comment threats that will be invisible. Like in this case.

This one fitted in one screenshot. The longest threat has 19 comments on a proposition that some crime is not anti-social. They read pretty much like the average online discussion minus insulting.

Participants can vote on the impact strength of a proposition. Statements with more impact are listed on top. They can also flag statements because they are: unsupported, not a claim, unclear, vulgar/abusive, duplicates, unrelated, or not sincere. Can that really work?

See it work

It’s best to go through one particular threat of one conversation to talk about how the dynamics of such a debate evolve. So, we start with the original proposition. I number the statements to be able to reference to them later.

(1) The primary focus of prisons should be rehabilitation and reintegration, not punishment.

I now select the first rejection to this proposition. It reads.

(2) The most essential element of any prison system is that the convicted criminal is removed from society while they serve their time. As a result, it is not possible to construct a system that does not primarily focus on punishment.

From my perspective, this argument simply states the opposite assumption to the opening proposition. Some people will consider (modern) prisons to be places of rehabilitation under supervision. Others will not. I also notice that the term punishment is used with different meanings. I infer that the opening statements talks about punishment as a continued series of actions that take place in prison – on top of being incarcerated. In the rejection, the act of incarceration is what constitutes the punishment. Anyways, this rejection is followed by one supportive statement and a number of rejecting statements.

Here’s the one in support.

(3) While I agree that rehabilitation and reintegration should be attempted whenever possible however punishment is a factor of this. In my personal opinion those in prison for drug crimes should be rehabilitated and reintegrated, we don’t punish people with other chronic diseases, so why should we punish those who suffer from the disease of addiction?

To this date, this statement has not been flagged, even though the first statement is essentially an unsupported claim. It was originally a statement rejecting the original proposition (1), then was moved to this position because the moderator wanted to reduce the number of top level claims. The moderator also adds a comment to the author that the first two sentences are two separate claims, and that in fact the second sentence supports the original proposition. The author never follows up on this.

Proposition (2) is also followed up with a large number of rejecting statements. For instance, there is this one.

(4) Separation from society is only punishment if performed in the spirit of punishment. Punishment is specifically a type a negative intervention designed to give wrongdoer what “xe deserves.” Even in a rehabilitation model, some interventions will be unpleasant for a convict. But to say punishments are unpleasant, separation from society is unpleasant, therefore separation from society is punishment is fallacious. It is akin to saying dogs have claws, cats have claws, therefore dogs are cats.

This statement was originally posted as a supportive statement to (2). The moderator made it a rejecting statement. The author very likely was confused and picked “supportive” because he supported the original proposition (1). In terms of substantial influence, this proposition hints at my earlier statement that both (1) and (2) are assumptions about the paradigm of prisons that one can agree or disagree with. There will not be a solution for this debate other than figure out what is the paradigm that the majority accepts.

There is a fourth level following up. One statement in support of (4).

(5) Being separated from society does not necessitate punishment, e.g. boarding schools, communes, living “off the grid”.

And one in rejection of (4).

(6) Humans are ultimately primates, homo sapiens largely are reliant on their social groups. You’re saying that in the “spirit of punishment” separation from society is punishment. Yet isolation-solitary confinement is proven psychologically damaging, indicating the further from society and others we are, the more unhealthy we will be. Without a rehabilitative model the criminal is ostracized, isolated from their learned society, and often adapt to their new one inside prison walls.

Neither (5) or (6) are moderated at this point. Remember, (4) was about the idea that separation is not punishment if not executed as punishment, but as rehabilitation. (6) says that separation is always punishment and therefore needs to be accompanied by rehabilitation. Would I flag it? It’s not strictly unrelated. It’s just related in the wrong way. It’s the last one argument in the threat.

In other threats of the same conversation, I encountered again that supporting and rejecting statements were not always marked correctly. Here is another example where one statements is followed by a supporting statement that actually rejects it. In this case, the moderator didn’t move it. This just really makes it very confusing do deal with the hierarchical system.

Another threat includes this statement.

(7) Humans are not the arbiters of justice. That is a role reserved for God. We should focus on reducing the number and scope of evil actions for practical reasons, but it is not up to us to decide what is a just punishment. Instead, we should seek an effective punishment.

This statement is in support of a statement that supports (1). It is, however, another paradigmatic statement that can only be supported and rejected based on opposing views on the correct paradigm. However, that makes debates rather useless, because the best outcome will be to agree to disagree.

In this debate, most branches of the hierarchy went no lower than level 3 (level 1 is the original statement). Some went down to level 6, and only a handful moved beyond that. The lowest level argument that I saw was 12 (it’s the con argument on the lower right – I show it in this way so the context is still rather clear).

It’s actually a fair point. Reading it in isolation, it seems entirely unrelated to the original proposition. It is not. Kant was used to back an argument in support of the original statement, and a debate evolved about how valid this use of Kant was. Of course, that can be a point of debates. The strength of Kialo is that there is a place where these aspects can be debated as well. The weakness is that a normal user will see it. I arrived here only because I did some statistics.

And in other threats, I wouldn’t say that all these side discussions really add to the original discussion and are not simply off-topic. On level 8 and 9, this debate also discussed gun control. This is as unproductive as it is on any other platform. Kialo’s advantage is that to this date its users are disciplined enough to make only small statements that are rather easy to reply to with yet another short statement.

Working numbers

I then was interested to get some data for this particular conversation. Essentially, I created a data set that includes all propositions, their level in the debate, whether or not they were positive or negative, how they were rated, how many comments they received, if they were flagged, and how many positive/negative comments followed up on them on the next level.

What I do not capture is the sentiment of text entries. I can’t copy/paste the text easily and I’m not an expert in natural language processing anyways. However, the tone is more heated in those discussions that start with a rejection on level 2. I also can’t quantify you exactly how much people made references to Norway, but they made these references a lot. It was basically the only international reference that appeared. I suspect, but haven’t verified, that all these references come from the same (Norwegian?) guy.

The amount of propositions in this debate was massive. I’ve counted 430 reactions to the original proposition. The initial proposition (level 1) was followed by 12 positive and 14 negative propositions. The bulk of reactions came on level 3 (82/55) and on level 4 (36/79). While level 3 was very much leaning towards supporting propositions on level 2, on level 4 and on lower levels rejections were much more common than support. Between level 5 and 12, supporting propositions account for only 20% of all propositions.

Put simply: at some level of detail in the debate, this conversation just is a discussion like any other, in which opinions confront.

167 propositions (incl. level 1) are supportive, while 264 propositions reject the proposition on the preceding level. I’ve split the data into propositions that follow up on supporting propositions on level 2 (221) – the green side of the tree – and those that follow up on rejections on level 2 (183) – the red side. While more propositions are made on the green side of the tree, the ratio of supportive to rejecting propositions is markedly different. On the green side of the tree, it’s all rather balanced; there’s a minor majority of rejections. On the red side of the tree, rejections are almost three times as frequent as supporting statement.

My take on this: If at all, the debate that Kialo dreams of takes place on the green side, between level 2 and 4. On the red side, the debate is much more ordinary: one viewpoint collides with the other.

Finally, I wanted to know which attributes of a proposition might make it more likely that it’s the last in its branch. Is a post more likely to be the last if its parent post already did not have much debate? Do comments on a post indicate interest that will also be reflected in more follow-up propositions? Does the impact rating matter at all, or how many sisters a proposition has on the same level? Let’s summarize this as: nothing of this really matters. I’ve tried a few different statistical models (logistic regressions, where the dependent variable states if the observation is a final post or not), and while I found some minor significant values here and there, the model fit statistics are so low that I would not dare to sell a story to anyone. That doesn’t mean there is no story. Just that I’m asking the wrong questions.

With respect to Kialo’s debate performance variables, conversations end random. Branches are always more likely to end than to continue. If at all, consistently positive branches are a tiny bit more likely to end.

Does it work?

I’m undecided about this. On the one hand, I see that through the entire discussion, there are no clear vulgar expressions. However, I think that this is very much the result of sample selection. Kialo’s user tend to highly educated. They are also users who distinctively search for a structured debate that follows the rule of debating clubs. And who debate for the sake of debating. In fact, as the debate evolves, more and more sub-threats are headed, but no where is actually a point where one could visibly say: Yes, you are right.

From my perspective, Kialo is highly artificial and already now suffers from the same overflow of information that kills online debating. For instance, it is possible that a rejection to a specific proposition at the highest level appears again at a lower level. Specific statements can be re-used at other positions of the same debate (or in another debate). Hence, circular discussions are very well possible, and complex discussions require a massive effort of moderators to keep everything linked.

That doesn’t even mention that I need to remember visually which green or red box represents the proposition I had been debating, because without clicking I won’t know. It looks visually appealing, but is effectively not very user-friendly for massive discussions. This debate on prisons had 17 active authors. That’s tiny. I can’t see how it should work if it scales up. To be honest, I can’t see that it will scale up. Internet debates take place where they pop up from nowhere.

I do not see that it’s going to bridge the ideological divide between conservatives and progressives. The prison debate is inherently leaning to the progressive side. That’s not a surprise. Kialo attracts a progressive audience, because they are those who complain about the state of debates. The reason that internet debates don’t work is because most participants want to express their opinion to prove the other opinion wrong, not because they want to listen. That’s not different with Kialo.

I do a see a value in Kialo as a tool during creative processes, where groups come together that ultimately have a shared basis. It’s essentially a feature-rich pro/con list. I see how it can be used to teach debating and logic in an educational context. I see how it can be interesting to read the first three levels. Anything that happens below level 3… I would safely expect to be out of scope of the debate and not to be well articulated anymore.

Hence, I don’t think it’s offering a solution to the internet shouting machine. If I want to shout and I’m not allowed to do it in Kialo, I just won’t bother about the platform. If I’m smart enough, I just troll the debate with formally acceptable statements, only to keep those debate nerds busy. It’s shouting in a silent way. And if I’m not smart enough, I will still just try to defend my opinion with arguments who are not arguments anymore, but just opinions. Such as these two that appear in the debate: