Ye Old Echo Chamber

Ye (formerly known as Kanye West) is buying Parler, a social network that some describe as an alt-tech alternative to Twitter (focusing on the conservative side of the political map), and others as a massive echo chamber of right-wing loyalists. 

But even among right-wing social media, it hasn’t done all that well:

“But it has struggled to build an audience. According to Similarweb, an analytics company specializing in web traffic and performance, Parler’s ranking compared to other platforms popular with conservatives has been decreasing over time. It saw just over 1.2 million visits in September, compared to GETTR, with over 7.1 million visits, Trump’s own Truth Social, with over 8.9 million visits, and another conservative site, Gab, with over 12.8 million visits.”

While Parler is an echo chamber, it’s not the only one. 

In a recent paper, researchers looked for evidence of echo chambers, which they defined as:

“online environments in which users’ opinions get reinforced by interacting mostly with like-minded sources. … a quantitative definition, relying on two dimensions: each user’s leaning on a given topic, and direct interactions between users.” 

The main result of the paper:

“The group found higher segregation on Facebook than on other platforms, and a clear distinction between social media with a feed algorithm tweakable by the users (such as Reddit) and less tweakable ones. Facebook, for example, does not offer a simple chronological option to see what your contacts share. Twitter lets you opt out of the algorithm by choosing to see ‘most recent’ tweets. Reddit and Gab work differently, and the former, in particular, relies less on algorithms and gives users more freedom as to what posts they see.”

So the more involved the algorithm is in curating the feed that users see, the more of an echo chamber it is. Not a causal argument but an interesting observation. 

But this is not only true for social networks. The same goes for social media that are managed by algorithmic curation. A recent article shows that similar issues occur on YouTube:

 “Ideological echo chambers refer to a distribution of videos recommended to a user that is both ideologically homogeneous and centered on the user’s own ideology. … We find that YouTube’s algorithm pushes real users into (very) mild ideological echo chambers. As we can see in the figure, by the end of the data collection task (the top part of the figure, traversals 15-20), liberals and conservatives received different distributions of recommendations from each other (we see that the three different color distributions in the top part of the figure do not perfectly overlap) in the direction we would expect: Liberals (blue) see slightly more liberal videos and conservatives (red) see slightly more conservative videos.”

The differences were not huge, but statistically significant. In a polarized world, small differences magnify over time. 

The Question 

First, let me be clear that I don’t blame these algorithms alone for creating these echo chambers and the increased polarization we observe in society. In a paper I co-authored with Kimon Drakopoulos and Vahideh Manshadi, we show that consumers of content who are trying to decide which information to consume, when given limited time but a constant barrage of information, will choose to consume information that they consider more credible. This sounds trivial. The problem? We evaluate what’s more credible from our point of view. So, we tend to consume information closest to our point of view. 

In other words, we show that confirmation bias is not a bias but rather a rational decision made in a world with too much information and too little time. And indeed, another recent paper shows that a big part of the polarization we observe on social networks is driven by partisan sorting.

But this is a given. Humans will always be human. This post aims to show that the algorithms can be tweaked to correct and reduce polarization while also making these places more engaging. 

So what role do these algorithms play in creating these echo chambers?

Algorithms provide recommendations on how to rank the feed. Usually, these feeds are ranked based on the content that the platform (or the algorithm) thinks you’ll like. Why? Because years of optimizing retail assortments (e.g., Amazon) and entertainment content (e.g., Netflix) have taught algorithm developers to focus on “feeding” consumers what they are more likely to enjoy. 

But this orthodoxy is exactly what is worth challenging: In a discussion-type platform, showing people what they like to see hurts engagement and creates a more polarized society. 

What Really Drives Engagement? 

In a recent paper I co-authored with Yoni Gur and Joseph Carlstein (a doctoral student who is on the market, in case your institution is hiring), we focus on platforms that facilitate discussions (like Facebook and Reddit) but in a slightly different setting: one in which there is more control over who is participating. The setting is a class discussion board or a workplace discussion forum, and we begin by asking what drives more engagement. 

We use data from ment.io, a firm that runs an interesting type of discussion board: for every question (which is posed by faculty or students), there are multiple answers offered by other students. While this may sound similar to other forums, the main difference is that the only way to respond to an existing answer or comment is to first indicate whether you agree or disagree.

This has two effects: it forces people to read what others have written (which is quite rare these days), and helps score the “support” each answer receives. Ment.io creates a Bayesian tree that results in a score for each answer. This score indicates the level of “support” for the current answer. Imagine that every answer has a score of 5/10 when it is initially posted. With the first agreeing student, it will go up to 7.5, for example. When the next student disagrees with the agreeing student, it will go down to 6.75. Things are a bit more complicated than this, but that’s the gist of it. 

Ment.io displays users as they enter the discussion, and presents them with the two most popular answers, along with their scores. Respondents are then ranked by their scores. So the added benefit of working with Ment.io was that we could study how users behave in such a dynamic setting, with the additional data on the level of support of each response (and not only its engagement). Of course, our data include all interactions as well as the evolution of many discussions across different environments. 

Our first task was to identify what drives engagement*, so in our empirical study, we found that the main drivers of engagement are the Score (naturally), the Ranking (of course), as well as a new metric (that we seem to be the first to identify), the Consensus Effect. 

*NOTE: It may seem we are focused on engagement, but that’s not the only goal. Some people use these discussion boards to get to the “right” decision or answer while others aim to achieve a divergence of opinions and raise as many opinions as possible for discussion to emerge. But regardless of the final goal, increased engagement is necessary. Some think it’s better to keep the discussion among users who are knowledgeable or informed, but even then, it’s still about more engagement (but of a subset of users). 

What is consensus? We all have some idea about it, but how can we measure it? We find that consensus is the variance between the support of the responses. In other words, if one answer has a score of 10 and all others have zero, then there is a clear consensus. If all responses are around 5 (or even 10), there’s no consensus. 

But what does it mean that consensus is an important engagement driver? By accounting for the level of consensus at any point in time, we can better predict engagement in the next period. 

In fact, we find that for most groups, the lower the consensus, the more engaging the discussion. To bring this back to the notion of the echo chamber, we show that discussions which created less of an echo chamber drove more engagement. Note that this is not only an argument about correlation. We show that at any point, and accounting for all other drivers, discussions that exhibited a lower level of consensus were more likely to allow someone to engage either by commenting on an existing answer or by voting. 

Note that the notion of consensus is new and doesn’t exist in settings such as retail (where it’s one product at a time) or entertainment (one clip at a time), and in that sense, it’s unique to discussion-based platforms where there is a feature (consensus) that brings together different elements. 

Better Feed Design

Knowing that people are more likely to engage with discussions that, as a whole, exhibit less consensus, we ask: How should discussion boards and fora structure their feed ranking? 

The most important result of the paper is that any ranking that promotes an echo chamber actually drives less engagement.

More precisely, we show that any algorithms that (only) rank answers based on previous engagement, score, or recency of the answer won’t be optimal. Any ranking that doesn’t take the level of consensus that emerges into consideration, is going to be sub-optimal.

The issue is that the optimal algorithm is very hard to compute in general, let alone in real-time. 

So we design a new algorithm that is consensus aware: it not only looks at the engagement in the current period, but also at the implication of the ranking on the behavior in the next period of time (where we usually prefer less consensus). The result is that ranking is no longer always sorted by score. The best option is to show people comments they may disagree with, or answers that the entire community may not agree with. 

Note that this is a limited statement regarding the existing algorithms. I am not claiming that they do or don’t drive engagement. I’m merely suggesting that if you want to drive more engagement (as a means of achieving other ends), you should avoid creating an echo chamber.

To test the viability of this algorithm, we run a randomized field experiment with Ment.io where different discussions follow different rankings. We find that the discussions that were ranked by the “consensus aware” algorithm had significantly higher discussion levels, and 25% more comments than those whose answers were ranked and displayed in the order of their scores. 

A very significant and reassuring result.

To summarize, when building an echo chamber, two things happen: you deter people from “the other side” from even checking the platform, and you drive less engagement among those who are already there since there is more consensus in the emerging discussions. 

By showing participants a diverse set of opinions and promoting a balanced discussion, you create not only a more informative discussion, but also a more engaging one. 

So, Ye reader: if you are reading this, call me. I know how to help you.