The data divide between perception and reality

It has been a bad year for numbers. At the start of 2016, Howard Wainer, a much revered Princeton-educated statistician, eulogised that “prediction models combined with exit polls let us go to sleep early with little doubt about election outcomes”. By the end of a year in which data — in the shape of facts, polls and forecasts — struggled to deal with an uncertain world, Mr Wainer’s words seemed out of touch with what has become known as the “post-truth” era.

In fact, it has been a bad year for our perception of numbers. People often find it difficult to interpret political polls and the associated concept of probability — a 30 per cent chance of rain does not necessarily mean you will stay dry. And our understanding of the world that we use statistics to measure is also very often at odds with the numbers themselves: perception does not match reality. Not even for readers of the Financial Times.

This month, Ipsos Mori published the latest in its annual Perils of Perception series, a 40-country survey of public perceptions about “key global issues and features of the population”. The survey found widespread social misperceptions, with Bobby Duffy, managing director of Ipsos Mori’s Social Research Institute, writing in The Guardian that this latest set of results reflected that “objective facts are less influential in shaping public opinion than appeals to emotion and personal belief”.

A cocktail of personal experience, circumstances and external influences — from social networks to media and advertising — mean that everyone will have their own perception of reality.

Fascinated by the Ipsos findings, David Blood and Ændrew Rininsland at the Financial Times devised the “How well do you really know your country?” quiz, challenging FT readers to compare their own perceptions of their country with both the public’s perceptions (as provided by the Ipsos Mori survey) and the “actual” figures used by Ipsos Mori.

Some two weeks after the quiz was published, and with thousands of results from FT readers across the world, we decided to take a look at some of the emerging patterns.

We extracted FT reader responses for each country/question combination for comparison with Ipsos’s figures, excluding those countries with a low level of reader responses. We then used the median figure for each question for comparison, as this is more resistant to extreme individual guesses influencing the average.

In many ways, that FT readers have a different view of the world from the general public should not be a surprise: the Ipsos Mori survey uses stratified samples to try and provide a “representative” view of each country’s population.

FT readers are not likely to be as representative of the broader population. And the results of the quiz should not be considered as statistically robust as a well-designed survey.

Nevertheless, the quiz does provide a fascinating glimpse into how different groups of people can have distinct and, at times, diverging views of the same reality. Failure to acknowledge this — or the fact that our own social interactions may not reflect the diversity of perception — may contribute to the “filter bubble” effect, or “echo chamber” of similar views.

As many become worried about the rise of fake news, the need for informed debate based on reality, and the various perceptions of it, has never been greater. So this is a good time to ask, “How well do you really know your country?”