The Verge published an interesting survey a few weeks ago about the popularity of the big tech companies and the level of trust towards them. The topic is really exciting, especially when one of the top charts states no less than ’People trust Amazon almost as much as their bank’.
/cdn.vox-cdn.com/uploads/chorus_asset/file/9547237/Amazon_01_01.jpg)
I know from personal experience this is the type of question that keeps banking executives awake during the night. Unfortunatelly the conclusion here is just as useless as it is bold, and suffers heavily from a handful of biases.
Issue 1: Distorted visualisation
First, this visualisation shows nothing of the message in the title. The only information this chart can clearly convey is that banks had the highest rate of people who did not contribute an answer (the rightmost yellow bars). All the other bars are disaligned and distorted by these meaningless answers and there are no labels to the rescue either.
I tried to decode the percentages by taking the pixel width of the bars (for the sake of simplicity I counted ’Greatly’/‘Somewhat distrust’ categories together). It turns out that actually Microsoft had the highest rate of ’Greatly trust’ answers in the survey, and Google seem to be ahead of Amazon in this aspect, too.
If we adjust the percentages for the people who did not have opinion, the actual ratios of (usable) opinions would look something like this:
Amazon, Microsoft and Google has almost the same rate of trust (53%, 52%, 52%, respectively) amongst people who can decide whether they trust them or not, and they are visibly behind banks (58%) in this respect. Mind you, this adjusment has a tricky bias too, as it might selectively exclude some (but not all) answers of certain individuals, and would also decrease sample size (which in turn increases uncertainty).
Issue 2: Too much noise, too weak signal
Uncertainty is a nasty beast, that eats survey results for breakfast. No matter how we try to fix the visualisaton, these percentages really mean nothing if we take into account the uncertainty of the survey (emphasis is mine):
This survey, conducted from September 28th to October 10th, included 1,520 people nationally representative of the US, based on 2016 US Census estimates. The margin of error is ±3 percent, with a confidence level of 95 percent.
So the differences between the original percentages for Microsoft, Google and Amazon all fall in the margin of error, and things get even blurrier with my rudimentary adjustments, as I could easily miscalculated by as much as half or even one percentage point.
This all means one thing: based on this survey we actually don’t know which of Amazon, Google, Microsoft or banks in general has the highest level of trust. The numbers for these companies are virtually indistinguishable from each other.
Issue 3: Confirmation bias
But for some reason the creator of the chart still wanted to single out Amazon as a contender for banks. Not Microsoft and not Google, even though they are likely in the same ’league of trust’. Why?
We will never know for sure, but maybe because in the breakout report concerning Amazon results also seem to deliver some positive messages on the company, and this chart had to fit in, even by misrepresenting the facts (not necessarily on purpose, this is just how our mind works).
Do not trust surveys
The conclusion here is not that trust towards Amazon is not comparable to banks and not the opposite either. I don’t know anything about that.
The takeaway is that we should not trust surveys packing blockbuster conclusions into distorted charts, based on questioning a handful of people on some mostly unconscious feelings.