(Graphic courtesy of http://xkcd.com/795/)
There are 3 types of lies - lies, damned lies, and statistics.
This quote is sometimes attributed to Mark Twain or Benjamin Disraeli, a famous PM of 19th century Great Britain. My next few blog posts will show, using real world examples, how misrepresenting data occurs all the time, how to spot it, and how to avoid it. Most of the examples involve finance and investment since this is near and dear to my heart. And some examples use charts and numbers from load tests we performed using LoadRunner.
For today's post we will focus on...
Using Statistics Inappropriately or For Qualitative Analysis
But does the data support this?
Here is a case where raw numbers/statistics can't tell you that information reliably. We can't know if the driver is normally a careless driver or not. Let's say you get into your car on a rainy night and discover your seatbelt doesn't work for some reason. What would you honestly do? Would you say that since the seatbelt doesn't work you will not drive? Or are you more likely to say that you will drive more carefully? If you say the latter then you are implying that you drive less carefully when you do actually wear your seatbelt. In that case I would argue a person like you should NEVER
wear a safety belt.
Another example is drunk driving
disagrees with me on this point. If I am pulled over at a DUI checkpoint and blow a 0.10 I will be arrested for drunk driving. Most people agree that I should be arrested. The theory is that when I am drunk I am impaired and hence a menace to others on the road
. Statistics show that drunk drivers kill more people than sober drivers and we should encourage sobriety driving. I agree that the statistics do show this, however, I disagree that drunk driving makes me instantly (more of) a menace to others on the road. In fact, many people drive drunk, albeit infrequently, and qualitatively do drive more safely than when sober. Why would a rational person ever drive drunk? A person does this knowing full well that what he is doing is wrong and dangerous and he tend to be more attentive, usually telling himself the whole time, "if I get through this I'll never drive drunk again." Not every drunk driver does this, and not every rational drunk driver really is as attentive to the road as he thinks he is. Please, don't misunderstand me, I don't condone drunk driving
, but the case can be made that this person will actually drive safer when drunk because he knows he is impaired. So again, the statistics might lead you to make a qualitative judgment that would be erroneous. So some people then say, "Well, how do we charge drunk drivers who commit vehicular manslaughter?" Quite simply by charging them with vehicular manslaughter! Well then, isn't the fear of being caught drunk driving at a DUI checkpoint a deterrent? Possibly, but a seriously doubt it. The "occasional" rational drunk driver I outlined above certainly wouldn't be deterred.
Be careful when using statistics to prove qualitative arguments. In my opinion this is a borderline lie. Some presentation-skills consultants even suggest that every statistic should be backed up with an anecdotal story
to make your point even more boldly. If that doesn't prove my point I'm not sure what will. Remember that anyone can manipulate data to get a statistic that proves his argument. That doesn't make the data any more or less accurate than another study. Be especially wary of scientists and engineers who spout statistics to make their case, they rarely understand what they are talking about
. More to come