DaveWentzel.com            All Things Data

How to Deceive with Charts and Graphs

(Graphic courtesy of http://xkcd.com/795/)

There are 3 types of lies - lies, damned lies, and statistics.

This quote is sometimes attributed to Mark Twain or Benjamin Disraeli, a famous PM of 19th century Great Britain.  My next few blog posts will show, using real world examples, how misrepresenting data occurs all the time, how to spot it, and how to avoid it.  Most of the examples involve finance and investment since this is near and dear to my heart.  And some examples use charts and numbers from load tests we performed using LoadRunner. 

For today's post we will focus on...
Using Statistics Inappropriately or For Qualitative Analysis
Often you will read a newspaper article about someone killed in an auto accident.  If the person was not wearing a seatbelt this will likely be pointed out in a fashion that uses this as evidence of the value of wearing a safety belt.  Usually a quote from the local rescue squad chief states his chances of surviving would have been higher had he been using a safety belt.  Often the quote is presented in such a way that the reader is led to believe the driver who doesn't wear a belt is more careless than the driver who does wear one
But does the data support this?  Here is a case where raw numbers/statistics can't tell you that information reliably.  We can't know if the driver is normally a careless driver or not.  Let's say you get into your car on a rainy night and discover your seatbelt doesn't work for some reason.  What would you honestly do?  Would you say that since the seatbelt doesn't work you will not drive?  Or are you more likely to say that you will drive more carefully?  If you say the latter then you are implying that you drive less carefully when you do actually wear your seatbelt.  In that case I would argue a person like you should NEVER wear a safety belt.  
Another example is drunk driving.  Everyone disagrees with me on this point.  If I am pulled over at a DUI checkpoint and blow a 0.10 I will be arrested for drunk driving.  Most people agree that I should be arrested.  The theory is that when I am drunk I am impaired and hence a menace to others on the road.  Statistics show that drunk drivers kill more people than sober drivers and we should encourage sobriety driving.  I agree that the statistics do show this, however, I disagree that drunk driving makes me instantly (more of) a menace to others on the road.  In fact, many people drive drunk, albeit infrequently, and qualitatively do drive more safely than when sober.  Why would a rational person ever drive drunk?  A person does this knowing full well that what he is doing is wrong and dangerous and he tend to be more attentive, usually telling himself the whole time, "if I get through this I'll never drive drunk again."  Not every drunk driver does this, and not every rational drunk driver really is as attentive to the road as he thinks he is.  Please, don't misunderstand me, I don't condone drunk driving, but the case can be made that this person will actually drive safer when drunk because he knows he is impaired.  So again, the statistics might lead you to make a qualitative judgment that would be erroneous.  So some people then say, "Well, how do we charge drunk drivers who commit vehicular manslaughter?"  Quite simply by charging them with vehicular manslaughter!  Well then, isn't the fear of being caught drunk driving at a DUI checkpoint a deterrent?  Possibly, but a seriously doubt it.  The "occasional" rational drunk driver I outlined above certainly wouldn't be deterred.  
Let's assume I like to text while driving.  Studies also show the reaction time for drivers while texting is actually worse than for drunk drivers.  Yet the punishment is far less severe.   In my opinion there should be no drunk driving laws.  If I kill someone on the road due to negligence it should not matter if it was because I was drunk, hungover, texting, putting on makeup, adjusting the radio, getting directions from my TomTom, or anything else.  
Be careful when using statistics to prove qualitative arguments.  In my opinion this is a borderline lie.  Some presentation-skills consultants even suggest that every statistic should be backed up with an anecdotal story to make your point even more boldly.  If that doesn't prove my point I'm not sure what will.  Remember that anyone can manipulate data to get a statistic that proves his argument.  That doesn't make the data any more or less accurate than another study.  Be especially wary of scientists and engineers who spout statistics to make their case, they rarely understand what they are talking about.  More to come.  


Well, yeah, but this is an apapel from a State court decision involving the interpretation of the 4th Amendment. Since the Fed'l Constitution provides the floor for rights, if it violates the 4th Amd., then it violates state law as well.

Add new comment