By Chet Yarbrough
Written by: Charles Wheelan
Narrated by: Jonathan Davis
As the title Naked Statistics implies, Charles Wheelan strips the clothes off social, economic, and scientific studies that use statistics to prove a point. Wheelan expresses confidence in probabilities, based on statistical analysis, while cautioning the public. Wheelan eschews blind acceptance of reported conclusions based on faulty statistical analysis. In choosing this audio book, prepare to learn the vocabulary of statistics.
Wheelan explains why Mark Twain suggested “Facts are stubborn, but statistics are more pliable.” When organizations like the Pew Research Center report on political discontent, the safety of measles vaccine, or how scientists engage the public, they are sampling public opinion and drawing conclusions.
What Wheelan clarifies is how important the population sample becomes when proffering public opinion as a basis for truth. Wheelan suggests questions and studied answers are required when a researcher uses a public opinion to make a point; e.g. is the sample population representative of the entire public? How many people are needed to represent a complete population? What questions must be asked to fairly reflect public opinion being proffered? If the sample is not representative, the conclusions are unreliable. If the sample is not large enough, the conclusions are unreliable. If the questions are not focused on the subject and commonly understood, the conclusions are unreliable.
Wheelan, though a strong proponent of statistical analysis, offers evidence to shake one’s confidence in any conclusions drawn by statistical analysis. Shaking one’s confidence is not Wheelan’s intent but the variables in statistical analysis seem legion and often un-quantifiable. It reminds one of S. I. Hayakawa, the language expert, who wrote a book titled “Language in Thought and Action”. Hayakawa notes that a question is framed by a speaker that has little control of how a listener understands what is being asked.
The meaning of words is frequently misunderstood because of differences in culture. Being able to frame questions, so that all socio-economic classes understand the question in the same way, makes public surveys inherently suspect.
Add to that concern, the multiplicity of variables that actually measure public opinion and confidence in analysis falls and skepticism rises.
Another area of concern is Wheelan’s frequent use of intuition as criteria for statistical analysis. As Kahneman writes in “Thinking Fast and Slow”, intuition is a form of “fast thought” that is frequently incorrect. One becomes suspicious of statistical interpretation based on intuition. However, here is where Wheelan begins to argue for a better understanding of the process of statistical analysis. Wheelan argues that intuitive guessing is validated by probabilities, based on proper procedural statistical analysis.
Wheelan begins explaining the vocabulary of statistics. Words like “sample”, “mean”, “median”, “regression to the mean”, “regression analysis”, “null hypothesis”, “standard deviation”,
“standard error”, “t-statistic”, etc. are defined. The definitions become somewhat overwhelming but Wheelan makes the point that “reproducible result” is validation of all scientific exploration; including statistical analysis. If proper procedure is followed, statistical analysis is a critical part of scientific progress. In other words, read the fine print (procedural documentation) when relying on the conclusions of a report based on statistical analysis.
Wheelan’s book is spiked with interesting vignettes of statistical analysis gone astray. It is a good introduction to, and cautionary tale about, the science of statistics. Wheelan reinforces the fear of information accumulation and availability that can mislead the public as well as invade privacy.