Stat Spotting (9/2008)
A Field Guide to Identifying Dubious Data

1: Getting
Started
A. Spotting
questionable numbers B. Background (B1 Statistical
Benchmarks, B2 Severity and Frequency)

2: Varieties of
dubious Data
C. Blunders: C1. The slippery decimal point, C2.
Blotched transactions, C3. Misleading graphs, C4. Careless
calculations
D. Sources: who counted and why? D1. Big
round numbers, D2. Hyperbole, D3. Shocking claims
D4. Naming the problem
E. Definitions, what did they count? E1.
Broad definitions, E2. Expanding definitions, E3.
Changing the definitions, E4. The uncounted.
F. Measurements: how did they count? F1. Creating
measures, F2. Odd units of analysis, F3. Loaded
questions,
F4. Raising the Bar, F5. Technical measures.
G. Packaging: what are they telling us? G1.
Impressive formats, G2. Misleading samples, G3.
Convenient time frames
G4. Peculiar Percentages, G5. Selective comparisons, G6.
Statistical milestones
G7. Averages, G8. Epidemics, G9. Correlations, G10.
Discoveries.
H. Debates: what if they disagree?
H1. Causality debates, H2. Equality debates, H3. Policy
debates.

3: StatSpotting on
your own
I. summary; common signs of dubious data
J. Better data: some characteristics
K. If you had no idea things were that bad, they
probably aren't.
L. Suggestions for those who want to continue
statspotting.

Acknowledgements, Notes
and Index
Are four million women really battered to death by
their husbands or boyfriends each year? Does a young person commit
suicide every thirteen minutes in the United States? Is methamphetamine
our number one drug problem today? Alarming statistics bombard our daily
lives, appearing in the news, on the Web, seemingly everywhere. But all
too often, even the most respected publications present numbers that are
miscalculated, misinterpreted, hyped, or simply misleading. Following on
the heels of his highly acclaimed Damned Lies and Statistics and
More Damned Lies and Statistics, Joel Best now offers this
practical field guide to help everyone identify questionable statistics.
Entertaining, informative, and concise, StatSpotting is essential
reading for people who want to be more savvy and critical consumers of
news and information.
StatSpotting features:
* Pertinent examples from today's news, including the
number of deaths reported in Iraq, the threat of secondhand smoke, the
increase in the number of overweight Americans, and many more
* A commonsense approach that doesn't require advanced
math or statistics
Review by Bernie Madison, Univ. of Arkansas: "As we now swim
in information, much of it bogus or biased, spotting dubious data is
super important. In StatSpotting, Joel Best plays off the format
of field guides to give readers good commonsense ways not only to sense
bad data but also to understand what's wrong. Broken up into short
independent sections, the book is easy and enjoyable to read. I
will recommend it to my students, and to others, as a resource for
critical consumers of numbers."
Review by Neil Lutsky, Carleton
college: "Statspotting proposes to help readers become more
critical consumers of statistical claims. It is an important work
that addresses a significant problem in contemporary society:
thoughtlessness about numerical claims. Joel Best provides a
direct, accessible guide to critical readings of statistics."
Review by Alan Jasper, Graduate
Center, City University of New York: "If you ever scan the
newspaper, watch the TV news, or surf the blogs, you should read this
charming book. If you're a journalist, read it twice."
More Damned Lies and Statistics (8/2004)
How Numbers Confuse Public Issues
PDF

Table of Contents

Preface: People
Count

Ch 1: Missing Numbers

Ch 2: Confusing Numbers

Ch 3: Scary Numbers

Ch 4: Authoritative
Numbers

Ch 5: Magical Numbers

Ch 6: Contentious
Numbers

Ch 7: Toward
Statistical Literacy
Ch 7 Statistical Literacy: Excerpts
Shouldn't we be able to teach
"statistical literacy"basic skills for critically interpreting the
sorts of statistics we encounter in everyday life?
what if we call statistical
literacy a basic skill? Certainly a plausible argument exists for
considering it in these terms. After all, we are talking about teaching
people to be more critical, to be more thoughtful about what they read
in the newspaper or watch in a news broadcast, to ask questions about
claims from scientists, politicians, or activists. Being better able to
assess such claims is certainly valuable; we might even argue that it is
fundamental to being an informed citizen. Why not consider statistical
literacy a basic skill?
"Why not consider statistical literacy a skill? But this raises
another question: what sort of basic skill is it?" "This
competition [between departments] means that teaching basic skills often
is devalued." "College instructors are well aware that
substantial proportions of students have trouble reading  let alone
thinking critically about  basic graphs and tables. This is a
very important skill because graphs and tables are certain to appear in
much of the reading a student will need to do in the course of college.
And yet, no one wants to teach this skill, or at least spend much time
doing so. Many have the sense that students should already
be proficient in these skills when they get to college (even though it
is clear that many are not). To many others, it seems to simple,
too basic  a waste of time for professors who would prefer to teach
the more advanced topics in their disciplines." "Thus statistic
and mathematics instructors are unlikely to have any more interest in
teaching statistical literacy than English professors have in teaching
firstyear composition. Nor are other departments eager to teach
this material. I teach sociology courses, but I know that most
sociology professors tend to dismiss statistical literacy as "not really
sociology"; faculty in psychology and other disciplines probably have
the same reaction. Statistical literacy falls between the stools on
which academic departments perch." "The lessons involved in
teaching statistical literacy are not so terribly difficulty; rather,
the difficulty lies in finding someone willing to teach them."
THE STATISTICAL LITERACY MOVEMENT
Despite these obstacles, a small
educational movement advocating statistical literacy has emerged.
Professor Milo Schield, director of the W. M. Keck Foundation
Statistical Literacy Project at Augsburg College in Minneapolis, is the
movement's leading voice. Schield operates the Statistical Literacy Web
site (www.StatLit.org); for those interested in statistical literacy as
an educational movement, the site includes a section on teaching.
Although this is a promising development, the campaign to promote formal
instruction in statistical literacy is in its early phases.
But perhaps statistical literacy
doesn't have to be taught in classrooms. Recently, there seem to be
increasing calls to promote statistical literacy outside the educational
establishment. <snip>
In short, it may be true that
"everyone" agrees that improving statistical literacy is desirable, but
it isn't clear that they can agree on what statistical literacy means,
what improving it might involve, or what the consequences of this
improvement might be.
Even if no one opposes statistical
literacy, serious obstacles remain. There is disagreement about which
skills need to be taught, and, at least so far, no group has offered to
take responsibility for doing the necessary teaching. Plenty of
information is out thereany interested individual can learn ways to
think more critically about statisticsbut the statistical literacy
movement has yet to convince most educators that they need to change
what the educational system is doing.
As things stand, we constantly
find ourselves exposed to lots of statistics. Some of those numbers are
pretty good, but many aren't. As a result, we worry about things that
probably aren't worth the trouble, even as we ignore things that ought
to warrant our attention. Improving statistical literacy  if we can
manage it  could help us tell the difference and, in a small way, make
us wiser.
Reviews:
In this sequel to Best's Damned Lies and Statistics
(2001), the premise is simple: there are vast quantities of statistics
being bandied about in all walks of life, and we frequently rely on them
to form our own opinions about things. Often, however, neither we nor
the experts understand how those numbers work. "People need to agree
about what to count before they can start counting," the author tells
us, explaining why different people often disagree about the same
statistics. Some journalists say childabduction cases are up; others
say they're down; but no one has bothered to agree on what they mean by
child or abduction. Another problem: news media perpetuate inaccuracies
by citing each other's statistics without checking for accuracy. This is
why, for example, we keep hearing that 150 people die every year after
being hit by falling coconuts. (In fact, there is no such statistic
because no one tracks coconut deaths.) The book is packed with helpful
tips for understanding statistics, and it even manages to make a usually
dull topic entertaining. David Pitt Copyright © American Library
Association. All rights reserved Review
"Through his devastating work on common myths about
social problems, Joel Best has established himself as a brilliant
observer of our national fads and scares. In this latest book, Best
confronts yet more of the pseudostatistics by which we are bamboozled
day by day. One obvious question comes to mind. If he can deal with
highly significant topics in such lucid and enjoyable prose, why can't
other social scientists begin to match him?"—Philip Jenkins, author of
The New AntiCatholicism: The Last Acceptable Prejudice
"Joel Best continues to confront us with the delicious
lunacy of statistical gaffes and fantasies. Whether discussing 'deaths
from falling coconuts,' teenage bullying, or likelihood of contracting
breast cancer, Best teaches us to avoid the dangers of statistical
illiteracy. As his cogent and comic examples from the media amply
demonstrate, there is much teaching yet to be done. While we like to
believe that it is our opponents who are fools with figures, this volume
demonstrates that liberals, conservatives, libertarians, lawyers,
physicians, and educators fall in the same numerical traps."—Gary Alan
Fine, coauthor of Whispers on the Color Line: Rumor and Race in
America
"Best provides us with another telling compendium of
misleading statistics about a variety of topical issues. His approach to
explicating them is lucid, instructive, and quite engaging."—John Allen
Paulos, author of Innumeracy
About the Author: Joel Best is Professor and Chair of
Sociology and Criminal Justice at the University of Delaware. He is the
author of Damned Lies and Statistics: Untangling Numbers from the Media,
Politicians, and Activists (California, 2001), Random Violence: How We
Talk about New Crimes and New Victims (California, 1999), and Threatened
Children: Rhetoric and Concern about ChildVictims (1990).
Reviews by
Augsburg students: " I think that [reading] this [book] makes me a
more informed person and one less easily duped. I feel like I am less
confused, now, by conflicting claims. It wasn’t a book I would have
read outside of class, but I’m glad I did read it."
Damned
Lies and Statistics (5/2001)
Untangling Numbers from the Media, Politicians and Activists
Following
excerpts chosen by Milo Schield
P. 10. “Statistics can become weapons in political struggles over
social problems and social policy.”
P. 19. “Innumeracy is the mathematical equivalent of illiteracy; it
is ‘an inability to deal comfortably with the fundamental notions of
numbers and chance’.”
P. 26. "The lesson should be clear: Statistics  even official
statistics such as crime rates, unemployment rates, and census counts
are products of social activity.”
P. 27. "All statistics are social products, the results of people's
efforts."
P. 28. "There are three basic questions we should ask whenever we
encounter a new statistic. 1. Who created this statistic? 2. Why was
this statistic created? 3. How was this statistic created?" P. 28
P. 29. "Plan of the Book: The following chapters discuss some of the
most common and important problems with the creation and interpretation
of social statistics. Chapter 2 examines four basic sources of bad
statistics: bad guesses, deceptive definitions, confusing questions, and
biased samples. Chapter 3 looks at mutant statistics, at ways even good
statistics can be mangled, misused, and misunderstood. Chapter 4
discusses the logic of statistical comparison and explores some of the
most common errors in comparing two or more time periods, places, groups
or social problems. Chapter 5 considers debates over statistics.
Finally, chapter 6 examines three general approaches to thinking about
statistics."
P. 30. "Chapter 1 argued that people produce statistics. Of course
they do. All human knowledge  including statistics  is created thru
people's actions; everything we know is shaped by our language, culture
and society. Sociologists call the the social construction "of
knowledge."
P. 31. "Our knowledge about society tends to be "softer" than our
knowledge of the physical world."
P. 32. "Often the ways that people produce statistics are flawed:
their numbers may be little more than guesses; or the figures may be the
produce of poor definitions, flawed measurements or weak sampling. These
are the four basic ways to create bad social statistics."
P. 33. "Every social problem has a dark figure because some instances
(of crime, child abuse, poverty, or whatever) inevitably go unrecorded.
P. 34. "Because activists sincerely believe that the new problem is
big and important, and because they suspect there is a very large figure
of unreported or unrecorded cases, the activists' estimates tend to be
high, to error on the side of exaggeration. Their guesses are far more
likely to overestimate than underestimate a problem size."
P. 36. "Once a number appears in one news report, that report is a
potential source for everyone who becomes interested in the social
problem of statistics; officials, experts, activists, and other
reporters routinely repeat figures that appear in press reports. The
number takes on a life of its own, and it goes through 'number
laundering.'"
P. 39. "Definitions: Whenever examples substitute for definitions,
there is a risk that our understanding of the problem will be
distorted."
P. 39. "People promoting social problems sometimes do offer
definitions. When they do so, the tend to prefer general, broad,
inclusive definitions. "
P. 40. "This [broad versus narrow definitions] has obvious
implications for social statistics, because broad definitions support
much larger estimates of a problem's size."
P. 40. "No definition of a social problem is perfect, but there are
two principal ways such definitions can be flawed... False positives
(they mistakenly identify case as part of the problem)... False
negatives (incorrectly identified as not being part of the problem)."
P. 44. "Statistics about social problems always depend on how we
define the problem. The broader the definition, the bigger the
statistic."
P. 44. "There are, then, two questions about definitions that ought
to be asked whenever we encounter statistics about social problems.
First, how is the problem defined? <snip> Second, is the definition
reasonable?"
P. 45. "Measuring: Public attitudes toward most social issues are too
complex to be classified as simple pros or cons, or to be measured by a
single survey question."
P. 52. "Sampling: Virtually all social statistics involve
generalizing from samples."
P. 53. "There are two problems with sampling  one obvious, and the
other more subtle. The obvious problem is sample size. <snip> The second
issue: the representativeness of a sample is actually more important
than sample size."
P. 54. "Statisticians can calculate the probability that such random
samples represent the population... The real problem is that few samples
are random."
P. 55. " Statistics about social problems are usually based on
samples tht fall far short of randomness."
P. 57. "In short, the process of generalization is at the centre of
sampling."
P. 58. "The difficulties of drawing accurate samples also invite
another sort of generalizing claim. <snip> They [activists] may argue
that the problem threatens 'everyone', that it affects 'people of all
sorts,' even that it strikes at random."
P. 58. "In general, social problems are patterned... But people
promoting social problems often find it advantageous to gloss over these
problems, to imply that everyone shares the same risks and therefore we
all have the same, substantial stake in solving the social problem."
P. 59. "Characteristics of Good Statistics: First, good statistics
are based on more than guessing.... Second, good statistics are based on
clear, reasonable definitions. ... Finally, good statistics are based on
good samples."
Source: http://portal.tpu.ru:7777/SHARED/k/KITAEVA/statistics/book/Tab3/damnedstatistics.pdf
==========================================================================
TABLE OF CONTENTS

Introduction: The Worst Social Statistic
Ever

Ch 1: The Importance of Social Statistics

Ch 2: Soft Facts: Sources of Bad
Statistics

Ch 3: Mutant Statistics: Methods for
Mangling Numbers

Ch 4: Apples and Oranges: Inappropriate
Comparisons

Ch 5: Stat Wars: Conflicts over Social
Statistics

Ch 6: Thinking about Social Statistics:
The Critical Approach
