Mike Lehr's Blog

How we position things greatly influences the outcome. In the April 7, 2012 edition of The Economist the article, “Dressing Up,” uncovers that women’s sizes have inflated by four sizes since the 1970’s. Unlike men’s sizing which is based on inches, women’s sizing is purely arbitrary and often varies by brand. Thus, depending on the size, a pair of women’s pants might have increased as much as four inches at the waist and three inches at the hips since then.

The generally accepted assumption for allowing this size inflation is that if consumers feel good about themselves they are likely to buy, thus why the fashion industry calls it “vanity sizing.” However, even though it seems like a topic to take lightly or with which to have fun, vanity sizing plays in all aspects of statistics. That is why it’s important to challenge definitions and assumptions in order to understand and solve problems.

For instance, the article “Botox and Beancounting” of the The Economist’s April 27, 2011 edition, discusses how official U.S. economic statistics might be overinflating its performance relative to Western European economies. Ironically, the article’s title makes an appropriate analogy to vanity sizing.

U.S. unemployment figures present another excellent example. They not only conflict with one another on occasions but they are difficult to figure. Additionally, their accounting changed in the 1980′s, making them appear lower than before.

Thus, while it’s commonly said that “numbers don’t lie,” that’s true; however, an ignoramus isn’t lying either if he believes his own ignorance. If we’re ignorant to numbers’ origination, we are more likely to accept them if they tell us our glass is half full rather than half empty, thus reinforcing our own perceptions . . . also known as “vanity believing.”

Even though rankings are extremely subjective, they seduce us as strongly as the sirens did sailors in Greek mythology. Consequently, we often wreck ourselves on the rocky shores of fantasy island.

In order to understand the lure of rankings, we need to understand the lure of numbers. When we quantify something, it becomes easier to grasp. However, easier doesn’t mean that what we are grasping is real. It’s often easier to understand what we want to believe than it is to understand reality. For example, in reality a woman’s measurements don’t tell us much about her, but that doesn’t prevent them from triggering our fantasies.

Applying this illusionary power to rankings, they tap into our insecure desires for:

1. Simplifying a complex world
2. Defining limits to large or limitless knowledge pools
3. Quantifying the unquantifiable
4. Delivering certainty in an uncertain world

Rankings perform complex thought for us by determining which is better by deciphering many, many variables. They imply we can get by on much less knowledge by giving importance to the top ten rather than the top million or billion. Their parameters and measurements are subjectively determined, trying to measure something that normally is immeasurable. Finally, as implied above, the quantification inherent in rankings provides certainty; “these are the important ones and that’s it.”

For instance, consider these Google searches:

“Top 10” = 743 million results
“Top 100” = 1,083 million
“Top 1,000” = 46 million
“Top 10,000” = 17 million
“Top 100,000” = 2 million
“Top 1,000,000” = .6 million
“Top 1,000,000,000” = 5,250

Clearly, our focus is on the simple with limits; so, the problem is this: How are we going to ever appreciate the billions of unique people, places, creatures and things in this universe if we’re so focused on the top ten?

I ran across a good article by Malcom Gladwell in the February 14 & 21 issue of The New Yorker titled, “The Order of Things.” The detail with which he explores rankings of colleges, hospitals and cars demonstrates the immense subjective potential rankings have. What is even more astounding is Gladwell’s discovery of the degree to which many organizations hold their leaders accountable for their place in these rankings.

From an intuitive perspective, people tend to have an emotional connection to statistics; they satisfy feelings for certainty, clarity and knowledgeableness. Thus, when we express arguments statistically, they tend to carry more weight than if we simply express them in words. Rankings clearly define for us what is best, better and good. However, they are more akin to magic where reality is but a trick. Thus, the feelings we receive from rankings (certainty, clarity, knowledgeableness) are satisfied because we want to believe their magic is real.

As a rule, unless the ranking is comparing very similar things against a single, measurable criterion, it is highly subjective. Therefore, here are some important questions to ask about the ranking to discover how its trick works:

• Is it really comparing similar things?
• Is the ranking based upon multiple criteria?
• How important is each criterion and is it valid?
• How does it weight the criteria?
• Is it using some criteria as proxies for things that are difficult to quantify or research?
• What important criteria are absent because of these difficulties?
• Is the difference between one rank and each of those immediately above and below it that significant?
• How accurate was the data collected for each criterion?
• What problems might have retarded data quality?

Applying these questions will demonstrate that our affinity for rankings is more emotional than pragmatic.