Play Things

Asking without really asking

I will let master statistician Stanley L. Warner begin this blog post. From the first few paragraphs of his landmark 1965 paper "Randomized Response: A Survey Technique for Eliminating Evasive Answer Bias":

"For reasons of modesty ... or merely a reluctance to confide secrets to strangers, many individuals attempt to evade certain questions put to them by interviewers... This paper suggests an alternate method for increasing cooperation... built on the premise that cooperation should be naturally better if... the interviewee responds with answers that furnish information only on a probability basis."

What Warner is trying to explain is an interesting way of carrying out surveys. Let me explain this idea without going too much into the mathematics. Instead I'll focus on how its done, why it's useful and what happened when we ran a little Lok Sabha exit poll here in the office using Warner's Randomized Response method.

The Randomized Response Technique is used to carry out surveys when you expect respondents not to be honest. For instance:

1. Do you secretly hate the Zoozoos and want to see them shot but keep the emotion to yourself?

2. You don't really get Twitter do you?

Seriously speaking, the technique is used when you want to research the prevalence of issues that people feel uncomfortable talking about. A college would ever be able to survey its students for prevalence of drug use and cheating. A student would never risk the probability of being identified saying: "Yes. And sometimes both at the same time."

What Warner suggests is using a technique where merely using the math of probabilities, especially conditional probabilities, you make estimations. The survey works in two stages: first you ask the respondent to roll a dice, pick a card and so on. Depending on the result you ask them to answer one of two questions without yourself knowing which question is being asked. (Hold on.) The respondent merely says if they agree or disagree with the question before them. Then you survey the next respondent. And so on.

Then the math comes in. Using a few formulas which look at the chance of a question being picked and the regularity with which an answer was given, you could approximate what the whole selection of respondents felt. Obviously bigger the sample, better the accuracy. (More clear math in the second paper linked below.)

But let me tell you what we tried in the office. We took 16 identical pieces of paper and on 12 we wrote the statement "I did not vote for the UPA: Congress or allies" and on the remaining four we wrote "I did vote for the UPA: Congress or allies". Testy questions indeed. Then we shuffled the cards statement-side down and asked employees to pick one each. (The card was returned and the deck shuffled after each employee. Only people who actually voted were allowed to pick.)

Then I ran the math. Using a sample size of 34 voters (very small, but good enough to blog about) and the 16 cards we were able to approximate that 32.53% of the office voted for the UPA and the rest did not vote for the UPA. (So we really can't say who they voted for. That wasn't the question see.)

Warner's paper here.

A more easier to understand one here.

p.s. In a similar survey done in college many years ago, over a sample of some 90 people, we discovered that 73% of the student population would cheat on their spouse, significant other if they knew they could get away with it.