Sampling Errors & Response Bias vs Human Stubbornness

Polling entities, all of them, face a basic problem: it’s very difficult to make the sample truly random. This problem is even more difficult when the society in question has multiple fault lines within that are a few thousand years old: like Dalits or caste/religious groups or linguistic groups. Assuming they do over come this insurmountable problem with acceptable levels of error, there is the question of Response Bias. If that too can be somehow overcome to arrive at a reasonable vote share estimate, there is the problem of the physics of a multi cornered fight in a first past the post system.

Instinctively, one is tempted to therefore suggest a continuous polling solution that’d side-step many or all of these issues. By virtue of polling the same people instead of seeking a different and still incorrect random sample, the idea is that a continuous poll will internally adjust both sampling and response bias in the successive polls. What we’ll then have is a better capture of the trend and a far less volatile result. The most famous example of this was of course the Rand Corporation’s continuous poll of the 2012 Presidential elections in the USA. It was, as the numbers speak for themselves, fairly accurate.

One is not aware of research that compares the outcomes of such polling with the traditional ways of picking a fresh random sample every time one polls. Someone must have already done that given it’s such an obvious target area — but most resources draw a blank on the subject, strangely[1].

There is however one problem with this polling approach that any lay person will tend to raise. Almost all of us must have had the famous Monty Hall problem thrown at us when we went for our first job interview after College. Perhaps many of us now use it on people whom we interview — as test against BS if not ability in actual Probability theory. But the simple fact, as the problem illustrates quite beautifully, is that human beings like to rationalise their decisions after it’s been made. And stick with it unless something dramatic happens. And this phenomena makes one wonder whether it applies to those being polled repeatedly as well. And whether the significantly lower volatility is not merely a sign of self adjusting system for sampling bias but is an indicator of a completely new kind of error.

This may be quite a nice topic of research in the Indian environment even it has been studied elsewhere. How does one volunteer in a Political Science Department in India for such a study?


[1] — If you are aware of such a paper, please point me to it.