Monthly Archives: March 2014

Tracker Poll Bihar: Defying the Law of Large Numbers

In the Bihar instalment of the poll released by CNN-IBN, the pollsters set a new record for volatility. In a direction counter to conventional wisdom at that. All in a month.

There was a 32 percentage point increase in the satisfaction levels with the UPA! Something even Dr Manmohan Singh won’t claim.

bihar

If the same experiment in such a short time period is resulting such a dramatic movement it only means there is a serious methodological flaw. Even if there were an Earth shattering news event, this’d be difficult to defend. A case in point would be that President Bush’s approval ratings after September 11, 2001 moved by roughly as much.

Tracker Poll UP: Fewer Votes = More Seats?

The incongruity between their own data and the discussion on CNN-IBN has now come to be expected; the instalment of the program for Uttar Pradesh  was no different. The anchor continued to assume all that the polls showed was a Modi wave and brushed aside his own panelists trying to add nuance.

If we looked at the tracker poll’s data benchmarked to itself from last month, the vote share results for the State of Uttar Pradesh looks like this,

 

UP Tracker

 

That seems slightly counter-intuitive given the SP government has been getting a lot of negative press. But that’s why it’s better to get actual data than judge based on press reports. Perhaps, the SP is recovering its support from the lows of January. But the significant point is, both the BJP and the INC have a slight movement away from them. And as the pollster himself has said, if anything the tracker is useful only to measure a trend. This, as it happens, is the trend.

The calculation methodology of seats from the vote share is not made transparent. But the number of seats projected last month for the BJP with a higher vote share was lower. So how the BJP managed to increase its seats from last month’s projection on the back of a falling vote-share is something neither the Panelists nor Dr Kharandikar has explained.

Tamil Nadu’s Volatile Polling Numbers

The CNN-IBN Poll for Tamil Nadu was published yesterday. This instalment of the poll continued to raise eyebrows as it did for Maharashtra.

Let’s begin with the DMDK. According to the poll, the party has a vote share of 4% in the State. That’s roughly 1/3rd of what the BJP and the INC are polling. If you were to walk any random street in Tamil Nadu and say this, you’re likely to be laughed off. However, people can be bad judges of strength and it’s better to trust a poll than “wisdom.” So let’s assume that were true. But what’s incredibly hard to believe is the self adjusting nature of polling numbers among challengers in the state while the leader remains unchanged.

Consider the table  below. It uses the poll summary table and adds the last two columns,

Party

July 2013Survey estimate

January 2014Survey estimate

February 2014Survey estimate

Percentage Point Difference for Jan- Feb

Percentage Difference for Jan- Feb

AIADMK+Left

31

30

30

0

0.00%

DMK

16

18

26

8

44.44%

INC

18

17

10

-7

-41.18%

BJP

10

16

12

-4

-25.00%

DMDK

7

3

4

1

33.33%

PMK

5

1

2

1

100.00%

AAP

3

2

-1

-33.33%

Others

13

12

14

2

16.67%

All over the world, any meaningful change in polling numbers is simply expected to happen between the top contenders. Of course the magnitude of change in terms of absolute percentages for parties with low numbers may be high owing to a low base. But it’s very very strange to see 0% movement of the leader and all the movement is towards the immediate challenger. Further, this increase in strength of the challenger is 8 percentage points or an absolute change of 44.4%. This has happened in a period of 1 month and no major news event happened at that time[1]. Even more difficult to believe is that the previous base of July 2013 held in conventional terms all the way up to January 2014 — for six months — while a single month has now resulted in this bizarreness[2].

But that’s hardly the most surprising part of the poll. What’d stun people far more on the streets compared to the DMDK’s low numbers is that there is a 15 percentage point increase in the satisfaction levels towards the performance of the UPA Government in Delhi! Consider the table below,

July 2013 Survey estimate

January 2014 Survey estimate

February 2014 Survey estimate

Percentage Point Difference for Jan- Feb

Percentage Difference for Jan- Feb

Satisfied

39

32

47

15

46.88%

Dissatisfied

38

46

42

-4

-8.70%

Can’t say

23

22

12

-10

-45.45%

One explanation in the pollster’s defence is that the people who were previously undecided are now deciding in favour of the UPA. But even that doesn’t sufficiently explain the huge jump. Again there has no been major news event and if anything the political consensus as the political aligning indicates is against the UPA and not for it.

Even if the movement were in the direction of conventional wisdom, the quantum of jump in this poll in such a short time period will trouble anyone who’s sat in a class involving basic statistics. The change and its direction in this instance within a month almost holds up as a counter to the law of large numbers. Especially given the relative stability of the previous six months.

[1] – This 8 percentage point jump is suspiciously close to the 9 percentage point drop of Modi in Maharashtra. If they were both 2 percentage points exactly, no one would bat an eyelid. But for such large jumps to cluster together in seemingly unrelated data points is very very very strange.

[2] – In qualitative terms, this means at least some of those who were BJP Voters in January have now decided to vote for the DMK in February. One cannot think of such a shift unless one decided to treat DMK and BJP as data entries in an excel sheet. Maybe one is wrong.

Regressing to an imaginary mean

There have been recent allegations that opinion polls are up for sale. Typically suggesting they could manipulate aspects of the poll such as favorability of a specific candidate for PM if not the actual vote share.

In that context, the CNN-IBN/CSDS poll for Maharashtra deserves some attention. The first thing that anyone would notice about the poll is the dramatic fall in Narendra Modi’s support for choice of Prime Minister. The poll shows a 9 percentage point drop in a 1 month period — a steep enough fall in a short enough time period for any presenter to make this the primary object of discussion. However, if one listened to the panel discussion on Television instead of reading their summary tables, one’d think Modi were still on the upswing. The anchor even went on to say things like “it’s Modi all the way.”

The next interesting and never discussed fact was that the NDA camp had only one PM choice in the summary tables. The NDA has obviously a declared candidate and hence there is only one. However, the UPA has not officially declared its candidate. And it’s reasonable to assume the choice of Rahul Gandhi, Sonia Gandhi and Manmohan Singh are all interchangeable. If one adds their three numbers together along with their ally, Sharad Pawar, it comes up to 30%. That’s just a one percentage point difference between the two major camps according to the poll; Narendra Modi polled 31%. Yet, the entire discussion on Television was how a “majority” of the people wanted Mr Modi as PM and how this was driving the BJP. That could mean only one of few things — either the panelists had not read the poll numbers or don’t know how to or worse, have been asked to toe a certain line as alleged by Caravan.

If one assumes the benefit of doubt goes to the panellists and they were merely ignorant on how to read successive polls, one then is left with the poll itself. As anyone who has watched polling numbers can attest, nine percentage point movement in one month is phenomenal. So it has to come from somewhere in this zero sum game. If it were a natural variation in the data, one’d then expect the numbers to probably be the result of one person catching up with the leader. As it usually happens in Gallup Polls in American primaries. But instead, in an odd looking table of comparison, each person in the INC camp — Rahul Gandhi, Sonia Gandhi and Manmohan Singh — has risen exactly two percentage points. The probability of everyone having the same percentage point raise is, well, remote. Add to this, the table also means Manmohan Singh has doubled his popularity as a choice of PM in the past month. This after expressly declaring himself out of the race and not being seen in public in that period. Baffling is a major understatement.

If one were to assume this to be a freak result of normal variation in data, what’s even more interesting is that in the latest instalment of the Maharashtra poll the choice of PM question is shown in the data region-wise. While in the previous month’s poll it wasn’t. So there isn’t any chance of seeing where the catching up has been for such a uniform two percentage point increase favouring Sonia Gandhi, Rahul Gandhi and Manmohan Singh. Especially Manmohan Singh.

Finally, the panellists on the show may not be statisticians. But they are literate enough to understand 2009 actual result has no meaning as a base value when the past month’s poll exists. Yet they insist on looking at the absolute number instead of the direction of the poll movement incrementally, given the latter is self adjusting and thus probably a far a better reflection. Strange are those who are wrong once and knew they were. Especially when they are in the business of predicting the future and so do not know what the right answer was. Or is.