More Referendums, not Fewer

India, as a nation state, does not care much for philosophical rectitude of the source of its sovereignty. After all, it promised Kashmir a plebiscite in 1948 and has since reneged on it. The consent of the governed has rarely ever been discussed; not even in the Constituent Assembly Debates. To live in such a country and look at the two recent referendums that Britain has conducted leaves one wondering about the moral legitimacy of a lot of things that modern societies have come to accept.

Consider the state of our democracy now: we elect a person who then goes on to vote on our behalf in the Parliament. That’s not a great system by any stretch of anyone’s imagination. It is a vestige that can be traced to the Magna Carta: elected MPs have replaced Barons but the concept of electing a person as opposed what we actually want is still a feudal idea that is beholden to the ‘great man theory.’ The moment we have individuals, their decisions are incentivized by their career prospects.

The alternative, direct democracy or the Athenian model as opposed to the Magna Carta version, has its own flaws. Quite often, indirect democracy saves us from ugly bigotry of the electorate. It’s quite likely that if direct democracy were practiced we’d not have outlawed untouchability or sati or child marriage. But human societies have evolved enough to deal with this complexity and not fall victim to such a binary solution space.

A common refrain of those who want to refuse the Kashmiris a plebiscite is to cite the voter turnout of Kashmir in elections that India conducts. The argument is: participation in elections that India conducts is proof of the voter’s affiliation to India. That’s absurd. It then follows that wherever participation falls below a certain threshold, the population there automatically rejects its affiliation to India. But more importantly, such reasoning conflates the detail of governance with source of sovereignty. A Kashmiri may well dislike being part of India but may also want a say in local administration because it affects him/her.

The consent of the governed as we now understand in the context of electoral democracy derives mainly from the English Civil war, French Revolution and the American War of Independence. None of them resolve the question completely. But they all uniformly drive at the unit of governance being distinct from source of moral legitimacy for sovereignty.

An elected representative in 2016 is not even a logistical compromise. Technology can circumvent that. It’s not necessary for an elected individual to go to an Assembly to cast a vote on a piece of legislation. People can do that from the comfort of their homes. It’s possible to imagine a system that strikes a balance between efficiency of delivering administration and philosophical rectitude while still managing to side step extreme bigotry. To start with, people could be given N votes in their lifetime where N is significantly less than the number of elections they are likely to cast their votes on in their lifetime. That makes people really value the vote they are casting. Such a system will perhaps result in a reasonably stable body politic in a few iterations and can eliminate the need for elected representatives.

Will such a system result in a referendum seeking independence for Jammu & Kashmir? Likely. For Tamil Nadu? Possibly. But at least it will be more just and less feudal; there won’t be a Boris Johnson for instance. It will also likely treat minorities better: after all those who’re really oppressed will be more willing to use their limited votes to address that than larger population.

Buying Elections vs Winning Elections

The Election Commission is India’s second most respected public institution, right behind the Supreme Court. The reputation is well earned; all one needs to do is compare our elections with those of other lower-middle income countries. Electoral results are rarely disputed in India which isn’t the case with most peers. Just in case we forgot, our lower-middle income peers include countries like Sudan, Congo, Honduras, Nicaragua and Pakistan. Not exactly models of Democracy. India’s elections, whatever else the country’s flaws may be, are considered free and fair. Quite remarkable for a nation with such a fractured polity, low income and poor levels of education.

Does the success justify the Election Commission’s methods, though? Consider what elections have become in the last decade and a half. There’s very little campaigning on the street. Political graffiti on the street vanish during elections. It’s now possible to live in a middle class neighborhood in an Indian city and not encounter any campaigning in the campaign period. Whom does this affect? Surely, the small sub-regional political party or the independent candidate. They depend on support on the street to get their message across. But the Election Commission by banning graffiti on walls, even on private walls, has disadvantaged the smaller players. And, the opposition. After all, conventional wisdom is that the incumbent needs less of political discourse while the opposition needs more of it to win an election.

The other most discussed aspect of the recent elections in Tamil Nadu was distribution of cash to voters. The Election Commission has even stopped the elections in two of the 234 constituencies, citing distribution of cash. It’s obviously illegal for a candidate to buy votes. But does that translate to the warrant-less searches that the Election Commission conducts? The irony of suspending the very rights the process aims to guarantee for the purpose of the said process takes some restating.

The Election Commission, at least its Tamil Nadu arm, also said something to the effect of selling one’s vote makes one liable to criminal prosecution. One is tempted to auction off one’s vote on eBay just to annoy the Election Commission’s authoritarianism.

Let’s consider the practice voters being given cash to vote. Firstly, this phenomenon is of recent vintage. There may have been instances of some voters being given some money earlier, but the organized and targeted ways in which Tamil voters are given cash is only 15 or so years old. Incidentally, this is also the period in which the Election Commission has capped constituency level expenditure by the candidate and enforced it with maniacal self-righteousness. The official expenditure limit for candidates contesting an Assembly Election is 16 Lakhs. That’s atrociously low. Most weddings in middle class households cost more. That being the case, and given most candidates who contest elections are generally wealthy, is it any surprise that expenditure above the prescribed paltry limit goes as cash bribes to voters? The overzealous monitoring of the expenditure and making sure official limits aren’t exceeded is a direct contributor for cash distribution. That being the case, the Election Commission then spending ever more resources in violating the rights of ever more people by subjecting them to search without warrant is truly absurd.

A reasonable hypothesis for why cash distribution happens in Tamil Nadu far more than other states is that the state is relatively prosperous and each voter in a given constituency has a much higher impact on the election owing to falling TFR compared to other states. When measured as a probability of impact, the Tamil voter is worth some 1.4 times the median Indian voter. If 20-30% of these voters are targeted for cash distribution, that’s an easy Rs 1,000 per vote for a contestant with a budget of 3-4 crores.

Then there’s the issue of the ethics of accepting such cash. To argue, as the Election Commission has, that voters accepting cash is the reason corrupt politicians exist is to mix cause with effect. Political corruption exists because opportunities for it exist owing to a flawed system. Fixing cash distribution to voters to fix systemic corruption sounds a lot like solving the Kashmir problem by insisting on how maps are to be printed in foreign publications. In a deeply unequal society with a serious structural problem in allocating resources, the one place where the poor get to be treated equal is the ballot box. It therefore also ends up being a place where there’s a trade. The politician’s corruption, in some senses, can be understood as one of seeking rent on behalf of the masses and using that to win their support come elections. To blame the poor in this for seeking their share of resources in a system stacked against them is something only middle class bureaucrats are capable of.

Three former CECs from Madras all live within a 2 Km radius in the City after retiring. They all belong to the same caste, incidentally. Maybe that tells us something.

Scotland’s Past and Tamil Nadu’s Future

Why do countries exist? Why should they? And what is their definition? Are they subject to change? Why do countries not hold referendums to decide the future of disputed regions around the world? Did not India promise a plebiscite at the time Kashmir was acceded to India in 1948? Why can’t Kashmiris vote like the Scots did in 2014? What about Tamil Nadu? Why is it part of India? It wasn’t ever part of India until the British came in, was it? Aren’t many of the states in India similar to Scotland? These are some of the questions that keep popping in one’s head as one reads through TM Devine’s Independence or Union: Scotland’s Past and Scotland’s Present.

Sir Tom Devine, yes he was knighted, is a professor of History at the University of Edinburgh. So he does give us a much needed lesson in Scotland’s history. The Union of Scotland with England, we are made to understand, was complex, complicated and a result of multiple competing interests in the late 17th and early 18th Century. The English wanted a buffer in the north against the scheming French. Presbyterian Scots feared the return of Stuart Kings and their Catholicism more than they hated the English. The Staurt Kings were preparing to take over Scotland with French help. The result of all this was that in 1707, the ancient Kingdom of Scotland merged with England to form Britain.

At the time of Union, Scotland was impoverished by successive droughts in the 1690s and owing to disastrous wars. It was, Devine argues, the poorest country in Europe at the time. But the Union made it possible for Scotland to find a market for its produce, primarily linen and agricultural produce, in England. With the last of the Jacobite rising being crushed in 1745, Scotland became truly prosperous shortly thereafter. Devine again points to how Scotland moved from being the poorest to being one of the richest in Europe in the 18th Century, culminating in the Scottish Enlightenment. Scots, with their long and storied martial lineage, formed the backbone of British military forces in the 19th Century, strengthening bonds with England. Devine presents data to point to how the Scots were overrepresented compared to their population in the military and as staff of East India Company. The small country also built most of the world’s ships in this period.

But questions of Scottish identity, Edinburgh’s friction with Westminster and a long declining economy in the second half of 20th Century seem to have found a perfect target in Margret Thatcher. She was the embodiment of everything the Scots hated about the English. It’s at this point one wonders if commentators aligned with the BJP who now compare Narendra Modi with Margret Thatcher may be more right than they wish to be. Aggressively Hindu Delhi functionaries of the BJP sound eerily similar (and far worse) than the Tories in Westmister that Devine describes.

Kashmir, however, did not benefit from its union with India the way Scotland did in the first 250 years. So perhaps that’s a bad comparison. But a state that did is Tamil Nadu. Tamil Nadu was comparable to rest of India in most aspects at the time of independence. However, it now ranks alongside Kerala as one of India’s best governed states in terms of various indicators in health, education and economic growth. It also happens to be a region that always stood distinct from India both in terms of a territory and in terms of cultural identity.

That brings us to the question of: why should unions of regions into a nation state be treated as inviolable? Shouldn’t they be a question that the populace gets to  vote on at least once every generation? At this point, it’s clear Tamil Nadu has little to gain from India. It generates more revenue than it gets back from Delhi, it has much better health and education indices than India and the Lok Sabha is likely to end Tamil Nadu’s honeymoon of higher per capita representation through frozen delimitation in the next decade. The last factor will be an ugly flashpoint when it happens; it is an irreversible demographic trend that’s going to put Tamil Nadu at significant odds with north India. Shouldn’t political parties in the state therefore have an SNP equivalent? Even the Tamil Nationalist parties in the fray for 2016 Assembly Elections, Naam Tamilar or MDMK for instance, don’t seem to be approaching this question at all. The DMK, which was the SNP in India before the SNP got to be itself in the UK, has now walked a long way back from its Dravida Nadu demand.

The solution, one thinks, is that people should be allowed to hold a referendum if enough of them agree to hold a referendum in the first place. And that should include options of everything including independence. Maybe this could result in a problem when it’s used frequently; simple safeguards like limiting the number of votes per person in a lifetime for such referendums could be added to it. A simple rule of a modern Democracy ought to be that it doesn’t rule over a people who don’t wish to be ruled over. It’s about time mankind moved there. Be it Kashmir today or Tamil Nadu 10 years from now.

Writing vs Thinking vs Writing

If one were to read/re-read JA Baker’s beautifully written 1967 classic, The Peregrine, one is reminded how distinct it is from most of contemporary writing. Baker’s writing so carefully and completely excludes the writer. His prose, in fact, does the opposite; the author seeks to become one with the birds he watches, never once corrupting that world with anthropocentrism.

It’s a useful lesson and helps one stay honest. The standard template of modern writing in non-fiction, one that The New Yorker seems to fancy, is the opposite. Where the use of first person singular and thrusting oneself into what one is writing about are virtues. The obvious problem with this is obvious: it’s more susceptible to error and conflict of interest. But what is even more worrying is the problem this technique engenders when it succeeds. It tends to reduce thought to the proponents of it. Or, at least, associate the two closely.

Twitter, at least the Indian instance of it, is the extreme example of this going wrong even when done right. The need and the comfort of belonging to a group – be it feminist, liberal, conservative, funny, cool or whatever else – seems to triumph devotion to thought from first principles. And that, one could reasonably hypothesize, is an effect of thrusting the person ahead of thought.

The atrocious examples of it, when the bigotry is obvious, are hardly interesting. What is though is when well meaning groups use this shorthand; such as dismissing the human condition of the rich/privileged because they are rich/privileged. Or, academics justifying themselves to justify their work’s conclusion or possible bias. Or, politicians being expected to embody the virtues they espouse on behalf of their electorate.

DMDK: Why is it sought after?

Alliances in Tamil Nadu, conventional wisdom dictates, often decide electoral outcomes. In the 80s and 90s, an alliance with the INC was considered crucial. In the late 90s and early aughts, the PMK sought to portray itself as the deciding factor. In the past decade, the DMDK has tried to capture that space. Three of the possible four major formations have publicly sought the DMDK as an alliance partner for 2016 Assembly Elections. But how effective are these allies? In particular, how useful an ally is the DMDK?

Let’s consider the DMDK. It was part of the victorious ADMK combine in 2011 and was the second biggest party in the legislative assembly. But that is not a full measure of how good an ally the DMDK was. In constituencies that the DMDK contested, it got 44.84% of the votes polled. While its dominant partner, the ADMK, got 53.93%. That is, the DMDK underperformed its dominant partner by 9 percentage points. But that doesn’t give a full picture either. The four other junior partners in that alliance were: CPI, CPIM, PT and MMK. They only contested in 27 seats, put together, as opposed to the DMDK having contested in 41. But three of those four parties polled much better in the constituencies they contested in compared to the DMDK. The DMDK underperformed every party in the electoral alliance except the MMK in 2011.

Now, consider the Lok Sabha elections of 2014. DMDK had moved away from the ADMK alliance to the NDA; it was the dominant partner in that alliance contesting in 14 of the 39 Lok Sabha seats. In those 14 seats, the party got an average of 15.94% of the votes. The other NDA allies, in the seats they contested, got an average of 26.06% of the votes polled. That is, the DMDK underperformed its allies in an alliance in which it was the dominant party by over 10 percentage points!

It’s not even the DMDK’s case that it’s stronger in 2016 than it was in past elections. For a party that’s been a drag on all the alliances it has been part of, it’s a bit confusing as to why it’s sought after so much. If the DMDK were in a direct contest with ADMK, and it continues the trend of underperforming its own alliance by 10 percentage points, it needs a wave to win. Under regular circumstances, the ADMK will win comfortably in a direct contest with just its core vote.

The other complication that the DMDK presents to its alliance partners is: it seeks too many seats. The DMDK, unlike say the PMK, does not have a caste base as its core vote. This means its votes are diffused; that partly explains why it underperforms compared to more sub-regional parties. If the DMK were to ally with the DMDK and allotted 60 seats for the latter to contest(as it’s being speculated), that’s too much of a risk given the ADMK may well win most of them. Further, the DMDK voter by not being a caste bloc is more likely to not be additive to the dominant partner. The two Dalit parties (VCK and PT) and PMK, claim with some evidence, that their vote is transferable. That is their dominant ally can count on them. The DMDK, at least on the basis of past results, can make no such claims.

A good example of the complicated caste dynamic making PMK a good ally is the 2014 Chidambaram Lok Sabha result. The ADMK won that seat; but what is interesting is that its vote share for the constituency is one of the lowest among the seats it won at 42.5%. The DMK alliance did particularly well as Thol Thirmavalavan contested here as an ally and got 30.26% of the votes. But what is of importance is that the PMK did much better than its average performance elsewhere; it got 27% of the votes at over 5 percentage points higher than the alliance’s mean vote share. The only explanation that people who’ve studied Chidambaram will give is: the Vanniyars rallied behind PMK because they did not want Dalits to win. That kind of effect, had the elections been close, would have been crucial. The DMDK simply does not have a story like that to sell.

There are only 5 parties in Tamil Nadu that bring with them a transferable vote: PMK, VCK, PT, CPI and CPM. Of these, the VCK and PMK are incompatible as allies owing to caste factors; the PMK has also announced it won’t ally with either Dravidian party. So, if the DMK wants to win, allying with the Left and two Dalit parties appears a much better option compared to the DMDK. But those smaller parties have announced a new alliance among themselves called PWF with the VCK wanting a Dalit Chief Minister; which, one agrees, has to happen.

Unless there’s some secret sauce that the DMDK hasn’t made public yet, the only reason well informed politicians have to talk the DMDK’s stock up is to make it seem dearer to the opposition.

PS: Here’s the Data Source.

Young vs Old Voters: Cleavages in Impact

In the aftermath of Lok Sabha elections in 2014, youth and first time voters were thought to be crucial to winning elections in India. The BJP built its campaign around that idea and its success lent much credence to the young voter theory; exit poll data supported this assertion as well. It was easy to claim a generational shift in Indian polity.

But is this true across India? Does the youth vote make even a marginal difference in states that have had fertility rates below replacement levels? In states that still have above replacement fertility rates, what’s the extent of their impact?

Let’s consider India’s ‘big states’ and their electoral rolls from 2008 and 2015. It is quite stunning what the data reveals: there was actually a decline in the number of voters for Kerala and Tamil Nadu, two states that have had below replacement fertility rates for a generation or more, in this period. No other large state recorded this phenomenon.

It is likely the Election Commission cleaned up electoral rolls sometime in-between. Perhaps that partly explains the decline. The natural rate of growth of population, while low compared to other states, is still positive in both these southern states. They don’t have a declining population; at least not yet. But such a clean up, one can assume, was normally distributed. There’s no reason to think voter fraud skewed towards the states where fertility is low. So what’s of significance is the relative distance between the two southern states and others in terms of growth in electoral roll.

If we were to calculate the annual percentage growth rate of the electoral roll, both Kerala and Tamil Nadu have negative growth rates as one’d expect from declining number of voters. While that may overstate the phenomenon, what remains indisputable is that northern states that elected the BJP to power are all experiencing a surge in youth vote. And the two southern states in question – where the BJP hardly made a dent – are not. If we extrapolate this and calculate the growth for a 5 years, Bihar and Uttar Pradesh will expand their electoral rolls every election cycle by 10.56% and 14.74% respectively, while Tamil Nadu and Kerala would have shrunk theirs. 

Almost all of the expansion in rolls in the northern states, is likely to have come from new voters. Their skew towards the youth vote accelerates thus. However, these young voters join a pool of electorate that’s already lost representation in relative terms to the two southern states owing to freezing of delimitation in 1976. Therefore, while the skew of UP and Bihar maybe towards youth, the value of each additional vote and those of existing votes, is set to decline.

Youth joining the electoral rolls in Tamil Nadu and Kerala though, will be joining an electoral roll that appears to be shrinking. Their parents’ generation enjoyed the merits of freezing delimitation by becoming more important in terms of representation ratio in Lok Sabha compared to the northern voter. Though their worth within their constituency was declining every cycle a generation ago because the electoral roll was still expanding in these two states. Now, however, the youth joining the shrinking rolls will get more important even within their constituency; over and above becoming ever more important than the northern voter in terms of representation ratio. Unlike the skew in north India towards the youth vote, the skew in Kerala and Tamil Nadu is towards every existing voter becoming as powerful as the new entrant every election. Thus, if anything, there’s a skew towards older voters in terms of importance, since there are more of them.

If we calculated the probability of each vote impacting the result in UP’s median Lok Sabha Constituency in the next elections, assuming the growth rates above, it worsens by a factor of e^(-32)[2]. Just the worsening factor alone will make the individual’s vote almost meaningless. This is of course on top of an absurdly low original impact probability of e^(-463). The youth in UP might as well not vote, whether or not the trend of the youth vote is a political phenomenon. And this worsening factor is likely to get applied for successive elections for the foreseeable future.

For Kerala, the impact probability will increase by a factor of e^(2.1) in the next election, over an already slightly larger but still absurdly low impact probability of e^(-389). And given fertility has been less than replacement for long, this factor of improvement in impact is likely to get higher each cycle.

In other words, in north India there are more number of increasingly less powerful young people joining the system to elect fewer representatives per person. In Kerala and Tamil Nadu, there are less number of relatively more powerful young people joining the system to elect more representatives per person. These two polities are so vastly different from each other that they simply cannot exist in the same electoral system for long. Either this’d mean mass migration to normalize the impact probability or secession. The former carries with it the risk of nativist surge in destination constituencies which will only result in the demand for latter.

[1] – Data sources: 1, 2 and 3.

[2] – Method of Calculation for impact probability of each vote here.

Prohibition, TASMAC & Floods in Tamil Nadu

There was minor kerfuffle in an irrelevant and petty corner of the internet, betraying a general inability to parse sentences. That in itself is irrelevant to progress of thought. But its underlying assumptions and motivations are interesting.

Groups that are generally associated with progressivism, such as women’s rights groups, are often at the forefront of demanding prohibition. This was true in India a generation ago, it’s true to some extent now and it was true in bringing the Eighteenth Amendment to the United States Constitution. Suffragists and abolitionists were the core groups that demanded prohibition in America.

In Tamil Nadu, the distribution of alcohol is a state monopoly. It’s also the highest revenue earner for the state. This complicates the political climate in many ways. The state has seen several phases of total prohibition in the last century; it was lifted after a large number of deaths owing to consumption of illicit, unsafe liquor each time. The repeated lesson learnt was that illicit and unsafe liquor will replace the legitimate one when there’s a ban. This was sadly re-learnt over and over again in the 70s, 80s and 90s at the cost of many lives. The formula was depressing: a complete prohibition followed by deaths in hooch tragedies which resulted in relaxing the prohibition. And a subsequent election where prohibition became a campaign issue.

Almost to the word, the tropes used by those bemoaning TASMAC in Tamil Nadu follow the script of  of those in America a century ago: the illiterate poor don’t know better; they drink to their own ruin and are a threat to women and society. That alcoholism was considered a threat to the white race and clan members were recruited to enforce prohibition is startling in its similarity to the caste dynamic at play in Tamil Nadu. Kalefah Sanneh, citing Lisa McGirr, writes,

Lisa McGirr believes that this is a mistake. She is a historian who studies grassroots political movements in twentieth-century America, and she has concluded that our fascination with the boozy, semi-clandestine world that Prohibition created has led us to ignore its more lasting effects. In her view, Prohibition was not a farce but a tragedy, and one that has made a substantial contribution to our current miseries. In “The War on Alcohol” (Norton), she urges us to put aside our interest in the many ways involuntarily temperate citizens sought relief, so that we can consider the federal government’s strenuous attempts to stop them. Her book’s subtitle is “Prohibition and the Rise of the American State,” and by “state” she means in particular what she calls the “penal state”: the Prohibition Bureau and its many enforcers, some of them drawn from the ranks of the Ku Klux Klan; the laws and prisons required by a federal government newly alarmed about crime; the reality of a country in which addicts were treated not as victims but as perpetrators. Prohibition was patchily enforced, and certain groups were more likely to find themselves tossed into the rough patches: “Mexicans, poor European immigrants, African-Americans, poor whites in the South.” Nearly a century later, she argues, the legacy of Prohibition can be seen in our prisons, teeming with people convicted of violating neo-Prohibitionary drug laws. Many at the time viewed Prohibition as an outrage, and, in McGirr’s view, we are missing its true meaning if we are not outraged, too—and ready to resist its equally oppressive descendants.

There are many arguments against alcohol. Research elsewhere points to about 10% of all people who ever tasted alcohol becoming dependent on it at some level. Addiction to and abuse of substances or alcohol is as old as civilization. But so is the basic knowledge of making a fermented drink; almost anyone who knows how to boil rice will know how to ferment a sugary beverage into an intoxicant. No ban can be effective. And alcohol probably results in fewer deaths and chronic ailments than refined sugar.

In 1981 when MGR went back on his campaign promise to lift prohibition, the following years were the state’s longest spell without mass causalities from hooch tragedies in a long time. The ban not only affects the poor in terms of making them more susceptible to poisonous and unsafe alcohol, it also taxes Dalits/others by enforcing a certain version of ritual brahminism. The assumption that others don’t know better gets even worse when its couched in caste hegemony.

Though TASMAC has been a creation of the state, it still carries out a business. It’s not a utility. To demand that this particular business be subject to sudden and temporary requests for shut down, especially by the elite, is what it is: an imposition of social order. When floods cause disruption in essential services, that’s understandable. Soup kitchens and hospitals need hygienic places, power and a large workforce. Storing bottled alcohol needs nothing. It functions even when there’s flooding inside the store. To construe this as somehow a reflection of government priority is to miss the point entirely. Worse, to attribute sexual violence in flood times to alcohol because the elite go there on relief work is the kind of prism that’s its own parody.

The one criticism of TASMAC and prohibition that’s indisputable is that it has shifted the burden of taxation, as a relative measure, from the rich to the poor. MSS Pandian’s Marxist critique thus of MGR’s policy is perhaps the most well argued case for prohibition in Tamil Nadu. Absent a strong state with an ability to implement progressive taxation, an excise duty on consumption that’s skewed toward the lower end of the economic spectrum is unfair. That critique has now become a reality, however much it pays for other services of the state.

The Sixteenth Amendment of the United States Constitution was a necessary condition for the Eighteenth to even happen when it did. The opposite, that is an ability to raise local and state level taxes, is a pre-condition for arguing towards prohibition in Tamil Nadu. That still is merely addressing economic viability.

Problems with Opinion/Exit Polls in India

After the recent failure of exit polls in Bihar, pollsters from Delhi have decided to form a self-governing body of sorts. This is unlikely to be of much use.

The problems with opinion and exit polling in India aren’t unique. But they are uniquely complicated by the fact that it’s the world’s second largest population with a FPTP system in a multi party electoral democracy in a vastly unequal society. The basic ways in which a survey, which is what a poll is except it asks questions on voting, can go wrong are finite. There can be either be sampling errors or response bias or a combination of the two.

A simple random sample will be one in which a pollster takes every nth voter from a voting list. Except, in real life we run into problems that caused an upheaval in America’s nascent polling industry in the 1940s; George Gallup had to testify in a Congressional hearing about response rates of his polling. That is in America of the 1940s where only 10% of the population was black and most of them were disenfranchised anyway owing to Jim Crow laws. But even for that relatively homogeneous white voting population, the response rate had fallen to single digits. That is, if every nth voter is called, less than 10% of those called told the pollster their opinion/whom they voted for. This is for polling via telephone.

In India, the problem is complicated manifold. Firstly, telephone surveys don’t work at all. There’s a selection bias that’s too huge to account for if only people with phones were polled. And, there are cultural and linguistic reasons as well: it’s very difficult for someone who speaks in a certain urban way to get a Dalit in a rural area to answer questions on political opinion even if one assume the said Dalit possesses a phone and speaks the same language. Even if the pollster actually goes to where the voter is, it gets extremely complicated given how disparate the voting population and their living conditions are.

Firstly, a lot of people are going to refuse to answer. That’s a fundamental problem for pollsters everywhere. This problem when overlaid with caste and religious fault lines in a largely feudal society, gets impossible to account for. The only kind of Dalit, for instance, who’s likely to answer a survey is the kind of Dalit who’s politically active. This is true for most most non-dominant castes and other social groups in various degrees. This is over and the problem of basic sampling;  that in itself is difficult to get right given Census data does not provide caste information. At least until the last set.

And finally, even if they do answer, they can and often do lie. As it’s been established, socially disadvantaged groups in feudal societies tell their interviewers what they think the interviewer wants to hear. So the problem thus far: we can’t get a random sample, we can’t adjust for selection bias because we can’t get the social break up of the voting population right, we can’t account for response rates and the resultant sampling bias and we don’t account for response biases given we don’t know the break-up in the first place.

The only way in which any of the above problems can be addressed is if the data is made public and there are more and more such polls so that some kind of adjustment factor for each of the above problems for each of the above groups can be found. These are academic endeavors that for-profit companies with short term goals don’t solve well.

The other possibility, less likely but not implausible, is that one of the polling organizations makes data up or fudges it. Many of us when faced with such a difficult task may resort to that. Added reasons such as political bias and the ability of polls to influence people make it even more tempting. So such a regulatory body may be useful on that count.

Fudging can happen in many ways. The most simple form of it can be the kind that was found in the Research 2000/ Daily Kos scandal. It was simple manual fudging; exposed because of the inability of humans to be random even if they wanted to. The data was found to have even numbered percentage responses for both genders. The probability of that in one poll, which can happen, is 0.5*0.5 = 0.25. But they had even numbered response percentages for both genders for 24 successive polls. That results in an impossibly low probability; so low, it’s less than one in all the atoms in the Universe. Most reasonable people will conclude, that’s fudged. But that’s just one way to fudge it. No one is going to do this again given people will now look for this.

The other method of fudging data is by having a base data set and adding noise to it to make it seem like it’s a different data set. A famous example of this kind of data falsification was the gay marriage study which everyone was talking about in 2014. It’s relatively easy, again, to find out this kind of falsification. The Kolmogorov-Smirnov test does precisely this. And it’s something Physicists use to do far more important things, because of which it’s a well developed test. Even for those of us who did not take courses in Statistical Physics, the test is easy to administer these days given it takes one line of code. It’s tempting to think, those who have political biases will indulge in precisely this kind of falsification. It’s relatively easy and serves the purpose really well to take a base of a previous election you won and falsify it as new opinion/exit poll.

A self-regulating body, one fears will hardly be pointing fingers at itself. This is why opening up the data is important since the scientific method will address these aspects much better. The problem pollsters have of course is two fold: their secret sauce gets revealed and the secret sauce in this case often tends to be a series of assumptions which may be wrong.

Finally, a far better summing up of the American situation can be found here.

Gaming the FPTP: Should Dalits in Haryana move to UP?

In recent years, Haryana has seen a steep increase in crimes against Dalits. The rate of increase in crimes against Dalits is about 16% every year compared to the previous year, in the period 2000-2014.

It’s reasonable to imagine a Dalit politician in the place of VK Singh wouldn’t have resorted to his recent choice of words. Even if the hypothetical Dalit politician did, it’d hardly have carried the same weight of implied bigotry as that of a person belonging to an oppressing caste. It’s also true that Dalits in Haryana don’t have representation that’s anywhere close to capturing power; as is the case in much of India.

Haryana sent two Dalit MPs to Lok Sabha and both were elected from reserved constituencies. Its politics at the state level does not have a dominant Dalit party either, unlike in Uttar Pradesh. A lack of electability for Dalits in non-reserved constituencies, as seen in Tamil Nadu, betrays both prejudice on the part of others and the polity not being sufficiently fractured to overcome that prejudice. Uttar Pradesh has about 22% of its population as Dalit; that corresponding figure for Haryana is 19%. The difference however is, in Haryana there is a single caste Hindu group that’s got more people than all Dalit castes put together; namely the Jats[1]. 

In Uttar Pradesh, there is no caste group as dominant as Jats in Haryana. And, Muslims form a much larger group at 18.5% of the population; whereas only 7% of Haryana’s population is Muslim and they are largely concentrated in one district. So the typical political coalition of the Gangetic plains, which usually is Dalits or Muslims with one other caste group, simply does not work well in Haryana. Consider the Dalit population distribution in Haryana by District,


With the exception of Sirsa and Fatehabad, no other district has a large enough Dalit population for them to be a political force by themselves. And as it’s been established multiple times across multiple states, a coalition involving Dalits is a lot more difficult to build owing to prejudices[2]. Especially when the dominant caste is this numerous and the contest is not four cornered like it is in Uttar Pradesh.

In such a scenario, when one group faces escalating violence perpetrated by dominant groups who also corner much of the political representation, what can Dalits do? Even if every Dalit votes one way, in a two cornered contest their entire voting block will have a probability of impacting an MP’s election ~ 5.6*10^(-174)[3]. If one were to look at merely maximizing this impact probability, one’d tell Dalits in Haryana: just relocate yourself to Uttar Pradesh. That will instantly achieve two things, a high likelihood of Dalit party winning elections and an increase in impact probability of individual Dalits in elections by multiple orders of magnitude.

Is that feasible? Firstly, Haryana is a far more wealthy state compared to Uttar Pradesh. But it’s an agricultural state[4] and Dalits historically haven’t been land owners. So, perhaps the effect of mass migration will be less acute than the socio-economic indices of Haryana and Uttar Pradesh suggest. The more important question is: what will so many people do in Uttar Pradesh, an already poor and struggling state? Maybe this isn’t such a good strategy after all. But is targeted action possible? Even if not everyone, those in districts with particularly low Dalit population – like Panipat, Faridabad and Mewat – can move to the many districts in Uttar Pradesh that have ~25% Dalit population.

The BSP clearly has an incentive to encourage this. No other group in India has so little to lose in terms of possessions and so it’s unlikely to cause migration in the opposite direction. Perhaps if not across the state, Dalits in each state can migrate to other districts within their own state such that they can game FPTP.


[1] – Surely, there are multiple sub-castes among Jats; it’s likely the sub-castes don’t all vote the same way. Estimates of Jat population vary between 20-25%, given there’s no caste census.

[2] – Andrew Wyatt’s Party System Change in South India explains this phenomenon.

[3] – Extending the arithmetic here.

[4] – Despite the rapid move to service economy in the districts adjoining Delhi, 70% of Haryana’s population is still engaged in agriculture.

Muslim Population Growth: Sex Ratio & Rural Populations

The common refrain of many when Census data showed high Muslim growth rates was: look at Kerala. Let’s look.

Two factors that were reported by the Census data and pointed to as factors that could, at least in part, explain the higher growth rates of Muslims were: sex ratio and percentage of rural population. It’s a reasonable hypothesis. After all, fewer women mean fewer women who can bear children and vice-versa. Similarly, a greater percentage of rural population means more people with lower socio-economic status pointing to higher fertility rates. But is that hypothesis, while well founded in literature[1], hold up within the data set[2] released?

The first thing one does when told one variable may be predicting another is to simply plot the data, before any analysis. A plot of rural population in various states as X and Muslim growth rates in population as Y looks thus.



A similar plot against sex ratio is,


There really is nothing discernible to the naked eye from these scatter plots. If anything, it appears more women correlates to lower growth rates. Not higher.

The next thing we’d do is simple linear regression. There seems to be no linear relationship between rural population and growth rates for Muslims. There isn’t an exponential relation either. But the data for sex ratio among Muslims is a tiny bit more interesting. At least within this flawed model which predicts nothing, if at all there is a wager it’s that more women seems to point to lower rates of growth. 

It’s useful in such cases to just see the two variables interact on a response surface. For this purpose, let’s model this as a Gaussian Process[3].



As we’d expected, the response surface doesn’t move much except for the places X2 acts up. X1 is rural population and X2 is female population as a percentage. The basic point of the data that Muslims have higher growth rate is captured well, though. To understand that even better, let’s look at how the corresponding response surface for Hindus looks,

HinduGPAgain, there is no evidence for rural populations outpacing their urban peers, much. But female population seems to suggest a concentration in the middle and a plunge when the ratio of women increases. The scatter plot of Hindu women and their population growth is in this direction as well,


Of course we know from living in the real world that more women are not resulting in lower population growth. The reason is likely to be the reverse: socially advanced states have better sex ratios and low fertility rates, resulting in low growth.

With the data that’s been released by Census, there isn’t a conclusive model for anyone to conclude reasons within this set. There isn’t even a bad model. But we at least know the hypothesis that lower IMR and hence better sex ratio is a contributing factor to high Muslim growth rates has no evidence. Rural population ratios don’t seem to matter, either.

[1] – Demography as a Science is far too advanced for lay people to just make stray observations. When searching for the model Demographers use to estimate human population, I found this thesis; it’s quite useful as background reading material. Hopefully, I’ll understand the models better in the near future.

[2] – Data used here is the from the same source as the previous post: Census of India.

[3] – Please do not take this to mean anything more than an illustration of the variable’s effects. The GP here is not a real model given how limited the data is. And don’t blame me if you get a failing grade for doing this in class.

[4] The Model details, if you are interested in wrong models that is, of the two GPs.