Why Should Data Not Go Private?

The development of private data sources is a hugely positive development.
It should serve as a challenge to the government to improve its own record on producing timely and reliable statistics, points out T N Ninan.

Is India's data market going increasingly private?

It would seem that way, though of course not entirely so.

Consider the following: The most commonly cited employment numbers are those put out by the Centre for Monitoring Indian Economy (CMIE), using surveys whose sample size matches those used by the government.

Probably the most trusted source of data on the true state of Indian school education is Pratham, a non-profit organisation, which puts out a widely anticipated survey of education annually.

Other numbers that are tracked closely include IHS Markit's purchasing manager's index, which serves as a reliable guide to the economic tempo.

An analytics firm, CRISIL, is the go-to source on the creditworthiness of companies. And there is no rival to the Centre for Technology, Innovation and Economic Research (CTIER) for a range of data on technology issues.

Meanwhile, a private monsoon forecaster like Skymet has in at least some years done better than the government's meteorology department.

And when the World Health Organization says that Covid-related deaths in India are 10 times the government's tally, many will wonder about the official statistics.

Even the Reserve Bank of India, which remains a reliable source for quality data, finds it hard to match the speed and range of data on corporate sector performance. One could go on with such examples.

CMIE, Pratham, CRISIL, Skymet, IHS Markit, CTIER and others like them have come into existence, or into prominence, in the past two or three decades.

This is as it should be; a developing economy will have new and multiple sources of data, including digitally sourced ones.

The government on its part may question some of the country rankings put out by international non-government bodies, like the recent one on a hunger index, but its own data are increasingly mired in controversy.

The result is questionable quality and undependable frequency for some of the important economic statistics put out by the government -- for much of which there can be no comparable private substitute.

The country has no consumption survey data beyond 2011-2012 (the numbers for 2017-2018 were suppressed).

It has been the same story with employment statistics: The government has suppressed numbers while trying to substitute the old accepted ones (imperfect as they were) with partial substitutes like the number of people with provident fund accounts. And GDP numbers for the past have been revised repeatedly.

Even a foundational data-gathering exercise like the census, done every decade since 1881 and last due in 2021, has been delayed.

The reason cited is Covid, but people have been moving about relatively freely for a year or more, without China-style lockdowns. Yet survey work will not begin till next year.

The census delivers many socio-economic numbers, not just the population figure.

The harm done to the statistical system, and to policy analysis, because of the absence of such basic data should be obvious.

Everything is not a change for the worse.

Some data releases by the government have become more frequent (like the quarterly GDP data, which did not exist before); some data packs have seen improvements in methodology; other numbers come out faster than before, like the trade statistics, or are more transparent -- like tax and fiscal data.

Still, you could visit almost any economic ministry's web site to discover how out of date and inadequate much of the data presented is, and on occasion how slanted to cover up inconvenient facts.

Like many other things, statistics have become more political.

Digitisation has made a huge difference, and thrown new light on otherwise data-dark corners of the economy.

As one example, the new credit bureaus that gather information on companies' and individuals' credit records help lenders of all kinds.

Similarly, the growth of online selling and payment systems has created new sources of data on consumer behaviour.

An economy that aims to be the third-largest in the world in under a decade cannot plan and function properly without data that meets the tests of speed, frequency, reliability and completeness.

The development of private data sources is therefore a hugely positive development. It should serve as a challenge to the government to improve its own record on producing timely and reliable statistics.

Feature Presentation: Ashish Narsale/Rediff.com