NMI relies on 1968 research design

KUALA LUMPUR: The formal proceedings of the Sun Media Corporation's case versus global market researcher The Nielsen Company (Malaysia) Sdn Bhd commenced on June 13 in the court of Justice Lee Swee Seng.
Lee utilised the first four days to execute an interesting hot-tub session. Independent expert witnesses nominated by both parties were invited to share their knowledge and opinions regarding the core technical issues of the case, also referred to as the terms of reference.
These expert witness statements and rebuttals are now deemed as public documents which can be accessed and downloaded by readers, industry professionals and media.
Nielsen has appointed its former employee and veteran researcher, Sydney-based Jonathan Jephcott to represent its perspectives and his expert opinion. He has over 45 years of industry experience.
Jephcott insisted that the research design used by Nielsen Media Index (NMI) to measure media consumption, still conformed to global best practices. That it had undergone only minimal changes in the last four decades.
Filing the legal suit in 2011, Sun Media Corporation Bhd had questioned the research design and methodology of the NMI, besides disputing that the NMI had under-represented the readership of its newspaper theSun between 2006-2010.
The suit claims for negligence, defamation and malicious falsehood.
This followed as a consequence of NMI reporting that theSun had an Average Issue Readership (AIR) of only 131,000 daily readers.
This amounted to less than 45% of its independently audited circulation of over 300,000 copies , as certified by ABC Malaysia in 2010.
theSun pursues a free distribution model with strict circulation management and tracking, which in turn is audited by ABC's certified accountants. Advertising revenue is its primary source of income.
Mumbai-based Suresh Nimbalkar has been appointed by Sun Media. He has over 23 years in the research industry, with comprehensive ground level experience across all facets of research design and execution including sampling, fieldwork supervision, data collation and validation, identifying and arresting unethical field influences, weighting and projection of final audience estimation.
For over a decade, Nimbalkar has successfully directed the comprehensive execution of the Indian Readership Survey (IRS), the largest and most complex continuous national media survey in the world involving annual respondent samples of over 260,000.
Nimbalkar in his reports has identified the perceived flaws in the NMI research design, sampling, fieldwork execution, quality control, data validation, weighting and projections.
He also criticised that a research design last reviewed in 1973 would fail to take into consideration all the facets of socio-economic development, including population and income growth plus the evolution of living standards. These would have a direct impact on the survey.
He also questioned the 50% to 60% interview completion success rate claimed by Nielsen through face-to-face interviews. Particularly considering current research-response levels having come down to less than 30%, even for telephone interviews.
Nimbalkar also stated that Nielsen had failed to provide the raw data i.e the respondents' original answers to the survey questionnaire.
Further that, Nielsen had failed to consider that NMI was used as a research currency for the planning and selection of advertising investment by advertising and media agencies. This would have a direct impact on theSun's ad revenue income.
The trial has in fact been delayed due to Nielsen's reluctance to furnish the basic respondents' raw data, from which the NMI final estimates are processed. The Court of Appeal in 2013 had directed Nielsen to comply.
Jephcott agreed that the weighting used in NMI was not optimal. Confirming that "this criticism has some relevance from a technical perspective, due to the complexity of the NMI weighting strategy in terms of its scale and criteria". Also stating that "... certainly if the sample size was increased and the cluster size was reduced, the precision of the survey results would increase and they would be less volatile across reports".
"On the other hand, such actions would involve issues of cost and in turn increase subscriber's fees and, at the same time, would be unlikely to reduce any systematic bias, if it exists," he said.
Plaintiff's counsel Datuk Malik Imtiaz Sarwar told the court that Nielsen has failed to take adequate steps to ensure the integrity of the survey process underlying the NMI and that its integrity has been compromised. "Serious doubt have however been cast over the correctness of these representations. The NMI has come to be seen as controversial in so far as the fairness of its estimates of readership of newspapers is concerned," he said.
Defence counsel Yee Mei Ken said the NMI is a multimedia and product consumption survey, which includes estimation of readership, viewership, listenership and other audiences for newspapers, television, radio, cinema, outdoor point of sale.
Yee said Nielsen does face to face interviews using an annual sample of 10,000 people aged 15 years and above. The results are projected to the estimated population of Peninsular Malaysia.
Jephcott expressed his doubts on the correlation between audited circulation and readership. Particularly voicing his concerns regarding the calculation of Readers Per Copy (RPC).
Nimbalkar stated that over a long period of time audited circulation and readership estimation would reflect similar trends. Irrespective of the intellectual debates on the subject, RPC calculations are still executed by media planners and media sellers using two basic methods. Either the respondent can be directly asked how many other persons (pass-along readers) also read his copy or one can divide the estimated readership by the audited circulation number to derive a rough RPC estimation.
In 2015, for instance, The Star with a weekday (audited) print circulation of 248,413 has been projected by Nielsen to have a daily AIR of 1.41 million. This would amount to approximately 5.7 readers per copy.
In a country like the United Kingdom, where English is the primary language even widely distributed newspapers like Metro and London Evening Standard, reflect around two to three readers per copy, when their estimated readership (UK-NRS) is divided by their (ABC-UK) audited circulation.