Challenges faced in conducting the Phase I of the consumer survey:
The CLASP and BEE teams expect that bidding agencies will model their methodology and research process taking into account the challenges that surfaced during the first phase of Impact Assessment.
During the first phase, it was difficult to develop a sample design and framework that could establish pan India representation at 95% confidence level within an acceptable margin of error of 5%. In order to do so, the team started with a sample of 200 per area. In targeting a total of 400 consumer interviews in each region, the same margin of error needed to be maintained across bigger and small cities. However a higher margin of error of up to 10% was assumed to be acceptable in rural areas as the overall awareness was anticipated to be low.
During execution of research in the rural areas, the actual awareness and intended purchase levels as per the definitions were far lower than anticipated. It was difficult to obtain a sample size of 60 interviews per rural area, because the team had not factored in the highly prominent incidence of gifting and resale of durables. Moving forward, criteria for establishing rural purchasers will need to re-considered and redefined (especially time period elapsed since purchase) so that actual rural usage levels are represented.
Secondly, the research was required to test the efficacy of a communication burst for BEE that had aired for 6 months and thereafter was decayed. The instruments initially designed to study the communication burst did not cater to a detailed qualitative analysis that could delve down deep into information and messaging relays. This was attempted during the consumer focus groups post a forced exposure in order to elicit consumer speak about the advertising. However, analysis of information and messaging relays is better done with a larger pool of people and for such analysis to be done, there should be minimal communication decay. For instance, owing to considerable decay before taking recall levels, during reporting, it was found that for the Television Campaigns (TVC), stimulus comprising blind shots was not boosting recall levels. Perhaps use of blind shots as an aid for boosting TVC recalls also needs to be reconsidered.
Thirdly, in conducting the manufacturers’ survey, it was observed that contacting respondents through the conference organized by BEE for Manufacturers enabled responses, whereas contacting respondents outside of the conference did not yield positive responses within the timeframe. It is recommended that a consistent record of contacts in the manufacturers sphere is maintained for further exercises of this nature in order that such feedback is obtained within timelines set for the research.
Finally, the final figures with respect to energy consumption were estimated from secondary sources as well as validated with primary interviews with manufacturers. Sources from which the estimation is done need to be confirmed as valid by BEE/CLASP partners in order to ensure the final outputs are legitimate.