RFP #4-10

Consumer Awareness Survey for Impact Assessment of BEE’s S&L Program in India – 2nd Phase

Introduction

The Collaborative Labeling and Appliance Standards Program (CLASP) is an international non-profit organization that promotes energy efficiency standards and labeling in commonly used appliances and equipment. CLASP and its partners in India have provided technical support to India’s Bureau of Energy Efficiency (BEE) on various aspects of the Indian S&L Program, including the design and implementation of the 5-Star Comparative Label.

This document requests proposals for a consumer, retailer and manufacturer awareness survey, as part of a larger impact assessment analysis of the Energy Efficiency Standards and Labeling program implemented by the Bureau of Energy Efficiency (BEE), Government of India.

This market research is a part of the second phase of analysis undertaken by CLASP. CLASP and its partners in India conducted similar research in January 2009 during the first phase of the study, which included consumer, retailer and manufacturer surveys specifically for refrigerators and air conditioners.

Background

Energy efficiency standards and labels shift appliance, lighting and equipment markets toward more energy-efficient technology while dramatically reducing national climate change impacts. The Government of India introduced the Energy Conservation (EC) Act 2001 and created the Bureau of Energy Efficiency as a statutory body to implement the Act. Standards and labels are two major program areas being implemented by BEE. BEE works through committees of experts and stakeholders, including representatives from industry, consumer organizations, and industrial associations, who determine how the program should be implemented on a product-by-product basis. BEE launched the first voluntary comparative energy label for refrigerators in May 2006. Other labeled domestic products to-date are air-conditioners, tubular florescent lamps, storage water heaters (geysers), color televisions, ceiling fans and cooking gas stoves.

Energy efficiency labels are informative labels that are affixed to manufactured products and describe a product’s performance (usually in the form of energy use, efficiency, or energy cost) to provide consumers with the data necessary for making informed purchases. The labels allow consumers to select efficient appliance models. Labels also provide a common energy efficiency benchmark that makes it easier for governments to offer consumers incentives to buy energy efficient products. The effectiveness of energy labels is highly dependent on the awareness of consumers about the labels.

Scope of Work

The focus of the study is on consumer awareness and education and retailer/manufacturer perception about the energy efficiency labeling program for:

  1. Refrigerators
  2. Room air conditioners
  3. Color televisions
  4. Ceiling fans
  5. Geysers
  6. Tubular florescent lamps (tube lights)

It is intended to identify the impact that BEE’s voluntary labeling program has had on the consumers, retailers and manufacturers.

The selected market research company will assist BEE, CLASP, and CLASP’s partners to conduct a survey for assessing consumer awareness, retailer and manufacturer participation in the appliance labeling program. A secondary and more holistic goal of the study is to observe the annual trend in the consumer awareness of the labeling program. A similar survey was conducted in January 2009. This proposed market research is expected to build upon this survey and demonstrate the growth/decay of the awareness of the labeling program.

The consumer oriented research will look into the following qualitative as well as quantitative criteria:

  1. Current priorities of consumers while buying specific appliances – refrigerators, ACs, tube lights, ceiling fans, colour televisions and geysers – including economics, efficiency, size, visual appeal, and after sales service.
  2. Attitude towards energy and environment (awareness).
  3. Influencing factors in purchase decision, including labels, advertisements, selling technique, information sources and price.
  4. Importance of energy cost saving in making purchase decisions.
  5. Awareness of energy issues.

Findings from qualitative research will be used to develop the survey instrument, which need then be administered to a large section of consumers to get statistically measured and validated recommendations.

BEE already has a list of company wise labeled models sold in the market, along with respective star rating and total number of units sold. The support from BEE required for manufacturer survey includes: 

  1. Data related to measures undertaken by manufacturers for attaining energy efficiency:
    1. Upgrades being done to produce labeled or high efficiency products
    2. Research and development investments for energy efficiency
    3. New models being introduced with higher energy efficiency
  2. Data related to quantification of market transformation effect in India for labeled appliances:
    1. Total number of various appliances (labeled and unlabeled) manufactured annually
    2. Total number of appliances (labeled and unlabeled) sold annually and total number of labeled appliances sold annually
    3. Number of labeled products sold region wise. For the purpose of the study the sales data is required separately for both: urban, sub-urban, rural regions as well as geographic regions – northern, eastern, southern, western and central
    4. Data related to the expenses borne by the consumer for energy efficient products – the additional cost of a particular star rating model over an unlabeled model of same capacity.

    The methodology adopted for the study will be decided in consultation with BEE, the CLASP team and the selected market research company. The methodology will broadly include the following:

    • Finalization of sample strategy
    • Designing of the survey technique so as to get the desired information from the consumers
    • Designing questionnaires for the interviews with consumers
    • Format of the analysis report on the consumer survey
    • Soft copies of all the raw data

    Timeline

    The total time period of the study will be 8 weeks (including the reporting period) from the date of commissioning and it is expected that the participating firms should adhere to the timelines strictly. A detailed breakdown of tasks, deliverables and time frame is provided below:

    Activity 1

    Award of Contract

    4 weeks from the issuance of RFP

    Activity 2

    Identifying survey area, sampling method and survey techniques for all tasks

    1 week from the date of commissioning

    Determination of survey area taking into account the country’s diversity
    Determination of sampling methodology to ensure adequate representation considering the following factors:

    • Geographical
    • Economic

    Finalizing survey techniques. These may include the following:

    • Structured interviews
    • Sample strategy
    • Questionnaire/check lists

    Activity 2 Deliverable

    Report defining survey area, sampling methodology and survey techniques

    1 week from the date of commissioning

    Activity 3

    Designing survey model to understand consumer awareness

    1 week from date of commissioning

    Designing a survey to address the following issues:

    • Awareness about energy efficiency
    • Impact of BEE’s promotional campaign
    • Familiarity with Efficiency label
    • Inclination to buy energy efficient product

    Activity 3 Deliverable

    Report defining survey design to get the desired information.

    1 week from date of commissioning

    Activity 4

    First Draft of Survey

    6 weeks from date of commissioning

    A detailed report on the survey conducted including the following:

    Consumer Awareness Research:

    • Factors that drive choice of appliances (considered in the survey) and importance on energy conservation
    • Current usage of appliances
    • Extent of awareness of energy efficiency standards and labels
    • An empirical test of consumer understanding of the label and ability to select the most efficient products among an array of label options
    • Knowledge levels about attitude towards and importance placed on energy conservation
    • Future usage patterns of these appliances and role of energy efficiency label in choice
    • Profiling data (age, education, occupation, income class, usage behavior)

    Manufacturer Research:

    • Related to measures undertaken by manufacturers for attaining energy efficiency
    • Related to quantification of market transformation effect in India for labeled appliances
    • Related to the expenses borne by the consumer for achieving energy efficiency in the product

    (See Scope of Work above for methodology)

    Activity 4 Deliverable

    A compiled report of the whole survey providing the desired information.

    8 weeks from date of commissioning

    Activity 5

    Consultations and Discussions on the first draft

    8 weeks from date of commissioning

    A review of the survey draft

    Activity 6

    Final report on Consumer Awareness Survey

    10 weeks from date of commissioning

    A final report to be submitted incorporating the suggestions made in the review

    Activity 6 Deliverable

    Report on final survey results including the suggestions made in the review

    10 weeks from date of commissioning

    Deliverables

    1. Report defining survey area, sampling methodology and survey techniques, due one week after the commissioning of the project.
    2. Report defining the survey design for obtaining the desired data from consumer, retailers and manufacturers.
    3. Summary Report on consumer survey results, including formats of survey questionnaires
    4. Interim presentation on the collected data
    5. Final Report

    Criteria

    The applicant should have the following qualifications:

    • Expertise in market research including both qualitative and quantitative methods with staff who are experienced focus group moderators, survey designers and interviewers. Applicants without significant track record in this area should not apply.The applicant should include examples of market research, media campaigns, etc. with goals and results. Experience of conducting similar Market Research for BEE is essential.
    • Strong creative capabilities including concept development, copywriting, layout art and graphic design. The proposer must show demonstrated ability to communicate technical information (e.g. energy efficiency) to consumers in an engaging and motivating way, using a variety of communication tools.
    • Media mix/research skills to reach the target audiences.
    • Budget/reach/frequency allocation modeling software and market prioritization model (key to market research)
    • Should prescribe to industry rating reports
    • Access to primary and secondary market research in relevant subject areas, and capabilities in applying research to this market research objective.

    The proposals will be evaluated on the best value to the program according to the relative importance of each criteria indicated and its assigned value for a total of 85 points. All bidders will be invited to the bid opening, and the averaged scores from the evaluating committee will be announced. Thereafter, the financial bids will be opened and scores assigned as follows: The lowest bidder will be awarded the maximum (15) points. The rest of the bids are all awarded points based on a percentile. The study will be awarded to a company whose proposal provides the best value. The criteria and their corresponding weights are:

    • Cost factors: 15 points
    • Technical factors (on the basis of proposed methodology): 85 points

    Submittal

    Interested firms are required to register as CLASP Implementing Partners at before final submittal. You may register to become an Implementing Partner here. Any questions about CLASP’s Partner Process may be directed to Corinne Schneider via email at cschneider@clasponline.org.

    Technical proposals may be submitted electronically in pdf format to Bodhisatya Datta, bdatta@clasponline.org. Hard copies of financial proposals should be sent to the address below:

    B. Datta
    79 Navjeevan Vihar
    Ground Floor
    New Delhi 110017
    India

    The envelope should clearly state ‘Financial Proposal,’ the task Impact Assessment Market Research, and the bidder’s name.

    Annex 1

    Challenges faced in conducting the Phase I of the consumer survey:

    The CLASP and BEE teams expect that bidding agencies will model their methodology and research process taking into account the challenges that surfaced during the first phase of Impact Assessment.

    During the first phase, it was difficult to develop a sample design and framework that could establish pan India representation at 95% confidence level within an acceptable margin of error of 5%. In order to do so, the team started with a sample of 200 per area. In targeting a total of 400 consumer interviews in each region, the same margin of error needed to be maintained across bigger and small cities. However a higher margin of error of up to 10% was assumed to be acceptable in rural areas as the overall awareness was anticipated to be low.

    During execution of research in the rural areas, the actual awareness and intended purchase levels as per the definitions were far lower than anticipated. It was difficult to obtain a sample size of 60 interviews per rural area, because the team had not factored in the highly prominent incidence of gifting and resale of durables. Moving forward, criteria for establishing rural purchasers will need to re-considered and redefined (especially time period elapsed since purchase) so that actual rural usage levels are represented.

    Secondly, the research was required to test the efficacy of a communication burst for BEE that had aired for 6 months and thereafter was decayed. The instruments initially designed to study the communication burst did not cater to a detailed qualitative analysis that could delve down deep into information and messaging relays. This was attempted during the consumer focus groups post a forced exposure in order to elicit consumer speak about the advertising. However, analysis of information and messaging relays is better done with a larger pool of people and for such analysis to be done, there should be minimal communication decay. For instance, owing to considerable decay before taking recall levels, during reporting, it was found that for the Television Campaigns (TVC), stimulus comprising blind shots was not boosting recall levels. Perhaps use of blind shots as an aid for boosting TVC recalls also needs to be reconsidered.

    Thirdly, in conducting the manufacturers’ survey, it was observed that contacting respondents through the conference organized by BEE for Manufacturers enabled responses, whereas contacting respondents outside of the conference did not yield positive responses within the timeframe. It is recommended that a consistent record of contacts in the manufacturers sphere is maintained for further exercises of this nature in order that such feedback is obtained within timelines set for the research.

    Finally, the final figures with respect to energy consumption were estimated from secondary sources as well as validated with primary interviews with manufacturers. Sources from which the estimation is done need to be confirmed as valid by BEE/CLASP partners in order to ensure the final outputs are legitimate.

    Summary of Bids and Decisions

    In keeping with CLASP’s commitment to a fair and transparent evaluation of all bids, we have provided a basic summary of the bids received under this RFP as well as the basic rationale for the selections.

    This was a competitive solicitation and responses were received from two groups:  

    • Market Xcel Data Matrix Pvt. Ltd.
    • Kadence International

    A sub committee comprising CLASP Consultant & Staff and External Technical Experts assessed and rated each proposal based on the technical submissions of the bidders.

    In the evaluation committee meeting an opportunity was extended to the bidders to present their proposal to the 5 reviewers. While both the finalists’ submitted excellent proposals and considered strong and likely to result in a valuable final product, the submission by Kadence International was ranked 3.3 higher than the other, on two key evaluation criteria:

    1. International reputation and credibility of the team, especially the exposure of the bidder towards research for public policy and energy efficiency related projects
    2. Comprehensiveness of the proposed methodology based on the understanding of the project requirements as stated in the RFP.