RFP #2-11

The Super-efficient Equipment and Appliance Deployment (SEAD) Initiative: North America Television Test Laboratory Request for Proposal

Introduction

The Collaborative Labeling and Appliance Standards Program (CLASP) is an international non-profit organization that promotes energy efficiency standards and labeling (S&L) for commonly used appliances and equipment. CLASP serves as the Operating Agent for the Super-efficient Equipment and Appliance Deployment (SEAD) Initiative, a Clean Energy Ministerial initiative that leverages high-level political dialogue to advance global market transformation for efficient equipment and appliances. CLASP facilitates interaction among the governments participating in SEAD, as well as the SEAD International Steering Committee and Leadership Council.

Through its activities and projects, SEAD is engaging governments and the private sector to tap the potential of appliance and equipment efficiency. SEAD partners are working together in voluntary activities to: (1) "raise the efficiency ceiling" by pulling super-efficient appliances and equipment into the market through cooperation on measures like incentives, procurement, awards (the focus of this RFP), and R&D investments; (2) "raise the efficiency floor" by bolstering national or regional policies like minimum efficiency standards and labels; and (3) "strengthen the foundations" of efficiency programs by coordinating technical work to support these activities.

Background

In 2012, the SEAD initiative will launch its first international awards competition to recognize the most efficient televisions worldwide. The SEAD Awards Competition strives to advance efficiency improvements in flat-panel televisions by (1) recognizing products with the best energy efficiency performance; and (2) signaling to industry and policy officials what levels of efficiency are achievable with commercially available and cost-effective technology. Television manufacturers will be invited to nominate flat-panel television models sold in Australia, Europe, India, and North America for regional awards.

As an important step in the identification and qualification of test laboratories to support the SEAD Awards program, CLASP is conducting an international round robin test of flat panel televisions to verify regional lab capability and identify potential testing issues prior to verification testing. As the Operating Agent for SEAD and the Administrator of the SEAD Awards Program, CLASP will contract a test laboratory in North America to test a single group of six televisions during round robin testing, at a time to be determined between December 2011 and March 2012. Following the successful conclusion of round robin testing, the test lab will perform verification testing on up to 10 additional televisions between June and July 2012.

CLASP will supply an impartial technical expert (“expert witness”) to observe and oversee round robin testing at the contracted test laboratory. The laboratory will work with the technical expert to ensure proper test conduct.

Scope of Work

As the Operating Agent for SEAD, CLASP has developed a plan (see Appendix) (pdf) that provides details on guidelines and objectives for the SEAD round robin testing project. CLASP seeks to contract a qualified test laboratory in North America to measure the energy use of product samples. The results will be compared to tests conducted at peer laboratories in Europe, India and Australia. Information on laboratory responsibilities, reporting, and deliverables is provided below, along with the project timeline.

The contracted test laboratory will be responsible for the following activities:

  • Agree to a test schedule in collaboration with CLASP, the expert witness, and the other round robin test laboratories. Testing is tentatively planned to occur on 16-24 January 2012.
  • Perform six tests on each of the six round robin test samples according to the IEC 62087 test method, as specified in the SEAD awards rules . [1]
  • Record all test results as specified in program documentation and in consultation with the expert witness.
  • Work with the expert witness to resolve discrepancies and verify results.
  • Examine and verify various conditions for testing, including installation conditions, testing environment conditions, etc. so that all laboratories can test uniformly.
  • Package sample products according to pre-established shipping specifications and pay for customs and shipping costs to ship sample products via air freight to the next test laboratory.
  • Pending successful completion of round robin testing, perform verification testing on up to 10 additional televisions between June and August 2012.
  • Record and report test results from verification testing to CLASP.
  • Abide by confidentiality provisions.


[1] The draft Official Rules for the 2012 SEAD Television Awards Competition can be accessed via the SEAD website at http://superefficient.org/Activities/Awards.aspx

Timeline

Round robin testing is expected to commence in December 2011 and will conclude in March 2012. Round robin testing in North America is tentatively scheduled for 16-24 January 2012.

Deliverables

The contracted test laboratory, in coordination with CLASP, will be required to provide the following outputs over the course of the project: 

  • Report of testing conditions, to include environmental conditions, calibration, test procedures, etc.
  • Report of test results – Provide results of all tests and include in the final report.


Contractor will deliver test results to CLASP within 10 days after testing.  

All deliverables will be provided in unlocked, accessible electronic file formats (preferably Microsoft Office files).

Criteria

To be eligible for a contract with CLASP, you must first register as a CLASP Implementing Partner. Registering is easy, and can be done here.

A committee appointed by CLASP, consisting of CLASP staff and external advisors, will evaluate project and budget proposals received from respondents. Selection of qualified laboratories will be based upon the following criteria: 

  • Laboratory qualifications, including:
    • Located in North America
    • IO/IEC 17025 accreditation
    • Previous experience testing flat panel televisions to the IEC 62087 test method
    • Examples of prior round robin testing experience and reporting results
    • Team composition and qualifications
  • Cost proposal


Proposals will be evaluated using the Quality and Cost-Based Selection (QCBS) method, with weights of 80 percent towards team and organization experience and qualifications, and 20 percent towards proposed costs.  

Additional information about CLASP's selection process may be viewed here.

Submittal

The Consultant is hereby requested to provide a detailed proposal that includes the following: 

  • Test laboratory profile, including a summary of qualifications relevant to this assignment and proof of ISO/IEC 17025 accreditation.
  • Specify the number of tests performed according to test procedures outlined by IEC 62087.
  • Provide examples of prior test reports.
  • CVs and related summaries of experience and qualifications of applicable staff who would perform the round robin testing.
  • Detailed timeline of deliverables and milestones. The contracted laboratory must be able to complete round-robin testing between December 2011 and March 2012 and verification testing between June and August 2012.
  • Detailed budget that includes a breakdown of costs for staff, equipment, and test execution, with test costs listed on a per-test basis.


If necessary for the selection process, CLASP may request additional information from any applicant.

Proposals should be submitted online using the “Submit Bid” button above and filling out the requested information. The deadline for submissions is December 2, 2011.

All questions may be directed to Anna Lising at alising@clasponline.org. We request all inquiries be made via email and not by phone.

Funding for this project is provided by the U.S. Department of Energy and the project shall be performance in accordance to 10 CFR Part 600.

Appendix

Round Robin Test Plan for 2012 Televisions Awards

Introduction
Round robin testing, also known as inter-laboratory testing, is a process whereby individual product samples are tested at two or more different laboratories, and results are compared to ensure consistency in test results, address potential issues prior to the Television Awards verification testing, and identify any improvements needed in testing capabilities.

Contents

  • Objectives
  • Round Robin Testing Scheme
  • Testing Guidelines
    • Test Laboratory Selection
    • Procurement of Test Samples
    • Reporting
    • Transportation of Samples
  • Timeline

I. Objectives of Round Robin Testing

Just as processes that produce a product may vary, the process of performing a test and obtaining measurements is subject to variation. RRT between laboratories is a way of identifying many of the issues that cause variation in measurement. RRT is concerned primarily with identifying issues with:

  • Laboratory practice;
  • Measurement process;
  • Calibration and traceability of equipment;
  • Measuring conditions;
  • Improving measurement standards;
  • Repeatability and reproducibility of the test method; and
  • Test laboratory capability.


II. Round Robin Testing Scheme

SEAD will utilize a sequential testing scheme for television round robin testing. Each group of test samples will be transported from one laboratory to the next. Tests will be performed with samples in “out of box” condition to minimize opportunity for variation in test results. The first laboratory to test the product samples will record the “out of box” settings and screen luminance levels and report the information to subsequent laboratories. Each following laboratory will check the product settings and luminance levels before measurement and report any discrepancies. The international expert will advise for resolution of any discrepancies.

III. Guidelines for Round Robin Testing

An impartial technical expert (also referred to as the “expert witness”) will observe and oversee testing at each test laboratory. Testing will be conducted with suitable, properly calibrated instrumentation and equipment at each laboratory. All test samples will be tested as a group, starting with a sample validation process.

Test laboratories will perform strict sample performance homogeneity testing with identical test methods and procedures, and will conduct additional examination and verification of testing conditions (installation, testing environment, etc.). Six reproducibility tests are required at each test laboratory to facilitate detection of variations not due to laboratory conditions or techniques (e.g. sample stability), and to provide better information with which to assess laboratory capability. Product samples procured for round robin testing will require a power supply of 230 VAC / 50 Hz. The North American test laboratory will utilize a 230 VAC / 50 Hz power supply to ensure homogeneity in testing conditions across all participating regions.

An instruction manual will be distributed to participating laboratories prior to the RRT commencement. The manual, which will be developed by the technical expert, will detail specific instructions on the following aspects of the round robin testing: 

  • Test laboratory and expert witness responsibilities;
  • Product sample receipt and validation observation and measurement;
  • Test procedure requirements;
  • Reporting uncertainty and uncertainty calculation details;
  • Reporting test results and observations;
  • Criteria for assessment of results, including a statistical analysis;
  • Requirements for packaging and shipment of product samples.

A. Test laboratory selection
A total of four laboratories (one in each region participating in the 2012 SEAD Television Awards – Australia, Europe, India, and North America) will be selected for round robin testing and potentially for verification testing for candidate award-winning television models.

Prospective test laboratories with ISO/IEC 17025 accreditation with a scope that covers the IEC 62087 Ed. 3.0 test method will be preferred. Test laboratories will be required to complete questionnaires about their technical qualifications and expertise, and to provide examples of prior test reports. Test laboratories will be required to specify the number of tests performed according to test procedures outlined by IEC 62087. A Request for Proposal (RFP) will be issued to laboratories that meet the screening criteria. Final laboratory selections may require in-person evaluations by the technical expert.

Selection of a laboratory for round robin testing does not guarantee that the test laboratory will also be contracted for verification testing. Laboratory selection for verification testing will be based on results of the RRT process, which will be summarized in a SEAD-internal report. 

B. Procurement of test samples

Approximately six sample products will be tested. Test samples shall be procured from different retailers, and, if more than one sample of a single model is used, from different manufacturing regions and batches. All sample products will be capable of handling high definition digital broadcast signals and will meet other pertinent eligibility criteria for the SEAD TV awards.

C. Reporting

Each test laboratory must complete a test report to ensure consistency in testing conditions, calibration, test procedures, etc. Results will be reviewed by the expert witness to detect any anomalies or errors prior to shipment of the test samples to the next test laboratory. The reports are to be submitted electronically to CLASP for consolidation and analysis. The expert witness will investigate possible causes of any inconsistencies in test results. Once round robin testing is completed, the expert witness will produce a final RRT report to summarize test results from each test laboratory. The report will include a robust statistical analysis of the test results along with a summary of product sample model details and settings, test equipment details including calibration dates, and environmental conditions for each test laboratory. 

D. Transportation of test samples

Prior to testing, each laboratory must measure and record the “out of box” settings and luminance measurements. In addition, the settings and luminance measurements of any other modes used for testing will be recorded and reported. After testing, each test laboratory will return samples to original factory settings. The expert witness will review each test laboratory’s results to ensure that any necessary re-testing is conducted prior to packaging and shipment of test samples to the next laboratory.

To avoid damage to the test samples, each test laboratory will package the sample products using wooden crates that will store two or more televisions each. The expert witness will specify the shipping crate design and packing procedures for transporting the test samples. Custom packaging will add extra protection for the televisions and will keep the samples together to reduce the possibility of losing any test samples during transportation. The first test laboratory will provide the shipping crates and packaging materials specified by the expert witness.

Test laboratories will adopt necessary measures to ensure test sample packaging is sound enough to withstand typical international shipping hazards. Test samples will be shipped via air freight to expedite the RRT process and ensure reliability in delivery. Upon receipt of the test samples, test laboratories are to record any damage that may have occurred during shipment. 

IV. Timeline

The laboratory selection process will occur in November. Round robin testing will commence in early December, once contracts are in place and sample products are procured. Although actual testing of sample products is expected to take no longer than one week per laboratory, four (4) weeks of testing is allotted per laboratory to allow for transportation, customs, and results reporting. Approximately 4 months has been scheduled for RRT with an anticipated completion date of 30 March 2012.

Sample products will be procured and custom shipment packaging will be procured by the technical expert. Contracted test laboratories will be responsible for shipping and customs costs to transport the product samples to the next test laboratory.

Summary of Timeline and Deliverables

Deliverable Start End
Procure product samples in Australia 28 Nov. 2011 9 Dec. 2011

Round Robin Testing

9 Dec. 2011 30 Mar. 2012
  • Australia
  • North America
  • Europe
  • India

1 Dec. 2011
16 Jan. 2012
13 Feb. 2012
20 Mar. 2012

13 Dec. 2011
24 Jan. 2012
24 Feb. 2012
30 Mar. 2012

Summary of Bids and Decisions

In keeping with CLASP's commitment to a fair and transparent evaluation of all bids, following is a summary of the bids received under RFP#2-11 “The Super-efficient Equipment and Appliance Deployment (SEAD) Initiative: North America Television Test Lab Request for Proposal”, as well as the basic rationale for the final selection.

This was a competitive solicitation, and technical and financial proposals were received from four organizations: Intertek Testing Services NA, Inc., MET Laboratories, Inc., Underwriters Laboratories (UL), and an anonymous organization.

While all four bidders submitted strong proposals and were considered viable candidates for successful implementation of the project, the review committee shortlisted proposals from UL and MET Laboratories for their technical expertise, experience and understanding of project objectives.  The submission from Underwriters Laboratories was considered superior due primarily to the experience and qualifications of the team, their experience in testing televisions in accordance with IEC 62087 and IEC 62301 test methods, their familiarity with round-robin testing, and the comprehensiveness of the proposed methodology based on an understanding of the project requirements as stated in the RFP.

Proposals were evaluated by a three person review team according to pre-defined criteria regarding the proposed approach, methodology, and project execution plan, extent of experience in television testing using IEC 62087 and 62301 test methods, and cost projections in accordance with the RFP.
Proposals were scored based on a Quality and Cost Based System (QCBS) totaling 100 points, with 80 points allocated to the technical proposal evaluation and 20 points allocated to the financial proposal evaluation.

Final scores for the shortlisted proposals are provided in the table below:

 

Organization

Total Score

MET

83

UL

88

 

Please contact Anna Lising, SEAD Program Coordinator, at alising@clasponline.org with any further questions.

CLASP and SEAD transform the global market for efficient appliances.

CLASP Supports the SEAD Initiative

Donec a nisl eget leo varius facili. Pellentesque sit amet. Donec a nisl eget leo varius facili. Pellentesque sit amet. Donec a nisl eget leo varius facili. Pellentesque sit amet. Donec a nisl eget leo varius facili. Pellentesque sit amet. Donec a nisl eget leo varius facili. Pellentesque sit amet. Donec a nisl eget leo varius facili. Pellentesque sit amet. Donec a nisl eget leo varius facili. Pellentesque sit amet. Donec a nisl eget leo varius facili. Pellentesque sit amet.

Read More