RFP_23NC0465_CPUC_EE_Group_B_Oversight.pdf

STATE OF CALIFORNIA GAVIN NEWSOM, Governor

PUBLIC UTILTIES COMMISSION

505 VAN NESS AVENUE

SAN FRANCISCO, CA 94102

California Public Utilities Commission (CPUC) Request for Proposal (RFP) – Secondary Number 23NC0465

For:

Energy Efficiency Program Oversight and Evaluation of Group B Program Sectors: Market Support; Equity; Codes & Standards Development and Adoption; and Community Choice

Aggregators, Regional Energy Networks and Local Government Partnerships

July 16, 2024

The California Public Utilities Commission (CPUC) invites you to review and respond to this Solicitation. In submitting your response, you must comply with the instructions found herein.

This is an open solicitation for private entities, non-profit organizations, the University of California, California State University Foundations, and other governmental entities.

CPUC currently has another RFP posted for Group A Energy Efficiency. Proposers may submit responses to both RFPs, but cannot be awarded both contracts. Group A will be awarded first, the Proposer selected for that contract will then become ineligible to be awarded the contract for Group B.

In this RFP, CPUC Energy Division is seeking a Prime Contractor to carry out program evaluation and related studies for various non-resource and cross-cutting program areas in the Market Support, Codes and Standards and Equity segments as well programs from other non-Investor Owned Utility (IOU) program administrators.

The Contractor will develop Research Sector Work Plans for each area of the contract, as described in Deliverable 1: Research Sector Work Plans. Proposers should identify their approach to address the following factors:

a. The basis for selection savings and program evaluation or research approaches;

b. Knowledge of and experience in various evaluation techniques; and

c. Tools and methodologies which the Proposer anticipates using in each Deliverable.

Before submitting a response to this solicitation, interested parties should read and review the entire solicitation including any attachments, all referenced web addresses, and any regulations, orders, and statutes cited.

Any questions regarding the solicitation or terms and conditions of the proposed contract must be submitted by the deadline listed in the key action dates. Terms and conditions will not be negotiated after contract award.

Responses are due Friday, August 30, 2024, by 5:00 PM Pacific Time. If you have questions or need any clarification of information contained herein, please send your questions via e-mail to the assigned CPUC Point of Contact below by the date as specified in the Key Action Dates.

Total proposal cost must not exceed $16,000,000.00 (sixteen million dollars) for the four-year term. Proposals exceding these amounts will be automatically disqualified.

CPUC Point of Contact:

Alex Ritchie alexander.ritchie@cpuc.ca.gov

California Public Utilities Commission RFP Number 23NS0465

TABLE OF CONTENTS

SCOPE OF WORK AND CONTRACT TERMS

Exhibit A – Scope of Work

1. Background

2. Scope of Work

3. CPUC Responsibilities

4. Location of Services and Work Environment

5. Amendments

6. Contact Information

EXHIBIT B – BUDGET DETAIL AND PAYMENT PROVISIONS

1 Invoicing and Payment 2 Budget Contingency Clause 3 Prompt Payment Clause

EXHIBIT C – GENERAL TERMS AND CONDITIONS (GTC 04/2017)

EXHIBIT D – SPECIAL TERMS AND CONDITIONS

1. MINIMUM QUALIFICATIONS

2. ADMINISTRATIVE INFORMATION

3. SUBMISSION REQUIREMENTS

4. PREFERENCE AND INCENTIVE PROGRAMS

5. EVALUATION CRITERIA AND PROCESS

6. AWARD AND CONTRACT EXECUTION

7. REQUIRED ATTACHMENTS

ATTACHMENT 1 – SOLICITATION CHECKLIST

ATTACHMENT 2 – CONFLICT OF INTEREST STATEMENT

ATTACHMENT 3 – REFERENCES SHEET

ATTACHMENT 4 – PAYEE DATA RECORD (STD 204)

ATTACHMENT 5 – PAYEE DATA RECORD SUPPLEMENT (STD 205)

ATTACHMENT 6 – CONTRACTOR CERTIFICATION CLAUSES (CCC 04/2017)

ATTACHMENT 7 – DARFUR CONTRACTING ACT FORM

ATTACHMENT 8 – BIDDER DECLARATION FORM (GSPD-05-105)

ATTACHMENT 9 – DVBE DECLARATIONS (DGS-PD-843)

ATTACHMENT 10 – COMMERCIALLY USEFUL FUNCTION (CUF) CERTIFICATION. 67

ATTACHMENT 11 – IRAN CONTRACTING ACT ATTACHMENT

ATTACHMENT 12 – CALIFORNIA CIVIL RIGHTS LAW ATTACHMENT

ATTACHMENT 13 – COST SHEET AND SAMPLE AGREEMENT

ATTACHMENT 14 – EVALUATION AND SCORING CRITERIA

SCOPE OF WORK AND CONTRACT TERMS

EXHIBIT A – SCOPE OF WORK

1. Background

In Decision 23-06-055, which Authorized Energy Efficiency Portfolios For 2024-2027 And Business Plans For 2024-2031, as well as Decision 23-08-005, which adopted Energy Efficiency Goals for 2024-2035, the California Public Utilities Commission (CPUC or the Commission) authorized an annual investment of approximately $1 billion of energy utility customer funding per year (2024-2027) for the implementation of energy efficiency programs by Energy Efficiency Program Administrators (PAs) in California, including investor-owned utilities and non-utility program administrators.1 This funding will pay for a portfolio of programs across resource, market, and equity segments that are intended to realize significant energy savings and greenhouse gas reduction, and make progress toward accomplishing the legislative mandates established in California Public Utilities Code Sections 455.5 and 381.2.2

In order to ensure the long-term success of these dynamic energy efficiency portfolios, the Commission is committed to an integrated program of ex ante energy savings measurements, estimations and ex post evaluation activities to forecast, track, and measure energy savings3 from resource programs (as well as total system benefits) and related progress in achieving various market and equity segment goals as laid out in the program administrators’ business plans for 2024-2027.

The main drivers of the scope and timing of work required for energy savings estimation and evaluation activities include Commission decisions and legislative mandates.

The California Long-term Energy Efficiency Strategic Plan (CPUC, 2008) and The Existing Buildings Energy Efficiency Action Plan (CEC, 2015) chart the overall policy vision for energy efficiency and align with legislative mandates. In relevance to this RFP, the common underlying goal of the strategic plans, legislative mandates and Commission decisions is to accelerate and scale the adoption of energy efficiency as a reliable resource with high assurance coming from ratepayer funded energy efficiency programs. The Commission continues to strive to improve the accuracy, speed and cost-effectiveness of the ratepayer funded program evaluation process. This RFP invites submittal of innovative proposals that will meaningfully refresh and/or replace elements of the existing evaluation program while maintaining enough continuity that a smooth transition to new processes is ensured.

1 The California Energy Efficiency Program Administrators are Pacific Gas & Electric, Southern California Edison, San Diego Gas & Electric, and San Diego Gas (the Investor Owned Utilities, IOUs or Utilities); Southern California Regional Energy Network and Bay Area Regional Energy Network; and Marin Clean Energy, a Community Choice Aggregator.

2 California Public Utilities Code:

http://leginfo.legislature.ca.gov/faces/codesTOCSelected.xhtml?tocCode=PUC&tocTitle=+Public+Utilities+Code+-+PUC 3 References to energy savings estimates and measurements throughout this document include all relevant parameters for calculating cost-effectiveness in the Cost Effectiveness Tool.

For example, earlier legislative mandates, including but not limited to Assembly Bill 802 (Williams, Chapter 590, Statutes of 2015)4 and Senate Bill 350 (De’ Leon, Chapter 547, Statutes of 2015)5, brought substantial changes to the Commission’s savings measurement, estimation and evaluation methods over the last few years. Likewise, newer legislation at the state level, targeting greenhouse gas reductions (Assembly Bill 3232, Friedman, Chapter 373, Statutes of 2018) and building electrification programs (Senate Bill 1477, Stern, Chapter 378, Statutes of 2018), as well as Federal legislation such as the Inflation Reduction Act (IRA), will influence those methods over the lifetime of this contract.

Additionally, the schedule of the necessary savings estimation and evaluation activities updated in Commission Decision 21-05-031,shown in Figure 1.A, below, remains in place Note that firm deadlines, referred to in the decision and this RFP as bus stops, are adopted for evaluation, measurement, and verification (EM&V) studies, Database of Energy Efficient Resources (DEER) updates, and Goals updates. Contractor due dates are typically 2 weeks to 1 months ahead of the Commission due dates shown in Figure 1.A below.

Figure 1.1 – Energy Efficiency Annual EM&V Schedule6

4 Assembly Bill 802 mandates that the Commission shall authorize IOUs to provide incentives “based on all estimated energy savings and energy usage reductions, taking into consideration the overall reduction in normalized metered energy consumption as a measure of energy savings.”

5 Senate Bill 350 mandates the doubling of energy efficiency impacts by 2030 based upon normalized metered electricity and natural gas consumption data.

6 Appendix 6 of D.15-10-028.

(Continued on next page)

Further, in D.09-09-047, the Commission adopted policy goals for EM&V to support its objectives and guide the work of the Commission’s Energy Division and the PAs. These goals and core objectives continue to guide the Energy Division’s EM&V work:

“EM&V activities shall be planned and implemented to achieve the following core objectives in order to support the Commission’s oversight function of ensuring the efficient and effective expenditure of ratepayer funds within the energy efficiency portfolios. All activities should be undertaken to meet the overarching goals of clarity, consistency, cost-efficiency, and timeliness.

The core objectives are:

I. Savings Estimation, Measurement, and Verification – Estimation, measurement, and verification of savings resulting from PA resource programs, including energy efficiency measures, programs, and portfolios serve the fundamental purpose of developing estimates of reliable load impacts delivered through ratepayer-funded efficiency efforts. EM&V work should reflect a reasonable balance of accuracy and precision, cost, and certainty, and be designed for incorporation into in procurement planning activities. Note that Market Support and Equity programs, though lacking a priority resource saving component, may nevertheless lead to energy savings that the IOU program administrators may count towards their respective Total System Benefit achievements (see Item II below).

II. Total System Benefit (TSB) – Commission Decision 21-05-031 moved Commission policy in the direction of setting energy efficiency potential and goals based on TSB. TSB aggregates the various benefits of energy efficiency, rather than assessing and assigning separate electricity, demand and natural gas savings goals that were traditionally used in past program cycles, into an annual dollar-based value for each IOU.

III. Inflation Reduction Act (IRA) – As noted in D.23-06-055, the IRA dovetails with existing and future market access resource programs, in that the residential and commercial downstream retrofit markets include projects targeted by the IRA.

Consequently, the IOUs (and other PA) may be able to complement ratepayer funding for these programs with IRA funding. Any program or programs deploying this complementary funding approach would require evaluation to determine the impacts on program participation and savings due to the augmented funding approach and potential for multiple attribution sources.

IV. Program Evaluation – Evaluation of program-specific qualitative and quantitative measures, such as the program performance metrics, key performance indicators (KPIs), and process evaluations, serves a key role in providing feedback for the purposes of improving performance and supporting forward-looking corrections to utility programs and portfolios. In order to maximize return on ratepayer dollars, program evaluations must be completed on a timeline, which informs mid-course corrections and/or program planning for the following cycle.

V. Market Assessment – In a constantly evolving environment, market assessments are an essential EM&V product needed to set the baseline for strategic design and improvement of programs and portfolios. Saturation studies, surveys of emerging technologies and other such analyses which inform estimates of remaining program potential and forward-looking goal-setting are key aspects of market assessment.

VI. Policy and Planning Support – Consistent with prior program cycles, it is essential to reserve funding to support overarching studies and advisory roles which support Commission policy goals. Over the last program cycle this has been inclusive of Potential and Goals studies, maintenance of DEER database, developing databases of best practices for program design and delivery, program design mix, and other means which support the Commission’s oversight role, but do not fall under the core EM&V categories described above.

VII. Financial and Management Audit – Supporting the Commission’s oversight function of ensuring the efficient and effective expenditures of ratepayer funds within the utilities’ energy efficiency portfolios is another objective of EM&V activities. Rigorous financial and management audits overseen by Commission Staff will be critical in ensuring that the utilities’ general and administrative costs and other program expenditures are prudent and reasonable.”7

Beginning in program year 2022, Decision 21-05-031 ordered that all energy efficiency program administrators shall segment their portfolios into three categories, with the primary purposes of resource acquisition, market support, and equity. The definitions of these segments can be found in Decision 21-05-0318 and are pasted below for your reference:

Resource Acquisition: Programs with a primary purpose of, and a short-term ability to, deliver cost-effective avoided cost benefits to the electricity and natural gas systems. Short-term is defined as during the approved budget period for the portfolio, which will be discussed further later in this decision. This segment should make up the bulk of savings to achieve TSB goals.

Market Support: Programs with a primary objective of supporting the long-term success of the energy efficiency market by educating customers, training contractors, building partnerships, or moving beneficial technologies towards greater cost-effectiveness.

Equity: Programs with a primary purpose of providing energy efficiency to hard-to-reach or underserved customers and disadvantaged communities in advancement of the Commission’s Environmental and Social Justice (ESJ) Action Plan9; Improving access to energy efficiency for ESJ communities, as defined in the ESJ Action Plan, may provide corollary benefits such as increased comfort and safety, improved indoor air quality, and more affordable utility bills, consistent with Goals 1, 2, and 5 in the ESJ Action Plan.

Further, it should be noted that the Commission expects to evaluate current 2024-2027 resource programs that employ either Pay-for-Performance or program-embedded measurement and verification.

Evaluation budgets for the IOUs have shifted from 27.5% to 40% of the total in recognition of the Program Administrators’ increased emphases on 1) Normalized Metered Energy Consumption and Pay-for-Performance programs, and 2) continuing planning and market assessment associated with the market transformation and other program features highlighted in SB 350 and AB802.10

All sectors and markets being evaluated under the EM&V Plan will be organized into multiple contracts for staggered release. The evaluation budget for this RFP, and all Commission staff led energy efficiency RFPs, has shifted from 72.5% to 60% of the total.

Therefore, the upper limit of this RFP is set at $12,000,000.00 (twelve million dollars), which corresponds to the cap of 60% for the Commission’s evaluation budget for all contracts.

This RFP is the second solicitation for evaluation under the EM&V Plan, and will address Group B Sectors: Market Support; Equity; Codes & Standards Development and Adoption;

7 D.09-09-047, page 299-301.

8 D.21-05-031, page 14-15.

9 For more information, see the following link: https://www.cpuc.ca.gov/esjactionplan/ 10 See Commission Decision D.16-08-019:

https://docs.cpuc.ca.gov/PublishedDocs/Published/G000/M166/K232/166232537.PDF and Community Choice Aggregators, Regional Energy Networks and Local Government Partnerships.

2. Scope of Work

The following section describes the overall work that will be completed under the Energy Efficiency Evaluation Contract for Group B Sectors: Market Support; Equity; Codes & Standards Development and Adoption; and Community Choice Aggregators, Regional Energy Networks, and Local Government Partnerships.

Energy Division Project Managers will be members of Commission Staff assigned to manage each research roadmap or sector, and serve as a point-of-contact for specific projects. No work will be performed under the Prime Contract without a prior approval by the assigned Energy Division Project Manager, except where specifically described in Sections 2.2.0 through 2.10.

In order to facilitate the current portfolio cycle, the prime contractor must complete work products to meet the bus stops or future proceeding due dates. This will require more integrated project management and coordination between the EM&V work products and energy efficiency proceeding updates through rulings, decisions or resolutions.

The Prime Contractor will be responsible for the execution of Commission Staff-led research outlined in the Contract Group B research roadmaps. All evaluation studies must be conducted in accordance with the California Energy Efficiency Evaluation Protocol.11

In addition, the Prime Contractor for Contract Group B will serve as the primary point of contact to coordinate with the Prime Contractors for the other Contract Groups on the other energy efficiency evaluation contracts to collaboratively update estimated energy savings values; plan, execute, and oversee EM&V studies; and engage with stakeholders in public meetings and workshops to effectively communicate both the content and impacts of the work outlined in this RFP. This contract will also include coordinating the planning, oversight, or implementation of market studies and/or process evaluations run by the Commission’s contractors or program administrators.

For the research roadmap listed in Section 1, Background, the Contractor and Commission Staff will review the research priorities in the research sector, including updates to savings parameter estimates. Commission Staff have initiated priority and gap analyses to define the proposed studies in the plan, but expect that certain aspects of the work will remain for the Prime Contractors to complete. The main objectives for Group B Sectors: Market Support;

Equity; Codes & Standards Development and Adoption; and Community Choice Aggregators, Regional Energy Networks and Local Government Partnerships include the following:

a. Research Plan;

b. Data Collection;

c. Impact Analysis;

d. Process Analysis;

e. Market Studies;

11 California Energy Evaluation Protocols: Technical, Methodological, and Reporting Requirements for Evaluation Professionals.

f. Project Management;

g. Strategic Planning; and

h. Policy Advocacy and Education

Task 1 EM&V Plan and Work Plans

The Group B Prime Contractor will work with Energy Division Project Managers, other Commission energy efficiency evaluation contractors, and other Commission Staff to maintain and update the EM&V Plan for the sector or program areas they are assigned. This plan serves to guide the scope of evaluation work to be conducted by Commission program staff and their contractors, and outlines specific projects that are expected to be completed and timelines for execution. Energy Division is responsible for maintaining the EM&V Plan and may request modifications to Deliverable 1: Research Sector Work Plans as necessary throughout the contract term. The contractor tasks associated with maintaining and updating the research plans includes:

a. Developing a Research Sector Work Plan for each of the specified research sectors;

b. Assisting with the inventory and prioritization of savings estimation and EM&V projects defined in the research roadmaps and required and recommended by Commission orders, which may include:

i. Modifying the Research Sector Work Plans to streamline studies,

ii. Leveraging data and work performed for other studies in the portfolio,

iii. Reducing cost, and

iv. Setting priorities;

c. Developing detailed work plans for each Deliverable, reflecting prioritization and timelines for implementing the specific projects described in the research roadmaps;

d. Incorporating savings estimation data needs into research plans where applicable;

e. Defining technical and procedural protocols for conducting research projects;

f. Developing consistent methods for conducting the evaluation and measurement of specific types of energy efficiency program activities (examples include analyzing free ridership or spillover);

g. Participating in Project Coordination Groups or other stakeholder groups on an on-going basis as directed by Commission Staff leads; and

h. Conducting reviews of research needs with the Project Coordination Groups on an annual basis, and updating the Research Sector Work Plans; as necessary, which includes:

i. Assisting with the annual inventory and prioritization of EM&V projects proposed by program administrator staff, Commission Staff, stakeholders, and Prime Contractors, and

ii. Maintaining records of data and analysis inventory, and prioritization for future needs assessments in updates to the research roadmaps.

Deliverable 1: Four Research Sector Work Plans

(Narrative Required)

The Contractor must develop a Research Sector Work Plan for each research sector which will describe the methodology and approach the Contractor will take to complete all subsequent Deliverables and studies for each sector addressed under this contract. The research sectors are as follows:

1) Market Support

2) Equity

3) Codes and Standards;

4) Community Choice Aggregators, Regional Energy Networks and Local Government Partnerships

Each Research Sector Work Plan must include:

a. A detailed description of the Contractor’s approach and methodology for each research sector-specific Deliverable, and how the approach is statistically valid;

b. A list of major activities, tasks and sub-tasks, and milestones for each Deliverable;

c. A timeline for completion of each task and a schedule for both one-time and recurring activities;

d. A list of resources and personnel necessary for each Deliverable;

e. The relevant skills and experience of key personnel who will work on each Deliverable; and

f. A detailed budget for each Deliverable with estimates broken down by task and activity.

Possible Activities in the Research Sector Work Plan may include, but are not limited to, the following:

a. Gather data and verify the accuracy of data and data sources;

b. Engage stakeholders;

c. Evaluate, choose, and implement tools and methods;

d. Identify the skills needed on a team to perform the work;

e. Manage the project and communication with Commission Staff;

f. Coordinate with commission staff to obtain approval for all evaluation decisions made at the protocol level;

g. Describe potential barriers and opportunities to providing annual results;

h. Achieve a balance among:

i. Completeness of data,

ii. Representativeness of sample,

iii. Meaningful results,

iv. Cost of reporting,

v. Collaboration with Commission Staff and contractors,

vi. Stakeholder engagement and review, and

vii. Disputes over the methods and/or results; and

i. Update savings estimation and evaluation approaches in the Evaluation Plan to address specific requirements reflected in AB802 and SB350, including:

i. Baselines for the deemed savings estimates for the highest impact measures,

ii. Requirements of the legislation and the Commission’s High Opportunity Projects and Programs framework for review,

iii. Acknowledgement of the Program Administrators’ proposals for methods to conduct a normalized metered energy consumption assessment and the results from those proposals,

iv. Benefits and challenges of the proposed approach, and

v. A plan for using evaluation results to inform future savings estimation and program design.

The Contractor must submit each of the final drafts of the Research Sector Work Plans within 30 days of contract execution. The Contractor must work with Commission Staff to update the Research Sector Work Plans annually to ensure alignment with Energy Division’s EM&V Plan and Commission priorities.12

12 Energy Efficiency Evaluation, Measurement, and Verification Plan Version 9.

Task 2: Administration and Project Management

Administration and Project Management includes planning and leading specific studies as well as coordinating activities within and across the program areas assigned to the contractor. The Contractor shall provide a Project Manager capable of managing the general administration, proactive project coordination, project planning, and internal and external communication necessary to complete the Deliverables described in this section. The Contractor’s Project Manager will be the primary point of contact for the contract and must comply with the following scope of administration and project management requirements when performing services described in this solicitation:

a. Provide and implement a well-organized and complete plan for all activities, tasks, and milestones;

b. Monitor the schedule, budget, and specifications;

c. Monitor resources and expenditures to ensure fulfillment of project goals and objectives within the approved budget;

d. Obtain approval from the appropriate Energy Division Project Managers for all work performed;

e. Obtain approval from the appropriate Energy Division Project Managers for expenditures that were not individually listed in the Cost Sheet, prior to undertaking such expenditure;

f. Obtain approval from the appropriate Energy Division Project Managers for any replacement, substitution, or addition of Contractor staff under the contract;

g. Obtain approval from the appropriate Energy Division Project Managers for any replacement, substitution, or addition of sub-contractors under the contract;

h. Carry out provisions of the contract according to the agreed upon specifications, budgets, and schedules;

i. Identify and report any problems in a timely manner, and recommend and implement solutions;

j. Hold project meetings with Commission Staff as directed by the appropriate Energy Division

Project Managers to discuss progress, needs, problems, solutions, and Deliverables;

k. Monitor and track quality of performance, staff, subcontractors, and other relevant factors to ensure contract compliance and excellence;

l. Interact with the appropriate Energy Division Project Managers, and other Commission Staff and managers professionally and efficiently;

m. Understand and comply with the Commission’s administrative processes and procedures;

n. Respond to the identified workload in a timely manner and consistently meet all work product and event deadlines;

o. Maintain a working knowledge of existing and new Commission programs applicable to the contract;

p. Maintain and utilize existing administrative systems needed to manage the contracted work; and

q. Perform general administration and project coordination functions as guided by Deliverable 02:

Progress Reports and Updates for the purposes of administering and coordinating all the work performed under the contract.

Deliverable 2: Monthly Progress Reports and Updates

Prime Contractor must submit a monthly progress report to the ED Contract Manager and ED Project Managers and/or other entities at the discretion of the ED Contract Manager that will include the following information: progress made on each deliverables, deliverable related activities, invoicing updates, or other information upon request.

At the discretion of the ED Contract Manager for additional financial or deliverable tracking purposes, the Contractor will provide an additional report, up to once per month.

Deliverable 3: Contract Manager Meetings The Contractor will attend periodic check-in meetings with the ED Contract Manager to report on contract invoicing, evaluation project timelines, and other contract or evaluation matters. The ED Contract Manager will determine the appropriate frequency for meetings, up to two meetings per month, for the duration of the contract.

Deliverable 4: Kickoff Meeting

Upon execution of the contract, a virtual kickoff meeting will be held between CPUC Energy Division staff and the Contractor’s team. This meeting is to:

a. Orient the Contractor’s project management team to the administrative and technical expectations of the Energy Division project managers;

b. Initiate discussion of prioritization of tasks;

c. Affirm expectations, communication protocols, and objectives for the contract;

d. Determine what actions are necessary to finalize the EM&V Master Plan by

December 31, of each year.

Deliverable 5: Project Coordination Meetings

The Contractor will coordinate and attend 1 to 4 monthly project coordination group meetings (up to 48 meetings per year) with Commission Staff, program administrators, program implementers, other Commission contractors, and other parties on an ad hoc basis or regular schedule—up to one meeting per month per research sector throughout the duration of the contract—as directed by each research sector’s Energy Division project manager. The purpose of the project coordination group meetings is to provide a venue to discuss the status of sub-tasks across the research sectors with Commission Staff and stakeholders; strategize for coordination between Prime Contractors as needed; and coordinate evaluation activities with the program administrators’ staff.

Task 3: Stakeholder Engagement

The Contractor Team will participate or host various types of stakeholder engagements to ensure a proper public vetting process throughout the life of the contract. Expected activities to complete this task include:

a. Presenting at public meetings and workshops, including quarterly EM&V stakeholder meetings, in person at CPUC in San Francisco and via webinar;

b. Participating and collaborating in relevant monthly Project Coordination Groups and other stakeholder groups;

c. Addressing issues that members of the public raise at the stakeholder workshops with specific documentation of how each comment from the Stakeholder Workshop has been considered or addressed;

d. Managing written and oral communication, including public briefing materials, summarizing work products to decision-makers and stakeholders, as requested by Commission Staff;

e. Conducting stakeholder workshops and webinars to solicit feedback from stakeholders.

The stakeholder workshops dates are to be determined upon contract execution.

f. Explain ongoing studies and upcoming reports, and results and findings upon study completion.

Deliverable 6: Stakeholder Workshops and Webinars

The contractor will participate in a maximum of 200 stakeholder engagement activities throughout the contract, at which they must be able to explain their savings estimation and evaluation approaches and results to an audience of both technical and non-technical individuals. At least two key team members will be required to participate in each stakeholder engagement activity. Upon completion of the workshops, the Contractor will provide a summary of the data and information gathered.

Task 4: Data Collection and Transfer

Transforming data into meaningful, comprehensible information is necessary to inform Commission Staff and stakeholders. A data collection plan must be developed for each study based on the goals and objectives of the study. These collection methods may include direct contact or observations with program participants, interviews, surveys, experiments, or other industry approved collection methods. The observations or results must be recorded, analyzed and reported to appropriate Commission Staff and other stakeholders. The Proposer is responsible for determining the method of collecting information from each source. The data collection plan is the first stage of developing results for a specific study approach using data sets and data variables. The Contractor is responsible for developing data collection plans and determining the method of collection and gathering of data from various sources. The Contractor is expected to:

a. Understand standard data collection industry practices;

b. Follow a stratified sampling methodology dictated by the available data, and other protocols determined by the Commission;

c. Gathering data from the appropriate sources;

d. Similar operations developed to obtain the quantitative information for the analysis;

e. Selecting a method for sampling a subset of a population;

f. Review tracking data for all programs and measure in their respective sectors;

g. Review any new programs or measures in their respective sectors that may require additional data collection;

h. Review the Business Plan Metrics for each of the Contract Group B Sectors, and determine if additional data collection is needed;

i. Ensure collected data conforms to the formatting requirements developed by the data contractors;

j. Review Program Administrators’ savings projections in portfolio and/or program applications;

k. Create and submit data requests for the evaluation to ensure sufficient data;

l. Develop data collection forms/surveys and provide clear survey designs;

m. Ensure quality control of all data collected;

n. Provide training to field staff to ensure data is collected consistently and in compliance with the evaluation protocols and any other commission guidance;

o. Provide documentation and training to Commission Staff on the collection of data;

p. Develop sampling strategies that meets evaluation protocols requirements.

The contractor will document any specifications and descriptions of the datasets including metadata (such as definitions for data fields) throughout the life of this contract and provide all data collected to the CPUC’s data management contractor. Any primary data generated as a part of this contract shall be considered the property of the CPUC and will be made available to the public subject to the CPUC’s data privacy and confidentiality provisions. The ED Contract Manager and/or ED Project Managers may require the Contractor or its subcontractors to submit interim data as needed for ongoing quality control reviews.

Deliverable 7 (DT-01): Data Collection, Transmission, and Storage Plan

The Contractor is responsible for developing a data collection plan which clearly describes the approach and timeline for data collection throughout the life of the contract.

Deliverable 8 (DT-02): Data Transfer

The Contractor must on an annual basis securely transmit all data collected in the course of any evaluation studies to CPUC’s data Contractor for long-term archiving in a format readily accessible to CPUC staff. This includes the following:

i. Descriptive statistics of the data collected for each data collection period;

ii. Data requests associated with evaluation studies;

iii. Data collection forms and surveys;

iv. Methodology on quality check and results of the quality check;

v. Training materials;

vi. Documentation of all data sets and any calculations

In addition, the Contractor must securely retain copies of evaluation data for at least two calendar years following the end of the contract. The Contractor must take extra precautions to ensure any personally identifying information collected for the purpose of any study is not released or compromised.

Sector-Specific Task and Deliverables Overview

Decision 21-05-031 adopted a new structure to the energy efficiency program portfolios by segmenting programs into Resource Acquisition, Market Support, Equity, and Codes and Standards. Decision 23-06-05513 approved portfolio administrators’ applications (2024-2027) and business plans (2024-2031) using these segments. Studying programs in the Market Support, Equity, and Codes and Standards segments will be a key focus of this Group B contract.

Measurement and verification of savings resulting from energy efficiency measures, programs, and portfolios serve the fundamental purpose of developing estimates of reliable load impacts delivered through ratepayer-funded efficiency efforts. Measurement and verification work should reflect a reasonable balance of accuracy and precision, cost, and certainty, and be designed for incorporation in procurement planning activities.

13 Decision Authorizing Energy Efficiency Portfolios for 2024-2027 and Business Plans for 2024-2031

Evaluation of program-specific qualitative and quantitative measures, such as the program performance metrics and process evaluations, serves a key role in providing feedback for program designers and policy makers. Evaluation findings are also incorporated into updates of the EM&V Plan for the purposes of improving performance and supporting forward-looking corrections to utility programs and portfolios. In order to maximize return on ratepayer dollars, program evaluations must be:

1. Be completed on a timeline which informs mid-course corrections and/or program planning for the following cycle;

2. Provides sufficient resources for continued in-depth research on the impacts of energy efficiency measures, programs, and portfolios;

3. Enables portfolio-level analysis of costs and impacts, with the goal of assessing portfolio cost effectiveness and improving savings estimates in the future.

The following Deliverables in the next four tasks are intended to invite innovative approaches to study the portfolio segments and sector below. Upon execution of a contract, all segment and sector-specific Deliverables are subject to approval by the Energy Division Contract/Project Managers for each section, and all Deliverable schedules and due dates are subject to change at the discretion of the Energy Division Contract/Project Managers.

Task 5: Market Studies

In a constantly evolving environment, market assessments are an essential EM&V product needed to set the baseline for strategic design and improvement of programs and portfolios, identify and track appropriate metrics of market change, and measure progress toward achieving the goals of the California Long-Term Energy Efficiency Strategic Plan. Contractors shall review all planned Market Support program implementation plans posted on the CEDARS14 website and demonstrate the full breadth of creative approaches (e.g., topics, design, methodologies) relevant to the evaluation of these programs, while also allowing for flexibility during contract execution. The winning bidder will ultimately complete numerous evaluations of Market Support programs over the course of the contract, subject to approval of the Energy Division Contract Manager. It is expected this deliverable will include studies that address such programs as finance, workforce education and training, marketing, and emerging technologies, to name a few.

The main objective of Market Support programs, as stated in D.23-06-055, is to help support the long-term success of the energy efficiency market. Examples include:

a. Educating customers

b. Educating and training contractors

c. Building, enabling, and maintaining greater, broader, and/or more equitable access to capital, e.g., financing

d. Building partnerships with consumers, governments, advocates, contractors, community-based organizations, etc.

e. Building, enabling, and maintaining innovation – moving beneficial technologies towards greater cost-effectiveness or declining costs

14 https://cedars.sound-data.com/

General examples of activities required to complete this tasks includes but is not limited to the following:

1. Planning and conducting studies on the program oversight by third-party implementers or IOU program administrators;

2. Reviewing budgets and descriptions of programs and implementation plans;

3. Creating and submitting data requests for the evaluation of programs;

4. Customer application and project approval process;

5. Creating methodologies and customer interview guides or surveys;

6. Conducting qualitative and quantitative analysis; and

7. Submitting program recommendations in time to inform annual program changes by program administrators.

8. Presenting draft and final results to the public and Energy Division.

Deliverable 9: (MS-01) – Market Support Real-Time Evaluations

The contractor will be responsible for designing and conducting four to six evaluations per year that provide real-time feedback and recommendations for in-flight Market Support programs that are in the early to mid-implementation stage. Completion timelines for these evaluations shall be less than a year and could target monthly, quarterly, or semi-annual reports. The types of evaluations pursued could include short term process evaluations or embedded15 evaluations.

Deliverable 10: (MS-02) – Market Support Annual Evaluations

The contractor will be responsible for designing and conducting six to twelve evaluations of Market Support segment programs that take place on an annual basis. Evaluation types include impact evaluations, process evaluations, and market studies.

For impact evaluations, in describing the approach, the Contractors shall adhere to the California Energy Efficiency Evaluation Protocols and provide a plan for both Basic and Enhanced levels of rigor.16

Deliverable 11: (MS-03) – Market Support Portfolio-Level Evaluations (Narrative Required)

Once during the contract, with timing at the discretion of the ED Project Manager for Market Support, the contractor shall conduct a portfolio-level study of the Market Support segment.

The evaluation shall analyze Market Support segment performance and spending levels, and provide recommendations – both for the overall portfolio as well as individual program administrators.

15 CPUC here defines “embedded” evaluations as those focusing on supporting real-time data-based analysis, developmental decision-making, and providing real-time lessons learned and options for course correction.

16 Levels of Rigor are defined in California Energy Efficiency Evaluation Protocols: Technical, Methodological, and Reporting Requirements for Evaluation Professionals.

Task 6 Deliverables – Equity Programs

The Contractor shall review all planned Equity program implementation plans posted on the CEDARS17 website and demonstrate the full breadth of creative approaches (e.g., topics, design, methodologies) relevant to the evaluation of these programs, while also allowing for flexibility during contract execution. The winning bidder will ultimately complete numerous evaluations of Equity programs over the course of the contract, subject to approval of the Energy Division Contract Manager.

The purpose of Equity segment programs, as stated in D.23-06-055, is to provide energy efficiency to hard-to-reach or underserved customers and disadvantaged communities, and program administrators must design their equity segment programs to reach, serve and ultimately benefit hard-to-reach and/or underserved customers and/or disadvantaged communities. Examples include:

a. Addressing disparities in access to energy efficiency programs.

b. Promoting resilience, health, comfort, safety, energy, affordability, and/or energy savings.

c. Reducing energy related GHG and criteria pollutant emissions.

d. Providing workforce opportunities.

Deliverable 12: (EQ-01) – Equity Real-Time Evaluations

The contractor will be responsible for designing and conducting four to six evaluations per year that provide real-time feedback and recommendations for in-flight Equity programs that are in the early to mid-implementation stage. Completion timelines for these evaluations shall be less than a year and could target monthly, quarterly, or semi-annual reports. The types of evaluations pursued could include short term process evaluations or embedded18 evaluations. General example objectives for these evaluations could include but are not limited to:

1. Evaluating a program’s real-time performance

2. Evaluating a program’s outreach efforts

3. Measuring customer satisfaction

4. Measuring customer engagement

Deliverable 13: (EQ-02) – Equity Annual Evaluations

The contractor will be responsible for designing and conducting six to twelve evaluations of Equity segment programs that take place on an annual basis. Evaluation types include impact evaluations, process evaluations, and market studies.

17 https://cedars.sound-data.com/ 18 CPUC defines “embedded” evaluations here as evaluations that focus on supporting real-time data-based analysis, developmental decision-making, and providing real-time lessons learned and options for course correction.

General examples of activities include but are not limited to the following:

1. Planning and conducting studies on the program oversight by third-party implementers or IOU program administrators;

2. Reviewing budgets and descriptions of programs and implementation plans;

3. Creating and submitting data requests for the evaluation of programs;

4. Customer application and project approval process;

5. Creating methodologies and customer interview guides or surveys;

6. Conducting qualitative and quantitative analysis; and

7. Submitting program recommendations in time to inform annual program changes by program administrators.

8. Presenting draft and final results to the public and Energy Division.

For impact evaluations, in describing the approach, the Contractors shall adhere to the California Energy Efficiency Evaluation Protocols and provide a plan for both Basic and Enhanced levels of rigor.19

Deliverable 14: (EQ-03) – Equity Portfolio-Level Evaluations

Once during the contract, with timing at the discretion of ED Project Manager for Equity, the contractor shall conduct a portfolio-level study of the Equity segment. The evaluation shall analyze Equity segment performance and spending levels, and provide recommendations – both for the overall portfolio as well as individual program administrators.

Task 7 – Codes and Standards Programs

The contractor will be responsible for completing the following Deliverables addressing Codes and Standards programs (C&S), subject to approval of the Energy Division Project Manager for C&S.

Deliverable 15: (C&S-01) – Codes and Standards Impact and Process Evaluation (Narrative Required)

The contractor will be responsible for one evaluation per year of Codes and Standards programs and sub-programs, and to ensure compliance with the Federal standards and State building standards per Title 20 (appliance standards) and Title 24 (energy efficiency building codes), established by the California Energy Commission. All Codes and Standards programs are listed on the CEDARS website.20

If relevant, the contractor’s approach shall comply to the Enhanced rigor, applying IPMVP option D, according to the Codes and Standards and Compliance Enhancement Evaluation Protocol, and the Impact Evaluation Protocol, both from the California Energy Efficiency Evaluation Protocols. The contractor should review previous C&S impact evaluation reports and focus on areas of Codes and Standards that warrant focused evaluations, including

19 Levels of Rigor are defined in California Energy Efficiency Evaluation Protocols: Technical, Methodological, and Reporting Requirements for Evaluation Professionals.

20 https://cedars.sound-data.com/ compliance studies, or studies on specific sectors, such as multi-family. This deliverable should combine both impact and process evaluation methods to recommend improvements to the C&S programs and sub-programs. The contractor should also examine historical C&S savings, lifecycle savings, and cost effectiveness calculations.

Task 8 Deliverables – Evaluations of Community Choice Aggregators, Regional Energy Networks and Local Government Partnerships

The contractor will be responsible for completing the following Deliverables addressing, Community Choice Aggregators (CCAs), Regional Energy Networks (RENs), and Local Government Partnerships (LGPs) subject to approval of the Energy Division Project Managers for each category.

Deliverable 16: (GOV-01) – Assessment of CCAs and LGPs

(Narrative Required) The Contractor will be responsible for one evaluation per year that tracks and analyzes the energy efficiency programs administered by current and any future Community Choice Aggregators (CCAs) and, if directed by energy division staff, Local Government Partnerships (LGPs). This Deliverable will include an analysis of the benefits of non-resource programs, including reviewing and summarizing process evaluations, but any impact evaluation of resource programs will be evaluated under a separate Prime Contract. The consultants support should also consider the impact of all CCA and LGP programs on the overall portfolio, including a determination of any impacts the CCA and LGP programs have on participation in, or effectiveness of, programs administered through the IOUs. The evaluation and report will be completed under the direction of the assigned Energy Division Project Manager.

Deliverable 17: (GOV-02) – Impact/Process Evaluations for RENs

The Contractor will be responsible for one evaluation per year that tracks and analyzes the impacts and effectiveness of energy efficiency programs administered through Regional Energy Networks (RENs). This Deliverable will include an analysis of the non-resource programs and analyze how these programs provide benefits in terms of increased performance of contractors, greater customer awareness of programs and how to reduce energy consumption, etc., support non-resource programs provide to ongoing utility and REN resource programs and especially how they support program implementation and program delivery for resource programs.