RFP_Secondary_23NC0404_-_Energy_Efficiency_Savings.pdf

STATE OF CALIFORNIA GAVIN NEWSOM, Governor

PUBLIC UTILTIES COMMISSION

505 VAN NESS AVENUE

SAN FRANCISCO, CA 94102

California Public Utilities Commission (CPUC) Request for Proposal (RFP) – Secondary Number 23NC0404

For:

Group A: Energy Efficiency Evaluation, Measurement, and Verification

June 19, 2024

The California Public Utilities Commission (CPUC) invites you to review and respond to this Solicitation. In submitting your response, you must comply with the instructions found herein.

This is an open solicitation for private entities, non-profit organizations, the University of California, California State University Foundations, and other governmental entities.

CPUC currently has another RFP posted for Group B Energy Efficiency. Proposers may submit responses to both RFPs, but cannot be awarded both contracts. Group A will be awarded first, the Proposer selected for that contract will then become ineligible to be awarded the contract for Group B.

This RFP invites submittal of innovative proposals that will meaningfully refresh and/or replace elements of the status quo while maintaining enough continuity that a smooth transition to new processes is ensured. CPUC expects to evaluate current 2023-2027 resource programs that are either Pay-for-Performance or deployed program-embedded measurement and verification. Evaluation may also touch programs in the market and equity segments that provide energy savings and/or TSB.

Before submitting a response to this solicitation, interested parties should read and review the entire solicitation including any attachments, all referenced web addresses, and any regulations, orders, and statutes cited.

Responses are due August 5, 2024 by 10 A.M. Pacific Time. If you have questions or need any clarification of information contained herein, please send your questions via e-mail to the assigned CPUC Point of Contact below by the date as specified in the Key Action Dates.

Total proposal cost must not exceed $ 40,000,000.00 Proposals exceding this amount will be automatically disqualified.

CPUC Point of Contact:

Trisha Duchaine

Trisha.Duchaine@CPUC.ca.gov mailto:Trisha.Duchaine@CPUC.ca.gov

TABLE OF CONTENTS

EXHIBIT A – SCOPE OF WORK

EXHIBIT B – BUDGET DETAIL AND PAYMENT PROVISIONS

EXHIBIT C – GENERAL TERMS AND CONDITIONS (GTC 04/2017)

EXHIBIT D – SPECIAL TERMS AND CONDITIONS

1. MINIMUM QUALIFICATIONS

3. SUBMISSION REQUIREMENTS

4. PREFERENCE AND INCENTIVE PROGRAMS

5. EVALUATION CRITERIA AND PROCESS

6. AWARD AND CONTRACT EXECUTION

7. REQUIRED ATTACHMENTS

ATTACHMENT 1 – SOLICITATION CHECKLIST

ATTACHMENT 2 – CONFLICT OF INTEREST STATEMENT

ATTACHMENT 3 – REFERENCES SHEET

ATTACHMENT 4 – PAYEE DATA RECORD (STD 204)

ATTACHMENT 5 – PAYEE DATA RECORD SUPPLEMENT (STD 205)

ATTACHMENT 6 – CONTRACTOR CERTIFICATION CLAUSES (CCC 04/2017)

ATTACHMENT 7 – DARFUR CONTRACTING ACT CERTIFICATION

ATTACHMENT 8 – BIDDER DECLARATION FORM (GSPD-05-105)

ATTACHMENT 9 – DVBE DECLARATIONS (DGS-PD-843)

ATTACHMENT 10 – COMMERCIALLY USEFUL FUNCTION CERTIFICATION

ATTACHMENT 11 – CALIFORNIA CIVIL RIGHTS LAW ATTACHMENT

ATTACHMENT 12 – COST SHEET AND STANDARD AGREEMENT SAMPLE

ATTACHMENT 13 – FIREWALL REQUIREMENTS FOR ENERGY EFFICIENCY

ATTACHMENT 14 – EVALUATION AND SCORING CRITERIA

California Public Utilities Commission RFP Secondary Number 23NC0404

EXHIBIT A – SCOPE OF WORK

1. Background

In Decision 23-06-055, which Authorized Energy Efficiency Portfolios For 2024-2027 and Business Plans For 2024-2031, as well as D. 23-08-005, which adopted Energy Efficiency Goals for 2024-2035, the California Public Utilities Commission (hereinafter referred to as “the CPUC”) authorized an annual investment of approximately $1 billion of customer funding per year (2024-2027) for the implementation of energy efficiency programs by Energy Efficiency

Program Administrators (PAs) in California, including the investor-owned utilities1 and non-utility program administrators. The funding will pay for a portfolio of programs across resource, market and equity segments that are expected to realize significant energy efficiency savings and greenhouse gas reductions and make progress toward accomplishing the legislative mandates established in the California Public Utilities2 Code Sections 455.5 and 381.2.3

In order to ensure the long-term success of these dynamic energy efficiency portfolios, the CPUC is committed to an integrated program of ex ante energy savings measurements, estimations and ex post evaluation activities to forecast, track, and measure energy savings4 from resource programs (as well as total system benefits) and related progress in achieving various market and equity segment goals as laid out in the program administrators’ business plans for 2023-2027.

The main drivers of the scope and timing of work required for energy savings measurement, estimation and evaluation activities include state energy efficiency strategic plans, legislative mandates and Commission decisions and legislative mandates.

The California Long-term Energy Efficiency Strategic Plan (CPUC, 2008) and the Existing Buildings Energy Efficiency Action Plan (CEC, 2015) chart the overall policy vision for energy efficiency and align with legislative mandates. In relevance to this RFP, the common underlying goal of the strategic plan, legislative mandates and CPUC decisions is to accelerate and scale the adoption of energy efficiency as a reliable resource with high assurance coming from ratepayer funded energy efficiency programs. The CPUC continues to strive to improve the accuracy, speed and cost-effectiveness of the ratepayer funded program evaluation process.

For example, earlier legislative mandates, including but not limited to Assembly Bill 802 (Williams, Chapter 590, Statutes of 2015)5 and Senate Bill 350 (De’ Leon, Chapter 547, Statutes

1 The California Energy Efficiency Program Administrators are Pacific Gas & Electric, Southern California Edison, San Diego

Gas & Electric, and San Diego Gas (the Investor Owned Utilities, IOUs or Utilities); Southern California Regional Energy

Network and Bay Area Regional Energy Network; and Marin Clean Energy, a Community Choice Aggregator.

3 The California Public Utilities Code is available through:

https://leginfo.legislature.ca.gov/faces/codesTOCSelected.xhtml?tocCode=PUC&tocTitle=+Public+Utilities+Code+-+PUC 4 References to energy savings estimates and measurements throughout this document include all relevant parameters for calculating cost-effectiveness in the Cost Effectiveness Tool.

5 Assembly Bill 802 mandates that the Commission shall authorize IOUs to provide incentives “based on all estimated energy savings and energy usage reductions, taking into consideration the overall reduction in normalized metered energy consumption as a measure of energy savings.”

of 2015)6, brought substantial changes to the Commission’s savings measurement, estimation, and evaluation methods over the last few years. Likewise, newer legislation at the state level, targeting greenhouse gas reductions (Assembly Bill 3232, Friedman, Chapter 373, Statutes of 2018) and building electrification programs (Senate Bill 1477, Stern, Chapter 378, Statutes of 2018), as well as Federal legislation such as the Inflation Reduction Act (IRA), will influence those methods over the lifetime of this contract.

Additionally, the schedule of the necessary savings estimation and evaluation activities updated in CPUC Decision 21-05-031, shown in Figure 1.A-B, below, remains in place. Note that firm deadlines, referred to in the decision and this RFP as bus stops, are adopted for evaluation, measurement, and verification (EM&V) studies, Database of Energy Efficient Resources (DEER) updates, and Goals updates. Contractor due dates are typically 2 weeks to 1 month ahead of the CPUC due dates shown in Figure 1.A-B below.

Figure 1.A Rolling Portfolio Cycle Schedule7

6 Senate Bill 350 mandates the doubling of energy efficiency impacts by 2030 based upon normalized metered electricity and natural gas consumption data.

7 Appendix 6 of D.15-10-028.

CPUC Decision D.21-05-031 updated a bi-annual schedule for the integration of the recent evaluation data into savings estimates in measure packages and DEER, as illustrated in Figure 1.

The decision provided further guidance on the role of EM&V and ex ante processes within the proceeding, which proposers should review prior to submitting proposals.

CPUC decisions 09-09-047, 11-07-030, 12-05-015, as well as a November 18, 2009, Ruling in A.08-07-021, set forth processes for determining and freezing energy savings values that are used for planning, estimating savings claims, and program tracking. The values are considered either deemed (i.e., typical) or custom (i.e., site-specific) and are currently housed in DEER, non- DEER workpapers, or custom project documentation. At the CPUC, estimated energy savings values are utilized for:

1. Developing the energy efficiency potential and goals that feed into the California Energy

Commission’s (CEC) demand forecast, and inform the Integrated Energy Policy Reports.8

2. Forecasting anticipated energy impacts at the measure, program, and portfolio level prior to implementation, as well as optimizing implementation strategies.

3. Calculating cost-effectiveness at the program-level and for the overall resource segment of the portfolio for tracking and approval purposes.

4. Reporting and tracking program accomplishments during implementation.

8 More information about the IEPR can be found at http://www.energy.ca.gov/energypolicy/ http://www.energy.ca.gov/energypolicy/

Further, in D.09-09-047, the CPUC adopted policy goals for EM&V to support its objectives and guide the work of the CPUC’s Energy Division (ED) and the PAs. In addition to the overarching goal of ensuring that program influence is accurately distinguished from naturally occurring market adoption, these goals and core objectives continue to guide the ED’s EM&V work:

“EM&V activities shall be planned and implemented to achieve the following core objectives in order to support the Commission’s oversight function of ensuring the efficient and effective exenditure of ratepayer funds within the energy efficiency portfolios.

All activities should be undertaken to meet the overarching goals of clarity, consistency, cost-efficiency, and timeliness.”

The core objectives are:

I. Savings Estimation, Measurement and Verification – Estimation, measurement and verification of savings resulting from PA resource programs, including energy efficiency measures, programs, and portfolios serve the fundamental purpose of developing estimates of reliable load impacts delivered through ratepayer-funded efficiency efforts.

Measurement and verification work should reflect a reasonable balance of accuracy and precision, cost, and certainty, and be designed for incorporation into procurement planning activities. Note that Market Support and Equity programs, though lacking a priority resource saving component, may nevertheless lead to energy savings that the IOU program administrators may count towards their respective Total System Benefit (see item I below) achievements.

II. Program Evaluation – Evaluation of program-specific qualitative and quantitative measures, such as the program performance metrics (PPM), key performance indicators (KPI), and process evaluations, serves a key role in providing feedback for the purposes of improving performance and supporting forward-looking corrections to utility programs and portfolios. To maximize return on ratepayer dollars, program evaluations must be completed on a timeline which informs mid-course corrections and/or program planning for the following cycle.

III. Market Assessment – In a constantly evolving environment, market assessments are an essential EM&V product needed to set the baseline for strategic design and improvement of programs and portfolios. Saturation studies, surveys of emerging technologies and other such analyses which inform estimates of remaining program potential and forward-looking goal setting are key aspects of market assessment.

IV. Policy and Planning Support – Consistent with prior program cycles, it is essential to reserve funding to support overarching studies and advisory roles which support CPUC policy goals. Over the last program cycle this has been inclusive of potential and goals studies, maintenance of DEER database, developing databases of best practices for program design and delivery, program design mix, and other means which support the CPUC’s oversight role, but do not fall under the core EM&V categories described above.

V. Financial and Management Audit – Supporting the CPUC’s oversight function of ensuring the efficient and effective expenditures of ratepayer funds within the utilities’ energy efficiency portfolios is another objective of EM&V activities. Rigorous financial and management audits overseen by CPUC staff will be critical in ensuring that the utilities’ general and administrative costs and other program expenditures are prudent and reasonable.”9

Additionally, the following should also be considered core objectives:

I. Total System Benefit (TSB) - Commission Decision D. 21-05-031 moved CPUC policy in the direction of setting energy efficiency potential and goals based on TSB.

TSB aggregates the various benefits of energy efficiency, rather than assessing and assigning separate electricity, demand and natural gas savings goals that were traditionally used in past program cycles into an annual dollar-based value for each

IOU.

II. Inflation Reduction Act (IRA) - As noted in D. 23-06-055, the IRA dovetails with existing and future market access resource programs, in that the residential and commercial downstream retrofit markets include projects targeted by the IRA.

Consequently, the IOUs (and other PA) may be able to complement ratepayer funding for these programs with IRA funding. Any program or programs deploying this complementary funding approach would require evaluation to determine the impacts on program participation and savings due to the augmented funding approach and potential for multiple attribution sources.

III. Equity - Per D.21-05-031, the portfolio has been segmented into resource acquisition, market support and equity segments. A central component of this contract will be to include as a primary focus of the equity sector. Equity is defined as: “Programs with a primary purpose of providing energy efficiency to hard-to-reach or underserved customers and disadvantaged communities in advancement of the CPUC’s Environmental and Social Justice (ESJ) Action Plan;10 Improving access to energy efficiency for ESJ communities, as defined in the ESJ Action Plan, may provide corollary benefits such as increased comfort and safety, improved indoor air quality, and more affordable utility bills, consistent with Goals 1, 2, and 5 in the ESJ Action Plan.”11

Note that although the CPUC has adopted core objectives and policy processes, the policy context driving the energy efficiency portfolios is subject to change substantially as the State works to achieve its greenhouse gas reduction goals. For instance, several decisions adopted since D.09-09- 047 created an ex-ante review process that will now be performed by both ex-ante and EM&V contractors under the direction of CPUC staff. Additionally, although Decision D.15-10-028 removed the Program Portfolio Metric requirements12 , D. 23-06-055 authorized a new set of PA business plan indicators focused on market support and equity in addition to encouraging PAs to revisit the common metrics adopted in D.18-05-041 to recommend metrics that should be suspended, modified, or removed.12 The roles, process, and timing of analytical method and model reviews may also shift as the CPUC continues to place greater emphasis on “meter based” program designs (see D. 23-06-055).

9 D.09-09-047, page 299-301.

10 For more information, see the following link: https://www.cpuc.ca.gov/esjactionplan/ 11 D.21-05-031, p.14 12 See Attachment 17- Glossary of Terms.

https://www.cpuc.ca.gov/esjactionplan/

Based on that increased emphasis on meter-based savings for resource programs, the need for upfront planning with the goal to transform current program delivery and evaluation methods remains paramount. The Research and Evaluation Plans will need to employ innovative and possibly first-of-kind methods to evaluate such programs that target meter-based savings and/or include measurement and verification methods. For these meter-based resource savings programs, we anticipate the Research and Evaluation Plans to clearly articulate the methods for assessing program influence (such as randomized control trials), and other methods that complement program design.

Contractors hired through this RFP must be able to demonstrate the ability and capacity to propose, develop, review as well as adapt to new methods and models for impact evaluation approaches over the course of the contract.

Further, it should be noted that the CPUC expects to evaluate current 2023-2027 resource programs that are either Pay-for-Performance or deployed program-embedded measurement and verification.

Evaluation may also touch programs in the market and equity segments that provide energy savings and/or TSB.

The CPUC intends to release other RFPs through the course of this year that will collectively implement the necessary tasks for carrying forward the goals and objectives outlined in this section.

The contract resulting from this RFP falls under Contract Group A. Note that evaluation budgets for the IOU’s have shifted from 27.5% to 40% of the total, in recognition of the increased emphasis on 1) Normalized Metered Energy Consumption and Pay for Performance programs, and 2) continuing planning and market assessment efforts associated with the market transformation and other programmatic emphasis in SB 350 and AB 802 (Decision 16-08-01913). Consequently, the evaluation budget for this RFP has shifted from 72.5% to 60% of the total. Therefore, the upper limit of this RFP is set at $30 million, which corresponds to the cap of 60% for the Commission’s evaluation budget.

2. Statement of Work

The nature and timing of this contract is aimed to be responsive to current research and evaluation needs while at the same time ensuring that the Commission prepares for emerging industry paradigms and recent legislative mandates that trigger a remarkable shift to current practices.

The following section describes the overall work that will be completed under Group A contract. For the proposal, the Proposer will create a work plan for the research roadmap described for each sector and market area included in this RFP to specify the activities, work products, schedules, and budgets for all deliverables required under this contract. The CPUC’s Energy Division will assign Project Managers (EDPM) to manage each sector and market area and serve as a point of contact for specific projects. No work will be performed under the Prime Contract without prior approval by the EDPM, except where specifically described in the Scope of Work and Deliverables. The contractor is expected to make program design recommendations as part of the scope of this contract but should not make policy recommendations on issues that are typically addressed by the CPUC through due processes.

13 CPUC, 2016. “Decision Providing Guidance for Initial Energy Efficiency Rolling Portfolio Business Plan Filings.”

http://docs.cpuc.ca.gov/PublishedDocs/Published/G000/M166/K232/166232537.PDF#page81 http://docs.cpuc.ca.gov/PublishedDocs/Published/G000/M166/K232/166232537.PDF#page81

In order to make the rolling portfolio cycle operable, the Prime contractor must complete work products to meet the bus stops. This will require integrated project management and coordination between the EM&V work products, ex ante estimate work products and any potential meter-based savings tools and approaches that may be developed as part of the Research and Evaluation plans.

Task 1: Research and Evaluation Plans

As directed by the EDPMs, the Prime Contractor and/or its subcontractors will work with EDPMs, other CPUC Contractors, and other CPUC Program staff to finalize, maintain and update the Research and Evaluation plan for the sectors, measure groups and/or program areas they are assigned. These plans will guide the scope of the ex ante and EM&V work to be conducted through the term of this awarded contract, outlining specific projects that are expected to be completed and timelines for execution. Upon completion, CPUC staff is responsible for maintaining the research and evaluation plans and may make modifications to them as necessary throughout the contract term. The contractor activities associated with maintaining and updating the plans may include, but not be limited to:

1. Developing a plan for all estimation activities under this RFP;

2. Developing a plan for impact evaluation studies;

3. Assisting with the inventory and prioritization of savings estimation and EM&V projects defined in the research roadmaps and required and recommended by CPUC orders, which may include:

a. Modifying the research and evaluation plans to streamline the number of studies;

b. Leveraging other studies in the portfolio;

c. Reducing cost; and

d. Setting priorities.

4. Developing detailed work plans reflecting prioritization and timelines for implementing the specific projects described in the research roadmaps;

5. Incorporating ex ante savings estimation data needs into research and evaluation plans;

6. Defining technical and procedural protocols for conducting ex ante and EM&V research projects;

7. Participating in Project Coordination Groups or other stakeholder groups on an on-going basis as directed by EDPM;

8. Developing consistent methods (where necessary) for conducting the evaluation and measurement of specific types of energy efficiency program activities;

9. Conducting review of research needs with the Project Coordination Groups on an annual basis, and updating the plans as necessary:

a. Assisting with the annual inventory and prioritization of ex ante and EM&V projects proposed by program administrator staff, CPUC staff, stakeholders, and Prime Contractors; and

b. Maintaining record of inventory and prioritization for future needs assessments in updates to the research roadmaps.

Deliverable 1: Research and Evaluation Workplans and Updates (Narrative Required)

The contractor will complete 10 Research and Evaluation Workplans for each year’s submission and update. The multiple research plans are meant to reflect broken-out components of this project for staged release of deliverables. Workplans shall be in Microsoft Word and PDF format and delivered electronically.

The research workplans shall take into consideration the following sectors with resource and equity programs consistent with the business plan decision.

Additionally, the research workplans shall also take into consideration market support and equity programs with a resource component. Although these programs do not have resources savings as a primary goal, the CPUC allows any program in either segment to contribute energy savings to a PA’s overall portfolio and count towards TSB goal attainment.

The Research and Evaluation workplan will describe in detail the methodology and approach to complete all deliverables and studies for each sector/measure group required under this contract. Contractor must include a narrative detailing the approach for each deliverable, a detailed work plan that outlines all activities and tasks, a timeline for which the activities will be completed, resources and personnel necessary for each deliverable, and a detailed budget aligned to the deliverables. The contractor shall establish a detailed scope of the evaluation for the initial year of this contract, based on the most updated uncertain measures list released as of the release date of this solicitation. The Research and Evaluation workplan shall clearly describe methodology for activities including but not limited to the following:

1. Gather, document and vet data sources;

2. Engage stakeholders;

3. Choose, evaluate, and use tools and methods;

4. Identify skills needed on a team to perform the work;

5. Manage project and communication with Commission staff;

6. Coordinate with commission staff to obtain approval for all evaluation decisions made at the protocol level;

7. Describe the potential barriers and opportunities to providing annual results and demonstrate how you would balance:

a. Completeness of data;

b. Representativeness of sample;

c. Meaningful results;

d. Cost of reporting;

e. Collaboration with Commission staff and contractors;

f. Stakeholder engagement and review; and

g. Disputes over the methods and/or results.

8. Detail savings and Total System Benefits measurement, estimation and evaluation approaches in the Evaluation Plan to address specific requirements as reflected in AB802 and SB350, additional and relevant state legislation, and Commission directives, including:

a. Update baselines for the deemed savings estimates for the highest impact measures;

b. Address the requirements of AB 802 and the Commission’s High Opportunity Projects and Programs framework for review;

c. Assess the program administrators’ proposals for methods to conduct a normalized metered energy consumption assessment and the results from those proposals;

d. Assess the benefits and challenges of the proposed approach; and

e. Determine how the adaptations will fit into a feedback loop where evaluation results inform future savings estimation, assessments and program design.

f. Determine how energy savings and Total System Benefit compare with potential (I.e. do resource acquisition program and sector savings roll-up in alignment to forecast potential).

The contractor will work closely with CPUC staff to ensure that the research workplans will sufficiently meet all technical and administrative needs of the CPUC. Contractor must submit drafts of the research workplans within 45 days of contract execution and update the workplans annually and to align with ED’s bi-annual EM&V master plan.

Deliverable 2: Final Impact Evaluation Reports

The Contractor shall prepare ten (10) Final Impact Evaluation Reports per year. The Final Impact Evaluation Report must clearly and thoroughly address each of the Objectives, in the manner defined in the Final Research Plan. In order to successfully do so, the Final Impact Evaluation Report must clearly answer each of the Final Key Research Questions in the Final Research Plan. The Executive Summary component of each draft and final report must be developed using language that is understandable by a non-technical audience. The evaluation reports must also follow a protocol rigor level as determined by the CPUC.

The contractor will coordinate with the EDPM to determine due dates for draft reports to ensure sufficient time for final revisions. The EDPM will determine which components of the Final Impact Evaluation Report will be submitted each year. At a minimum, the final reports will include impact evaluation and findings for the uncertain measures list each year for each sector/market in this contract group. Each subsequent annual impact evaluation shall also consider the emerging issues for each sector that feed into the Annual EM&V Plan

The Contractor shall provide the Commission a draft evaluation report for comments and edits and conduct a stakeholder briefing before the report is finalized. Both the draft and the final evaluation report will include the following components:

1. Executive Summary: A brief, easily understandable document, written in plain language, written for a non-technical audience, e.g., policy makers and members of the public, to quickly understand the design, execution, and results of the study, written according to the

Executive Summary writing guide.14 Executive Summaries should be no more than 10% of the length, by page count, of the entire report (or 10 pages).

2. Approach and Methodology: The methodology shall allow professionals in the industry to replicate their calculations.

3. Data Source: A fully detailed document of data sources used in this report.

4. Submission of electronic PDF formatted document: Upon Contractor submission, the

PDF shall be in a format that is ready to be uploaded by CPUC Staff to the CPUC website for public consumption of the final graphic design and layout of the evaluation.

5. Submission of a print-ready format document: The Contractor shall electronically transmit the graphic design and layout of the Final Evaluation report document so the CPUC can readily submit to a printing entity for mass printing.

6. Furthermore, the draft and final evaluation report shall take into consideration the following critical elements:

a. Readability: The executive summary shall be written in a way that is understandable to a non-technical audience;

b. Copy-editing: The draft and final report must be copy-edited by a professional copy-editor. The contractor must provide a resume of the copy editor to the CPUC and obtain approval before finalizing the reports;

c. Peer-review; At the discretion of the CPUC, specific studies within the Impact Evaluation Report will undergo the peer review process prior to finalization. The peer review process must meet current industry standards, and the peer review panel must be selected and approved by the CPUC; and

d. Graphic Design: The draft shall be formatted in a professional manner with an effective design that meets or exceeds typical standards for high level public reports.

Upon approval of the CPUC Staff on the final report, the formatted Final Evaluation report will incorporate all edits from the previous drafts and will be submitted to the Commission within a timeframe specified by the CPUC.

Deliverable 3: Impact Evaluation Report Stakeholder Briefings

Each of the ten (10) yearly reports will be briefed at a stakeholder workshop or executive briefing. Each briefing will be approximately one hour long, and the Contractor must respond to each comment received from the stakeholders and/or executives by making changes to the draft report or addressing the specific comments in a summary report.

Energy Division staff are responsible for setting up stakeholder meetings. Energy Division staff or the contractor will provide the actual public meeting conference information (webex, call-in, etc.). ED staff will set up executive briefings (usually with CPUC management and Commissioners).

14 “A Guide to Writing an Effective Executive Summary.” Navy and Marine Corps Public Health Center, Environmental

Programs.

Deliverable 4: Annual EM&V Master Plan Update

The contractor will assist CPUC staff in updating ED’s EM&V Master Plan timeline and budget distribution across activities by December 31st of every other year. The EM&V Master Plan details the key questions, general methodologies, and evaluation method based on information provided in the gaps and emerging issues report.

Task 2: Administration and Project Management

Administration and Project Management includes planning and leading specific studies as well as coordinating activities within and across the program areas assigned to the contractor. The Prime Contractor will provide a Senior Project Manager capable of managing the general administration, proactive project coordination, project planning, internal and external communication, and deliverables described in this section. The Project Manager will provide monthly progress reports and attend project coordination meetings. The Project Manager will be the primary point of contact for the contract and must comply with the following scope of administration and project management requirements when performing services described in this solicitation:

1. Provide and implement a well-organized and complete plan for all activities, tasks, and milestones;

2. Monitor the schedule, budget, and specifications;

3. Monitor resources and expenditures to ensure fulfillment of project goals and objectives within the approved budget;

4. Acquire approval from the EDPM for all work performed;

5. Acquire approval from the EDPM and Contracts Officer for expenditures that were not individually listed in the Cost Sheet, prior to undertaking such expenditure;

6. Acquire approval from the EDPM for any replacement, substitution, or addition of staff under the contract;

7. Acquire approval from the EDPM and Contracts Officer for any replacement, substitution, or addition of sub-contractor under the contract;

8. Carry out provisions of the contract within specifications, budget, and schedule;

9. Identify and report problems in a timely manner and recommend and implement solutions.

Deliverable 5 Monthly Progress Report

Prime Contractor must submit a monthly progress report to the EDPM and/or other entities at the discretion of the EDPM that will include the following information: progress made on each deliverables, deliverable related activities, or other information upon request.

Deliverable 6 Kickoff Meeting

Upon execution of the contract, a virtual kickoff meeting will be convened for CPUC staff and Contractor’s Team to discuss and refine the submitted draft workplan, orient the Contractor’s Team to the administrative and technical expectations of the EDPM and/or CPUC Sector Leads, and initiate discussion of prioritization of tasks. The contractor will also work with the

EDPM to establish project management foundation for the contract and support the finalization of the EM&V Plans, as required by the schedule set in D.21-05-031.

Deliverable 7 Project Coordination Meetings

The Contractor will attend between 1 to 4 monthly project coordination meetings (up to 48 meetings per year) with CPUC staff and other CPUC contractors, on an ad-hoc basis or regular schedule, as directed by the EDPMs and/or CPUC Sector Leads. The purpose of the project coordination meetings is to provide a venue to discuss the status of tasks and activities across the entire contract with CPUC staff and CPUC management; strategize for coordination between Prime Contractors as needed; resolve administrative and reporting matters concerning the entire contract and coordination with other Prime Contractors; and coordinate interactions with program administrators’ savings estimation, EM&V, and program management staff.

The project coordination meetings required under this deliverable include administration check in (in-person or via conference call) approximately every month with CPUC staff to report on contract invoicing, evaluation project timelines, and other contract matters. Contractor will take notes and maintain materials on a shareable site with the EDPM.

Task 3: Stakeholder Engagement

The Contractor Team will participate or host various types of stakeholder engagements to ensure a proper public vetting process throughout the life of the contract. Expected activities to complete this task include:

1. Participating and collaborating in stakeholder workshops/webinars;

2. Addressing issues that members of the public raise at the stakeholder workshops and specifically document how each comment from the stakeholder workshop has been considered or addressed;

3. Developing executive summaries or presentations for all technical documents in language accessible to stakeholders and decision-makers who may not be industry or technical experts, as defined by CPUC staff

4. Must be able to explain their savings estimation, measurement and evaluation approaches and results to a non-technical audience;

5. Developing of briefing materials accessible to public audience upon request of CPUC staff;

6. Managing written and oral communication, including public briefing materials, summarizing work products to decision-makers and stakeholders, as requested by CPUC staff; and

7. Providing a summary of data and information gathered from the completed workshops/webinars.

At least two key team members are required to participate in each stakeholder engagement activity. Research and Evaluation Work Plans must state the anticipated number of stakeholder engagement events under each proposed activity.

Deliverable 8: Stakeholder Workshops / Webinars

The contractor will participate in and/or conduct up to 150 stakeholder engagement activities throughout the initial contract. The meetings may be online or in-person, depending on the direction of the EDPM at the time.

Task 4: Data Collection and Transfer

The Contractor is responsible for developing data collection plans and determining the method of collection and gathering of data from various sources. These collection methods can include direct contact or observations with program participants, interviews, surveys, experiments or other industry approved collection methods. The observations or results have to be recorded, analyzed and reported to appropriate CPUC Staff and other stakeholders.

Other data collection requirements include:

1. Understanding standard data collection industry practices;

2. Gathering data from the appropriate sources;

3. Developing data gathering operations to obtain the quantitative information for the analysis; and

4. Creating a method for sampling a subset of a population.

The contractor must allocate resources to combine and prepare the tracking data available and other necessary program or demographic data to complete all work related to this contract for the portfolio, including resource acquisition, equity and NMEC related evaluations, to name a few.

The contractor will document any specifications and descriptions of the datasets including metadata (such as definitions for data fields) throughout the life of this contract and provide all data collected to the CPUC’s data management contractor. Any primary data generated as a part of this contract shall be considered the property of the CPUC and will be made available to the public subject to the CPUC’s data privacy and confidentiality provisions. The EDPM may require the Contractor or its subcontractors to submit interim data as needed for ongoing quality control reviews.

Deliverable 9: Data Collection Plans

A written data collection plan has to be developed for each study based on the goals and objectives of the study. The plan must detail methodology the Contractor plans to use.

Contractor should budget for ten (10) of these plans per year. A data collection plan must be developed for each study based on the goals and objectives of the study and will be delivered to the EDPM for review and approval.

Deliverable 10: Data Transfer

The Contractors shall present all final evaluated data, for all studies, including resource acquisition, equity, and NMEC, to one dataset following the specifications for the “All Things Reported” longitudinal dataset as defined by the EDPM. This will include the final datasets that could be integrated in a format specified by the Energy Division as well as written documentation to explain the datasets.

The contractor shall provide the raw data, any secondary spreadsheets, databases, source codes, underlying scripts, algorithms, or other project-defining creations to the CPUC in coordination with the data management contractor and follow the data exchange specifications provided by the data management contractors. Note that the data management contractors will NOT be responsible for deciding the contents of the databases, only the format for putting the data into

CEDARS.

Task 5: Portfolio and Policy Impact Analysis and Recommendations

The Contractor will evaluate and analyze the entire IOU program administrator portfolio of programs and compare to existing CPUC policy and rules. Activities include but are not limited to the following:

1. Plan and conduct studies on the program oversight by third-party implementers or IOU program administrators;

2. Review budgets and descriptions of programs and implementation plans;

3. Create and submit data requests for the evaluation of programs, in conjunction with impact studies;

4. Analyze customer application and project approval process and business plan metrics;

5. Create methodologies and customer interview guides or surveys;

6. Conduct qualitative and quantitative analysis; and

7. Submit program recommendations for their respective sectors in time to inform annual program changes by program administrators. Recommendations should surround the following objectives on how administrators can improve the program(s):

a. Lower cost;

b. Increase innovation in program design and measure technology;

c. Increase energy savings;

d. Improve operations and efficiency;

e. Increase cost effectiveness; and

f. Increase participation and influence behaviors of potential, new customers.

Deliverable 11: Portfolio and Policy Impact Recommendations

The frequency of this deliverable may change due to portfolio activity but should at least include an ex-ante portfolio analysis each calendar year. The contractor is responsible for ultimately developing written program design recommendations for program administrators to improve their resource acquisition, market support and equity energy efficiency programs, which include the analysis of the program management and overview of current IOU energy efficiency resource, NMEC and equity programs.

Task 6: Gross and Net Savings Estimates

The Contractor will develop methodologies and on-site field work that will estimate the gross unit energy savings for kWh, kW and therms at the measure and program level, including metered programs.

The Contractor shall develop methodologies and survey instruments that will estimate the levels of program attribution and program influence at the measure and program level, including metered programs. The Contractor will specifically study potential new customers to improve the net savings estimates. This may include new methodologies at the market level that challenge traditional net-to-gross analysis.

Activities in this task include:

1. Review tracking database to extract the appropriate samples for site visits and interviews;

2. Develop the gross savings methodology/approach;

3. Develop survey instruments for the study;

4. Test the approach;

5. Field the survey;

6. Clean the data;

7. Determine the baseline energy use; (for gross savings)

8. Determine adjustments to installation counts and hours of operation; (for gross savings)

9. Provide gross savings of each sector and specified measures.

10. Develop the scoring algorithm; (for net savings)

11. Produce a net-to-gross value that meets commission protocol rigor requirements; (for net savings)

12. Provide net savings of each sector and specified measures.

The Proposer must clearly identify the number of measures for which gross savings estimates and net savings estimates will be developed, sample sizes by measure and sector in their proposal.

The Contractor should follow the California Protocols for Direct Impact Evaluation and Proposers are expected to refer to the evaluation protocols15 for a complete description of the rigor levels for gross and net impact studies and the corresponding relative precision required for each rigor level.

For Gross Impact studies (Deliverable 10), Proposer will submit an approach under the Gross Energy Evaluation Basic rigor level and an approach under the Enhanced rigor level.

For Net Impact studies (Deliverable 11), Proposers will submit multiple approaches under all Net Impact Evaluation rigor levels: Basic, Standard, and Enhanced.

15 CPUC, 2006. “California Energy Efficiency Evaluation Protocols: Technical, Methodological, and Reporting Requirements for Evaluation

Professionals”, pg. 22. https://www.cpuc.ca.gov/-/media/cpuc-website/files/uploadedfiles/cpuc_public_website/content/utilities_and_industries/energy/energy_programs/demand_side_management/ee_and_en ergy_savings_assist/caenergyefficiencyevaluationprotocols.doc https://www.cpuc.ca.gov/-/media/cpuc-website/files/uploadedfiles/cpuc_public_website/content/utilities_and_industries/energy/energy_programs/demand_side_management/ee_and_energy_savings_assist/caenergyefficiencyevaluationprotocols.doc https://www.cpuc.ca.gov/-/media/cpuc-website/files/uploadedfiles/cpuc_public_website/content/utilities_and_industries/energy/energy_programs/demand_side_management/ee_and_energy_savings_assist/caenergyefficiencyevaluationprotocols.doc https://www.cpuc.ca.gov/-/media/cpuc-website/files/uploadedfiles/cpuc_public_website/content/utilities_and_industries/energy/energy_programs/demand_side_management/ee_and_energy_savings_assist/caenergyefficiencyevaluationprotocols.doc https://www.cpuc.ca.gov/-/media/cpuc-website/files/uploadedfiles/cpuc_public_website/content/utilities_and_industries/energy/energy_programs/demand_side_management/ee_and_energy_savings_assist/caenergyefficiencyevaluationprotocols.doc

Deliverable 12: Gross Savings Report

Contractor shall prepare a written Gross Savings Report draft and submit to the Commission for review prior to finalization. There are multiple reports submitted over the life of the contract, each of which is related and specified by the EDPM. The format will be pdf, word and excel, to be determined by contractor with input from EDPM. Guidelines for the nature of the report(s) can be found at in the evaluation protocols.16

Deliverable 13: Net Savings Report

Contractor shall prepare a written Net Savings Report draft and submit to the Commission for review prior to finalization. There are multiple reports submitted over the life of the contract, each of which is related and specified by the EDPM. The format will pdf, word and excel, to be determined by contractor with input from EDPM. Guidelines for the nature of the report(s) can be found at in the evaluation protocols.17

Task 7 Ex-Ante Values Update

The CPUC updates the Database of Energy Efficiency Resource (DEER) Ex Ante values bi-annually with results from the final impact evaluation reports. The DEER Ex Ante values include energy impacts, technology costs and all other associated parameters (EUL/RUL, NTG, operating hours, etc.) used to determine net lifecycle savings for specific measures. Deemed savings estimations are provided in kW, kWh, and therms. DEER also provides Commission-approved methodologies for updating these values.

The DEER ex-ante values are used in several areas across the energy efficiency portfolio. For example, to assess cost-effectiveness of energy efficiency programs funded by the California ratepayers, and inform the CPUC-mandated deliverables, such as the Potential and Goals studies.

Under the direction of CPUC staff, the contractor will provide assistance to CPUC staff in the annual DEER update process. The contractor shall be responsible for developing and updating existing DEER Ex Ante values for all sectors (including residential, commercial and agricultural) and should show competence in energy modeling, building simulation and field verification, including use of software tools used to perform those analyses, such as Open Studio/EnergyPlus, eQuest /DOE-2, CBECC Com/ Energy Plus, CBECC Res/CSE and GitHub.

The contractor shall consider the following as principal source material for the DEER update (in no particular order):

16 https://www.cpuc.ca.gov/-/media/cpuc-website/files/legacyfiles/c/5212-caenergyefficiencyevaluationprotocols.doc 17 https://www.cpuc.ca.gov/-/media/cpuc-website/files/legacyfiles/c/5212-caenergyefficiencyevaluationprotocols.doc https://www.cpuc.ca.gov/-/media/cpuc-website/files/legacyfiles/c/5212-caenergyefficiencyevaluationprotocols.doc https://www.cpuc.ca.gov/-/media/cpuc-website/files/legacyfiles/c/5212-caenergyefficiencyevaluationprotocols.doc https://www.cpuc.ca.gov/-/media/cpuc-website/files/legacyfiles/c/5212-caenergyefficiencyevaluationprotocols.doc https://www.cpuc.ca.gov/-/media/cpuc-website/files/legacyfiles/c/5212-caenergyefficiencyevaluationprotocols.doc

1. Federal and State codes and standards, local regulations, industry standards, and best practices;

2. Impact evaluation results;

3. CPUC-issued decisions, resolutions, custom projects, measure package dispositions, and guidance documents; and

4. Market surveys, market data, and other available studies (not produced by CPUC).

Contractor shall pay particular attention to making public and transparent the underlying tools (including software and source code), algorithms, calculation methodologies, all assumptions, and values (as well as the basis for such engineering decisions), used when developing Ex Ante savings estimates. Contractor shall have regular check-ins with CPUC staff and provide status and progress reports throughout the biennial update process. Contractor shall conform to all CPUC direction and deadlines directed by the CPUC.

Contractor shall also coordinate with other Commission contractors (web domain administrators, database administrators, and other) to have DEER values updated in the Ex Ante database made ready for viewing immediately after the values are approved by the CPUC.

The Contractor will coordinate with other state agencies and public entities to develop standardized methods for estimating measure savings, updates and review methods for statewide adoption.

Contractor shall provide input to program impact evaluation teams during the EM&V planning phase by reviewing sampling specifications, monitoring methods, and strategies that should results in data readily available to be incorporated into the DEER update.

Deliverable 14: Ex-Ante Values Update Documentation

The Contractor shall develop a scoping document on the proposed updates for each year in a timeframe directed by CPUC staff and develop written documentation on all proposed changes to the Ex Ante values to be included in CPUC staff resolution. The scoping document should be delivered in both Word and PDF. The document should be developed on the Contractors SharePoint site to allow multiple users to access and edit the document at the same time. The document will include changes to the measure packages and procedures to upload to the electronic Technical Reference Manual (eTRM).