RFP_23NC0412_CPUC_Energy_Efficiency_Technical_Consultant.pdf

STATE OF CALIFORNIA GAVIN NEWSOM, Governor

PUBLIC UTILTIES COMMISSION

505 VAN NESS AVENUE

SAN FRANCISCO, CA 94102

California Public Utilities Commission (CPUC) Request for Proposal (RFP) – Secondary Number 23NC0412

For:

Energy Efficiency Technical Consultant Services

March 29, 2024

The California Public Utilities Commission (CPUC) invites you to review and respond to this Solicitation. In submitting your response, you must comply with the instructions found herein.

This is an open solicitation for private entities, non-profit organizations, the University of California, California State University Foundations, and other governmental entities.

The purpose of this solicitation is to source technical assistance and services for CPUC Energy Division (ED) Energy Efficiency (EE) staff to ensure that energy efficiency portfolios achieve portfolio segment objectives, reliably meet Total System Benefit (TSB) goals, and appropriately and reasonably use ratepayer funds. The deliverables associated with any Agreement coming from this solicitation will be used to improve instruments and procedures for EE staff’s oversight of energy efficiency portfolios. Deliverables will additionally be used to explore research and policy topics related to EE portfolio oversight.

Before submitting a response to this solicitation, interested parties should read and review the entire solicitation including any attachments, all referenced web addresses, and any regulations, orders, and statutes cited.

Responses are due Friday, May 31, 2024, by 5:00 PM Pacific Time. If you have questions or need any clarification of information contained herein, please send your questions via e-mail to the assigned CPUC Point of Contact below by the date as specified in the Key Action Dates.

Total proposal cost must not exceed $5,000,000.00 Proposals exceding this amount will be automatically disqualified.

CPUC Point of Contact:

Alex Ritchie alexander.ritchie@cpuc.ca.gov

California Public Utilities Commission RFP Number 23NC0412

TABLE OF CONTENTS

1. SCOPE OF WORK AND CONTRACT TERMS

EXHIBIT A – SCOPE OF WORK

1. Background

2. Statement of Work

3. Contractor Responsibilities

4. CPUC Responsibilities

5. Location

6. Time

7. Amendments

8. Contact Information

EXHIBIT B – BUDGET DETAIL AND PAYMENT PROVISIONS

EXHIBIT C – GENERAL TERMS AND CONDITIONS (GTC 04/2017)

EXHIBIT D – SPECIAL TERMS AND CONDITIONS

2. MINIMUM QUALIFICATIONS

3. ADMINISTRATIVE INFORMATION

4. SUBMISSION REQUIREMENTS

5. PREFERENCE AND INCENTIVE PROGRAMS

6. EVALUATION CRITERIA AND PROCESS

7. AWARD AND CONTRACT EXECUTION

8. REQUIRED ATTACHMENTS

ATTACHMENT 1 – SOLICITATION CHECKLIST

ATTACHMENT 2 – CONFLICT OF INTEREST STATEMENT

ATTACHMENT 3 – REFERENCES SHEET

1. SCOPE OF WORK AND CONTRACT TERMS

EXHIBIT A – SCOPE OF WORK

1. Background

a. Introduction

The California Public Utilities Commission (hereinafter referred to as the CPUC) regulates services and utilities, projects consumers, safeguards the environment, and assures Californians’ access to safe and reliable utility infrastructure and services. The essential services regulated include electric, natural gas, telecommunications, water, railroad, rail transit, and passenger transportation companies.

b. This Solicitation

The order that permits the CPUC to seek this service is D.23-06-055 O.P. 9. The parameters defined by the order provide the following restrictions:

I. Maximum amount of funding - $5,000,000.00 over 5 years.

II. The output from this project will be used to improve instruments and procedures for oversight of energy efficiency portfolios for the Energy Efficiency Branch, and to explore relevant research and policy topics related to EE portfolio oversight.

c. Objective

The purpose of this contract is to provide technical assistance and services to California Public Utilities Commission (CPUC) Energy Division (ED) Energy Efficiency (EE) staff to ensure that energy efficiency portfolios achieve portfolio segment objectives, reliably meet Total System Benefit (TSB) goals, and appropriately and reasonably use ratepayer funds. Under the guidance of EE Staff, the Energy Efficiency Technical Consultant will 1) support the Energy Efficiency Branch to improve its execution of portfolio oversight, 2) develop tools and resources to facilitate more consistent and streamlined staff review of Portfolio Administrator (PA) filings, programs, and other submissions 3) coordinate with the Reporting Program Coordination Group (PCG) to find opportunities to improve PA reporting mechanisms, 4) support visualization of information to improve and expedite staff analysis, and 5) provide recommendations for and related to certain studies to support how they complement general oversight.

d. Background

For over two decades, the CPUC has been engaged with utilities, local governments, communities, and various market actors to reduce energy consumption in homes and buildings using public purpose funds from ratepayers. Over that time, the regulatory and market environments evolved such that objectives, value streams, desired partnerships, and performance tracking for energy efficiency changed to become more specific and comprehensive. This evolution necessitates more sophisticated approaches, procedures, and instruments for the Energy Efficiency Branch to oversee PA achievements and to ensure reasonable expenditure of ratepayer funds.

The Commission’s role in energy efficiency is multifaceted. Its primary focus is to work with the PAs to deliver energy efficiency broadly and at reasonable expense to ratepayers. It is the PAs’ responsibilities to develop, execute, administer, and assess their portfolios. The CPUC oversees all these activities through regular meetings and reporting, as well as ad hoc requests when necessary, in addition to doing its own evaluation studies. ED Staff preserve PAs’ ownership of their portfolios by not providing specific direction or suggestion on how to improve or administer a program or other offering. Instead, ED Staff provide guidance to be responsive to new market and regulatory conditions as well as to achieve performance goals.

The Commission, in D.21-05-031, made significant changes to how the portfolio administrators (PAs) structure and value their portfolios. Major changes include:

Adopting Total System Benefit (TSB) as the primary metric to assess PA performance instead of energy savings alone, Changing budget approval to cycle every four years instead of annually, Apportioning the portfolios into segments based on resource acquisition, market support, and equity, and

Applying forecasted cost-effectiveness requirements only to the resource acquisition segment over four years on a lifecycle basis instead of to the entire portfolio on an annual basis

In D.23-06-055, the Commission provides new authorization for nine out of the ten energy efficiency PAs in California: four Investor-Owned Utilities (IOUs), five Regional Energy Networks1 (REN), and one Community Choice Aggregator (CCA) to manage nearly $4.3 billion in ratepayer funds to achieve a forecasted $3.5 billion in Total System Benefit2 (TSB) in addition to other unmonetized equity, workforce, and supply chain benefits over 2024-2027. This includes a new REN called Rural REN (R- REN), to deliver energy efficiency benefits to underserved customers and communities in rural areas across California in four incontiguous regions. In addition, the decision:

Implements previous Commission guidance to split energy efficiency (EE) portfolios into four segments:

o Resource Acquisition - focused on TSB, or system impacts from energy and capacity savings as well as greenhouse gas emissions reductions, o Market Support - focused on long-term success of the energy efficiency market and technologies, o Equity - focused on ensuring better access and more opportunity to engage with energy efficiency for hard-to-reach and underserved customers, and disadvantaged communities; and o Codes & Standards - focused on improving the energy efficiency of buildings and products through state and federal advocacy and other programs.

Approves new progress indicators and directs the development of long-term goals for market support and equity segments.

Approves a new Community Based Program Design Collaborative to create process mechanisms to enable communities to design their own energy efficiency programs and implement them.

Includes guidance for continued emphasis on the market access program as a flexible procurement approach that can leverage federal Investment Recovery Act (IRA) funding, the use of normalized metered energy consumption (NMEC) methods for measuring energy savings, and more opportunities to implement integrated demand-side management (DSM) in the energy efficiency portfolios.

Pertaining to this solicitation, Section 5.1 of D.23-06-055 discusses the Commission and stakeholders’ desire for more transparent accountability. Specifically, it states:

… we will make $1 million available annually in reimbursable funding for technical support and consulting, to be added to EM&V collected funds. Commission staff may carry forward the funds from year to year.

Reimbursement will be sought from the four IOUs in the same manner as for EM&V budgets.

1 I-REN did not file an application as part of this proceeding because its 2024-2027 budget was already approved in D.21-11-013. I-REN is included here for completeness.

2 Total System Benefit is the dollar value of the benefit that energy efficiency improvements provide to the electric and natural gas systems. It is the sum of the avoided costs of energy, capacity, greenhouse gas compliance and other grid needs.

This budget is necessary because of the level of program reporting requirements and analysis necessary to assess energy efficiency program progress, and will require consultant technical support. Commission staff anticipate technical support and consulting on tasks including, but not necessarily limited to, the following:

i. Program reporting assessment, analysis, and recommendations for process improvement: This task includes supporting Commission staff on establishing protocols and templates to review and evaluate the PAs’ annual reports, certain advice letters, and other filings specified by staff. The consultant may develop recommendations to streamline existing reporting processes and create new ones. This task includes identifying, analyzing, and developing recommendations to inform energy efficiency inputs, assumptions, and analytical frameworks for the oversight processes, as well as expert support and necessary facilitation of stakeholder processes to develop and vet inputs and assumptions needed for oversight implementation.

ii. Development of PA Performance Reporting Templates: This task focuses on developing templates for public dashboards maintained and refreshed by PAs on a quarterly basis using quarterly reports. The consultant will support Commission staff in seeking public feedback and developing the structure and components of the dashboard. Also included in this task is continued technical assistance to staff to review and improve the dashboards, including their delivery to staff, stakeholders, and the public.

iii. Data Visualization: The consultant will support Commission staff development of Tableau (or equivalent) instance(s) to convey reported information from PAs. The visualizations’ primary objective is to facilitate staff and stakeholders’ easier ability to compare and contrast PAs’ progress, as well improve understanding of individual PA portfolios.

iv. Study recommendations: To ensure information informs application of oversight, this task includes identifying research needs and providing expert guidance on the development of various studies’ scopes of work, which can include Awareness, Knowledge, Behavior, and Attitude (AKAB) surveys, Non-Energy Benefit Studies, goal development, and others, as determined by Commission staff.

e. Organizational Chart

To facilitate understanding of how the EE Branch is structured, the following organizational chart is shown. This contract, while supporting the entirety of the EE Branch, will be managed from the Energy Efficiency Procurement and Portfolio Management Section, as highlighted below. A brief description is included for each section.

(Continued on Next Page)

A flowchart is also provided to demonstrate how the CPUC’s EE cycle is expected to function:

f. Bidder Library

In order to make an informed bid on this solicitation, the proposer should focus on the Bidder Library which shows documents that are specifically relevant to the proposed task:

D.12-11-015 D.16-08-019 D.19-12-021 D.18-05-041 D.21-05-031 D.23-02-002 D.23-06-055

g. Impending Near-Term Events

The work of the Contractor may be impacted by the advice letters or their dispositions, required of PAs as directed in D.23-06-055. For example, Market Support and Equity Indicators and Goal Development, Community engagement indicators, IDSM advice letters, Non-Energy Benefits (NEBs) studies, and other anticipated activities could impact services performed under this contract. Additionally, goal setting in the EE rulemaking proceeding – R.13-05-011 or successor proceeding, EE Applications due in 2026 (as well as ad hoc applications), and potential changes or refinements to cost effectiveness could impact services provided under this contract.

2. Statement of Work

a. Reports

For any of the Tasks that requires analysis and report component, the Contractor must complete both an individual draft and final report, along with any evaluation data and parameter updates, with sufficient review time (see a summary of the review process below) to publish by the indicated due dates. Reports must clearly and thoroughly address each of the objectives, in the manner defined in the approved Work Plans. To successfully do so, the Closeout Report must clearly answer each of the questions as identified in Task 2. Any draft and final documents will be submitted to the CPUC Contract Manager for review, and to provide final edits and directions before finalizing and publishing the report.

Report Format

Draft reports will be complete versions of the final reports and should not require substantial editing.

Both the draft and the final evaluation report will include the following components:

I. Title Page: A single page on which the title and author(s) of the report are clearly presented, along with the date of publication, either the word “Draft” or “Final” as appropriate, and any additional textual or graphical elements the Contractor chooses to include. The title should be concise but must convey the subject matter of the study.

II. Contacts Page: A single page after the title page which presents the names, affiliations, and contact information for all key individuals associated with the study.

III. Table of Contents: A listing of sections of the report, and the numbers of the pages on which the sections begin.

IV. Executive Summary: A brief, easily understandable document, written in plain language for a non-technical audience, e.g., policy makers and members of the public, to quickly understand the design, execution, and results of the study, written according to the Executive Summary writing guide. Executive Summaries should be approximately 10% of the length, by page count, of the entire report.

V. Approach and Methodology: A written explanation of the Contractor’s study methodology must be included in each evaluation report and shall include sufficient detail to allow industry peers to replicate the Contractor’s sampling, collection, and calculations independently.

VI. Data Source: A fully detailed document of data sources used in this report.

VII. Findings and Recommendations: The contractor is responsible for developing recommendations to improve oversight of energy efficiency portfolios in each sector, as per the objectives of the task.

VIII. Submission of electronic PDF formatted document: Upon Contractor submission, the PDF shall be in a searchable format (not a scanned image) that is ready to be uploaded by CPUC staff to the CPUC’s website for public consumption of the final graphic design and layout of the evaluation. This may include references to electronic dynamic appendices or models which should also be made available for public consumption in the CPUC’s website.

IX. Table of “Comments”: A table of comments including the commenting party, the channel through which the comment was received, and the text of each comment, along with a description of how the Contractor addressed the comment or a rationale for taking no action.

X. Submission of a print-ready format document: The Contractor shall submit to the CPUC a digital copy of the final graphic design and layout of the final report document in a standard format that the CPUC can readily submit to a printing entity for mass printing.

Furthermore, draft and final reports shall incorporate the following critical elements:

XI. Readability: The executive summary shall be written in a way that is understandable to a non-technical audience, avoiding industry jargon, advanced concepts, and all acronyms specific to the energy or energy efficiency industries. The entire report shall also be well-written employing clear language, and a formal, objective tone.

XII. Copy-editing: The draft and final report must be copy-edited by a professional copy-editor. The Contractor must provide a résumé of the copy-editor to the CPUC and obtain approval before finalizing the reports. Draft and final reports should be well-written, logically organized, and cleanly formatted; CPUC staff may request rewriting or further editing at its discretion.

XIII. Review process: All reports are subject to a review and comment process agreed upon between the Contractor and CPUC staff. This review process may include the following types of reviews, which may occur in any order or concurrently:

XIV. Peer review—At the discretion of the CPUC staff, specific study reports will undergo a peer review process prior to finalization. The peer review process must meet current industry standards, and the peer review panel must be selected or approved by the CPUC.

XV. Stakeholder review— At the discretion of the CPUC staff, reports shall be distributed in draft among parties identified as stakeholders in the report subject matter. Stakeholders may include, but are not limited to: portfolio administrators, contractors, evaluators, individual customers.

XVI. Public review—At the discretion of CPUC staff, draft reports shall be made available to the public, and comments shall be accepted for a period of not less than ten (10) calendar days. The Contractor shall consider each public comment and address as deemed appropriate.

XVII. Energy Division Management Approval Process—The contractor shall allow for up to two (2) weeks of review by Energy Division Management. Energy Division Management Approval is required prior to releasing any version of an evaluation report to the public.

XVIII. Graphic Design: The draft shall be formatted in a professional manner with an effective design that meets or exceeds typical standards for high level public reports. The design should convey that the Contractor is responsible for the content, and no CPUC-branded material (e.g., the Seal of the

Commission or the CPUC logo) shall appear throughout the document in such a way as to suggest that the CPUC or CPUC staff wrote the report or conducted the study.

Report Presentations

Reports will potentially be presented at a stakeholder or public workshop, or an executive briefing, as described in the Tasks, as necessary. Each briefing will be approximately one (1) hour long, and the Contractor must record and respond to each comment received from the stakeholders and/or executives by making changes in the report or addressing the specific comments in a summary report. Upon approval by CPUC staff on the final reports, the formatted Final reports will incorporate all edits from the previous drafts and will be submitted to the CPUC within one (1) month of receiving feedback from the CPUC.

Report Contents

The following section describes the overall work that will be completed under Energy Efficiency (EE) Technical Consultant Services contract: Portfolio oversight mechanisms, tools and resources for Portfolio Administrator (PA) filings and project review, data visualization, research recommendations, and subject matter expertise on a variety of matters. The Contractor will submit a proposed workplan and provide a written narrative as a part of the proposal to specify the activities, work products, schedules, and budgets for all tasks and deliverables required under this contract. The CPUC will assign Project Managers to manage each sector and subject area, and serve as a point of contact for specific projects.

No work will be performed under the Prime Contract without a prior approval by the Project Manager, except where specifically agreed upon in advance.

The first deliverable due under this contract will be the Administration and Project Management Workplan for each task area that will guide the on-going requirements of the contract.

IMPORTANT: In order to meet the needs of the deliverables in this contract, the Prime Contractor must complete work products to meet the various deadlines. This will require integrated project management and coordination between various work products and deadlines related to the decisions listed in the Bidder Library and other filings in the energy efficiency proceeding (R.13-11-005). The performance of the contract associated with this RFP includes all administrative costs associated with the statement of work and any costs associated with the compliance of all government requirements.

b. Tasks and Deliverables

I. Task 1: Workplans (Narrative Required)

The Contractor is expected to provide ED Staff with plans detailing how and when the tasks included in the scope of work are intended to be completed. All work tasks shall comply with the CPUC style guide and take into consideration readability to a non-technical audience, copy-editing needs, and peer review incorporation.

Under the direction of the CPUC Contract Manager, the Contractor must develop a detailed workplan for every deliverable, this will be considered the submitted workplan. The Contractor must in addition develop a narrative for deliverables that are designated as “Narrative Required” describing the innovative approach proposed for these deliverables. The Contractor will also work with the CPUC Contract Manager to establish the project management processes for the contract and approach for finalizing the Work Plans.

Each workplan must include:

A detailed description of the Contractor's approach and methodology for each specific Deliverable, A timeline for completion of each task and a schedule for both one-time and recurring activities, A list of major activities, tasks and sub-tasks, and milestones for each Deliverable, A list of resources and personnel necessary for each Deliverable, The relevant skills and experience of key personnel who will work on each Deliverable; and A detailed budget for each Deliverable with estimates broken down by task and activity.

The Contractor must obtain ED Staff approval of the workplans before commencing work toward their completion. The Contractor should coordinate with appropriate ED Staff teams to develop the workplans, so they are guided by the needs of Staff.

List of Expected Deliverables

i. Final Workplan for each task and related deliverables incorporating feedback from CPUC staff review.

II. Task 2: Progress Reports and Updates

The Contractor will be responsible for timely interim progress updates and reporting milestones as negotiated with the CPUC Contract Manager. Required Progress Reports and updates will include the following:

Monthly Progress Reports

Prime Contractor will provide monthly progress reports and data sets to CPUC Contract Manager, CPUC Staff, and/or other entities at the discretion of the CPUC Contract Manager that could include but are not limited to the following issue areas:

Contract compliance, Deliverables progress and results, including the percentage of completion of each task, Project management’s notable findings and concerns, Project expenditures, Work activities, Progress, Work product deliverables, and Other information upon request

These reports shall be produced in a timely manner and must go through a quality control process before being submitted to the project manager. It is responsibility of the Contractor, not CPUC staff, to ensure accuracy and timeliness of the report.

Sub-Contractor Records and Other Government Records Reports

The Prime Contractor is responsible for collecting, retaining, and making available to CPUC staff all records associated with the scope of work, including subcontractor records and government reporting requirements, including but not limited to, Small Business and DVBE reports, progress reports, all deliverables, and presentations. At any point in the contract, the CPUC Contract Manager and relevant Project Manager may request immediate access to any of these records, and the Contractor must prioritize these requests. At the end of the contract, the Contractor is responsible for ensuring that all these records are provided in their entirety to the CPUC in a manner that gives access and control of the records to the CPUC.

Files

The Prime Contractor is responsible for creating and maintaining a database of all data, files, scripts, and outputs created while completing this deliverable and making that database available to the CPUC Contract Manager and relevant Project Manager. This database must also include documentation of how each output was created and how it was used.

Closeout Reports

The Contractor shall provide to CPUC at the end of the contract, a closeout report to summarize all activities, efforts, and findings throughout the contract, and the Contractor shall maintain all project records for a period of three years after the end of the contract.

Contents

The closeout report has two components, the project closeout and the data closeout.

The project closeout shall describe:

The plan, Timeline to ramp down work, and Any plan to transition activities between contract phases.

The data closeout shall describe:

All the data and files used over the course of the contract, A single, coherent, and comprehensive manual that explains what all the data is, identifies the files the data is in, and How the data and files were used over the course of the contract.

All files must be transferred in a secure electronic format (such as SFTP) to the CPUC Contract Manager.

Draft

A draft version of the comprehensive manual and related file architecture must be provided to the CPUC not less than 2 months before the conclusion of the contract, to allow CPUC staff to review, ask clarifying questions and request necessary changes.

The CPUC is responsible for requesting any changes to these materials no less than 3 weeks prior to the conclusion of the contract. The Contractor is responsible for implementing those changes and submitting the final comprehensive manual and related files within 1 week of the conclusion of the contract. If the final draft of the comprehensive manual and related files are not submitted within 1 week of the conclusion of the contract the Contractor shall forfeit 10% of the retention portion of this deliverable each business day until the final draft is received by the CPUC.

Final

If the final draft of comprehensive manual and related files have significant errors or have not addressed the draft feedback provided by CPUC, the Contractor will forfeit up to 50% of the retention portion of this deliverable.

Task 2 List of Expected Deliverables:

i. Monthly Progress Reports

ii. Narrative Reports as well as Monthly Datasets

iii. Project File Database

iv. Project Closeout Report

v. Data Closeout Report, including Manual

vi. Draft Report

vii. Final Report

viii. Closeout File Transfer

III. Task 3: Internal Kickoff Meeting (No Narrative Required)

Upon execution of the contract, an in-person, hybrid, or online kickoff meeting (to be determined) will be convened for CPUC staff and the Contractor’s Team to discuss and refine the submitted Work Plans and proposed research Work Plans, orient the Contractor’s Team to the administrative and technical expectations of the CPUC Contract Manager and/or CPUC staff Sector Leads, and initiate discussion of prioritization of tasks.

Activities include:

Meeting with the CPUC Contract Manager and CPUC staff EE sector leads to affirm expectations, communication protocols, objectives for the contract, and the proposed approach/methodology to complete said contract, Coordinating with the Prime Contractors on other contracts the CPUC may undertake to complete any related deliverables, as necessary and with approval from the CPUC Contract Manager.

Task 3 List of Expected Deliverables:

i. Internal kickoff meeting: includes planning, facilitation, and physical and/or virtual space.

ii. Meeting agendas, PowerPoint presentation, meeting notes, and action items.

IV. Task 4: Coordination Meetings with CPUC Staff (Narrative Required)

Bidders shall provide specific bids for items within this deliverable; see the Cost Sheet and Standard Agreement Example (often listed as Attachment 13 to an RFP).

As needed by ED Staff, the Contractor will coordinate, attend, and – in some cases - facilitate project coordination group and other meetings with CPUC Staff and other parties on an ad hoc basis or regular schedule:

Weekly EE Team/Consultant check-in calls to discuss work in progress and upcoming work.

Not to exceed fifty-two (52) call hours annually, and

Other regular check-ins and ad hoc calls and meetings to discuss specific projects and deliverables. Not to exceed a total of one hundred (100) hours of in-person meetings annually, and four hundred (400) conference call hours annually.

As needed support to staff internal briefings to decision-makers to explain methods, assumptions and to present results.

Since many calls/meetings are likely to require the attendance of multiple Contractor staff, the above mentioned 52-, 100-, 400-hour amounts are not person-hours, but rather meeting hours.

Each Contractor should cost out each meeting appropriately to cover the cost of the necessary attendees.

The purpose of these meetings is to provide a venue to discuss the status of tasks across the various work tracks with CPUC Staff and stakeholders; discuss and vet methodological or data issues; strategize for coordination between agencies; and resolve administrative, invoicing, and reporting matters concerning the contract.

The Contractor is responsible for keeping meeting agendas, notes, and action items accessible to CPUC Staff.

Task 4 List of Expected Deliverables:

i. Host up to fifty-two (52) check-in calls annually to discuss work in progress and upcoming work.

ii. Host other regular check-ins and ad hoc calls and meetings to discuss specific projects and deliverables. Not to exceed a total of one hundred (100) hours of in-person meetings annually, and four hundred (400) conference call hours annually.

iii. For each meeting: produce and share agenda prior to meeting and provide notes and action items after the meeting.

V. Task 5: Stakeholder Workshops and Webinars (Narrative Required)

The Commission values transparency and stakeholder input in the development of oversight practices, resources, and other research products. The work developed under the EE Technical Consultant contract will be no exception.

The Contractor will participate in a maximum of fifty-four (54) public stakeholder engagement activities each year, during which they must be able to explain their recommendations, methods, approaches, and results to an audience of both technical and non-technical individuals. These meetings serve as an opportunity to solicit formal or informal comments from stakeholders. Public engagements vary in length and complexity and may include half to full day in-person all party meetings or workshops, two-hour webinars, conference calls, or in-person meetings of various durations. CPUC staff shall determine the nature and duration of the public engagement, as well as the agenda.

The Contractor should expect to participate, which may include presenting at and facilitating stakeholder engagement, annually in up to:

Six (6) half-day in-person workshops, Thirty-six (36) 2-hour remote meetings, and Twelve (12) 2-hour in-person meetings

The Proposer shall provide pricing for various types of engagements. Responsibilities will include:

Participating and possibly presenting at public meetings and workshops at CPUC offices in either San Francisco, Los Angeles, or Sacramento, periodic technical advisory groups, or webinars associated with the Deliverables outlined in relevant Tasks.

Addressing issues that members of the public raise at the stakeholder workshops with specific documentation of how stakeholder comments have been considered or addressed, Managing written and oral communication, including public briefing materials, summarizing work products to decision-makers and stakeholders, as requested by CPUC Staff, Creating materials, including visualizations of complex information, that are provided to participants, Explaining ongoing studies and upcoming reports, as well as results and findings upon study completion, and

Facilitating public and stakeholder meetings and workshops using best practice facilitation techniques as well as appropriate audience engagement platforms and tools.

The Contractor will be responsible for keeping meeting notes, analyzing, and preparing recommendations for next steps based on feedback received.

Task 5 List of Expected Deliverables

i. Provide SME and/or hosting services at up to six (6) half-day in-person workshops annually.

ii. Provide SME and/or hosting services at up to thirty-six (36) 2-hour remote meetings annually

iii. Provide SME and/or hosting services at up to twelve (12) 2-hour in-person meetings

iv. Produce and share agenda for up to each engagement

v. Meeting notes and action items for up to each engagement

vi. Record, organize, and responds to stakeholder feedback for up to each engagement.

VI. Task 6: Presentations (Narrative Required)

The Contractor will be responsible for developing presentation materials and contributing to the development of CPUC Staff developed presentation materials on a variety of different topics. All presentations developed by the Contractor will include a draft version for CPUC staff feedback before a final version is produced.

Presentations can have different levels of effort to develop and present. To address this, this agreement includes different levels of presentations representing different approximate levels of effort. The Contractor should coordinate with ED Staff to determine the appropriate level of presentation for a given task.

Task 6 List of Expected Deliverables

i. In coordination with ED Staff, develop presentations for a variety of audiences. ED Staff will coordinate with the Contractor to specify for which tasks and deliverables the Contractor will develop a presentation. Prior to presentation development, the Contractor shall propose whether it will require Level 1, 2, or 3 effort, and obtain ED Staff approval for the presentation and associated level(s). The Contractor should develop pricing for the following:

1. Level 1: 1-2 hours of prep, up to 30 minutes presenting, no meeting facilitation

2. Level 2: 3-5 hours of prep, up to 60 minutes presenting, meeting facilitation possible

3. Level 3: 6+ hours of prep, 60+ minutes presenting, meeting facilitation possible

VII. Task 7: Resources and Tools for Review of PA Filings and Other Submissions (Narrative Required)

ED Staff and stakeholders to CPUC proceedings review various types of PA filings and other submissions in order to understand PA plans, assess their reasonableness and assess PA performance toward meeting goals and policy objectives. Based on ED priorities and guidance, the Contractor will develop tools to support Staff review of key aspects of PA EE portfolios and support enhanced reporting.

Sub-Task 7a: Develop Resources to Support Staff Review of PA Filings and Other Submissions (Narrative Required)

The Contractor will support Commission staff activities to document protocols and create/refine templates and checklists to develop, review, and/or evaluate the PAs’ portfolio applications, annual and quarterly reports, certain advice letters and compliance filings, and other submissions specified by staff. Templates may include those used by the PAs to complete their applications, as well as those used by Staff to review the applications and distill the information for various needs. This work will also include providing technical support and guidance on methods to improve Commission Staff’s use and implementation of CEDARS as part of oversight activities.

Additionally, as determined with ED Staff, the Contractor shall train staff on the use of new tools and resources every quarter.

Sub-Task 7a List of Expected Deliverables:

i. Mid-Cycle Advice Letter review checklist

ii. PA Application narrative and spreadsheet templates application review guidance, templates, and checklists for staff

iii. PA Annual Report template, staff checklist, and guidance

iv. PA Semi-Annual Report template and staff checklist

v. Quarterly Report checklist

vi. Monthly Report checklist

vii. True-Up Advice Letter template

viii. Support for Advice Letter review as assigned by ED staff

ix. Report with recommendations to improve Staff’s utilization of CEDARS

x. Other review tools as identified by ED staff

Sub-Task 7b: Visualizing Information (Narrative Required)

The Contractor will support Commission staff by making more information visual. Reviewing information that is converted into charts, graphs, animations, and other graphics improves Staff’s and stakeholders' ability to compare and contrast PAs’ progress, as well improves understanding of individual PA portfolios. This task includes development and maintenance of Tableau or PowerBI (or equivalent) instance(s) to convey reported information from PAs. Each deliverable includes:

Assessing EE Branch management and staff requirements, desires, and interests to gain understanding of how they intend to use visualized information for portfolio oversight.

Developing and support implementation of recommendations for interfacing between CEDARS and Tableau or PowerBI or another equivalent data visualization platform

Developing guidance and periodically training selected Commission Staff on how to best adjust, adapt, and/or utilize visualization tools for best oversight practices

Developing Tableau or PowerBI (or equivalent) instances to allow ED Staff to better understand information submitted via applications; certain motions; monthly, quarterly, and annual reports; certain compliance filings; and other sources identified by ED Staff. This includes providing staff with mockups prior to full production to allow structured ED Staff feedback along the way.

Continued support for improvements, new elements, and refinements.

Sub-Task 7b List of Expected Deliverables:

i. Tableau or PowerBI (or equivalent) instance conveying PA EE application information with associated guidance.

ii. Tableau or PowerBI (or equivalent) instance conveying PA progress/compliance report information beginning in 2022 or earlier based on associated guidance.

iii. Hourly post-production support as requested by ED.

Sub-Task 7c: Develop Public Dashboards Conveying PA Performance and Progress (Narrative Required)

Develop templates for public dashboards maintained and refreshed by PAs on a quarterly basis using quarterly reports based on ED Staff input. The dashboards will be housed on CPUC’s CEDARS website and will have links to the dashboards from CAEECC and all the PAs’ websites.

The Contractor should consider the feasibility of structuring the dashboards as an API or html widget (or equivalent) so their most up-to-date versions can be viewed in various websites simultaneously. Energy Division Staff, in consultation with the Contractor, may determine that the best method of dashboard development and delivery is associated with a Tableau or PowerBI (or equivalent) instance described in Section b. above; however, it is not making that determination yet. The Contractor should plan for this as an option. This sub-task includes:

Supporting Commission staff in seeking public feedback via the California Energy Efficiency Coordinating Committee (CAEECC), the Reporting PCG, and other means, to design the dashboards so they are useful for staff and stakeholder review of PA progress;

Developing the structure and components of the dashboard so they convey information that is key for assessing performance, expenditures, progress, and major concerns at the portfolio, segment, and sector levels; and

Providing technical assistance to staff to review and improve the dashboards, including their delivery to staff, stakeholders, and the public.

Sub-Task 7c List of Expected Deliverables:

i. Mockup of dashboards – based on ED Staff and PA feedback - for public comment

ii. Final dashboard template for PAs to populate

iii. Monitoring use of dashboards to assess usability resulting in periodic recommendations for their improvement

Sub-Task 7d: Support Enhanced Reporting (Narrative Required)

Develop and support implementation of recommendations to ED and the Reporting PCG for enhanced PA performance reporting to streamline existing PA reporting processes and create new ones based on EE staff and PCG input. This includes:

Identifying, analyzing, and developing recommendations to inform EE Staff about options for inputs, assumptions, analytical frameworks, data sources, thresholds, and outcomes for enhanced energy efficiency performance assessment;

Suggestions for improvements to reporting processes, procedures, templates, guidance, checklists, or other resources to improve functionality, usability, manageability, and/or display of information; and

Expert support and facilitation of stakeholder processes – if needed – to develop and vet inputs, outputs, and assumptions needed for performance assessment.

Sub-Task 7d List of Expected Deliverables:

i. Internal Memo to EE Branch: up to four annually

ii. Stakeholder facilitation: up to six annually

Sub-Task 7e: Staff Data Request Support (Narrative Required)

Staff occasionally need assistance obtaining record-level data that can only be accessed with database software such as PostgreSQL or SQL Server, or other software capable of handling large datasets, such as SAS or R. A technical consultant would be expected to perform database queries from either the CEDARS data server or the Energy Division central server and provide this to staff in a usable format as determined by the staff request. The consultant would be expected to document the query and explain any questions staff have about how the data is queried and used.

Sub-Task 7e List of Expected Deliverables:

i. Technical services – hourly

VIII. Task 8: Data Transfer and Documentation (Narrative Required)

The Contractor will document any and all specifications and descriptions of the datasets throughout the life of this contract and provide all data collected to the Contractors and to the CPUC. This deliverable is in addition to the closeout report detailed in Task 2. All datasets, data tools, and manuals describing their contents and how to use them, shall be transferred to the CPUC on an ongoing basis in a format readily accessible to CPUC staff. The file type of these datasets, tools, and user manuals, as well as the location to which the Contractor is to transfer them must be approved by the CPUC Contract Manager and the data contract group prior to transfer.

Any primary data generated as a part of this contract shall be considered the property of the CPUC and will be made available to the public subject to the CPUC’s data privacy and confidentiality provisions. The Contractor shall provide the raw data, manipulated data, any secondary spreadsheets, databases, source codes, underlying scripts, algorithms or other project-defining creations, as well as user manuals explaining all data and how it was used, to the CPUC. The Contractor will work with relevant CPUC staff to develop data specifications.

All materials falling under this deliverable must be complete, accurate, have gone through a quality control process, and be transferred to the CPUC’s designated location by the end date of this contract for the Contractor to receive payment for this deliverable.

Task 8 List of Expected Deliverables:

i. All data collected related to this contract;

ii. Documentation of program-specific analyses and related findings;

iii. Data collection instruments, such as forms, workbooks, and surveys;

iv. All algorithms, scripts, source codes, spreadsheets, databases, applications or files, etc., developed in the implementation of this contract;

v. Training materials and user manuals detailing what is included in each dataset/data tool and how it was used; and

vi. Documentation of all related documents for data collection.

vii. Delivery of other datasets and documents to be determined by the CPUC Contract Manager.

IX. Task 9: Support Staff Performance Monitoring: Indicators, Metrics, and Goals (Narrative Required)

D.18-05-041 approved what are known as the common metrics. These are a set of over three hundred metrics and indicators all the PAs should use to report progress on their programs. The PAs report different information monthly, quarterly, and annually. EE Staff are aware of several indicators and metrics that are either not used correctly, consistently, or at all by the PAs.

D.19-12-021 COL 6 requested the Regional Energy Networks (RENs) describe their unique value to California’s energy, climate, and equity goals. All the RENs submitted metrics and indicators that allow assessment of their unique value in their business plans. ED continues to assess whether and how the RENs’ unique value metrics and indicators provide utility and allow analysis of trends, and will seek the Contractor’s support for further evaluation.

D.23-06-055 approved a new set of indicators for the Market Support and Equity (MS&E) segments based on recommendations by the CAEECC Market Support and Equity Working Groups. That decision also directed the development of MS&E goals which are anticipated to be adopted for the 2028-2035 cycle. As new MS&E goals are developed, some MS&E indicators may become metrics that demonstrate progress toward the goals in the next eight-year cycle starting in 2028.

The Contractor will support Energy Division staff in their tasks to review, understand, and prioritize the indicators and metrics to better understand PAs’ progress. The Contractor is expected to provide EE Staff with analysis of the utility of and trends identified within indicators and metrics to aid in Staff’s oversight. Additionally, Staff need support to help analyze, visualize, and provide expertise as different constructs for goals are considered by PAs and stakeholders.

Further support may be required in a later phase to help quantify goals.

In May 2024, the PAs will file a joint Tier 2 advice letter adding clarity to the details of the MS&E indicators and recommending common metrics for suspension, removal, or modification. The Contractor should use the information in the advice letter to set context for analysis and recommendations of indicators and metrics. The following sub-tasks are phased, with subsequent sub-tasks building off the outcomes of previous sub-tasks. The full scope of work for sub-tasks 2 and 3 will be defined following phase 1 implementation.

Sub-Task 9a: Assessing Utility of Indicators and Metrics (Narrative Required)

Review and make recommendations on the utility of Energy Efficiency Portfolio Sector and Segment Indicators and Metrics to inform outcome-based oversight by ED and stakeholders. The Contractor shall consider the findings of relevant EM&V studies when making recommendations.

Phase 1: Develop report focusing on common metrics and RENs’ unique value metrics:

whether, how, when, in what circumstances metrics and indicators are used by which PAs.

Identify metrics and indicators that are under or unreported, provide inconclusive information, and/or reported inconsistently by PAs. Present ED approved content to CAEECC.

Sub-Task 9a Phase 1 List of Expected Deliverables:

i. Public Report

Phase 2: Working with ED Staff, develop memo suggesting improvements to utility of common metrics and indicators to achieve outcome-based oversight. The full scope of phase 2 will be developed with ED staff and may be based on outcomes from phase 1. Improvements may consist of any or all of the following: categorization, granularity, specificity, methodology, valuation, units, calculation, usability, delivery, meaningfulness, or other aspects of the metrics and indicators. Present ED approved content to CAEECC.

Sub-Task 9a Phase 2 List of Expected Deliverables:

i. Internal Memo

Phase 3: Support development of MS&E goals and develop a report focusing on MS&E indicators: whether, how, when, in what circumstances indicators are used by which PAs. The full scope of phase 3 will be developed with ED staff and may be based on outcomes from phases 1 and 2. Identify indicators that are under or unreported, provide inconclusive information, and/or reported inconsistently by PAs. Suggest improvements of the type that is requested in Phase 2. Present ED approved content to CAEECC. May include more than one meeting with CAEECC.

Sub-Task 9a Phase 3 List of Expected Deliverables:

i. Public Report

Sub-Task 9b: Assessing Trends in PA Reported Information (Narrative Required)

Conduct analysis to identify noteworthy trends, correlations, and associations in the reported information. Noteworthiness may relate to goal achievement, market conditions, unexpected performance, or other. The Contractor shall consider the findings of relevant EM&V studies in its analysis.

Phase 1: By September 2025, develop report identifying…