Attachment_16_Evaluation_and_Scoring_Criteria_Revised.pdf

ATTACHMENT 16: EVALUATION AND SCORING CRITERIA

Name of Proposer ________________________________________________

Name of Scorer/Rater ________________________________________________

SCORING SHEET

All proposals that advance past the Compliance Check (pass/fail) for meeting the Minimum Qualifications. (Section 3.1 of the RFP) and complying with Submission Requirements (Section 5 of the RFP) will be evaluated in the following scoring stages:

Summary of Categories and Point Value (1,022 Points)

Stage 1: Experience and Approach (Technical Evaluation): 642 Points Proposals must receive a minimum of 276 points in Stage 1 to advance Stage 2: Cost Evaluation: 380 Points

Scoring Definitions:

COMPREHENSIVE

Proposer’s overall response provides numerous details and/or examples that demonstrate the proposer's ability to meet all the requirement(s). The bidder addressed requirements with adequate details and the responses demonstrates that the bidder possesses the ability to meet the requirement(s).

SATISFACTORY

Proposer’s response provides sufficient details and/or examples demonstrating the ability to meet all the requirement(s).

MARGINAL

Proposer’s overall response provides details and/or examples but does not demonstrate the ability to meet all the requirement(s). The bidder may have addressed requirements with insufficient details, or not at all, but the overall response demonstrates that the bidder possesses the ability to meet the requirement(s).

POOR Proposer’s response provides insufficient details and/or examples demonstrating the ability to meet most requirement(s).

FAIL Proposer’s response is omitted completely or does not address or relate to the requirement(s).

Metzer, Ryan Definitions don’t match matrix.

Compliance Check Each proposal will be checked for the inclusion of required information in conformance with the submission requirements. Each proposal will also be reviewed to determine if the proposer team meets administrative compliance requirements. Proposers that meet all required criteria will begin the Proposal Evaluation Process. Proposers that fail any check will be deemed non-compliant and will not be eligible to receive an award.

Minimum Qualifications Criteria: Fail Pass

*See Sections. 1.1.3-1.1.5 under Proposer’s Team’s Qualifications to find a detailed break-down of Minimum Qualifications.

The Prime Proposer must identify all subcontractors proposed for participation in the contract, and the tasks or work areas that would be assigned to each subcontractor. A process does exist to add or change subcontractors over the course of the contract, but Prime Contractors are encouraged to identify all subcontractors in the bid proposal.

Proposers awarded a contract are contractually obligated to use the subcontractors for the corresponding work identified unless the CPUC Program Manager agrees to a substitution. See Attachment 6 – Bidder Declaration (GSPD- 05106). Additionally, if the subcontractor is a Disabled Veteran Business Enterprise (DVBE), then any replacement subcontractor must also be a DVBE.

No Yes

The Prime Proposer and all key members of the proposal team have certified and agreed to the bidding and subcontracting limitations that a firm submitting a proposal as the Prime Proposer for RFP [Title] may not be a subcontractor on any other proposing team for RFP [Title].

No Yes

RESULT:

Stage 1: Experience and Approach Proposers can receive up to 642 possible points in this stage. Points are combined from the Proposer’s Team Qualifications (incudes Project Team Description and Experience, and Minimum Qualifications), and Approach and Work Plan (includes all Deliverables).

Proposers who have passed Stage 1 with the required minimum point value of 276 will be evaluated further. Proposers that do not receive the minimum point value will be deemed non-compliant and will not be eligible to receive an award.

Below is a summary scoring sheet by section followed by scoring sheets for each item within all sections.

Section Point Value of Component Point Value Earned

Proposer’s Team Qualifications

1. Organization Description (20 points)

2. Desired Qualifications (50 points)

Approach and Work Plan

1. Overall Approach (75 Points)

2. Deliverable Narratives (497 Points)

Metzer, Ryan This table does not match MQs in solicitation

Total Stage 1: 642

1.1 Proposer’s Team Qualifications (70 points)

Organization Description (20 points) Fail Poor Marginal Satisfactory Comprehensive Score

1.1.1 Project Team Description and Organizational Chart (5 points):

The Proposer provides a detailed project organization chart and description of its team organizations and key personnel and clearly shows the specific role that each team member will have in performing the project functions.

0-1 2 3 4 5

1.1.2 Project Team Accountability (5 points):

The Proposer’s project organizational chart describes the administrative mechanisms by which each team member will be held accountable for performance.

0-1 2 3 4 5

1.1.3 Team Organization and Governance (10 points):

The proposer describes the project team’s experience and competency in the following areas:

1. Capacity expansion modeling

2. Experience and expertise in providing expert technical advice to clients on energy resource planning related topics

3. Managing employee turnover rate;

4. Managing subcontractors, if applicable;

5. Managing demand to scale up/down;

6. Managing employee bandwidth; and

7. Redundancy Plan (Plan to ensure continuity of service through turnover of personnel).

0-2 3-4 5-6 7-8 9-10

Minimum Qualifications (50 points) Fail Poor Marginal

Satisfactory

Comprehensive

Score

1.1.4 Project Administration Experience (10 points):

The Proposer’s Project Team should demonstrate understanding, expertise, capacity, and experience in successfully managing complex projects involving multidisciplinary teams comprised of the following:

a. At least one senior-level experienced overall project manager with at least 5 years experience managing complex projects;

b. At least one manager for each work track/project with at least 3 years managing projects;

c. Experience shall reflect the Proposer’s effectiveness in managing time and resources across multiple projects, setting work plan schedules and workflow tracking, and responding to client needs in a timely and competent manner.

d. Contractor must show 5 years experience of team level experience with matrix management on complex projects.

0-2 3-4 5-6 7-8 9-10

1.1.5 Analytical Skills and Experience (20 points):

The Proposer’s Project Team should demonstrate experience providing analytical

0-4 5-8 9-12 13-16 17-20 expertise, capacity, and experience for at least one cycle that spans 2-3 years that closely mirrors the CPUC’s IRP process. Team should provide examples and describing their role in the following energy programs evaluation related skills:

a. Full IRP-cycle like experience (2-3 years) that directly requires developing and implementing complex study designs and producing actionable recommendations for decision-makers;

b. Full IRP-cycle like experience (2-3 years) that directly requires collecting and organizing heterogeneous data from an array of sources;

c. Full IRP-cycle like experience (2-3 years) that directly requires utilizing and demonstrating proficiency with capacity expansion models and deep-understanding of production cost modeling, or equivalent models in conducting energy resource planning analysis.

1.1.6 Energy Resource Planning Skills and Expertise (20 points):

The Proposer’s Project Team members should demonstrate expertise, capacity, and experience, by providing examples and describing their role in the following energy resource planning related skills:

a. Direct experience and competency in capacity expansion modeling in a capacity that closely resembles a full IRP cycle (2-3 years).

b. An understanding of energy resource planning topics and issues, including those related to renewable resources, fossil fuel-based energy resources, electricity demand and load modifiers, transportation and building electrification, grid integration issues, distributed energy resources, long lead-time (LLT) resources, transmission and distribution, and reliability studies and principles, including loss of load expectation, loss of load probability modeling, planning reserve margins, and effective load carrying capabilities (ELCCs). Knowledge of these topics should be demonstrated in capacities that closely resemble the CPUC’s cycle (2- 3 years) where applicable.

c. Knowledge of the California power market including system, local, and flexible resource adequacy (RA) and RA’s slice-of-day (SOD) framework; utility regulation; and planning and procurement interactions between the CPUC, CAISO, CEC, and load serving entities (LSEs). Contractor should be able to demonstrate at least 5-years of collective knowledge on these topics and should be able to demonstrate experience with client management for entity similar to

CPUC.

d. Engineering expertise of the electric power distribution system to assist Commission staff in the representation and optimization of different resource types, including distributed energy resources (DERs), within IRP modeling.

Contractor should be able to demonstrate at least 5-years of collective knowledge on these topics and should be able to demonstrate experience with client management for entity similar to CPUC.

Maximum Points Possible: 70 TOTAL SCORE:

1.2. Deliverables Approach and Work Plans

Overall Approach (75 points) Fail Poor Marginal Satisfactory Comprehensive Score

1.2.1 Proposer sufficiently details the approach to execute project oversight and the plan to coordinate with subcontractors (if applicable) to ensure success of all deliverables.

0-2 3-4 5-6 7-8 9-10

1.2.2 Proposer sufficiently and clearly identifies specific interdependency and interaction points with the Commission and between deliverables. It is not sufficient to simply acknowledge the interaction points. The proposer must clearly articulate how the interaction points will be managed and must address how risks for meeting Commission specific deadlines and Commission needs will be mitigated.

0-2 3-4 5-6 7-8 9-10

1.2.3 Proposer sufficiently and clearly addresses and incorporates interdependencies with planning organizations (including CEC, CARB, and CAISO) and other Energy Division sections, and how the integration of data will be done in an accurate and efficient manner.

0-2 3-4 5-6 7-8 9-10

1.2.4 Proposer sufficiently details the approach to address all identified deliverable submission requirements for all deliverables under this RFP, taking into considerations such as its readability to a non-technical audience, copy-editing needs, and peer-review incorporation. Proposer also addresses how reports will be produced in a format easily accessible to the public, including models utilized in all tasks.

0-1 2 3 4 5

1.2.5 Proposer’s narrative and approach to each deliverable sufficiently demonstrate knowledge of Commission processes and proceedings as well as an understanding of energy resource planning issues, including renewable resources, conventional energy resources, load modifiers, electrification, grid integration issues, transmission and distribution, and system and local reliability.

1.2.6 Proposer’s narrative demonstrates the proposer’s approach for dealing with uncertainties due to evolving research needs such as: changes in policy objectives, Commission directions in both integrated resource planning or other proceedings such as EE, IDER, RA, related process alignment with IRP. Proposer’s narrative also demonstrates approach for dealing with uncertainties from other planning agencies’ inputs, including CEC’s IEPR and CAISO’s TPP.

0-2 3-4 5-6 7-8 9-10

1.2.7 All Work Plans provide a step-by-step account of how the Proposer plans to complete all work outlined in the RFP, including all deliverables, with Deliverable 14 requiring description of management of personnel, budget, and coordination with commission staff. The Work Plans clearly delineate in a logical and realistic manner the main activities and resources required including activities, tasks, methodology, resources, timeline, personnel, budget, and coordination with commission staff) to meet the Commission mandated schedule and all deliverable requirements for each sector.

0-2 3-4 5-6 7-8 9-10

1.2.8 All Work Plans identify the team member(s) responsible for each activity and any associated needs from or impacts on CPUC staff and/or other Contractor(s). Tasks are appropriately staffed at proper levels of authority and proper number of personnel.

0-2 3-4 5-6 7-8 9-10

Maximum Points Possible: 75 TOTAL SCORE:

Transition Memo (5 points)

Fail Poor Marginal Satisfactory Comprehensive Score

1.2.9 Proposer sufficiently details a logical and effective approach for working with the existing Prime Contractor and/or its subcontractors to transition existing items. Proposer details strategies to transition all applicable administrative, inputs & assumptions/modeling, and process items to their firm.

0-1 2 3 4 5

Maximum Points Possible: 5 TOTAL SCORE:

Deliverable 06: Updated Capacity Expansion Model and Inputs & Assumptions (85 points)

Fail

Poor

Marginal

1.2.10 Proposer sufficiently details a logical and effective approach for reviewing and updating data used in the capacity expansion model. This includes describing how to reconcile data input between capacity expansion

0-14 15-17 18-20 21-22 23-25 and production cost models used in IRP. Also describes approach for vetting external inputs to capacity expansion model, including data from CEC, CAISO, and CARB.

1.2.11 Proposer sufficiently details a logical and effective approach for improving the capacity expansion model used for IRP, this includes both a plan for improving functionality, including for analyzing local capacity areas, as well as a plan for potentially moving towards a new model should one be needed.

Bidder should indicate what level of model ownership the bidder has.

Bidders cannot achieve a maximum score in the evaluation process without providing their own capacity expansion model.

0-9 10-13 14-17 18-21 22-25

1.2.12 Proposer sufficiently details a logical and effective approach for developing an Inputs and Assumptions Package to accompany the capacity expansion model used for the IRP proceeding.

0-9 10-13 14-17 18-21 22-25

1.2.13 Proposer sufficiently details training materials for technical IRP concepts. Description of training materials should cover, at least, concepts in reliability planning, technical concepts in IRP modeling, and how IRP modeling inputs are developed.

0-2 3-4 5-6 7-8 9-10

Maximum Points Possible: 85 TOTAL SCORE:

Deliverable 07: Resource Potential, Land-Use, Transmission, and Busbar Mapping (100 points)

1.2.14 The proposer sufficiently details an innovative analytical approach for developing a Resource Potential assessment, including the potential methodologies, questions that would be analyzed, and how the outputs produced could be utilized in Transmission Planning Process and/or Preferred System Plan modeling.

0-4 5-8 9-12 13-16 17-20

1.2.15 The proposer sufficiently details an innovative analytical approach for developing a Resource Cost assessment, including the potential methodologies, questions that would be analyzed, and how the outputs produced could be utilized in the development of a Preferred System Plan and/or Transmission Planning Process portfolio.

0-4 5-8 9-12 13-16 17-20

1.2.16 The proposer sufficiently details an innovative analytical approach for developing an Environmental Impacts and Land Use Assessment, including the potential methodologies, questions that would be analyzed, and how the outputs produced could be utilized in the development of a Preferred System Plan and/or Transmission Planning Process portfolio.

1.2.17 The proposer sufficiently details an innovative analytical approval for developing an Available Transmission and potential transmission upgrades assessment, including the potential methodologies and questions that would be analyzed, how the outputs produced could be utilized in development of a Preferred System Plan and/or Transmission Planning Process portfolio.

The proposer should also describe how it would support ongoing staff work already underway.

0-4 5-8 9-12 13-16 17-20

1.2.18 The proposer sufficiently details an innovative analytical approach for developing a Busbar Mapping assessment, including the potential methodologies, questions that would be analyzed, and how the outputs produced could be utilized in the development of a Preferred System Plan and/or Transmission Planning Process portfolio.

0-4 5-8 9-12 13-16 17-20

Maximum Points Possible: 100 TOTAL SCORE:

Deliverable 08: IRP Scenarios and Other Analysis Design Frameworks (25 Points) Fail Poor Marginal Satisfactory Comprehensive Score

1.2.19 The proposer sufficiently details the activities for developing scenarios for a given IRP cycle and for other out-of-cycle analysis, and lays out how they would approach working with the CPUC to explore a mix of scenarios. Proposers should include:

a) Clear policy, economic, or regulatory question(s) to be studied using scenario and sensitivity analysis;

b) A proposed method for studying each question, including the model functionality that may be required for specific scenarios;

c) A description of potential data sources;

d) A description of how the various scenarios may be coded into the capacity expansion model so that they are accessible to parties; and A description of how results may be presented to various audiences.

0-5 6-10 11-15 16-20 21-25

Maximum Points Possible: 25 TOTAL SCORE:

Deliverable 09: State and Commission Priority Analyses (50 Points)

1.2.20 The proposer sufficiently details an approach for analyzing and characterizing reliability within the IRP process. At minimum, the description details how to analyze current system reliability, translating IRP reliability modeling with the Commission’s RA program, and comparing different State reliability analyses.

1.2.21 The proposer sufficiently details an innovative approach to analyzing emerging technologies, long lead-time and other diverse resources. The description includes details of how analytical futures of different procurement futures would be conducted and how the results would be used to inform scenarios modeled in the capacity expansion model.

0-2 3-4 5-6 7-8 9-10

1.2.22 The proposer sufficiently details an innovative approach to analyzing reliability and existing thermal generation. The description includes details of how changes to the gas system would impact planning, would consider local reliability requirements and should consider local resources and transmission alternatives.

0-2 3-4 5-6 7-8 9-10

1.2.23 The proposer sufficiently details how each of analyses listed in the “Other State and Commission Policy” would be satisfied using an innovative analytical approach. The proposer also demonstrates knowledge and understanding of Commission-wide proceedings and how to analyze and incorporate these in the realm of IRP.

0-2 3-4 5-6 7-8 9-10

1.2.24 The proposer sufficiently details an innovative approach to analyzing electrification and distributed energy resources. The description includes details of how distribution-level costs and benefits of DERs and electrification can affect reliability and IRP modeling.

0-1 2 3 4 5

1.2.25 The proposer sufficiently details an innovative approach to analyzing energy and storage trends. The description includes details of how trends in energy and storage may be analyzed and how such assumptions may be incorporated into the IRP capacity expansion model.

0-1 2 3 4 5

Maximum Points Possible: 50 TOTAL SCORE:

Deliverable 10: Technical, Policy, and Program Research and Evaluations (40 Points)

Fail Poor Marginal Satisfactory

1.2.26 Cost and Affordability: The proposer sufficiently details how each of the six “Cost and Affordability” requirements would be satisfied.

0-2 3-4 5-6 7-8 9-10

1.2.27 Air Quality. SCT, and ACC Analyses: The proposer sufficiently details how each of the requirements for these analyses would be satisfied.

Proposer demonstrates sufficient understanding of active Commission proceedings on these topics.

0-2 3-4 5-6 7-8 9-10

1.2.28 IRP Program Evaluation: The proposer sufficiently details an approach for the evaluation of the IRP program design and implementation.

Including what method would be used to answer the following questions:

1. How does the IRP program effect change on cost, affordability, GHG reductions, reliability, air quality, disadvantaged

0-1 2 3 4 5 communities?

2. How is the IRP program performing in terms of transparency, stakeholder participation, and ease of administration?

1.2.29 Non-optimized, Passthrough, or Non-modeled costs: The proposer sufficiently details the methodology for considering costs not included in the capacity expansion modeling.

0-2 3-4 5-6 7-8 9-10

1.2.30 Integrated system planning review + gap analysis: The proposer sufficiently details its approach for integrated system planning. Plan should include how Proposer plans to compare state practices to other locations.

0-1 2 3 4 5

Maximum Points Possible: 40 TOTAL SCORE:

Deliverable 11: Capacity Expansion Model Runs, Results Reports, and Preferred System Plan, Transmission Planning Process Modeling Support (90 Points)

Fail Poor Marginal Satisfactory Comprehensive Score

1.2.31 The proposer details its approach to conducting capacity model expansion model runs for proposed portfolios and for supporting sensitivity runs. Description includes how results will be displayed.

0-14 15-18 19-22 23-26 27-30

1.2.32 The proposer details an effective approach for aggregating data from multiple sources to recreate a system plan in preparation for production cost modeling to assess operational and reliability performance, GHG emissions, curtailment, and other relevant metrics, according to the tasks outlined in the deliverable.

0-14 15-18 19-22 23-26 27-30

1.2.33 The proposer details an effective approach for implementing and executing the Reliable and Clean Power Procurement program, including determining and allocating need to load serving entities.

0-14 15-18 19-22 23-26 27-30

Maximum Points Possible: 90 TOTAL SCORE:

Deliverable 12: Supporting Transmission and Interconnection Planning (36 Points) Fail Poor Marginal Satisfactory Comprehensive Score

1.2.34 The proposer details an effective approach to reviewing interconnection studies performed by participating transmission owners and the California Independent System Operator to determine if the inputs, assumptions, and methodologies used to conduct the studies are reasonable, accurate, and conducive to safe and reliable interconnection, and cost-effective utilization of the transmission system, and whether any improvements can be made

0-2 3 4 5 6

1.2.35 The proposer details an effective approach to determining the extent to which non-energy impacts should be incorporated into the CPUC’s IRP process, and the best qualitative and quantitative approaches to doing so

0-2 3 4 5 6

(e.g., a societal cost test based on modifications to the total resource cost test, etc.).

1.2.36 The proposer details how it will research opportunities for the CPUC’s IRP process to further engage in interregional transmission planning.

0-2 3 4 5 6

1.2.37 The proposer explains the impact of FERC Order No. 1920 on transmission planning in California and on the CPUC’s IRP process.

0-2 3 4 5 6

1.2.38 The proposer details an effective approach to evaluating whether and how Grid Enhancing Technologies (GETs) and Advanced Transmission Technologies (ATTs) (e.g., advanced conductors, power flow controllers, dynamic line ratings, storage as a transmission asset, etc.) should be incorporated into the CPUC’s IRP process.

0-2 3 4 5 6

1.2.39 The proposer explains how it will research the benefits and costs of using Power Line Systems – Computer Aided Design and Drafting (PLS- CADD), or other relevant PLS engineering software, for transmission analysis in the CPUC’s IRP process. The proposer indicates whether it is able to provide direct services using the software, train staff to use the software, or if it is unable to do either.

0-2 3 4 5 6

Maximum Points Possible: 36 TOTAL SCORE:

Deliverable 13: Proposer-Defined Studies (10 Points) Fail Poor Marginal Satisfactory Comprehensive Score

1.2.40 Proper details up to three studies it recommends conducting in the IRP process. If the studies are different than the ones listed in the RFP, proposer addresses why it is not submitting responses to those study topics.

0-2 3-4 5-6 7-8 9-10

Maximum Points Possible: 10 TOTAL SCORE:

Deliverable 14: Support Integrated Resource Planning Policies, Practices & Regulatory Oversight (10 Points)

Fail Poor Marginal Satisfactory Comprehensive

1.2.41 Proposer defines how it will support the CPUC in accounting for proposed legislation, regulations, and other State agency programs.

Proposer demonstrates clear understanding of State regulatory and legislative ecosystem.

0-2 3-4 5-6 7-8 9-10

Maximum Points Possible: 10 TOTAL SCORE:

Deliverable 15: Analytical and technical support for developing and/or refining RCPPP and IRP Procurement Program implementation tools and programmatic process improvement (36 Points)

Fail Poor Marginal Satisfactory Comprehensive Score

1.2.42 The Proposer sufficiently details innovative methods, tools, and 0-1 2-3 4-5 6-7 8-9 approaches to assessing, developing, and refining RCPPP and IRP Procurement Program compliance tools and guides. These tools will assist with the analysis and evaluation of LSEs’ procurement showings in accordance with the implementation of RCPPP and IRP procurement program rules.

1.2.43 The proposer must detail the methods and plans for development of a compliance filing portal and external facing user interface that LSEs may use in submission of their RCPPP and IRP Procurement Program compliance filings. The Proposer details an innovative plan to for maintenance of compliance portal and how best to minimize these costs.

0-1 2-3 4-5 6-7 8-9

1.2.44 Proposer must demonstrate their approach and experience in developing similar tools in addition to understanding of compliance with RCPPP and IRP Procurement Program requirements.

0-1 2-3 4-5 6-7 8-9

1.2.45 The Proposer outlines the resources, including staff, needed to perform the task. Support development of RCPPP and IRP Procurement Program implementation tools and materials as identified.

1. This includes streamlining existing process and implementing existing and new rules in the most efficient manner;

2. Evaluate RCPPP and IRP Procurement Program effectiveness as needed;

3. Develop workplans as needed; and

4. Development of Compliance filing portal and external facing application

0-1 2-3 4-5 6-7 8-9

Maximum Points Possible: 36 TOTAL SCORE:

Deliverable 17: Contract Closeout (10 Points) Fail Poor Marginal Satisfactory Comprehensive Score

1.2.46 Proposer outlines a logical strategy for maintaining project records and findings over the course of the contract and for a period of three years following the end of the contract.

0-2 3-4 5-6 7-8 9-10

Maximum Points Possible: 10 TOTAL SCORE:

Maximum Points Possible for Deliverables Approach and Work Plans: 572

TOTAL SCORE: