IMRFR Internal Scoring and Procedure doc 4.7.16.docx

4/7/16 Internal Scoring and Procedure Document Vehicle Inspection and Maintenance Program RFR COMMBUYS Bid#: BD-16-1030-0R100-0R001-00000006553 (for the procurement file)

Introduction This document outlines the procedures that the Massachusetts Department of Environmental Protection (MassDEP) and the Massachusetts Department of Transportation’s Registry of Motor Vehicles Division (MassDOT/RMV) (the Agencies) will follow in the evaluation of responses and the selection of a Contractor for the Vehicle Inspection and Maintenance Program (COMMBUYS Bid#: BD-16-1030-0R100-0R001-00000006553) (the Program). The objective of these procedures is to identify the Bidder most capable of providing high quality professional services and equipment within a competitive cost range. The Strategic Sourcing Team (SST) will rank the responses based on a comparison of the responses using the Evaluation Criteria established in Section 3 of this document.

This guidance consists of the following four sections:

Section 1. Strategic Sourcing Team identifies the staff from the Agencies that will participate on the SST.

Section 2. Evaluation Process describes the steps that the SST will follow to screen and assess the Responses.

Section 3. Evaluation Criteria provides guidance to the SST for its discussions in assessing and rating the Responses. (Criteria 1 through 6, including Subcriteria)

Section 4. Quality Standards are the categories that the SST will use to describe the quality of the Responses (Unacceptable, Acceptable, Good, and Outstanding).

Section 1 - Strategic Sourcing Team The SST will include both voting and non-voting members from the Agencies. The purpose of the SST’s evaluation is to select an apparent successful Bidder who will provide best value to the Commonwealth, after completion of the review of all Bidders’ technical and cost responses.

In order to facilitate its review of the Bidders’ RFR responses, the SST reserves the right to seek technical and administrative assistance from the Agencies’ staff, and may also elect to utilize the Agencies’ contractors if appropriate and if the SST deems this support is necessary to facilitate its review of this procurement.

Voting Members of the SST The voting members of the SST will evaluate the findings of the initial screening panel, review the responses, discuss the analyses of the subgroups, rate the quality of each response through the use of a scoring process, reserve the right to conduct interviews of any or none of the Bidders, and select a Bidder that will provide the best value to the Commonwealth. The voting members of the SST are expected to be:

Sharon Weber of MassDEP and TBD of RMV will serve as executive secretaries to the SST. They will be responsible for convening meetings and recording the ratings of the SST for each Bidder. They will also be responsible for documenting the SST’s findings and preparation of its final report and selection.

Nonvoting Members of the SST To assist the SST in its evaluation of Bidders’ RFR responses, nonvoting members of the SST or their designees will provide analysis to the voting members of the SST regarding the merits of certain aspects of the responses.

If necessary to facilitate the review of responses, the non-voting members of the SST will be supported with technical, business, financial, and cost analyses conducted by the Agencies’ staff and may also be supported by contractors working for the Agencies.

The voting members of the SST may call upon the non-voting members of the SST to answer questions and provide clarifications. The non-voting members of the SST may, at the request of voting members of the SST, provide written recommendations to the voting members of the SST regarding any aspect of the Bidders’ RFR responses. The nonvoting members of the SST and the SST Subgroups are expected to be as follows:

(MassDEP) MassDOT: Terry Hayes, Mary-Jo Griffin, Mike Devaney MassDEP: Craig Woleader

(MassDOT) MassDEP: Craig Woleader, Marc Bennett MassDOT: Paul LaValle, Mike Devaney, Mary-Jo Griffin

(MassDEP) MassDOT: Terry Hayes, Mike Devaney, Mary-Jo Griffin MassDEP: Julie Ross, Marc Bennett

(MassDEP) MassDOT (for all four categories): Terry Hayes, Mike Devaney, Mary-Jo Griffin

Motorist Assistance Centers:

MassDEP: Paul Davis, John Flemmi

Vehicle Maintenance Initiative:

MassDEP: Paul Davis

Communications:

MassDEP: Sarah Weinstein and/or Tom Higgins

Inspector Training:

MassDEP: Julie Ross, John Flemmi

(MassDEP) MassDOT: Theresa Moran MassDEP: Tim Cahill

(MassDOT) MassDEP: Yitling Slayman

Screening Panel The members of the screening panel are expected to be:

MassDEP - Marion Marcucelli RMV – Terry Hayes

Section 2 – Evaluation Process This section describes the evaluation process from receipt of RFR Responses to the negotiations with an apparent successful Bidder. The Agencies reserve the right to reject any or all responses if they determine that it is in the best interest of the Commonwealth to do so.

Opening the Responses As explained in RFR Section 3.2.2.1, Responses will be opened after the closing of the IMRFR to determine if there are any attachments to the Bidder’s Commitment Letters that identify any current or former state employees of the Agencies who participated or will participate in the preparation of a Bidder’s Response. These attachments will then be posted on COMMBUYS.

The original hardcopy RFR Responses, including the Bid Bond, shall be kept by MassDOT/RMV.

STEP 1. Initial Screening Selected SST members from the Agencies, which may include non-voting SST members, will screen all RFR Responses upon receipt, using the Bidders’ Response Checklist found in Attachment K, to determine if the Bidders’ RFR Responses are complete. The Response Checklist is the form that the SST will use to help determine whether Bidders’ RFR Responses provide the information requested in the RFR in the format prescribed.

The initial screening of the RFR Responses will consist of three parts: (1) an initial screening to determine whether the Response meets the packaging, format, and page limit requirements in the RFR; (2) a detailed screening to determine if the Response contains all of the RFR Response Elements required in the RFR; and (3) a review to determine whether minimum Specifications and Bidder qualifications set forth in the RFR have been met.

STEP 2. Clarifications/Corrections to Submitted Responses The voting members of the SST will review the results of the Initial Screening.

As stated in Section 4.2.2.1 of the RFR, the Agencies expect that Bidders will submit responses that are complete and in substantial conformance with the RFR. In its discretion, the voting members of the SST will determine whether to allow a correction of minor informalities in a response. Minor informalities are matters of form rather than substance and include clerical errors or minimal or insignificant mistakes that can be corrected without prejudice to other Bidders. The voting members of the SST may, upon request of a Bidder, allow a correction of a minor informality in a response, which is clearly evident, such as a typographical error, transposition error or arithmetical error where the correct answer is obvious. If the mistake is discovered by the SST, the SST may note the correction on the response.

If the SST requires a clarification of any particular section of a response, the SST will provide all Bidders with the same notice and opportunity for clarification of the identified section of the response. Clarifications are explanations of what is stated in a response and may not be used as an opportunity to submit supplemental information or a change in a response. No clarification of response prices, terms and conditions or the submission of supplemental information prejudicial to the interests of other Bidders or to fair competition shall be permitted.

If the voting members of the SST determine that the nature of a mistake, deficiency, or omission in a response is such that it cannot be corrected without prejudice to other Bidders, the voting members of the SST will determine whether to evaluate the response as originally submitted, or disqualify the response. A response may be disqualified if, in the judgment of the voting members of the SST, the response: (1) fails to meet any of the requirements of the RFR; (2) is incomplete, conditional or obscure; (3) contains deletions, erasures, alterations, or other irregularities; (4) contains errors; or (5) violates any of the provisions of 801 CMR 21.06(10).

The SST will document the basis for its decision to disqualify a response. The SST will notify the Bidder in writing of the decision to disqualify the response and the reasons therefore at the end of the evaluation process, after Contract award.

STEP 3. Preliminary Evaluation The SST will distribute the non-disqualified RFR Responses to the appropriate members of the SST, including as applicable, designated non-voting members, for preliminary evaluation. The purpose of the preliminary evaluation is for the SST to review the Bidders’ responses and rate the responses, on a preliminary basis, based upon the Bidder’s responsiveness to the evaluation criteria requirements as set forth in this Step 3. The SST will first review and score, on a preliminary basis, the Bidders’ Technical responses, and then review and score, on a preliminary basis, the Bidders’ Cost responses.

As part of the preliminary evaluation process, the SST may verify information contained in a response, including but not limited to, each Bidder’s qualifications and stated experience. The SST may direct a member of the SST to consult with personnel from the Agencies, or other jurisdictions familiar with the work of any or all of the Bidders. After obtaining the pertinent information, the SST member will report the results to the voting members of the SST.

The voting members of the SST will evaluate the strengths and weaknesses of complete Bidder RFR Responses using the Evaluation Criteria from RFR Section 4, listed here in descending order of importance, with percentage values, and further elaborated upon below in Section 3.

Evaluation Criteria:

Criteria 1: Overall Management Capability, Accountability, and Experience with similar Programs – 25% Criteria 2: Data Management – 25% Criteria 3: Provide Equipment, Equipment Maintenance, and Inspection Station Service – 15% Criteria 4: Program Support Elements (including Motorist Assistance Centers, Vehicle Maintenance Initiative, Communications, and Inspector Training) – 15% Criteria 5: Cost and Financial Plan – 10% Criteria 6: Supplier Diversity Plan – 10%

STEP 4. Project Proposal Presentations and Interviews After completion of the preliminary evaluation, the SST, in its sole discretion and at its option, may require any or none of the Bidders to conduct presentations of their proposal and interviews of key project personnel. The purpose of the presentation/interview is to enable the SST to better understand and evaluate the Bidder’s RFR Response, the Bidder’s proposed teams, and the Bidder’s ability to be successful.

The SST will determine the structure and content of the presentations/interviews and will contact the Bidder(s) individually, with at least 1 week’s notice, to schedule the session, which will take place during the timeframe specified in the procurement calendar. Bidders who are asked to participate in the presentations/interviews will be expected to prioritize this in their schedules. The SST will make every effort to find a mutually convenient time for the Bidder and the SST. However, failure to appear at the scheduled time of the presentation/interview may result in disqualification, lower evaluation, or other actions that the SST deems appropriate.

The Bidder’s key personnel identified in its RFR response are expected to attend the session. The SST will not inform a Bidder of its scores or ranking at the time of the session. This material and all written submissions become part of the Procurement File.

Presentations may be videotaped by the Agencies at their expense and thereby become public records and part of the Bidder’s RFR Response after the conclusion of the RFR process. All other expenses associated with the presentations and/or demonstrations will be borne by the Bidder.

STEP 5. Final Evaluation Once the project presentations and interviews have been conducted, the voting members of the SST will develop their final evaluation for each RFR Response.

The SST will fully evaluate the Technical and Cost Responses, determine and provide a final score for each Bidder’s RFR Response, and select an apparent successful Bidder.

At the option of the voting members of the SST, non-voting SST members may be requested to participate in the ratings discussions.

STEP 6. SST Selection of Apparent Successful Bidder After completion of the final evaluation and scoring for all Bidders, the SST will select the apparent successful Bidder for recommendation to the Agencies’ designated authorized contract representatives, and will write a memorandum summarizing the Bidder selection process and the reasons for selection of the apparent successful Bidder. After the Agencies have selected an apparent successful Bidder, the SST will post the name of the apparent successful Bidder on COMMBUYS, and will notify all Bidders in writing of the results of the RFR procurement process.

After announcement of the apparent successful Bidder, the Agencies and the apparent successful Bidder will enter into negotiations in order to finalize the scope of services, and enter into a Contract Award.

Section 3 – Evaluation Criteria The following series of criteria and sub-criteria have been designed as guidance for the SST members to help them deliberate, and make professional judgments about each response. The voting members of the SST will use these criteria and sub-criteria to produce a sound, well-reasoned, and well-justified rating of responses.

Criteria 1: Overall Management Capability, Accountability, and Experience with Inspection and Maintenance Programs – 25%

A. The Bidder’s ability to manage and implement the Program, including providing key personnel and staffing plans and demonstrating how its key personnel and subcontractors will be responsible and accountable to the Agencies on critical performance criteria.

B. The Bidder’s ability to develop and implement management and tracking systems to ensure contract performance.

C. The Bidder’s depth and breadth of experience in developing and implementing IM programs (including the experience of any proposed subcontractors).

D. The Bidder’s ability to develop and have testing equipment, Workstations, data systems, and Motorist Assistance Centers operational and on schedule to meet the October 1, 2017 startup date.

E. The Bidder’s ability and capacity to be flexible and responsive to potential Program changes during the length of the Contract.

F. The overall quality of the response, including how well the Bidder understands the challenges of managing the Program and how well the Bidder incorporates a systematic approach to quality assurance and quality control, as well as continuous improvement throughout the contract.

Criteria 2: Data Management – 25%

A. The Bidder’s ability to develop and maintain reliable data systems that properly, reliably, and quickly upload data for the Agencies to use.

B. The Bidder’s ability to revise data systems as needs change, such as MassDOT/RMV’s modernization of its ALARS system.

C. The Bidder’s ability to address improper and fraudulent inspections and vehicle tampering, and the Bidder’s ability to provide the Agencies with recommendations on how to revise inspection procedures in software coding to improve the detection and management of fraudulent practices and tampered vehicles.

Criteria 3: Provide Equipment, Equipment Maintenance, and Inspection Station Service – 15%

A. The Bidder’s ability to provide reliable testing equipment and Workstations meeting Contract specifications to inspection stations at Program start-up.

B. The Bidder’s ability to provide service to the inspection stations in a reliable and timely manner throughout the life of the Contract.

Criteria 4: Program Support Elements - 15%

A. The Bidder’s ability to provide services through a convenient network of Motorist Assistance Centers, including provisions for mobile assistance where necessary for service delivery.

B. The Bidder’s ability to promote proper emissions repairs through the vehicle maintenance initiative including training and recruiting registered repairers and registered repair facilities.

C. The Bidder’s ability to effectively communicate with motorists, the inspection industry, and the repair industry with a particular emphasis on communicating what motorists need to do when they fail the emissions and/or safety test.

D. The Bidder’s ability to provide and manage an Inspector Training program, including training necessary for continued federal certification of the Massachusetts Commercial Vehicle Inspection.

Criteria 5: Cost and Financial Plan – 10%

A. Is the total cost of Bidder’s proposal (Base Bid) reasonable?

B. Are the Bidder’s Workstation costs, monthly fees, and Inspector Training costs to be charged to Inspection Stations reasonable?

C. Will the bid reasonably cover the costs to implement the Program? Are the costs reasonably distributed for Program elements?

D. Was sufficient information provided to determine potential cost impacts if (1) Bidder is sole supplier of Workstations or (2) other vendors are also allowed to supply and maintain workstations?

E. Does the response provide sufficient information to verify the financial capability of the Bidder? Does the Bidder meet the threshold for financial capability in the RFR?

Criteria 6: Supplier Diversity Plan – 10%

A. The Bidder has properly committed to meeting contract requirements for supplier diversity on SDP Plan Form #1.

Note: VISITS and mobile printing solutions are outside of the Criteria above. These items were not included as part of the Base Bid.

Section 4 - Quality Standards The voting members of the SST must first seek to reach consensus on the rating for each response for a criterion. If the voting members of the SST cannot reach a consensus, the rating will be the score that a majority of the voting members of the SST agree upon. The SST’s report will explain the basis for the rating that the voting members of the SST gave to the criterion. Nonvoting members of the SST may participate in the ratings discussions.

Rating the Technical Responses For the purposes of rating the Technical Responses (Criteria 1 through 4), the SST will apply the quality standards "unacceptable," "acceptable," "good," and "outstanding" using the following general definitions.

"Unacceptable" A Response will be rated "unacceptable" for a given criterion when the voting members of the SST determine that: (1) there is no substantial likelihood that the Bidder will be able to carry out the contract obligations described in the RFR in a responsible, professional, or technically competent manner, (2) the weaknesses in the Response clearly outweigh the strengths of the Response or (3) the Bidder does not satisfy the fundamental requirements for the RFR that relate to the criterion.

"Acceptable" A Response will be rated "acceptable" for a given criterion when the voting members of the SST determine that: (1) the Bidder is likely to perform the work under the contract in a professional and technically competent manner, (2) the strengths of the Response outweigh or equal the weaknesses of the Response and (3) the Bidder satisfies the fundamental requirements related to the criterion in question.

"Good" A Response will be rated "good" for a given criterion when the voting members of the SST determine that the Response satisfies the guidelines for an "acceptable" Response and demonstrates a level of experience and expertise such that there is a substantial likelihood that the Bidder will perform the work under the contract in a highly professional and technically competent manner, and either: (1) the strengths of the Response clearly outweigh the weaknesses of the Response, or (2) the Bidder exceeds the fundamental requirements related to the criterion in question.

"Outstanding" A Response will be rated "outstanding" for a given criterion when the voting members of the SST determine that the Response satisfies the guidelines for a "good" response and that there is a substantial likelihood that the Bidder will perform work under the contract in a professionally and technically excellent manner and either: (1) the strengths of the Response greatly outweigh the weaknesses of the Response, or (2) the Bidder meets the highest standards for requirements related to the criterion in question.

During deliberations, the voting members of the SST will rate each Technical Response by assigning points on a scale of 0 to 10 points for Criteria 1-4. The point scores will be weighted as indicated in the RFR Scoring Sheet and will correspond to the quality standards below. These standards are described above.