IT Professional Services Master Price and Services Agreement WORK ORDER CONTRACT EXHIBIT 3 STATEMENT OF WORK FOR iQMS

Table of Contents

Appendix A – Generic Software Project Quality Standard.............................................. A-2 Appendix B – Project Evaluation Template ..................................................................... A-13 Appendix C – QA Status and Improvement Report Template2 ........................................ A-23 Appendix D – QA Status and Improvement “Sample” Report ........................................ A-28 Appendix E – EIS/OSCIO Project Assessment Report and Project Budget & Schedule Variance Report .......................................................................................................................................... A-34

1 The OSCIO developed the Oregon Quality Standards from review and analysis of the Project Management Institute’s Project Management Body of Knowledge (PMBOK) in the context of the International Standards Organization (ISO) standard ISO12207 for system life cycle management and traditional software development projects. Depending upon the nature of an IT project’s underlying solution, these Oregon Quality Standards may be modified, omitted or supplemented with additional, applicable quality standards.

2 Appendices C through E provide guidance for QMS Task 4, Deliverable 4.1, Quality Assurance Status and Improvements Report. Authorized Purchaser will work with Consultant to finalize agreeable format(s).

1. Introduction to Quality Management Services

1.1 Purpose

The purpose of the QMS Consultant’s work is to assure that appropriate levels of Quality Management Services are performed for the Commercial Vehicle Registration System Replacement (“Project”). These Services must provide the Authorized Purchaser with appropriate visibility into the processes being used, and the products being built. They must be sufficient to assure that the Project satisfies the needs for which it was undertaken, and that Project risks are well understood and appropriately mitigated or managed.

1.2 Quality Management Approach

The approach to ensure that the appropriate quality management and risk management activities are conducted must be based, as necessary and appropriate in the context of a particular project, on the Project Management Institute’s (PMI) Standard described in the current edition of the Project Management Body of Knowledge (“PMBOK” https://www.pmi.org/pmbok-guide-standards); and the International Standards Organization standard ISO12207 for system life cycle management. This includes Quality Management Services that determine the quality policy, objectives, and responsibilities; and implements them by means such as quality planning, quality control, and quality assurance.

Quality Management is a subset of project management that includes the process required to ensure that the Project satisfies the needs for which it was undertaken. Quality Management consists of activities in quality planning, quality assurance, quality control (including independent testing) and risk management. Together, these activities constitute the Authorized Purchaser’s independent verification and validation (IV&V) requirements. They are independent in the sense that they are conducted independently of the development Consultant or systems integrator.

The term “Quality Standards” refers to a project’s “process” and “product” quality standards. “Process” quality standards cover organizational influences, management support, decision drivers, project management, scheduling, resourcing, experience, and so on. “Product” quality standards cover product content, design, development, deployment, environment, technology, security, maintainability, and so on.

The quality management approach for the Project must adhere to and include, but not be limited to, work activities in support of the following Quality Management Services that constitute the Authorized Purchaser’s IV&V requirements:

1. Task 1 - Risk Assessment

2. Task 2 - Quality Management Planning

3. Task 3 - Quality Control

4. Task 4 - Quality Assurance

5. Task 5 - Independent Solution Testing 6.

Work Order Contract Exhibit 1_Statement of Work

Together, the satisfactory performance of these Tasks by the QMS Consultant helps ensure the application of best practices in project management, including quality management, and the delivery of technical work products that meet or exceed project requirements for schedule, cost, functionality, reliability, security, and other relevant quality standards.

1.3 Definitions

“Independence” or “Independent”: The delivery of services by a third party free from real or perceived conflict with the interests of the Authorized Purchaser and its contractors involved in project planning, design, development, implementation, deployment and operational activity, and as otherwise defined in Statewide Policy 107-004-030.

“Independent QA Act”: Chapter 77, A 2014 ACT HB 4122, codified at ORS 291.035.

“Independent QA Program”: The program that uses the group of pre-qualified IT Quality Management Services providers under statewide Master Price and Services Agreements for delivery of diverse, Independent Quality Management Services. DAS and the OSCIO cooperatively administer the program. Each Independent QA Program consultant has executed a Master Price and Services Agreement with the State that includes negotiated provisions, such as standard contract terms and conditions, established hourly rate cards, and the Independent QA Program enterprise Scope of Services.

“Independent Verification and Validation” or “IV&V”: Interchangeable terms that have the meaning set out in the latest edition of ISO 12207. “Verification” relates to evaluation that establishes the compliance of project work products with approved business requirements, technical requirements, architecture, and design. “Validation” relates to determining that project work products fulfill intended business needs. “Independent” has the meaning set out above in connection with the defined term, “Independence” or “Independent”.

“Information Technology Initiative” or “IT Initiative”: A project to develop or provide, with an Authorized Purchaser’s own personnel and resources, or to obtain by means of a procurement or set of related procurements: (i) New hardware, software or services for data processing, office automation or telecommunications; (ii) An overhaul, upgrade or replacement of a substantial portion of the hardware or software in an existing data processing, office automation or telecommunications system; or (iii) A substantial expansion of existing data processing, office automation or telecommunications services.3

3 Information Technology Initiative does not include: (i) A procurement for preliminary quality assurance services or quality management services; (ii) A routine update to or purchase of hardware or software within an existing data processing, office automation or telecommunications system; (iii) A renewal of an existing contract for data processing, office automation or telecommunications services under terms and conditions that are substantially the same as in the existing contract; or (iv) A replacement of a component of an existing data processing, office automation or telecommunications system that is not essential for the system to function as designed or that occurs at the end of the component’s anticipated life cycle.

“Preliminary Quality Assurance Services” means a set of services in which a Consultant provides an independent and objective review of an Authorized Purchaser’s plans, specifications, estimates, documentation, available resources and overall purpose for an information technology initiative, including services in which the Consultant evaluates a proposed information technology initiative against applicable quality standards and best practices from private industry and other sources.

“Quality Management Services” or “QMS” means a set of services in which a Consultant provides an independent and objective review and evaluation of a state contracting agency’s, a public corporation’s or another Consultant’s performance with respect to an information technology initiative, such as services in which the Consultant: (A) Identifies quality standards that apply or should apply to the information technology initiative; (B) Suggests methods and means by which the state contracting agency, the public corporation or the other Consultant may meet quality standards identified in subparagraph (A) of this paragraph; (C) Reviews and evaluates the state contracting agency’s, the public corporation’s or the other Consultant’s performance regularly as the information technology initiative progresses from start to finish;

(D) Identifies omissions or gaps in the state contracting agency’s, the public corporation’s or the other Consultant’s planning, execution, control, methodology, communication or reporting as the information technology initiative progresses from start to finish; (E) Identifies risks in the state contracting agency’s, the public corporation’s or the other Consultant’s plans or approach to designing, developing or implementing the information technology initiative and suggests methods to reduce, mitigate or eliminate the risks; (F) Assists the state contracting agency or the public corporation in testing or otherwise evaluating the hardware, software or services that are developed, provided or obtained as part of an information technology initiative to determine whether the hardware, software or services conform with the quality standards identified in subparagraph (A) of this paragraph; (G) Advises the State Chief Information Officer, the state contracting agency or the public corporation as to whether the hardware, software or services that are developed, provided or obtained as part of an information technology initiative meet the contracting agency’s or the public corporation’s needs, specifications or expectations and otherwise enable the state contracting agency or the public corporation to achieve the objectives for the information technology initiative; or (H) Identifies unsatisfactory performance and suggests methods the State Chief Information Officer, the state contracting agency, the public corporation or the other Consultant might use to eliminate the causes of unsatisfactory performance.

2. Quality Management Services (“QMS”)

2.1 Task 1: Risk Assessment

The Risk Assessment Task defines how the QMS Consultant will support a project’s overall risk management efforts. The QMS Consultant should refer to the project’s Risk Management Plan, if one exists. Authorized Purchaser’s project team retains the primary responsibility for executing the project’s risk management activities, while the QMS Consultant provides a supporting role through its independent efforts. Within the QMS Consultant’s role of completing Task 2 Quality Planning, Task 3 Quality Control (including Task 5 Independent Solution Testing), and Task 4 Quality Assurance; it must identify risks and provide recommendations for risk avoidance and mitigation strategies related to work product, process, and the overall project. Risk management includes the identification of risks, the thorough assessment of the probability and the impact for the occurrence of risks, and the planning of viable responses that include, but are not limited to, mitigation, contingency, and avoidance strategies. Identification of Information Security risks should also be an essential part of the scope of the QMS Consultant’s initial and on-going risk assessment.

The iQMS Consultant performs Task 1 through the following Deliverables:

1. Deliverable 1.1: Risk Assessment The QMS Consultant shall conduct a Risk Assessment of the project to identify the current status of the project, identify risks and their likelihood of occurring, and provide an independent evaluation of the planned schedule, fiscal and personnel resources, and processes. If Consultant conducts a Risk Assessment early in the project lifecycle, Consultant may be requested to refresh the Risk Assessment one or more times throughout the project. At minimum, in developing or refreshing its Risk Assessment Report, the QMS Consultant must evaluate the project’s Integrated Project Plan for reasonableness, validity, thoroughness and accuracy with emphasis on the following:

1. Project Plan including State Resource Plan;

2. Project Requirements Management Plan;

3. Project Change Management Plan;

4. Project Issue Management Procedure;

5. Communication Plan;

6. Other critical Project processes; and

7. Any authorized Task 3 Quality Control Reviews.

As applicable and available, the QMS Consultant should perform a risk assessment on the project’s most recent Software Development Plan, Facility Plan, Configuration Management Plan, Data Conversion Strategy, Testing Strategy, Post-Implementation Support Strategy (for on-going operations and maintenance), Training Strategy, and Certification Checklist. For projects that are not for the purpose of traditional software development, the Risk Assessment should include artifacts that are comparable to those listed in this section within the context of the project’s actual purpose. This Risk Assessment should take into account work product and project-level considerations, including at a minimum:

1. Feasibility of the technical solution;

2. Sufficiency of security controls to safeguard protected or confidential information and to meet security requirements;

3. Sufficiency of project components and processes; and

4. Sufficiency of project budget, schedule and resources.

To satisfactorily produce this Deliverable, the QMS Consultant should be well versed in all aspects of the project, gaining such basic project knowledge through its review of the project’s available:

1. Business Case, Charter;

2. Work Plan and Work Breakdown Structure;

3. Approved (original or current) baseline project budget and schedule;

4. Integrated Project Plan and related supporting plans; and

5. Other major project deliverables produced by the Authorized Purchaser or its other project consultants that relate to the project’s functional, non-functional, and security requirements.

The Deliverable should include:

1. Risk Identification (providing a description, level of impact, probability of occurrence, and measurable threshold to trigger the risk);

2. Risk Avoidance Plan (recommending specific solutions to avoid the triggering of each risk);

3. Risk Mitigation Plan (recommending specific risk mitigation solutions for each identified risk, including a cost-benefit analysis for each mitigation option along with specific recommendations); and

4. For projects under the oversight of the OSCIO, the OSCIO Project Assessment Report (completing a “forward view” column for all metrics identified therein). (See Appendix E.)

2. Deliverable 1.2: Risk Notification

As needed between periodic reports, the QMS Consultant should immediately verbally report to the Authorized Purchaser’s authorized representative the discovery of problems, new risks, or previously known risks that have increased in risk probability or potential impact; and which pose a risk of failure or danger to the success of the project. Within three (3) business days, the QMS Consultant should follow up verbal reports with the submission of a written report.

The written risk notification report, as part of the on-going activities, should follow the same requirements as defined for the initial risk assessment, with an emphasis on managing risks that occur and change during the development and implementation process. The QMS Consultant should conduct a risk assessment while conducting periodic reporting under Task 3 Quality Control and Task 4 Quality Assurance. At a minimum, the risk assessment should:

1. Monitor the solution’s hardware, software, infrastructure, and data security requirements and risks as they apply to the project’s progress to ensure that the solution meets project requirements;

2. Review where applicable the Business Continuity Plan, the Disaster Recovery Plan and audit controls to ensure the security of any Authorized Purchaser protected or confidential information at risk in the project; and

3. Review implementation tasks and activities to ensure that the confidentiality, integrity and availability of the solution are not compromised.

4.

FOR THE TASK 1 DELIVERABLES RELEASED UNDER THE CONTRACT, REFER TO SECTION 3 OF THE STATEMENT OF WORK: DELIVERABLES AND PAYMENT SCHEDULE.

2.2 Task 2: Quality Planning

The QMS Consultant may be required to establish a quality management system for the Project. The typical system is comprised of a quality management plan that includes all requisite quality standards, quality checklists for quality control reviews, quality control and quality assurance report templates, and quality control and quality assurance processes. The established quality management system should define the Project work tasks and resourcing that will add value and reduce risk subject to the constraints of available Project resources and applicable organizational policies, Oregon Revised Statutes, Oregon Administrative Rules, and statewide policies and standards.

The QMS Consultant’s work should adhere to generally accepted industry practices for project management and system life cycle management. In addition, the QMS Consultant should consider the Project’s quality management guide or plan as a reference4. QMS Deliverables, should conform to the Project Management Institute’s (PMI) Standard as described in the current edition of the Project Management Body of Knowledge (PMBOK) and International Standards Organization standard ISO12207 for software development life cycle management.

Through its established quality management system, the QMS Consultant should ensure that the Project plans, standards, processes and work tasks fit the Project’s needs, and verify that they will be usable for performing quality control reviews, inspections, and uncovering quality risks throughout the life cycle of the Project. The emphasis of this task is to confirm that quality is planned into the Project versus a reactive quality approach that measures quality through audits completed after the work is done.

A quality management system usually includes the following Deliverables:

1. Deliverable 2.1: Quality Standards - Operational Definitions This Deliverable sets out the written operational definitions for the identified Quality Standards applicable to the Project products and processes The Deliverable describes in very specific terms, what the standards are, and how each will be measured by the quality control process. The QMS Consultant must consider the Oregon Standards set out in Appendix A in developing Deliverable 2.1. The QMS Consultant must apply to the Project those Oregon Standards denoted with an asterisk. For the other Oregon Standards, the QMS Consultant may apply, supplement, modify, or omit them from application to the Project as appropriate by performing the following services:

4 The Authorized Purchaser is ultimately responsible for the delivery of project work products with good quality; and its integrated project plan should comply with industry standards and include a quality management guide or plan.

a. Identifying from the template the relevant process and product quality standards, risk cues or measurements that are appropriate for the Project based on the Project’s current business and technical complexity assessment.

b. Recommending additional quality standards based on the type of project, current phase of the project, or expert opinion where the absence of a quality standard would present high risk to the project; e.g. information security.

c. Recommending elimination of unnecessary quality standards based on the Project’s business and technical complexity assessment or current phase of the Project.

Appendix A to the SCS, incorporated into this Contract by reference in Section 4 of the Statement of Work, currently sets out 82 Oregon Process and Product Quality Standards that may apply to Major IT Initiatives and other IT projects under OSCIO oversight. Some of these may be modified or omitted depending upon the nature and scope of the project, but the following Appendix A standards are required on Major IT Initiatives and other IT projects under OSCIO oversight:

a. Required Process Quality Standards:

(1) Definition of the project, development schedule, delivery commitment, cost controls, budget and resource size, project management approach (standards #10, 11, 12, 13, 23, 25, 26, 27)

(2) Political influences, organizational stability (standards #6, 37)

(3) Leadership, project management authority, executive involvement (standards #12, 17, 41)

(4) Team productivity, team member availability, user involvement, user justification (standards #28, 35, 44, 48)

b. Required Product Quality Standards:

(1) An appropriate, useful, and maintainable set of requirements (standard #50)

(2) An appropriate, useful, and maintainable set of engineering and design specifications (standard #58)

(3) Appropriate, useful, and maintainable code, utility, object, etc., construction (standard #80)

(4) Appropriate, useful, and maintainable test planning, test execution, and test corrective actions (standard #51)

(5) The product or application is fit for use (standards #46, 76)

(6) Alternatives Analysis, Commitment Process, Lessons Learned (standards #55, 56, 63)

(7) Appropriate, thorough and maintainable implementation (standards #53, 81)

2. Deliverable 2.2: Quality Checklists For each Authorized Purchaser or other consultant deliverable identified for quality control review under Deliverable 3.1, the QMS Consultant shall determine, and document in quality checklists, how to monitor and measure the applicable Quality Standards identified in Deliverable 2.1. These checklists should have the following attributes:

a. Indicate the schedule for reviews, including the deliverable(s) under review, the person(s) responsible for the deliverable(s), and the person(s) responsible for conducting the reviews.

b. Indicate a specific review procedure to follow, e.g. a procedure for schedule analysis, code walk-through, peer review, interviews, lessons learned, test methodology, or if the review will be performed informally.

c. If appropriate, verify and record that a set of required inspections have been performed.

d. Indicate that the minimum quality standard(s) has been met.

e. Record the measurements.

f. Identify the expected risk cue or measurement.

g. Indicate the expected acceptability or tolerance.

h. Indicate the rank of the quality standards where risk was found unacceptable.

i. Indicate change in risk rank since the previous review.

j. Indicate a reference that will describe what was reviewed, who was interviewed, and the information or reasoning that justifies a conclusion that non-adherence to a specific quality standard causes risk.

The process of measurement, setting tolerance, and risk ranking may be based on the QMS Consultant’s subject matter knowledge, domain experience, and expert opinions.

The quality checklists should conform to the specific choice of system life cycle model applicable to the Project, especially concerning detailed requirements definition, general and detailed system design, test design, test planning and execution, and system implementation.

3. Deliverable 2.3: Quality Management Plan (QMP)

The Quality Management Plan (QMP) defines how the QMS Consultant will conduct Quality Assurance (QA), Quality Control (QC), Independent solution or system testing, and Risk Assessment on the Project. The QMS Consultant must adhere to generally accepted industry practices for project quality management and solution development quality management; and should review and use as key input all available Project planning artifacts, including any guidance provided by the Authorized Purchaser in writing or orally; the current version of the PMBOK; and ISO 12207. At a minimum, the QMP should include the following:

a. Identification and definition of QMS Consultant, other consultant, OSCIO and Authorized Purchaser staff roles and responsibilities; and the procedures and resources needed from such personnel to implement the Quality Management Plan.

b. Identification of all the work tasks to be performed by the QMS Consultant and other Project consultants, and the activities required of the Authorized Purchaser and OSCIO needed to facilitate the QMS Consultant’s satisfaction of the specified and relevant process and product quality standards. The following items should be considered for inclusion in the QMP:

c.

(1) A detailed plan for Quality Assurance Review, including scope, criteria, and the methodology to be employed;

(2) A detailed plan for Quality Control, including scope, criteria, methodology, and, if applicable, Independent solution testing;

(3) The quality management tools to be used (may reference applicable, currently approved documents):

i. Quality standards;

ii. Quality checklists;

iii. Templates for all reporting types required during the lifecycle of the Project;

e.g. Quarterly QA Status & Improvement Report, QC Review Reports, Monthly Status Report, Weekly Status Report, On-Going Risk Notification Report, Project Assessment Report, Project Budget and Schedule Variance Report, and so on;

iv. Additional quality tools and methods as needed; and

(4) Risk Assessment Methodology

5. Deliverable 2.4: Baseline QMS Work Plan

a. This Deliverable specifies the QMS Consultant’s manner and means for developing and submitting, for the Authorized Purchaser’s review and acceptance, the required QMS Deliverables. The Consultant must update this plan as needed. The plan should describe the following elements:

(1) The general approach to achieve QMS Contract requirements;

(2) Schedule;

(3) A detailed work breakdown structure, with key milestones, critical path elements, and QMS Deliverables identified;

(4) The estimated resource requirements to complete the required work;

(5) A staffing plan with the Consultant’s key persons identified;

(6) An assessment of all levels of assistance needed from the Authorized Purchaser, its staff, and the other Project consultants, including but not limited to hands-on participation, facilities, and infrastructure; and

(7) A clear link to the overarching Project plan and schedule, including the solution consultant’s plans and schedule.

b. Depending upon the nature and timing of the Consultant’s involvement in the Project, the Baseline QMS Work Plan’s development and completion rely on the QMS Consultant’s conduct, or review, of some, most or all of the following activities:

(1) An initial risk assessment in compliance with Task 1;

(2) For projects under OSCIO oversight, the approved business case on file with the OSCIO;

(3) All available planning artifacts from the Authorized Purchaser, e.g. Integrated Project Plan, supporting plans, and quality management guidance;

(4) Applicable standards, including technology and security standards;

(5)

(6) The Project’s documented requirements, including functional, non-functional, and security requirements;

(7) The solution consultant’s statement of work and related documentation, especially its plan, solution development methodology, data conversion and interface plans, and training plan as applicable.

6. Deliverable 2.5: Special Requests – OPTION RESERVED TO AUTHORIZED PURCHASER

At the Authorized Purchaser’s request, and if not already required in connection with another QMS Deliverable, the Consultant may be required to appear before various committees or individuals to discuss the overall strategic direction and progress of the Project, or complete other “special” work. Special work includes independent analysis, recommendations on unforeseen problems, opportunities for improvement, research and recommendations on alternative courses of action, and additional risk management activities. In the instances requiring internal/external presentations, the QMS Consultant should accompany the Authorized Purchaser’s staff to make presentations, help answer questions, and provide the Consultant’s assessment of confidence on the Project’s ability to satisfy its declared goals and objectives.

In response to a Deliverable 2.5 request, and within the time prescribed in the request, the QMS Consultant shall provide the Authorized Purchaser with a written fixed cost proposal and schedule.

7. Deliverable 2.6: Project Evaluation/Lessons Learned Report

The Authorized Purchaser may require the QMS Consultant to facilitate a “Project Evaluation/Lessons Learned” session(s) at the end of each Project phase or at the close of the Project, and create a report of the findings in the context of Appendix B. Deliverable 2.6 may become a periodic Deliverable or the final Deliverable under the QMS Contract. At a minimum, this Deliverable should reflect review and analysis of the Project’s quarterly Quality Assurance Status and Improvement Reports. Its minimum content should reflect:

a. Progress made;

b. Likelihood of achieving business case and return-on-investment targets;

c. Planned versus accomplished tasks;

d. The risks, problems, and delays encountered in the Project;

e. Mitigation actions taken or not taken, and why;

f. Successes and mistakes made; and

g. Recommended actions.

h.

FOR THE TASK 2 DELIVERABLES RELEASED UNDER THE CONTRACT, REFER TO SECTION 3 OF THE STATEMENT OF WORK: DELIVERABLES AND PAYMENT SCHEDULE.

2.3 Task 3: Quality Control (“QC”)

This task monitors Project results to determine if they comply with stated Project requirements. Project results include both work product results, notably deliverables, and project management results, notably schedule and cost performance. In general, this Task includes the:

a. Review of Project activities and inspection of Authorized Purchaser and solutions consultant work product deliverables throughout the life of the Project in terms of their ability to meet the performance requirements for scope, schedule, and cost. This evaluation should be done at the level of a specific work product deliverable in the context of its impact on subsequent deliverables to verify compliance and provide the Authorized Purchaser with visibility into the Project’s adherence to its established plans, process and product standards, and work tasks;

b. Review of work product compliance with Authorized Purchaser security standards, especially in relation to system access.

c. Review of the solution consultant’s deliverables for completeness and accuracy, as well as identifying and assessing issues and risks at the work product level. In ascertaining whether the solution consultant’s work products meet Authorized Purchaser requirements, the QMS Consultant shall pay particular attention to the scope, execution, and results of the solution consultant’s test plans.

d. Identify and discuss issues within the Project for resolution, if possible. Address the escalation of issues not resolvable within the Project to the Project’s Steering Committee or otherwise in accordance with the Project’s governance processes.

1. Deliverable 3.1: Quality Control Reviews (of foundational project management documentation and solution contractor deliverables)

a. Detailed Quality Control reviews and reports are required for foundational project management documents and major solutions consultant deliverables, including the solutions consultant’s Project Plan, which should be reviewed as needed. Each Quality Control report should be treated as a separate Deliverable with all applicable Deliverable 2.2 Quality Checklists completed and attached.

b. Quality Control reviews and reports should comply with the Deliverable 2.3 Quality Management Plan, and the approved Deliverable 2.1 Quality Standards, and Deliverable

2.2 Quality Checklists. The Quality Management Plan, Quality Standards, and Quality Checklists should indicate the type of review expected, e.g. data analysis, schedule analysis, standards or process conformance analysis, and additional analysis based on subject matter expertise. For projects under OSCIO oversight, the completed Quality Control reviews and reports, including their related Quality Checklists, are inputs to the Deliverable 4.1, Quarterly Quality Assurance Status and Improvement Reports and related OSCIO Project Assessment and Project Budget and Schedule Variance Reports (Appendix E to the Scope of Services).

c. The precise nature of Quality Control reviews for particular solutions contractor deliverables may need adjustment to accommodate the specific choice of the applicable solution development life cycle model.

2. Deliverable 3.2: Security Review

The Authorized Purchaser may require the QMS Consultant to review details of system design, development, and implementation as they relate to security standards compliance and meeting security related functional and non-functional requirements. In carrying out this work, the Authorized Purchaser may require the QMS Consultant to participate in major reviews of the modules or the subsystems that are developed or customized specifically for the Project’s solution. The type of reviews may include architecture, design, system integration, and code level reviews as they relate to information security. The areas for review and examination may include:

a. User Authentication

b. Role Based Security

c. Database Connectivity

d. Password Validation

e. Encryption

The QMS Consultant should determine the appropriate modules and subsystems for review and the nature of the review. Also, the QMS Consultant should schedule and conduct these reviews, in consultation with the Authorized Purchaser and the solutions consultant, within the context of the Project’s overall plan and schedule.

3. Deliverable 3.3: Quality Status Reporting QMS Consultant must regularly attend Project status meetings in order to prepare a Quality Status Report that documents Project status in the following areas:

a. Project Quality Status

(1) For the overall Project;

(2) For each element on the work breakdown structure in the Baseline QMS Work Plan being executed; and

(3) Tracking of recommendations made through Deliverable 4.1 Quality Assurance Status and Improvements Report or other channels designated by the Authorized Purchaser and, as appropriate, the OSCIO.

(4)

b. Risk Notification Report per Deliverable 1.2.

FOR THE TASK 3 DELIVERABLES RELEASED UNDER THE CONTRACT, REFER TO SECTION 3 OF THE STATEMENT OF WORK: DELIVERABLES AND PAYMENT SCHEDULE.

2.4 Task 4: Quality Assurance (“QA”)

In general, the Quality Assurance Task focuses on the Project’s quality management of its processes. Through such Task the QMS Consultant should provide overall Project quality review; periodically examine Quality Control review results, Quality Checklists, change requests and tracking; and summarize the results of such review and analysis for executive review and oversight throughout the life of the Project. For this purpose, the OSCIO will obligate Authorized Purchasers to require the QMS Consultant to create and deliver quarterly Quality Assurance Status and Improvements Reports and Presentations summarizing the overall Project status, performance, risks and recommendations for process improvement.

1. Deliverable 4.1: Quarterly QA Status and Improvement Reports / Presentations5

The Authorized Purchaser and the QMS Consultant should use as a guide for this Deliverable the QA Status and Improvements Report template, Project Assessment Report template, and the Project Budget and Schedule Variance Report template. (See Appendix C for the templates and Appendix D for sample reports.)

The Deliverable - QA Status and Improvement Report should include the following items:

a. Overall risk rank for the Project;

b. An Executive Summary for the entire report, which includes information such as:

(1) A summary of Project progress and accomplishments since the last report;

(2) A summary of expenditures to date compared to budgeted Project dollars;

(3) Identification and resolution of all problems encountered;

(4) Plans, milestones and deliverables for the coming period;

(5) A 3-month rolling risk matrix by assessment area;

(6) Assessment of overall risk to the Project;

(7) Evaluation of the overall Project status, Project budget and schedule variance, schedule and stage of completion, indicating the causal factors, mitigation efforts and recommendations; and

5 The exact dates of the End of Quarter will be specified in the Baseline QMS Work Plan. These dates will take into account and include but not be limited to, Consultant start date, and Oregon’s Joint Legislative Committee on Information Management Technology hearing dates.

(8) For projects under OSCIO oversight, evaluation of the Project’s ability to deliver the benefits and results stated in the approved business case on file with the OSCIO;

c. Summary of the current progress followed by major Project milestones. At a minimum the report shall contain a progress, status and risk analysis for the following assessment areas, in addition to other areas determined by the Authorized Purchaser to be critical for monitoring purposes:

(1) Project Management;

(2) Customer Involvement;

(3) Technology;

(4) Facilities and Support;

(5) Project Scope;

(6) Business Impact;

(7) Deliverable Quality;

(8) Development and Implementation;

(9) Schedule;

(10) Resources and Staff;

(11) Actual expenditures compared to original (or current baseline) Project budget;

(12) Project schedule performance compared to original (or current baseline) schedule; and

(13) Security.

d. Brief Project effectiveness statements on all the high level categories from the approved Quality Standards:

(1) Categories where no risks were identified should simply be indicated in one line, for example: Decision Drivers – no risks currently identified, or not evaluated at this time;

(2) Categories that contained a significant risk shall describe the risk, describe the factors used to determine the risk, estimate probability of occurrence, describe the potential impact, indicate the risk severity rating, and provide a recommended resolution strategy. At a minimum the resolution strategy should include:

i. Status of risks that must be monitored, identifying probability of occurrence and potential impacts;

ii. Identification of trigger points for intervention for each risk;

iii.

iv. Recommended and alternative correction actions plans, with options identified, presented for each identified risk to prevent and/or reduce potential for the risk occurring or the danger escalating;

v. Intervention strategies to address any risk(s) that exceed a trigger point, to include a definition of options available to address the risk, the potential affects and costs of implementing the strategy, a comparative summary of the alternative strategies and identification of a recommended strategy; and

vi. Implications for the solution consultant’s Project Plan;

e. Risks resolved since last period. Provide the previous risk rank, and brief status or the actions taken on risks and results; and

f. For projects under OSCIO oversight, completed OSCIO Project Assessment Reports and Project Budget and Schedule Variance Reports since Project start. (See Appendix E.)

FOR THE TASK 4 DELIVERABLES RELEASED UNDER THE CONTRACT, REFER TO SECTION 3 OF THE STATEMENT OF WORK: DELIVERABLES AND PAYMENT SCHEDULE.

2.5 Task 5: Independent Solution Testing

The purpose of this QMS Consultant Task is to ensure that the solution and its components, as delivered by the solutions consultant, are functional, stable, and secured as defined by approved Project requirements.

1. Deliverable 5.1: Independent Master Test Plan (“IMTP”)

The QMS Consultant’s IMTP should include relevant plans, methodologies, development procedures, and bug tracking for Independent Testing on a sampling basis. Because the solutions consultant’s deliverables may contain software elements that have been previously deployed and proven stable, the QMS Consultant’s Independent Testing should focus on the testing of high risk and new code areas, especially security access and control. High risk areas include critical items such as sub-system integration, interfaces to other data systems, and other data processing systems. At a minimum, the IMTP should include the following elements:

a. Identification of potential high risk or new code areas;

b. Test Definition and Test Matrix;

c. Detailed Test Script Development Procedure;

d. Detailed Configuration and Build Control Procedure;

e. Testing Procedure;

f. Testing Environment;

g. Test Scripts; and

h. Testing Metrics and Reporting.

i.

The nature of Independent Testing may vary greatly depending on the solution and solutions consultant selected by the Authorized Purchaser.

2. Deliverable 5.2: Independent Test Execution and Status Reports

The QMS Consultant should prepare for and conduct Independent Testing, and provide Test Execution and Status Reports periodically according to the schedule established in the IMTP.

These reports should include items such as:

a. Number of total tests to be conducted for the Project;

b. Number of tests per high risk or new code areas of the solution;

c. Number of tests conducted, completed, and awaiting retest;

d. Types of bugs correlated to the areas of test; and

e. Summary of the results of each security test.

3. Deliverable 5.3: Independent Security-Related Testing

The QMS Consultant should prepare for and conduct Independent Security-Related Testing, and provide Test Execution and Status Reports periodically according to the schedule established in the IMTP.

FOR THE TASK 5 DELIVERABLES RELEASED UNDER THE CONTRACT, REFER TO SECTION 3 OF THE STATEMENT OF WORK: DELIVERABLES AND PAYMENT SCHEDULE.

3. Deliverables and Payment Schedule The Authorized Purchaser shall make payments to QMS Consultant based on successful delivery of completed Deliverables and accepted in writing by Authorized Purchaser. The Parties may use the Contract’s Change Control procedures under Section 7.2 of the Contract to document certain changes to the Deliverables and Payment Schedule, such as:

1. Authorized Purchaser’s release to QMS Consultant of Tasks previously reserved to Authorized Purchaser for such release; provided the Parties previously negotiated the cost for such Tasks and included such costs in Section 6.1 of the Contract, Maximum Payment Amount; the

2. Parties’ agreement to adjust Deliverable costs by moving money between Deliverables; provided such adjustments do not increase the current Maximum Payment Amount under the Contract; and the

3. Parties’ agreement to adjust Deliverable Due Dates or to completely cancel one or more current Deliverables.

4.

Deliverable Number Deliverable Description Due Date NTE Deliverable Cost

Task 1: Risk Assessment

1.1.1

Project Risk Assessment Report - initial WOC NTP + 30 Business Days

OR

As agreed to and documented in the accepted Task 2, Deliverable 2.4, Baseline QMS Work Plan.

1.1.3

Project Risk Assessment Report – refreshed after planning with selected Development Contractor has been completed

REQUIRED

1.1.4

Project Risk Assessment Report – refreshed after substantial design has been completed

1.1.5

Project Risk Assessment Report – refreshed to ensure legacy shutdown/changes can accommodate the rollout

SUBMISSION REQUIRED

1.2

Ongoing Risk Notification (this is an extension of the Status Reporting under section 3.3 and 4.1) As needed. Written report three days after verbal notification from

APM

Price included in periodic Deliverables

3.3 (QC Reports) and

4.1. (QA Reports)

Task 2: Quality Planning

2.1

Quality Standards – Operational Definitions Report See 2.3

$0.00

Deliverable Number Deliverable Description Due Date NTE Deliverable Cost

2.3.1

Quality Management Plan (of iQMS Contractor) Deliverable 1.1 Acceptance + 20 Business Days. The most recent Risk Assessment Report should inform each version of the Quality Management Plan and associated Quality Standards, Quality Checklists and Work Plan.

2.3.2

Quality Management Plan (of iQMS Contractor) - Refresh If Deliverable 2.3 is released during the planning and procurement stage, a QMP refresh (deliverable 2.3.1) should be released after the solution vendor comes on board.

2.4.1

Baseline QMS Work Plan (of iQMS Contractor)

(Updated by the parties as needed to reflect mutually agreeable adjustments, if any, to QMS deliverables.)

See 2.3

2.4.2

2.5

Special Requests TBD by subsequent task release order or WOC amendment

OPTION RESERVED TO ODOT NO COST SUBMISSION REQUIRED

2.6

Lessons Learned Report – Project Evaluation (See template in SOW Appendix B)

Due at end of project As agreed to and documented in the accepted Task 2, Deliverable 2.4, Baseline QMS Work Plan.

Task 3: Quality Control

Deliverable 3.1: Quality Control Review

3.1.1 Review of RFP. Solution requirements related information and data components of the sample contract in the Request for Proposal (RFP) or P3 approved solicitation.

Related information used to support this review may include:

• Requirements supporting RFP (functional & non-functional / technical requirements)

• SOW supporting the RFP

• Agency Project Plan

• Agency Business Case

• Solutions Analysis

Prior to release of RFP or P3 approved solicitation

Deliverable

Deliverable Description Due

NTE Deliverable Cost

3.1.2 Solution contract SOW review, including solution requirements related information and data components of the proposed contract with the successful Proposer.

Solution requirements related information and data include:

· Requirements supporting the proposed solution contract (functional & non- functional / technical requirements)

· SOW supporting proposed solution contract

· Updated agency Project Plan

· Updated agency Business Case

Prior to Solution Contractor contract execution.

3.1.3

3.1.4

Solution Contractor’s Implementation Plan (inclusive of details relating to design, development, implementation, and hosting) After contract execution with Solution Contractor

Agency may choose to split this deliverable into multiple deliverables, e.g. 3.1.5a, 3.1.5b, etc.

The deliverable may QC the Solution Contractor’s documentation, as accepted / approved by Agency (Authorized Purchaser) or Agency’s documentation if such a deliverable does not exist in Solution Contractor’s SOW.

As agreed to and documented in the accepted Task 2, Deliverable 2.4, Baseline QMS Work Plan.

OPTION RESERVED TO AUTHORIZED PURCHASER/ NO COST SUBMISSION REQUIRED

(Consult with EIS Cyber Security Services BISO)

3.1.7

3.1.7.1. Solution Contractor system testing plan

3.1.7.2 Agency (ODOT) User Acceptance Testing plan

As agreed to and documented in the accepted Task 2, Deliverable 2.4, Baseline QMS Work Plan.

Deliverable

Deliverable Description Due

NTE Deliverable Cost

3.1.8 Solution Contractor statement of work deliverables.

Relevant deliverables in Solution Contractor’s statement of work may include:

•Functional design documentation •Technical design documentation •Solution architecture

• Software build and release plan

• Data Dictionary

• Data conversion / migration plan & results

• Disaster Recovery Plan

• Training Plan

• Operations and Maintenance Plan

After Solution Contractor contract execution

Deliverable 3.2: Security Code Review

3.2.1

Security Review and Sampling Plan

PURCHASER/ NO COST

3.2.2

Security Code Review and Sampling Report(s)

OPTION RESERVED

OPTION RESERVED TO AUTHORIZED PURCHASER/ NO COST SUBMISSION

Deliverable 3.3: Quality Status Reporting

3.3

Monthly Quality Status Reporting and Tracking

For up to 39 Instances One instance per month (end of month + 5 business days) except the month when the quarterly is due.

As agreed to and documented in the approved Baseline QMS Work Plan.

($0.00 per instance)

Task 4: Quality Assurance Status and Improvements Reporting

4.1 Quarterly QA Status and Improvement Reports / Presentations

For up to 15 Instances

*Note: The exact dates of the End of Quarter will be specified in the Baseline QMS Work Plan. These dates will take into account and include but not be limited to Contractor start date. Figuring 5 instances.

End of Quarter + 10 Business Days*

($0.00 per instance)

Deliverable

Deliverable Description Due Date NTE Deliverable Cost

Task 5: IV&V Testing

5.1

Independent Master Test Plan (IMTP)

OPTION RESERVED*

OPTION RESERVED TO AUTHORIZED PURCHASER/ NO COST SUBMISSION

5.2

Independent Test Execution and Status Reports

OPTION RESERVED

OPTION RESERVED TO AUTHORIZED PURCHASER/ NO COST SUBMISSION

5.3

Independent Security-Related Testing

PUR…