Amendment_2_SOW_Appendices_A-E.pdf

Appendix A thru Appendix E to QA SOW

Version 3.0

September 2016

Appendix A: Generic Software Project Quality Standards

When delivering Quality Management Services (QMS), the Consultant is expected to use this Appendix to identify relevant quality standards and associated risk cues in connection with Deliverable 2.1 Quality Standards. Deliverable 2.1 Quality Standards is an important aspect of quality planning that supports Deliverable 4.1 Quarterly Quality Assurance Report.

Notes to Consultant:

1. A total of 82 quality standards under 12 quality categories (quality focus areas) are listed. Quality Standard marked with “*” are required. For those Quality Standards not marked with an “*”, only those relevant to a specific project need to be used.

2. Additional quality categories may be added to the extent relevant to a project. In

Oregon IT projects, quality categories that are not part of this Appendix include:

information security, business transition, and financial control (e.g. separation of roles). Each of these additional quality categories usually consists of multiple quality standards and associated risk cues.

3. For each Quality Standard, “Low Risk Cue”, “Medium Risk Cue”, and “High

Risk Cue” comprise a qualitative metric (measurement) of risk associated with the specific Quality Standard.

4. For each Quality Standard, the definition of risk cues provided in this Appendix may be changed or tailored for the needs of a specific project. This tailoring is to be performed independently by the QMS Consultant. Input from state personnel or the state’s contractors may be sought, but cannot substitute for industry standards, best practice, and applicable expert knowledge and judgment that the QMS Consultant must apply independently.

5. The quality standards in this table are organized with the following headers:

a. QS# - A sequentially assigned number for quality standards

b. Quality Category – Header that names the category in which the following

Quality standards belong

c. Quality Standard – Named areas of potential quality standards. “*” indicates recommended minimums

d. Low Risk Cues – Characteristics of this quality standard when it can be considered low risk to the project

e. Medium Risk Cues – Characteristics of this quality standard when it should be considered high risk to the project

f. High Risk Cues – Characteristics of this quality standards when it should be considered high risk to the project

g. Rating – Level of quality risk you think is true of this project

• Low – This project exhibits the low risk cue, or appears to have no risks in this area

• Medium – This project exhibits the medium risk cue, or something similar in threat

• High – This project exhibits the high risk cue, or something similar in threat

• N / A – This factor is not applicable to this project

• Need Info – The Contractor needs information from someone else (perhaps an expert) to make a judgment

• TBD – The project is not far enough along to make a rating; the

Contractor needs to review the quality standard at a later time

h. Risk Rank – The numerical rating for risk as it ranks with other identified.

For example the quality standard may have high risk cues, but for the project may be of low risk

Generic Software Project Quality Standards for Risks

Q S

Quality Standards

Low Risk Cues Medium Risk Cues High Risk Cues

L o w

M ed iu m

H ig h

N o t A p p lic

N ee d In fo

T B

D

Risk Rank

Process Standards Business Mission and Goals

1 Project Fit to Customer Organization directly supports customer organization mission and/or goals indirectly impacts one or more goals of customer does not support or relate to customer organization mission or goals

2 Project Fit to Provider Organization directly supports provider organization mission and/or goals indirectly impacts one or more goals of provider does not support or relate to provider organization mission or goals

3 Customer Perception customer expects this organization to provide this product organization is working on project in area not expected by customer project is mismatch with prior products or services of this organization

4 Work Flow little or no change to work flow will change some aspect or have small affect on work flow significantly changes the work flow or method of organization

Rating (check one)

Generic Software Project Quality Standards

Using this table to identify quality standards and risk:

A project should use this table to identify quality standards (QSs) and their risks. The project should decide which standards are relevant at what rating, considering the risks they suspect could affect their project.

Material in the quality standards table is organized with the following headers:

QS # - A sequentially assigned number for quality standards.

Quality Category - Header that names the category in which the following QSs belong.

Quality Standard - Named areas of potential quality standards. "*" indicates recommended minimums.

Low Risk Cues - Characteristics of this QS when it can be considered low risk to a project.

Medium Risk Cues - Characteristics of this QS when it provides a medium risk to a project.

High Risk Cues - Characteristics of this QS when it should be considered high risk to a project.

Rating - Level of quality risk you think is true of this project.

Low - This project exhibits the low risk cue, or appears to have no risk in this area.

Medium - This project exhibits the medium risk cue, or something similar in threat.

High - This project exhibits the high risk cue, or something similar in threat.

Not Applic - This factor is not applicable to this project.

Need Info - We need information from someone else (perhaps an expert) to make a judgment.

TBD - The project is not far enough along to make a rating; we need to review this later.

Risk Rank - Numerical rating for risk as it ranks with others identified. For example the QS may have high risk cues, but overall for the project be of low risk.

6__generic_software_quality_standards_template Page 1 of 9 8/10/2016

Quality Standards

Low Risk Cues Medium Risk Cues High Risk Cues

L o w

M ed iu m

H ig h

N o t A p p lic

N ee d In fo

T B

Rating (check one)

5 Goals Conflict goals of projects within the organization are supportive of or complimentary to each other goals of projects do not conflict, but provide little direct support goals of projects are in conflict, either directly or indirectly

Decision Drivers 6 *Political

Influences no particular politically-driven choices being made project has several politically motivated decisions, such as using a vendor selected for political reasons, rather than qualifications project has a variety of political influences or most decisions are made behind closed doors

7 Convenient Date date for delivery has been set by reasonable project commitment process date is being partially driven by need to meet marketing demo, trade show, or other mandate not related to technical estimate date is being totally driven by need to meet marketing demo, trade show, or other mandate; little consideration of project team estimates

8 Attractive Technology technology selected has been in use for some time project is being done in a sub-optimal way, to leverage the purchase or development of new technology project is being done as a way to show a new technology or as an excuse to bring a new technology into the organization

9 Short Term Solution project meets short term need without serious compromise to long term outlook project is focused on short-term solution to a problem, with little understanding of what is needed in the long term project team has been explicitly directed to ignore the long term outlook and focus on completing the short term deliverable

Project Management 10 *Definition of the project project is well-defined, with a scope that is manageable by this organization project is well-defined, but unlikely to be handled by this organization project is not well-defined or carries conflicting objectives in the scope

11 *Project Objectives verifiable project objectives, reasonable requirements some project objectives, measures may be questionable no established project objectives or objectives are not measurable

6__generic_software_quality_standards_template Page 2 of 9 8/10/2016

Quality Standards

Low Risk Cues Medium Risk Cues High Risk Cues

L o w

M ed iu m

H ig h

N o t A p p lic

N ee d In fo

T B

12 *Leadership project has active sponsor project has sponsor responsible for project, but unable to spend enough time to direct effectively project has no sponsor, or project manager concept is not in use

13 *PM Approach product and process planning and controls in place planning and controls need enhancement weak or nonexistent planning and controls

14 PM

Communication clearly communicates goals and status between the team and rest of organization communicates some of the information some of the time rarely communicates clearly to the team or to others who need to be informed of team status

15 PM Experience PM very experienced with similar projects

PM has moderate experience or has experience with different types of projects

PM has no experience with this type of project or is new to project management

16 PM Attitude strongly committed to success willing to do what it takes cares very little about project

17 *PM Authority has line management or official authority that enables project leadership effectiveness is able to influence those elsewhere in the organization, based on personal relationships has little authority from location in the organization structure and little personal power to influence decision-making and resources

18 Support of the

PM

complete support by team and of management support by most of team, with some reservations no visible support;

manager in name only

Project Parameters 19 Project Size small, non-complex, or easily decomposed medium, moderate complexity, decomposable large, highly complex, or not decomposable

20 Hardware Constraints little or no hardware-imposed constraints or single platform some hardware-imposed constraints;

several platforms significant hardware-imposed constraints;

multiple platforms

21 Reusable Components components available and compatible with approach components available, but need some revision components identified, need serious modification for use

6__generic_software_quality_standards_template Page 3 of 9 8/10/2016

Quality Standards

Low Risk Cues Medium Risk Cues High Risk Cues

L o w

M ed iu m

H ig h

N o t A p p lic

N ee d In fo

T B

22 Supplied Components components available and directly usable components work under most circumstances components known to fail in certain cases, likely to be late, or incompatible with parts of approach

23 *Budget & Resource Size sufficient budget and resources allocated questionable budget and resources allocated doubtful budget and resources are sufficient

24 Budget Constraints funds allocated without constraints some questions about availability of funds allocation in doubt or subject to change without notice

25 *Cost Controls well established, in place system in place, weak in areas system lacking or nonexistent

26 *Delivery Commitment stable commitment dates some uncertain commitments unstable, fluctuating commitments

27 *Development Schedule team agrees that schedule is acceptable and can be met team finds one phase of the plan to have a schedule that is too aggressive team agrees that two or more phases of schedule are unlikely to be met

Project Team 28 *Team Member

Availability in place, little turnover expected;

few interrupts for fire fighting available, some turnover expected;

some fire fighting high turnover, not available; team spends most of time fighting fires

29 Mix of Team Skills good mix of disciplines some disciplines inadequately represented some disciplines not represented at all

30 Application Experience extensive experience in team with projects like this some experience with similar projects little or no experience with similar projects

31 Experience with Project Hardware and Software high experience average experience low experience

32 Experience with Process extensive experience with this process some experience with this process or extensive experience with another little or no experience with a defined process

33 Training of Team training plan in place, training ongoing training for some areas not available or training planned for future no training plan or training not readily available

6__generic_software_quality_standards_template Page 4 of 9 8/10/2016

Quality Standards

Low Risk Cues Medium Risk Cues High Risk Cues

L o w

M ed iu m

H ig h

N o t A p p lic

N ee d In fo

T B

34 Team Spirit and Attitude strongly committed to success of project;

cooperative willing to do what it takes to get the job done little or no commitment to the project; not a cohesive team

35 *Team Productivity all milestones met, deliverables on time, productivity high milestones met, some delays in deliverables, productivity acceptable productivity low, milestones not met, delays in deliverables

36 Expertise with Application Area (Domain) good background with application domain within development team some experience with domain in team or able to call on experts as needed no expertise in domain in team, no availability of experts

Organization Management 37 *Organization

Stability little or no change in management or structure expected some management change or reorganization expected management or organization structure is continually or rapidly changing

38 Organization Roles and Responsibilities individuals throughout the organization understand their own roles and responsibilities and those of others individuals understand their own roles and responsibilities, but are unsure who is responsible for work outside their immediate group many in the organization are unsure or unaware of who is responsible for many of the activities of the organization

39 Policies and Standards development policies and standards are defined and carefully followed development policies and standards are in place, but are weak or not carefully followed no policies or standards, or they are ill-defined and unused

40 Management Support strongly committed to success of project some commitment, not total little or no support

41 *Executive Involvement visible and strong support occasional support, provides help on issues when asked no visible support; no help on unresolved issues

42 Resource Conflict projects within the organization share resources without any conflict projects within the organization schedule resources carefully to avoid conflict projects within the organization often need the same resources at the same time (or compete for the same budget)

6__generic_software_quality_standards_template Page 5 of 9 8/10/2016

Quality Standards

Low Risk Cues Medium Risk Cues High Risk Cues

L o w

M ed iu m

H ig h

N o t A p p lic

N ee d In fo

T B

43 Customer Conflict multiple customers of the project have common needs multiple customers of the project have different needs, but do not conflict multiple customers of the project are trying to drive it in very different directions

Customer/User 44 *User

Involvement users highly involved with project team, provide significant input users play minor roles, moderate impact on system minimal or no user involvement; little user input

45 User Experience users highly experienced in similar projects; have specific ideas of how needs can be met users have experience with similar projects and have needs in mind users have no previous experience with similar projects;

unsure of how needs can be met

46 *User Acceptance users accept concepts and details of system; process is in place for user approvals users accept most of concepts and details of system; process in place for user approvals users do not accept any concepts or design details of system

47 User Training Needs user training needs considered; training in progress or plan in place user training needs considered; no training yet or training plan is in development requirements not identified or not addressed

48 User Justification user justification complete, accurate, sound user justification provided, complete with some questions about applicability no satisfactory justification for system

Product Standards

Product Content 49 Requirements

Stability little or no change expected to approved set (baseline) some change expected against approved set rapidly changing or no agreed-upon baseline

50 *Requirements Complete and Clear all completely specified and clearly written some requirements incomplete or unclear some requirements only in the head of the customer

51 *Testability product requirements easy to test, plans underway parts of product hard to test, or minimal planning being done most of product hard to test, or no test plans being made

52 Design Difficulty well defined interfaces; design well understood unclear how to design, or aspects of design yet to be decided interfaces not well defined or controlled;

subject to change

6__generic_software_quality_standards_template Page 6 of 9 8/10/2016

Quality Standards

Low Risk Cues Medium Risk Cues High Risk Cues

L o w

M ed iu m

H ig h

N o t A p p lic

N ee d In fo

T B

53 *Implementation Difficulty algorithms and design are reasonable for this team to implement algorithms and/or design have elements somewhat difficult for this team to implement algorithms and/or design have components this team will find very difficult to implement

54 System Dependencies clearly defined dependencies of the software effort and other parts of system (hardware, process changes, documentation, ...)

some elements of the system are well understood and planned; others are not yet comprehended no clear plan or schedule for how the whole system will come together

Development Process 55 Alternatives

Analysis analysis of alternatives complete, all considered, assumptions verifiable analysis of alternatives complete, some assumptions questionable or alternatives not fully considered analysis not completed, not all alternatives considered, or assumptions faulty

56 Commitment Process changes to commitments in scope, content, schedule are reviewed and approved by all involved changes to commitments are communicated to all involved changes to commitments are made without review or involvement of the team

57 Quality Assurance Approach

QA system established, followed, effective procedures established, but not well followed or effective no QA process or established procedures

58 *Development Documentation correct and available some deficiencies, but available nonexistent

59 Use of Defined Engineering Process development process in place, established, effective, followed by team process established, but not followed or is ineffective no formal process used

60 Early Identification of Defects peer reviews are incorporated throughout peer reviews are used sporadically team expects to find all defects with testing

61 Defect Tracking defect tracking defined, consistent, effective defect tracking process defined, but inconsistently used no process in place to track defects

62 Change Control for Work Products formal change control process in place, followed, effective change control process in place, not followed or is ineffective no change control process used

6__generic_software_quality_standards_template Page 7 of 9 8/10/2016

Quality Standards

Low Risk Cues Medium Risk Cues High Risk Cues

L o w

M ed iu m

H ig h

N o t A p p lic

N ee d In fo

T B

63 Lessons Learned Lessons learned and improvements made at milestones or phases

Lessons learned conducted, improvements not incorporatated

No lessons learned conducted, improvements not incorporated

Development Environment 64 Physical

Facilities little or no modification needed some modifications needed; some existent major modifications needed, or facilities nonexistent

65 Hardware Platform stable, no changes expected, capacity is sufficient some changes under evolution, but controlled platform under development along with software

66 Tools Availability in place, documented, validated available, validated, some development needed (or minimal documentation) unvalidated, proprietary or major development needed; no documentation

67 Vendor Support complete support at reasonable price and in needed time frame adequate support at contracted price, reasonable response time little or no support, high cost, and/or poor response time

68 Contract Fit contract with customer has good terms, communication with team is good contract has some open issues which could interrupt team work efforts contract has burdensome document requirements or causes extra work to comply

69 Disaster Recovery all areas following security guidelines;

data backed up;

disaster recovery system in place;

procedures followed some security measures in place;

backups done;

disaster recovery considered, but procedures lacking or not followed no security measures in place; backup lacking; disaster recovery not considered

Technology 70 Technology

Match to Project technology planned for project is good match to customers and problem some of the planned technology is not well-suited to the problem or customer selected technology is a poor match to the problem or customer

71 Technology Experience of Project Team good level of experience with technology some experience with the technology no experience with the technology

72 Availability of Technology Expertise technology support and experts readily available experts available elsewhere in organization will need to acquire help from outside the organization

73 Maturity of Technology technology has been in use in the organization for quite some time technology is well understood in the organization technology is leading edge, if not "bleeding edge" in nature

Deployment

6__generic_software_quality_standards_template Page 8 of 9 8/10/2016

Quality Standards

Low Risk Cues Medium Risk Cues High Risk Cues

L o w

M ed iu m

H ig h

N o t A p p lic

N ee d In fo

T B

74 Hardware Resources for Deliverables mature, growth capacity in system, flexible available, some growth capacity no growth capacity, inflexible

75 Response or other Performance Factors readily fits boundaries needed;

analysis has been done operates occasionally at boundaries operates continuously at boundary levels

76 *Customer Service Impact requires little change to customer service requires minor changes to customer service requires major changes to customer service approach or offerings

77 Data Migration Required little or no data to migrate much data to migrate, but good descriptions available of structure and use much data to migrate; several types of databases or no good descriptions of what is where

78 Pilot Approach pilot site (or team) available and interested in participating pilot needs to be done with several sites (who are willing) or with one who needs much help only available pilot sites are uncooperative or in crisis mode already

79 External Hardware or Software Interfaces little or no integration or interfaces needed some integration or interfaces needed extensive interfaces required

Maintenance 80 *Design

Complexity structurally maintainable (low complexity measured or projected) certain aspects difficult to maintain (medium complexity) extremely difficult to maintain (high complexity)

81 *Support Personnel in place, experienced, sufficient in number missing some areas of expertise significant discipline or expertise missing

82 Vendor Support complete support at reasonable price and in needed time frame adequate support at contracted price, reasonable response time little or no support, high cost, and/or poor response time

Total Categories 12 Total QS's 82

6__generic_software_quality_standards_template Page 9 of 9 8/10/2016

Appendix B:

Project Evaluation Template

(at end of each project phase or at project closing)

Notes to Consultant

When delivering Quality Management Services (QMS), the Consultant may be authorized by the state to deliver Deliverable 2.6 Project Evaluation/Lessons Learned Report; typically at the end of each Project phase or at the close of the Project. This appendix provides a template for reference purposes.

If this template is used, a level of tailoring is generally expected based on project specifics and input from the Authorized Purchaser and state staff.

If this template is not used, the Consultant’s template must be pre-approved by the state before use. Unless otherwise approved by the state, the Consultant’s template must have contents that meet or exceed the level of contents described in this template.

Template for Deliverable 2.6 Project Evaluation/Lessons Learned Report

Approval Date: Month Day, Year

Initiating, Planning, Executing, Controlling

Evaluate Key Aspects of Project

Transition Project to Operations

Document Lessons Learned

Closing

Project Evaluation

Purpose of the Document

The purpose of the Project Evaluation is to evaluate the project (at end of each project phase or at project closing), evaluate transition of the project to operations, provide a basis for feedback to the project team and management, and to document the lessons learned to improve the process and future project potential.

Template Instructions

This template contains suggested boilerplate language and assumes that the project will make appropriate additions, deletions, and changes for their specific needs.

1 Insert information between left and right brackets - <> and then Delete Brackets 2 Information in italics is additional template instructions. Delete all italicized instructions 3 In file on the menu go to properties and in the summary folder enter the document title and author (person or group) 4 If the document is longer than 5 pages, you should insert an automatic table of contents

Degree of Attainment of Business Objectives

Document how the project performed against each objective established in the Product Description and Integrated Project Plan.

<Objective n>

Degree of attainment of objective

Success Factors

Nature and causes of variances

<Objective n>

Degree of attainment of objective

Success Factors

Nature and causes of variances

Degree of Attainment of Budget Objectives State the Planned Cost and Funding for the project, as approved in the Integrated Project Plan. State the Actual Cost and Funding at end of project phase or at completion. Document and explain all cost and funding variances, including approved changes to the cost baseline.

Expenditures ($000)

Planned Actual Variance Explanation

Internal Staff Labor

Service

Software Tools

Hardware

Materials and Supplies

Facilities

Telecommunications

Training

Contingency (Risk)

Total

Funding Source ($000)

Planned Actual Variance Explanation

General Fund

Non-General Fund

Federal

Other

Degree of Adherence to Schedule Compare the approved schedule baseline against the actual completion dates. Document and explain any schedule variances, including approved changes to the schedule baseline

Degree of Satisfaction of User Requirements Document any changes to the Requirements and their impact on Performance, Cost, or Schedule Baselines.

Degree of Realization of Anticipated Benefits (Business Case Realization) Document the primary benefits the agency projected would be realized/attained (benefits or ROI Targets). Document whether those benefits were obtained (or are expected to be obtained) by the project.

Degree of Productivity Experienced Indicate the productivity level of the project and factors that caused increased performance, as well as, decreased performance.

Degree of Delivery – Product Project Deliverable List the major Project Deliverables and the date each was accepted by the user. Identify any contingencies or conditions related to the acceptance.

Deliverable Date Accepted

Contingencies or Conditions

1.

2.

3.

Transition to Operations and Maintenance

Describe the plan for operation and maintenance of the product, good, or service delivered by the project below. In addition, state the projected annual cost to operate and maintain the product, good, or service. If the operation and maintenance plan is not in place, what is the target date for the plan and what is the impact of not having operations and maintenance for the product, good, or services in place.

Operations and Maintenance Plan Define what will be maintained, who will be responsible for maintaining, how changes will be made to the application, how regular upgrades to software, utilities, and hardware will be prioritized, what business unit is responsible and any other service agreements. You may want to define what are “functionality enhancements”, “Operations enhancements”, “Defect enhancements” and “Emergency Fixes” and how these requests will be prioritized in the future.

Operations and Maintenance Cost Expenditures ($000)

Yr1 Yr2 Yr3 Yr4 Yr5+ Explanation

Internal Staff Labor

Services

Software Tools

Hardware Materials and Supplies

Facilities

Telecommunications

Training

Contingency (Risk)

Funding Source ($000)

Yr1 Yr2 Yr3 Yr4 Yr5+ Explanation

General Fund

Non-General Fund

Federal

Release of Project Resources List the Resources used by the project. Identify to whom each resource was transferred and when it was transferred. Account for all project resources utilized by the project.

Resource (Describe or name the resource used)

Person or Organization Who Received Resource Turnover Date

Project Team

Customer Support

Facilities

Equipment

Software Tools

Transition of Project Documentation Identify all project documentation materials stored in the project library or other repository. Identify the type of media used and the disposition of the project documentation (see Communications Plan).

Report(s) and Document(s) Media Used Storage Location Disposition

Lessons Learned Identify primary Lessons Learned. Lessons Learned should be stated in terms of Problems (or issues) and Corrective Actions taken. Site any references that provide additional detail. References may include project reports, plans, issue logs, change management documents, or Lesson-learned checklist.

Statement of Lesson References Corrective Actions

1.

2.

3.

4.

Project (or Project Phase) Closeout Transition Checklist Complete the Status and Comments column. In the Status column indicate: Yes, if the item has been addressed and completed; No, if item has not been addressed, or is incomplete; N/A, if the item is not applicable to this project. Provide comments or describe the plan to resolve the item in the last column.

Item Status Comments/

Plan to Resolve 1 Have all the product or service deliverables been accepted by the customer?

1.1 Are there contingencies or conditions related to

the acceptance? If so, describe in the Comments.

2 Has the project (or project phase) been evaluated against each objective established in the product description and Integrated Project Plan?

3 Has the actual cost of the project (or project phase) been tallied and compared to the approved budget?

3.1 Have all approved changes to the cost baseline

been identified and their impact on the project documented?

4 Have the actual milestone completion dates been compared to the approved schedule?

4.1 Have all approved changes to the schedule

baseline been identified and their impact on the project documented?

5 Have all approved changes to the project requirement been identified and their impact on the performance, cost, and schedule baselines documented?

6 Has operations management formally accepted responsibility for operating and maintaining the product(s) or service(s) delivered by the project?

6.1 Has the documentation relating to operation and

maintenance of the product(s) or service(s) been delivered to, and accepted by, operations management?

6.2 Has training and knowledge transfer of the

operations organization been completed?

6.3

Has the projected annual cost to operate and maintain the product(s) or service(s) been approved and funded? If not, note and explain who is responsible to resolve.

7 Have the resources used by the project been reassigned to other units or projects?

8 Has the project documentation been archived or otherwise disposed as described in the project communication plan?

9 Have the lessons learned been filed with the Project Management Office?

Approvals

Position/Title Signature/Printed Name/Title Date Project Manager

Project Sponsor

Agency IT Maintenance /Operations Manager

Program/Agency Management

Appendix C: QA Status and Improvement Report Template authorized by the state to deliver Deliverable 4.1 Quarterly QA Status and Improvements Reports. In some projects, this deliverable may be more frequent than quarterly. If so, this deliverable may be renamed as needed.

If this template is used, a level of tailoring is generally expected based on project specifics and input from the Authorized Purchaser and state staff.

If this template is not used, the Consultant’s template must be pre-approved by the state before use. Unless otherwise approved by the state, the Consultant’s template must have contents that meet or exceed the level of contents described in this template.

In performing the work of Deliverable 4.1 Quarterly QA Status and Improvements Reports, the Consultant must attach completed OSCIO Project Assessment Report and Project Budget & Schedule Variance Report to this QA Status and Improvement Report.

Format of these two OSCIO reports may not be altered by the Project, and the latest approved version of these reports must be used.

Template for Deliverable 4.1 QA Status and Improvements Reports

Period: Month/Year

Low Medium High

Risk of Project Failure

Note: Attach completed DAS OSCIO Project Assessment Report and Project Budget & Schedule Variance Report to this QA Status and Improvement Report.

Format of these two DAS OSCIO reports may not be altered by the Project, and the latest approved version of these reports must be used. However, the format of this QA Status and Improvement Report may be customized to the needs of the Project, subject to the review and approval process specified in the QA Contract Statement of Work.

Summary This section should provide a paragraph summary of the entire report. It should contain information such as; A brief description the primary achievement in the last period. A brief description of the highest ranked primary (if any) quality risks covered in the detail of the report. A brief description of the impact or consequence of the risk if left unresolved. This section should not be more than a 1/3 of this page.

The following section should start with a summary of the Current Progress followed by major milestones.

Current Progress

Progress Point – Make specific progress point regarding schedule, and budget i.e., The project is on schedule

Remaining 73%

Complete 27%

Include a diagram as above that indicates % complete, % remaining.

Following diagram provide more detail relating to any progress points made.

At this time, resource usage, task completion and work remaining indicate the project is on schedule and can be completed on time and within planned budget.

Additional Progress point

Detail of Point.

Additional Progress point

Detail of Point.

Major Milestones – From <date to date>

Abbreviated Major Milestone Title

Status or Milestone.

Current Quality Risks – by Category The following sections should be summary results of the risks from the Quality Control Audits, using the Quality Standard checklist’s quality categories.

• Categories where no risks were identified should simply be indicated in one line, for example, Decision Drivers – no risks currently identified or not evaluated at this time.

• Categories which contained a significant risk should briefly describe the risk, describe the factors used to determine the risk, describe the potential impact, indicate the risk severity rating, and provide a recommended resolution strategy.

Business Mission and Goals

High Risk Cue, i.e.

Does not support or relate to customer organization mission or goals

Low Moderate High

Risk

Risk: <Quality Standard Title>, i.e., Project Fit to Customer Organization. Brief description of actual risk to project and current condition.

Primary point

Determining Factors: Briefly describe what was assessed and what indicated the risk condition, refer to expert opinion, industry standard, statistics, internal opinion, or analysis, i.e. specific schedule analysis method.

Primary point

Potential Impacts: Briefly describe the likely risk consequence and probability of occurring.

Actual Rating

Severity Rating: Indicate reason for rating and direction rating is going (if possible), i.e. moderate moving towards severe, or moderate moving towards low.

Primary Point

Resolution Strategy: Describe the mitigation, contingency, or actions recommended to resolve or prevent the quality risk. Recommend person responsible to resolve and suggested time frame in which resolve.

Repeat the above section for the following categories from the Quality Standards Control

Checklist below.

1 Decision Drivers 2 Project Management 3 Project Parameters 4 Project Team 5 Organization Management 6 Customer/User 7 Product Content

8 Development Process 9 Development Environment 10 Technology 11 Deployment 12 Maintenance

Specific Project Risks (Including Risks Resolved Since Last Period)

New Risk and Risk Rank

Risk Status: Brief description of status of actions taken on risk and results.

New Risk and Risk Rank

Previous Risk and

Previous Risk and

List of Attachments

1. DAS OSCIO Project Assessment Report

2. DAS OSCIO Project Budget & Schedule Variance Report

3. etc.

4. etc.

Appendix D: QA Status and Improvement “Sample” Report

The intent of this Appendix is to provide an example to assist Consultant in creating a new document based on the standard template given in the previous appendix. All project data provided are fictitious and is for reference purposes only.

Note: Attach completed DAS OSCIO Project Assessment Report and Project Budget & Schedule Variance Report to this QA Status and Improvement Report.

Format of these two DAS OSCIO reports may not be altered by the Project, and the latest approved version of these reports must be used. However, the format of this QA Status and Improvement Report may be customized to the needs of the Project, subject to the review and approval process specified in the QA Contract Statement of Work.

XYZ Project QA Status & Improvement Report

Period: January 2002

Low Medium High

Risk of Project Failure X

Summary The development team concluded Joint Application Development (JAD) sessions during the reporting period and delivered the draft requirement documents. Users have reviewed the requirements and provided feedback to the development team. In order to allow adequate time for revision and final review of the requirements the project team has agreed to extend the revision/review period. At this time it is believed this will not effect the final project dates. Risk identified last period associated with an informal communications structure has been mitigated. A risk associated with expectations for deliverable purpose and content was identified and resolved this period.

Current Progress The project is on schedule

Remaining 81%

Complete 19%

At this time, resource usage, task completion and work remaining indicate the project is on schedule and can be completed on time and within planned budget.

Requirements review and revision time frame extended.

Draft requirements produced and reviewed

The review and revision deadline for the requirement documents has been extended to assure that there is time to adequately review and revise these documents. The project team believe delay’s created by extending the review/revisions cycle for the requirements document will not affect final project dates. We believe this extension will result in a higher quality deliverable that will benefit dall parties.

The development team has completed and delivered the draft requirement documents (RAD’s). The project has reviewed these documents and provided feedback to the development team. The development team is in the process of making changes to the requirements documents. Final requirement documentation should be complete in February.

Major Milestones Draft Requirements Document Delivered

The draft requirement documents (RAD) have been delivered by the project.

Requirements Document Reviewed by Project

The project has completed it’s review of the requirement documents and provided feedback to the development team.

Current Quality Risks – by Category Risk discussed last period associated with an informal communications structure has been mitigated. One new risk associated with the expectations surrounding deliverables was identified and resolved this period.

1. Business Mission and Goals

Multiple demands on state resources

Potential project delays

Implement a standardized staffing

Low Moderate High

Risk X

Risk: As the Department is faced with multiple projects in 2001 - 2002, including TRACE construction, HIPAA mandates, MMIS replacement, and the department wide re-organization, there are potentially conflicting demands on department staff and management resources. Since these multiple projects are long-term in nature, this is likely an on-going risk for the duration of the project.

Potential Impacts: If department staff and management availability is less than needed to meet the vendor's 52-week schedule, there may be project delays.

Mitigating Strategy: Project Managers from each major project have adopted a standardized staffing matrix to assess risk and manage resources. The matrix can serve as a tool for the affected managers to matrix and regular communication and coordination across all projects

Unknown at this time, but should be closely monitored meet regularly and coordinate the use of resources as efficiently as possible. The Project Managers should also use the matrix and the scheduled meetings to plan and manage their own availability. These meetings will be held monthly during the project's duration. However, as the state's involvement intensifies, the project managers may need to meet more often to ensure regular communication and coordination.

Severity Rating: Until specific conflicts in state resources are identified, it is difficult to assign a severity rating. However, this risk should be closely monitored, as it is potentially long-term in nature and could adversely impact the schedule.

No Current Risks No risk identified for this period.

2. Decision Drivers

3. Decision Drivers

4. Project Management

5. Project Parameters

52-week schedule appears to be aggressive

Quality of final product may suffer

Closely monitor project status and progress

Risk X

Risk: The project plan identifies a 52-week project cycle, which appears to be aggressive for a project of this size.

Potential Impacts: Adequate time for all project activities, including state reviews, requirements definition and testing, are essential to ensuring the final product meets the state's business and user needs. By compressing the schedule, the quality of the final product may suffer.

Mitigating Strategy: Given the aggressive schedule, it is imperative that the project is monitored closely to ensure all major milestones, deliverables, and activities meet the state's requirements while adhering to the schedule. Any slippage in schedule or perceived quality of work will be evaluated and reported, including anticipated impacts.

Potentially high - a realistic schedule is essential

Severity Rating: Potentially high. A realistic schedule is critical to the overall success of the project.

6. Project Team

7. Organization Management

8. Customer/User

9. Product Content

Miscommunication about expectations of deliverable purpose and content.

Potential impacts are project delay, deterioration of relationships and diminished quality of final product.

Agree to deliverable content before submission of draft deliverables

This risk is severe and the potential

Risk X

Risk: With the release of the first project draft deliverable there was miscommunication about the purpose and content of the documents.

Without some mitigation it is likely this miscommunication will repeat with each deliverable resulting in the impacts noted below.

Potential Impacts: There are three significant potential impacts of this risk. The first potential impact is the delayed acceptance of deliverables and subsequent schedule slippage. The second potential impact is the deterioration of the relationship on the project as some believes the other failed to meet expectations regarding deliverables. The last potential impact is reduced quality of the final product (TRACE Data Warehouse) resulting from submission and acceptance of deliverables that do not adequately communicate user needs and proposed solutions to project.

Mitigating Strategy: The project and the state should agree on deliverable format and content before construction of the actual deliverable begins. This generally takes the form of a detailed or annotated deliverable outline created by the development team and reviewed and approved by the project as a whole. The project has already agreed to such a plan.

Severity Rating: If left unmitigated this risk is severe. Its potential impact is great. Recognizing this project has already developed an impact is great but the Project Team have mitigated this risk.

acceptable mitigation strategy. As long as the strategy is implemented we consider this risk closed. Further we commend both the project for recognizing this risk early and addressing it so quickly.

10. Deployment

11. Development Process

12. Development Environment

13. Technology

14. Maintenance

Specific Project Risks (Including Risks Resolved Since Last Period)

Risk Potential Impact Recommended Strategy

Limited on-site development

Resulting system does not address unique requirements of Oregon

• Ensure users participating in JAD session have clear understanding of and clearly communicate requirements to the project.

• Review and approve requirements document against user's stated requirements

• Require incremental prototypes of the system for review and approval by state

• Perform rigorous user acceptance testing

• Ensure SOW addresses each milestone/deliverable with adequate review time for the state

• Require regular progress reports against milestones and hold the development team accountable

• Perform on-site visits in Atlanta

Multiple commitments on behalf of the development team' Project Manager

Project drifts without full-time PM

• Monitor the project for a 100% commitment by the development team’ Project Manager

List of Attachments

• OSCIO Project Assessment Report

• OSCIO Project Budget & Schedule Variance Report

Appendix E:

• DAS OSCIO Project Assessment Report (Version 2.2)

• DAS OSCIO Project Budget & Schedule Variance Report (Version 2.0) authorized by the state to deliver Deliverable 4.1 Quarterly QA Status and Improvements Reports. In performing the work of Deliverable 4.1, the Consultant must attach completed OSCIO Project Assessment Report and Project Budget & Schedule Variance Report.

Format of these two OSCIO reports may not be altered by the Consultant or agency management and are subject to revision by OSCIO only. The latest version of these as approved by OSCIO must be used.

Project Assessment Report Version 2.2

Project Name:

Agency Name:

Person completing this form and affiliation:

Date of completing this form:

Technology Description Platforms:

% custom:

Funding Description

Rating Rating Explanation Rating Scale^ 1 Overall Project Health (rear view) (fwd view) Red, yellow, green*

1a Total Cost (Budget) Red, yellow, green* 1b Schedule Red, yellow, green* 1c Scope Red, yellow, green* 1d Resource Red, yellow, green*

1e Deliverables (Work Products) Quality Red, yellow, green*

2 Overall Delivery Risk high, medium, low* 2a Technology N/A high, medium, low* 2b Financial & Business Case N/A high, medium, low* 2c Business Transition N/A high, medium, low* 2d Funding N/A high, medium, low*

3 O&M Risk 3a Long-Term Supportability N/A high, medium, low* 3b Long-Term Maintainability N/A high, medium, low*

To ensure a consistent reporting approach and terminology used across IT projects and across agencies statewide, this template must be used for quarterly reporting without modification.

Note that Items (1a) through (1e) will be reported from two perspectives: "rear view" and "forward view".

Rear view refers to project status to date. "Forward view" refers to expectations in the future.

Items (2a) through (2d) and Items (3a) through (3b) will be reported in the context of the "forward view" only.

Notes:

^ See attached description of scale.

* Explanation not required when "green" or "low".

1a Total Cost (Budget) Green = actual is (is expected to be) at or under approved budget baseline, as reported on Project Varience Report.

Yellow = actual is (is expected to be) within 0 to 15% above approved budget baseline, as reported on Project Varience Report.

Red = actual is (is expected to be) more than 15% above approved budget baseline, as reported on Project Varience Report.

1b Schedule Green = project completion is (is expected to be) at or before approved schedule baseline, as reported on Project Varience Report.

Yellow = project completion is (is expected to be) delayed with project duration 0 to 15% above approved schedule baseline, as reported on Project Varience Report.

Red = project completion is (is expected to be) delayed with project duration more than 15% above approved schedule baseline, as reported on Project Varience Report.

1c Scope Green = all deliverables fulfill (are expected to fulfill) stated requirements.

Yellow = at least one deliverables does not fulfill (is not expected to fulfill) stated requirements.

Red = more than one deliverables do not fulfill (are not expected to fulfill) stated requirements.

1d Resource Green = All resources needed by internal and contractor staff are (are expected to be) available.

Yellow = Almost all resources needed by internal and contractor staff are (are expected to be) available.

Red = Almost all resources needed by internal and contractor staff are (are expected to be) unavailable.

1e Deliverables (Work Products) Quality (Work Products include all planning and design artifacts.)

Green = All deliverables are (are expected to be) "fit to use".

Yellow = at least one deliverables is (is expected to be) not "fit to use".

Red = more than one deliverable is not (is not expected to be)"fit to use".

2a Technology Low = all technology employed (including development and…