Attachment_K___________Engineering_Design_Review-_Technical_Stage_Gates_1A_and_1B_Process_Description.pdf

Attachment K Task Order: 0003 Engineering Design Review- Technical Stage Gates 1A and

1B Process Description

Version 2.0 ● 6/27/2016

Document Number: FSA_TO_PROC_TSS.DSGNRVW_001

Draft

Attachment K Task Order: 0003 Engineering Design Review- Technical Stage Gates 1A and 1B Process DescriptionDocument Version Control

Version: 2.0 ii 6/27/2016

Document Version Control

VERSION DATE AUTHOR DESCRIPTION

1.00 11/12/2014 FSA Initial Release

1.1 06/06/2016 FSA Updates to include: removal of User Interface Specification (UI Spec) references since the TQC assessment for the UI Spec has been retired; adding the Technical Objectives assessment

2.0 06/27/2016 FSA Revised Process to include Design Technical Objectives Assessments

Attachment K Task Order: 0003 Engineering Design Review- Technical Stage Gates 1A and 1B Process DescriptionTable of Contents

Table of Contents

SECTION 1. INTRODUCTION

1.1. PURPOSE

1.2. SCOPE

1.3. BACKGROUND

1.4. INTENDED AUDIENCE

1.5. DOCUMENT ORGANIZATION

1.6. REFERENCES AND RELATED DOCUMENTS

1.7. APPLICABLE STAGE GATE LINKS

SECTION 2. KEY ROLES AND RESPONSIBILITIES

SECTION 3. TECHNICAL STAGE GATE PROCESSES

3.1. OVERVIEW

3.2. STAGE GATE INTAKE PROCESS

3.3. STAGE GATE REVIEW PROCESS

3.4. STAGE GATE CLOSEOUT PROCESS

SECTION 4. TECHNICAL QUALITY CONTROL (TQC) ASSESSMENT

4.1. OVERVIEW

4.2. WHEN IS A TQC ASSESSMENT REQUIRED?

New IT Systems ................................................................................................................................... 11 4.2.1.

Other IT Systems ................................................................................................................................. 12 4.2.2.

4.3. THE TQC SCORING TEMPLATE

Tailoring the Scoring Template............................................................................................................. 12 4.3.1.

TQC Scoring......................................................................................................................................... 13 4.3.2.

4.4. PRELIMINARY DESIGN REVIEW (PDR) - TQC PROCESS

Overview and Objectives ...................................................................................................................... 14 4.4.1.

PDD TQC Assessment Focus Areas .................................................................................................... 14 4.4.2.

PDR TQC Review Process .................................................................................................................. 15 4.4.3.

4.5. DETAILED DESIGN REVIEW (DDR) - TQC PROCESS

Overview and Objectives ...................................................................................................................... 17 4.5.1.

DDR Assessment Materials ................................................................................................................. 18 4.5.2.

DDD TQC Assessment Focus Areas ................................................................................................... 18 4.5.3.

DDR TQC Review Process .................................................................................................................. 19 4.5.4.

APPENDIX A - ACRONYMS AND ABBREVIATIONS ............................................................................................ A-21

APPENDIX B - GLOSSARY .................................................................................................................................... B-22

APPENDIX C - PDD ASSESSMENT CRITERIA ..................................................................................................... C-24

C.1 PRELIMINARY DESIGN STANDARDS COMPLIANCE ASSESSMENT CRITERIA .............................................................. C-24 C.2 PRELIMINARY DESIGN TECHNICAL OBJECTIVES ASSESSMENT CRITERIA ................................................................ C-25 C.3 PRELIMINARY DESIGN QUALITY OF DESIGN AND FSA TECHNICAL STANDARDS COMPLIANCE ASSESSMENT CRITERIA C-28

APPENDIX D - DDD ASSESSMENT CRITERIA ..................................................................................................... D-29

D.1 DETAILED DESIGN STANDARDS COMPLIANCE ASSESSMENT CRITERIA ................................................................... D-29 D.2 DETAILED DESIGN TECHNICAL OBJECTIVES ASSESSMENT CRITERIA ..................................................................... D-31 D.3 DETAILED DESIGN QUALITY OF DESIGN AND FSA TECHNICAL STANDARDS COMPLIANCE ASSESSMENT CRITERIA ...... D-34

APPENDIX E - THE TQC REVIEW PACKAGE....................................................................................................... E-35

E.1.1 Slide 1- Cover Page .............................................................................................................................. E-35 E.1.2 Slide 2- Agenda ..................................................................................................................................... E-36 E.1.3 Slide 3- Score Sheet Legend................................................................................................................. E-36 E.1.4 Slide 4- Review Team & Scores ............................................................................................................ E-37 E.1.5 Slide 5- Assessment Summary Report .................................................................................................. E-37 E.1.6 Slides 6 & 7- Questions & Attachments ................................................................................................. E-38

Attachment K Task Order: 0003 Engineering Design Review- Technical Stage Gates 1A and 1B Process DescriptionList of Figures

List of Figures

Figure 1: Overview of LMM Stages and Stage Gates Figure 2: Preliminary Design TQC Scoring Template- Contracted View Figure 3: Detailed Design TQC Scoring Template- Contracted View

List of Tables

Table 1: Intended Audience Table 2: References and Related Documents Table 3: List of Applicable Stage Gate Links Table 3: Key Process Roles and Responsibilities Table 4: Stage Gate Intake Process Activities Table 5: Stage Gate Review Process Activities Table 6: Stage Gate Closeout Process Activities Table 7: TQC Criteria Evaluation Scoring Levels Table 8: TQC Results- Scoring Ranges Table 9: PDD Assessment Focus Areas Table 10: DDR Assessment Materials Table 11: DDR Assessment Focus Areas

Table A-1: Acronyms and Abbreviations .................................................................................................................. A-21 Table B-1: Glossary ................................................................................................................................................. B-23 Table C-1.1: Preliminary Design Standard Compliance Assessment Criteria Elements 1.0–1.14 ........................... C-24 Table C-1.1: Preliminary Design Technical Objectives Assessment Criteria Elements 1.0–1.07 ............................. C-25 Table C-2.2: Preliminary Design Technical Objectives Assessment Criteria Elements 1.07.01–1.12.03 ................. C-26 Table C-3.3: Preliminary Design Technical Objectives Assessment Criteria Elements 1.12.04–1.17.02 ................. C-27 Table C-1.4: Preliminary Design Quality of Design & FSA Technical Standards Compliance Assessment Criteria. C-28 Table D-1.1: Detailed Design Standard Compliance Assessment Criteria Elements 1.0–1.09.02 ........................... D-29 Table D-2.2: Detailed Design Standard Compliance Assessment Criteria Elements 1.09.03–1.14.01 .................... D-30 Table D-1.3: Detailed Design Technical Objectives Assessment Criteria Elements 1.0–1.07.01 ............................ D-31 Table D-2.4: Detailed Design Technical Objectives Assessment Criteria Elements 1.07.02–1.12.05...................... D-32 Table D-3.5: Detailed Design Technical Objectives Assessment Criteria Elements 1.12.06–1.17.02...................... D-33 Table D-1.5: Detailed Design Quality of Design & FSA Technical Standards Compliance Assessment Criteria ..... D-34

Attachment K Engineering Design Review- Technical Stage Gates 1A and 1B Process Description Introduction

Version: 2.0 1 6/27/2016

Section 1. Introduction

Engineering Design Review Technical Stage Gates are comprehensive quality reviews of Federal Student Aid (FSA) Information Technology (IT) project system design artifacts to include the Preliminary Design Document (PDD) and the Detailed Design Document (DDD). Stage Gates are key decision control points for IT project oversight, and an integral part of FSA Lifecycle Management Methodology (LMM) and IT Governance decision making. Aligned with CMMI-Dev Level 3, Stage Gates provide defined, repeatable, and mature processes. The objective of the Engineering Design Review Technical Stage Gates is to provide a “point in time” risk assessment of IT project design artifacts with respect to compliance with FSA design standards, quality of the design, and compliance to FSA architectural technical standards templates. The assessments, aka Technical Quality Control (TQC) reviews, employ templates that are tailored to the complexity of the project and the technology being implemented. When completed, the results of the reviews are provided to Integrated Project Teams (IPTs) for review and comment and to the Engineering Review Board (ERB) for IT Governance decision-making.

There are two Engineering Design Review Technical Stage Gates. Technical Stage Gate 1A (TSG1A), the Preliminary Design Review (PDR), encompasses the TQC of the PDD. Technical Stage Gate 1B (TSG1B), Detailed Design Review (DDR), encompasses the TQC of the DDD. For the purpose of clarity, and to avoid confusion, throughout this document the use of TSG1A and TSG1B refers to the “gates” in the LMM, the term Stage Gate is synonymous with the PDR and DDR control processes executed at those gates, and TQC is used to refer to the design artifact assessments.

1.1. Purpose

The purpose of this document is to provide effective guidance to FSA IT project teams, stakeholders, and other individuals involved in the development and review of FSA IT system design artifacts. It is also to provide awareness into the roles and responsibilities for conducting associated process activities and to identify the FSA standards and exemplars for developing complete and comprehensive system design artifacts for TQC assessments.

1.2. Scope

As determined in FSA’s Lifecycle Management Methodology (LMM) and Engineering Review Board (ERB) processes, IT projects may be subject to Engineering Design Reviews- TSG1A and TSG1B. This document defines the FSA approach used by system development and engineering support teams for conducting the activities to ensure the technical quality and efficacy of IT system design artifacts, specifically the Preliminary Design Document (PDD) and the Detailed Design Document (DDD), which are developed from templates that define FSA standardized design formats. In addition, it is the objective of this document to:

Describe the Technical Stage Gate Process and execution activities

Identify the roles and responsibilities of Design Stage Gate participants

Identify the available FSA online resources for producing IT system design artifacts

Describe the TQC review method employed for assessing FSA IT system design artifacts

Describe the TQC review method employed for non-standard format design artifacts

Describe the PDR and DDR TQC activities

1.3. Background

LMM, FSA’s project delivery and oversight methodology is required for all new IT projects. Structured to identify and mitigate solution development risks early in an IT project lifecycle, the LMM approach to solution delivery consists of seven project stages as shown in Figure 1 below.

Implemented at key points throughout the development lifecycle are Management and Technical oversight control processes, known as Stage Gates. Stage Gates are established key milestones in the LMM lifecycle used to ensure the success of a project. Essentially, a project does not move from one

Attachment K Engineering Design Review- Technical Stage Gates 1A and 1B Process Description Introduction

Version: 2.0 2 6/27/2016 stage to the next stage until the appropriate Stage Gate Review Body has deemed the activities and artifacts produced by an IPT for a review are sufficient for project continuation.

Figure 1: Overview of LMM Stages and Stage Gates

Early in the LMM, the LMM Tailoring process takes place at which time a determination is made of whether an IT project is required to pass through TSG1A and TSG1B. If TSG1A and TSG1B are required, this requirement is documented in the project LMM Tailoring Plan and then the Design Stage Gate control processes, PDR and DDR, comprised of TQC reviews / assessments of the appropriate IT system design artifacts are to be conducted.

FSA implements standardized design artifact templates for IT projects to develop their PDD and DDD.

Therefore, for TSG1A the PDR consists of a TQC review of the PDD and for TSG1B the DDR consists of a TQC review of the DDD, for compliance in FSA standardized formats. For projects that have design artifacts in non-standardized formats, TQC assessments are performed utilizing Design Technical Objectives criteria to assess the technical quality and efficacy of IT system design artifacts.

It is incumbent on the IT System PMs to initiate the Technical Stage Gate Process with an email request sent to the Design Stage Gate Owner at LMMDesignStageGate@ed.gov. For TSG1A and 1B, the TQC reviews are efforts coordinated by the Design Stage Gate Owner and performed by the Design Stage Gate Review Team (aka TQC Review Team), a team comprised of members of the Enterprise IT Architecture and Strategic Infrastructure Group (EITASIG) and other independent non-project related relevant technical SMEs. TQC reviews of design artifacts may also be initiated at any point in a system’s lifecycle, through direction from the ERB or requests from the IT System PM, to provide input into FSA IT Governance decision making processes.

1.4. Intended Audience

The table below lists intended users and the purpose for which they may utilize the information in this document:

INTENDED AUDIENCE USES

FSA Management Provides guidance to management on participation and sign-off responsibilities in the Engineering Design Review Process

FSA Engineering Review Board

(ERB)

Provides guidance to the ERB on IT design artifact development and stage gate processes prior to an ERB Review of the project

FSA Integrated Project Teams (IPTs) and IT System Project Managers (PMs)

Provides guidance to project team on FSA IT system design artifact collaborative development activities, the review requirements for Engineering Design Reviews, TQC assessments, and ERB governance processes mailto:LMMDesignStageGate@ed.gov

Attachment K Engineering Design Review- Technical Stage Gates 1A and 1B Process Description Introduction

Version: 2.0 3 6/27/2016

INTENDED AUDIENCE USES

Design Stage Gate Review Team

Provides guidance to the team of non-project related subject matter experts on FSA IT system design artifact collaborative development activities, the review requirements for Engineering Design Reviews, TQC assessments, and ERB governance processes

System Support Contractors Provides guidance to contractors on the Engineering Design Review, TQC, and governance processes in order to support FSA IT system design development and review activities

Table 1: Intended Audience

1.5. Document Organization

This document comprises the following sections:

Section 1 – Introduction: describes the reason and background for this document, the intended audience and applicable reference documents and links

Section 2 – Key Roles and Responsibilities: introduces and describes the roles and responsibilities of the participants in the Engineering Design Review Technical Stage Gate Process

Section 3 –Technical Stage Gate Process: Provides an overview of TSG1A and TSG1B in the context of the LMM Technical Stage Process, describes the Technical Stage Gate Process, and describes the activities performed in each sub process

Section 4 – Technical Quality Control (TQC) Assessment- provides an overview of the TQC, the general criteria of when a TQC is to be performed, the scoring methodology employed, the objectives for a performing Preliminary Design Review (PDR) and a Detailed Design Review (DDR) using TQC and the high level activities to perform a TQC review

Appendix A – Acronyms and Abbreviations: provides a list of acronyms and abbreviations

Appendix B – Glossary: defines terminology within the context of this process

Appendix C – PDD Assessment Criteria- detailed criteria found in a non-tailored TQC template and also includes the Technical Objectives assessment criteria

Appendix D – DDD Assessment Criteria- detailed criteria found in a non-tailored TQC template and also includes the Technical Objectives assessment criteria

Appendix E – The TQC Review Package- describes the PowerPoint presentation that summarizes the results of a TQC assessment in a Risk Assessment Summary Report for presentation to the Engineering Review Board (ERB)

1.6. References and Related Documents

The following documents are FSA online resources applicable to the Engineering Design Review- Technical Stage Gates 1A and 1B Processes. This list includes the FSA standards and exemplars for developing complete and comprehensive system design artifacts.

DOCUMENT T ITLE AND VERSION

DOCUMENT

DESCRIPTION

Lifecycle Management Methodology (LMM), Federal Student Aid- Version 1.2, 06/15/2012

Describes in detail FSA’s IT Lifecycle Management Methodology

Lifecycle Management Methodology Stage Gate Review Process Description - Version 1.2, 06/15/2012

Describes at a high level each of the LMM Stage Gate review processes

FSA Engineering Review Board Charter, March 2012 Charter that defines the purpose, authority, membership, and roles and responsibilities of the

ERB

Engineering Review Board Process Description Final, Federal Student Aid- Version 1.4, 03/16/2012

Descriptions of the ERB review workflows.

Attachment K Engineering Design Review- Technical Stage Gates 1A and 1B Process Description Introduction

Version: 2.0 4 6/27/2016

DOCUMENT T ITLE AND VERSION

DOCUMENT

DESCRIPTION

TASS Requirement Standards Overview, Federal Student Aid- Version 2.02, 06/20/2014

A guidance document that provides an overview of FSA’s IT system requirements documentation and includes an overview of the User Interface Specification

FSA Design Standards Guidance, Preliminary Design Document (PDD) and the Detailed Design Document (DDD), Federal Student Aid- June 2014

A guidance document that provides an overview of FSA’s IT system design standards documentation the PDD and the DDD

Preliminary Design Document (PDD) Template, Federal Student Aid- Version 1.5, 3/22/2013

FSA standard template used for producing an IT system Preliminary Design Document. (includes user guidance)

Preliminary Design Document (PDD) Exemplar, Federal Student Aid- Version 3.4, 4/4/2013

An example of a completed Preliminary Design Document for use as reference by the IPT

Detailed Design Document (DDD) Template, Federal Student Aid- Version 1.7, 02/6/2013

FSA standard template used for producing an IT system Detailed Design Document. (includes user guidance)

Detailed Design Document (DDD) Exemplar, Federal Student Aid- Version 3.2, 04/11/2012

An example of a completed Detailed Design Document for use as reference by the IPT

TQC Scoring Spreadsheet for UI, PDD, & DDD Artifacts, Federal Student Aid- Version 3.0, 12/22/2010

A spreadsheet used in performing a TQC assessment of a UI Spec, PDD, or DDD.

User Interface/ Preliminary/Detailed Design TQC Review Package

A briefing template used to capture and consolidate the results of a TQC for use by the IPT and ERB review.

Technical Stage Gate Design Review, ERB Briefing, Federal Student Aid

A briefing template used by the IPT to capture and consolidate the results of a TQC and project status for an ERB review.

Enterprise Architecture Program, BPA #ED-FSA-13-A- 0007 Goals, Objectives, Qualities and Validation Scenarios- Version 00.00.10, 9/8/2015

A document written to support FSA application system procurements and create a means for evaluating alternative architectures by increasing consideration of solution architecture in those procurements

Table 2: References and Related Documents

1- These documents are located online on the FSA LMM Library (see Section 1.7 Applicable Stage Gate Links).

2- These documents are located online as elements the Technology Office Standards Library (TOSL) (see Section 1.7 Applicable Stage Gate Links).

1.7. Applicable Stage Gate Links

The following table provides a list of relevant Technical Stage Gate Process links:

DESCRIPTION L INK

FSA LMM Library https://fsa.share.ed.gov/lmm/SitePages/LMM%20Welcome.aspx

Technology Office Standards Library https://fsa.share.ed.gov/TOSL/SitePages/Home.aspx

Design Stage Gate Outlook Mailbox LMMDesignStageGate@ed.gov

ERB Design Stage Gate Document Library https://fsa.share.ed.gov/teams/to/DCIO/ERB/Design%20Stage%20G ate%20Documents/Forms/AllItems.aspx

ERB Design Stage Gate Summary Report https://project.share.ed.gov/EPPM/StageGateSummaryProgressRep ort/Forms/AllItems.aspx

Table 3: List of Applicable Stage Gate Links https://fsa.share.ed.gov/lmm/SitePages/LMM%20Welcome.aspx https://fsa.share.ed.gov/TOSL/SitePages/Home.aspx mailto:LMMDesignStageGate@ed.gov https://fsa.share.ed.gov/teams/to/DCIO/ERB/Design%20Stage%20Gate%20Documents/Forms/AllItems.aspx https://fsa.share.ed.gov/teams/to/DCIO/ERB/Design%20Stage%20Gate%20Documents/Forms/AllItems.aspx https://project.share.ed.gov/EPPM/StageGateSummaryProgressReport/Forms/AllItems.aspx https://project.share.ed.gov/EPPM/StageGateSummaryProgressReport/Forms/AllItems.aspx

Attachment K Engineering Design Review- Technical Stage Gates 1A and 1B Process DescriptionKey Roles and Responsibilities

Version: 2.0 5 6/27/2016

Section 2. Key Roles and Responsibilities

The following list identifies the Key Roles and Responsibilities of the participants in the Engineering Design Review- Technical Stage Gates 1A and 1B processes:

ROLES RESPONSIBILITIES

FSA LMM Tailoring Team

Quality Assurance (QA)

Enterprise Project Management and Oversight (EPMO) Group

Technology Office Application Development Group (ADG) Project Management Office

Assists new project teams tailoring LMM compliance expectations based on project size, scope and complexity.

Validates LMM compliance expectations are established early for new project planning, are documented and are updated as the project progresses. This includes establishing the requirement for new projects to pass through stage gates TSG1A and TSG1B for TQC review of system design artifacts.

FSA Design Stage Gate Owner

Coordinates and oversees execution of Design Stage Processes

Coordinates TQC reviews with Design Stage Gate Review Team, subject matter experts and IPT

Produces ERB presentation with Risk Assessment Summary and Review package with recommendations and provides it to IPT and ERB

IT System Project Mangers (PMs)

Initiates Design Stage Gates with request to Design Stage Gate Owner

Coordinates with Stage Gate Owner to establish schedule for TQC review

Coordinates with Stage Gate Owner for Technical Objectives TQC template tailoring, reviews, and feedback during design document reviews

Performs system design self-assessment with Technical Objectives

TQC

FSA Integrated Project Teams (IPTs) and Contractors

Develops and updates IT system design artifacts

Performs system design self-assessment with Technical Objectives

TQC

Produces IPT Review package and presents to ERB for review

Design Stage Gate Review Team (aka TQC Review Team)

The Technical Stage Gate Review Body for performing TQC assessments of IT System design artifacts, typically made up of members of the Enterprise IT Architecture and Strategic Infrastructure Group (EITASIG) and other independent non-project related relevant technical SMEs

Reviews IT design artifacts and provides subject matter expertise and guidance to IPT

Participates in Technical Quality Control reviews as requested

FSA Engineering Review Board

(ERB)

Reviews TQC assessments, and recommendations

Makes project executive decisions and recommendations, including recommending “go/no-go” of projects.

Initiates a TQC assessment of systems as required for IT governance and decision-making processes

FSA Investment Review Board

(IRB)

Reviews ERB assessments and IPT Technical Stage Gate Design Review Briefings

Makes project executive decisions regarding project terminations

Subject Matter Experts (SMEs)

Participates in Technical Quality Control reviews as requested

Reviews IT design artifacts and provides subject matter expertise and guidance to IPT

Table 4: Key Process Roles and Responsibilities

Attachment K Engineering Design Review- Technical Stage Gates 1A and 1B Process DescriptionTechnical Stage Gate Processes

Version: 2.0 6 6/27/2016

Section 3. Technical Stage Gate Processes

3.1. Overview

The Technical Stage Gate Processes provide a “point in time” risk assessment of IT project design artifacts with respect to compliance with FSA design standards, quality of the design, and compliance to FSA architectural technical standards templates.

For new IT projects, the FSA LMM Tailoring Team establishes the requirement for a project to develop IT system design documentation using FSA standardized formats, then to pass through Technical Stage Gates TSG1A and TSG1B and have a TQC review of system design artifacts performed. The requirement for the Technical Stage Gates is documented and communicated via the project’s LMM Tailoring Plan.

For these Stage Gates, the TQC reviews have an emphasis on compliance with FSA standards.

For other IT projects, such as legacy systems, TQC reviews may also be initiated at any point in a system’s lifecycle. These projects may have design artifacts in non-standardized formats. For reviews of IT system design artifacts in non-standardized formats, the TQC assessment employs a set of Technical Objectives criteria to assess the technical quality and efficacy of IT system design artifacts.

The Technical Stage Gate Process is comprised of three sub processes:

The Stage Gate Intake Process– This process encompasses activities required to initiate, schedule, and track the Technical Stage Gate reviews

The Stage Gate Review Process- This process entails the activities required to coordinate and perform the TQC assessment of a project’s IT system design artifacts

The Stage Gate Closeout Process- This process encompasses the activities to aggregate the results of the TQC reviews into a Risk Assessment report and present the findings and a recommendation to the

ERB

3.2. Stage Gate Intake Process

It is the responsibility of the IT System PM or the IPT Lead to initiate the Stage Gate Intake Process.

The general requirements for entry into the Stage Gate Intake Review Process consist of:

An email request completed and submitted by the IT System Project Manager to the Design

Stage Gate mailbox at LMMDesignStageGate@ed.gov requesting a Technical Stage Gate review. The email will contain the type of review requested, the project name, release, date of release and when the stage gate needs to be completed

The email request must be made at least 12 weeks prior to the requested date of completion

Completed IT system design artifacts (PDD or DDD) and any supporting documentation

Activity

Name Responsible Group

Descript ion

1 Start Integrated Project Team

1. 12 weeks in advance of the needed completion date, the IPT System PM or IPT Lead submits a request to the Design Stage Gate mailbox at.

LMMDesignStageGate@ed.gov. The email will contain the type of review requested, the project name, release, date of release and when the stage gate needs to be completed. The request will contain electronic copies of all design artifacts within the scope of the request.

2 Receive Request

Design Stage Gate Owner

1. The email system will generate an automatic reply to the Requestor and the Design Stage Gate Owner acknowledging receipt of the request.

mailto:LMMDesingStageGate@ed.gov

Attachment K Engineering Design Review- Technical Stage Gates 1A and 1B Process DescriptionTechnical Stage Gate Processes

Version: 2.0 7 6/27/2016

Activity

Name Responsible Group

Descript ion

2. The Design Stage Gate Owner will create an Outlook folder in the Design Stage Gate mailbox named after the project. This folder will then serve to store and manage any email correspondence associated with this project

3. The Design Stage Gate Owner stores the email and any attachments in the new folder

4. The Design Stage Gate Owner will update the ERB

Design Stage Gate Summary list to include capturing the date the request was received and a status indicating it is currently in the intake status.

5. The Design Stage Gate Owner will create a

SharePoint intake folder in the ERB Design Stage

Gate Document Library and place copies of the email and the design artifacts received with the request into the folder. (See Section 1.7 for link to Design Stage

Gate Library)

3 Review Request

Design Stage Gate Owner

1. The Stage Gate Owner will review the project request and design artifacts submitted and determine if a

Stage Gate is required.

2. If a Stage Gate is not required:

a. The Stage Gate Owner updates the ERB Design

Stage Gate Summary list to indicate that the stage gate is not required pending ERB approval.

b. The Stage Gate Owner sends an email, from the

Design Stage Gate mailbox, to the ERB coordinator requesting ERB approval of the decision to not perform the stage gate

c. The email is copied to the Outlook folder established for the project’s stage gate correspondence and to the corresponding

SharePoint folder in the ERB Design Stage Gate

Library

d. If the ERB approves the recommendation:

i. The ERB Design Stage Gate Summary list is updated indicating that the stage gate is not required and has been approved by the ERB.

ii. The Design Stage Gate Owner sends an email, from the Design Stage Gate mailbox, to the Requestor confirming the decision of the ERB and stores the email in the corresponding Outlook and SharePoint folders. The process stops here.

3. If a Stage Gate is required or the ERB disapproves the recommendation:

a. The Design Stage Gate Owner sends an email, from the Design Stage Gate mailbox, to the

Requestor confirming that a Stage Gate is required and stores the email in the corresponding

Outlook and SharePoint folders

Attachment K Engineering Design Review- Technical Stage Gates 1A and 1B Process DescriptionTechnical Stage Gate Processes

Version: 2.0 8 6/27/2016

Name Responsible Group

Descript ion

4. The process continues to the next step, the Stage

Gate Review Process

Table 5: Stage Gate Intake Process Activities

3.3. Stage Gate Review Process

Activity

Name Responsible Group

Descript ion

1 Prepare Design Stage Gate Review

Design Stage Gate Owner

1. The Design Stage Gate Owner identifies the TQC

Review Team, notifies them via email from the Design

Stage Gate mailbox, and stores the email in the

Outlook folder

2. The Design Stage Gate Owner prepares the scoring and criteria selection in the TQC Scoring Template based on the requirements of the artifacts to be reviewed. Detailed instructions for scoring and criteria selection are provided in the TQC assessment tool

3. The “master copy” of the prepared TQC Scoring

Template is stored in the project folder of the ERB

Design Stage Gate Library

Perform TQC Technical Objectives Review (as required)

Design Stage Gate Owner and Integrated Project Team

1. If the design artifacts to be reviewed are not in the

LMM standard:

a. An email is sent to the IT System PM, from the Design Stage Gate mailbox, with a TQC Scoring Template attached that has been prepared by the Design Stage Gate Owner for a Technical Objectives review. The PM is instructed to forward the TQC Scoring Template to the project team to complete the required project specific Technical Objective categories and return it when complete.

The email is filed in the Outlook folder created for this project

b. The ERB Design Stage Gate Summary list is updated to indicate that the review is pending the project’s team mapping to the TQC Technical Objectives standard categories

c. When the TQC Technical Objectives mapping document is received, it is reviewed by the Design Stage Gate Owner for completeness. The email is filed in the outlook folder created for this stage gate and a copy of the mapping document is stored in the corresponding ERB SharePoint folder

d. The ERB Design Stage Gate Summary list is updated to indicate that the review is complete for the project’s team mapping to the TQC Technical Objectives standard categories

2 Perform Design Stage Gate Review

Design Stage Gate Owner and TQC Review Team

1. If the design artifacts to be reviewed are in the LMM standard or if the TQC Technical Objectives Review is complete by the project team:

a. The Design Stage Gate Owner notifies the TQC

Review Team, via email from the Design Stage

Gate mailbox, of the pending Stage Gate and the

Attachment K Engineering Design Review- Technical Stage Gates 1A and 1B Process DescriptionTechnical Stage Gate Processes

Version: 2.0 9 6/27/2016

Name Responsible Group

Descript ion date the review completion is required. The email contains a copy of all documents (or links to documents) necessary for the TQC review. The email is filed in the Outlook folder created for this project

b. The ERB Design Stage Gate Summary list is updated to indicate that the TQC review is “in process” with the projected date of completion

2. The TQC Review Team performs TQC reviews and completes the TQC Scoring Template following the steps indicated in Section 4 of this document.

a. The Design Stage Gate Team provides their feedback and completed assessments back to the

Design Stage Gate Owner via the Design Stage

Gate mailbox, LMMDesignStageGate@ed.gov. All emails are stored in the Outlook folder created for this project. Individual TQC assessments are stored in the corresponding project folder of the

ERB Design Stage Gate Library

3. The process continues to the next step, the Stage

Gate Closeout Process

Table 6: Stage Gate Review Process Activities

3.4. Stage Gate Closeout Process

The general exit requirements from the Stage Gate Closeout Process for TSG1 and TSG2 consist of:

A completed TQC review of IT system design artifacts by the TQC Review Team

A completed ERB project Risk Assessment Summary presentation

A completed Engineering Review Board (ERB) Review Meeting

A documented ERB determination / decision

Name Responsible Group

Descript ion

1 Consolidate TQC Results / Develop ERB Presentation

Design Stage Gate Owner

1. When all TQC assessments have been received from the TQC Review Team, the ERB Design Stage

Gate Summary list is updated to indicate that the

TQC review is “Complete- ERB Results in process” with the projected date of completion.

2. The Design Stage Gate Owner reviews the TQC

Assessments received to ensure the assessments and comments are completed.

3. The Design Stage Gate Owner then compiles the results of the TQC reviews consolidating the comments and prepares a draft ERB presentation.

The draft presentation is stored in the corresponding

Attachment K Engineering Design Review- Technical Stage Gates 1A and 1B Process DescriptionTechnical Stage Gate Processes

Version: 2.0 10 6/27/2016

Name Responsible Group

Descript ion project folder of the ERB Design Stage Gate Library

4. The Design Stage Gate Owner distributes the draft

ERB presentation to the TQC Review Team, via the

Design Stage Gate mailbox, and schedules a conference call with the team to review the results and obtain consensus on the results.

5. The Design Stage Gate Owner updates the draft

ERB presentation based on feedback from the TQC

Review Team and distributes the revised draft, via the Design Stage Gate mailbox, to the TQC Review

Team and the IT System PM for review and feedback. The email is saved in the Outlook project folder and the revised draft is saved in the project

ERB SharePoint folder. Any comments or feedback received are filed in the Outlook project folder

6. The Design Stage Gate Owner prepares a final version of the ERB presentation. The final version is sent to the ERB coordinator, via the Design Stage

Gate mailbox, for distribution to the ERB and scheduling of the ERB presentation. The IT System

PM and the TQC Review Team are copied on the email. The email is saved in the Outlook project folder and the final ERB presentation is saved in the project ERB SharePoint folder

Present Risk Assessment to ERB- Final Decision

Design Stage Gate Owner and the ERB

1. On the scheduled date, The Stage Gate Owner presents the TQC findings (aka Risk Assessment

Summary Report) presentation to the ERB

2. The ERB votes on the recommendation

3. The ERB publishes their Decision Record. The

Decision record is sent to the Design Stage Gate

Owner, via the Design Stage Gate mailbox. The email is filed in the corresponding Outlook project folder created for this Stage Gate and in the ERB

SharePoint folder

4. The Design Stage Gate Owner then updates the

ERB Design Stage Gate Summary list to indicated the Technical Stage Gate is “Complete”

Table 7: Stage Gate Closeout Process Activities

Attachment K Engineering Design Review- Technical Stage Gates 1A and 1B Process DescriptionTechnical Quality Control (TQC) Assessment

Version: 2.0 11 6/27/2016

Section 4. Technical Quality Control (TQC) Assessment

4.1. Overview

A TQC assessment is performed by a team of qualified design SMEs that are not part of the implementation team, the Design Stage Gate Review Team (more commonly known as the TQC Review Team). This impartial team of subject matter experts uses a TQC Scoring Template to assess the technical quality and efficacy of the IT system design artifacts for each IT project based on the complexity of the project and the implemented technology. Essentially the team evaluates if the design artifacts can stand on their own merit.

FSA employs the TQC framework to ensure that FSA IT system designs:

Are viable, complete, and well documented

Document Key Design Decisions

Address both functional and non-functional requirements

Comply with FSA design documentation standards

Comply with FSA technical standards

Provide appropriate use of Federal Student Aid infrastructure

Support code and unit test activities

Meet FSA Design Technical Objectives

FSA TQC assessments enforce quality control through review processes and design artifact inspection during:

Preliminary Design Review (PDR)

Detailed Design Review (DDR)

Additionally, FSA TQC assessments are used to evaluate the “current state” of IT system design artifacts in meeting FSA Design Technical Objectives. This can occur at any point within a system’s lifecycle through the Technical Objectives review mechanism. Examples include legacy systems presently deployed in Production and systems with design artifacts in non-standardized formats.

The output from a TQC assessment includes:

TQC Review Package / Risk Assessment Summary Report in an ERB presentation

4.2. When is a TQC Assessment Required?

New IT Systems 4.2.1.

Each new FSA IT system projects must follow the LMM. During the project’s LMM Tailoring process a determination is made of whether the project is required to pass through the Engineering Design Review Technical Stage Gates TSG1A and TSG1B. If the Stage Gates are required, then TQC assessments of the appropriate IT system design artifacts will be conducted.

4.2.1.1. TQC and the LMM Tailoring Plan

The LMM Tailoring Plan defines the lifecycle process and required artifacts for each new IT project. The Tailoring Plan is key to determining how system development and engineering support teams will conduct their respective activities towards successful project delivery. Smooth and successful Design Stage Gate reviews have a significant dependency on well-defined LMM Tailoring Plans. Therefore, it is imperative for an IT System PM to engage the Stage Gate Owner as early as possible in the project’s LMM Tailoring process to identify and plan for the Design Stage Gate reviews and TQC assessment criteria. The following are the example common criteria used to determine if a TQC Assessment of a project’s IT system design artifacts is required. However, these criteria are changing as FSA strives to improve its IT systems governance processes and audit posture. The criteria include:

Attachment K Engineering Design Review- Technical Stage Gates 1A and 1B Process DescriptionTechnical Quality Control (TQC) Assessment

Version: 2.0 12 6/27/2016

All Tier 1 projects as determined from LMM Tailoring by the FSA LMM Tailoring Team and documented in the project LMM Tailoring Plan

IT projects that are not aligned with the Target State Vision

IT projects introducing non-standard or new technology or software (including COTS) into FSA’s environment

IT projects introducing custom coding

IT projects that are hosted outside of the Virtual Data Center

Tiers 1, 2, or, 3 projects, as specified by the ERB or the FSA LMM Tailoring Team

Equally as important a TQC Assessment of a project’s design artifacts is typically not required if:

The project results in adding functionality to an existing application

The project simply utilizes existing toolsets and existing infrastructure

The project is adding requirements to an existing application

The project is implementing requirements using an existing technology platform like SharePoint

The project is regarding moving from one vendor to another

The project is regarding moving applications within the current infrastructure

The project is regarding hardware implementation

The project is regarding system upgrades

Other IT Systems 4.2.2.

For other IT systems, i.e. systems that are not “new” such as legacy Production systems or systems that may have design documentation in non-standardized formats, a TQC review called the Technical Objectives assessment is performed. The Technical Objectives TQC assessment may be performed at any point in an IT system’s lifecycle and is typically initiated through a directive issued to the Design Stage Gate Owner from the Engineering Review Board (ERB) or a request made from an IT System PM.

4.3. The TQC Scoring Template

The TQC Scoring Template is the tool used by the TQC Review Team in applying standard assessment criteria for determining the quality and completeness of submitted system design artifacts. The scoring template is flexible in design and is tailored by the Design Stage Gate Owner to accommodate the TQC assessment criteria requirements of a particular IT project. In addition, the template employs weighting and scoring level factors to generate a risk score. Each deficiency found will be documented along with the severity to aid in communication back to the IPT and the ERB.

Please refer to Appendixes C and D for the following sections. Appendixes C and D provide examples of the detailed assessment criteria for the PDD and the DDD respectively including the Technical Objectives assessment criteria for each.

Tailoring the Scoring Template 4.3.1.

The PDD and DDD design document templates are generic in scope and try to accommodate the full spectrum of architectures. If LMM Tailoring indicates a project must pass through the Design Stages Gates, the IT System PM engages with the Stage Gate Owner to determine FSA’s requirements for the project design documents. This is so the Stage Gate Owner can generate a tailored TQC template for evaluating the design artifacts and establishing assessment criteria elements that match the business requirements and the design goals.

For design artifacts in non-standardized formats, the process is similar for a Technical Objectives assessment for a system. The IT System PM engages with the Design Stage Gate Owner to determine FSA’s requirements for the review and to establish the design criteria to be evaluated. The Design Stage Gate Owner generates a tailored template and provides it to the IT System PM. The tailored template will

Attachment K Engineering Design Review- Technical Stage Gates 1A and 1B Process DescriptionTechnical Quality Control (TQC) Assessment

Version: 2.0 13 6/27/2016 be used by the PM to perform a self-assessment of the system’s design artifacts that is then used by the TQC Review Team in performing a Technical Objectives TQC.

TQC Scoring 4.3.2.

4.3.2.1. Weighting

To highlight areas of importance (and infer higher risk if not complied with), the criteria are weighted in two different ways. Each high level section is given an overall weight with each criteria sub-section given a referential weight ranging from 0 to 4. These weighting scores are based on what the design SMEs matter experts feel are the highest priority at each review cycle.

4.3.2.2. Scoring Levels

The TQC Review Team evaluates and scores each design criteria element. To score each element, the evaluator will score on a scale of 0 - 4. Each score corresponds to a deficiency level and is color coded to provide visual cues in the tool as indicated below:

Scoring 0 = Non-compliant

1 = Critical Defect

2 = Major Defect with no known work around

3 = Defect with known work around

4 = Minor or No Defect

Table 8: TQC Criteria Evaluation Scoring Levels

If the evaluator ranks any criteria element below a 4, a note for reasoning along with a citation to the referring document will be put in the Comments/Description field. These comments / justifications will be summarized for the Risk Assessment Summary Report of the ERB presentation.

For Technical Objectives assessments, a comment is captured in the Comments/Description field for each satisfied criteria describing how the criteria is met. It addition, the Criteria Location, document and page number is also captured for reference and use by the TQC Review Team. For criteria not met, the evaluator will select “NIS” (Not in Scope) for that criterion. An NIS yields an automatic score of zero for the selected criteria.

4.3.2.3. Scoring Range

As evaluators enter raw scores in to the scoring worksheet, aggregate scoring is applied with a weighting factor to each criterion to compute a score for each scorecard section. The totals for each section are weighted to give a total score as a percentage of perfect (0-100) scale. The range is also color coded too provide visual cues as indicated below:

Range 95-100 = Low to No Risk

80-95 = Moderate to Low Risk

65-80 = High to Moderate Risk less than 65 = Very High Risk

Table 9: TQC Results- Scoring Ranges

Attachment K Engineering Design Review- Technical Stage Gates 1A and 1B Process DescriptionTechnical Quality Control (TQC) Assessment

Version: 2.0 14 6/27/2016

4.4. Preliminary Design Review (PDR) - TQC Process

Overview and Objectives 4.4.1.

The Preliminary Design Review (PDR) is the first design-level standard system engineering review comprised of the PDD with a TQC assessment. The PDR:

Validates the design concepts with respect to the system requirements

Establishes a baseline architecture and component selection

Validates that the design conforms to the established Federal Student Aid standards

The TQC assessment reviews the high-level design architecture based on criteria derived from the Preliminary Design Document Template and reference architecture models. The TQC results are recorded in a Risk Assessment Summary Report in the ERB presentation delivered to the ERB and Integrated Project Team (IPT) point of contact.

4.4.1.1. PDR Objectives

The PDR reviews the high-level architecture documents, models, and requirements traceability. PDR objectives include:

Review and assess the high-level design document(s)

- Verify solution architecture concept by reviewing the various models and design elements presented.

Verify traceability

- The high-level design concepts and use case models should be traceable to the baseline system requirements.

Validate the technical baseline

- The baseline component-level architecture is validated to go forward to detailed design.

Identify technical risks

- Technology and standards

- Requirements and acceptance criteria ambiguity

- Solution feasibility and continuity

Assess any identified data or infrastructure migration plans

Assess architectural decision logs for assumptions and risk mitigation

Identify dependencies (external dependencies include technology, baselines, interfaces)

Assess test architecture (software and hardware)

Assess non-functional requirements identified and addressed through high-level analysis

Deliver a Risk Assessment Summary Report to the Solutions Delivery Team point of contact that details the strengths, weaknesses, and defects of the architecture.

PDD TQC Assessment Focus Areas 4.4.2.

IPT artifact developers and members of the TQC Review Team should focus on the areas detailed in Table 9 as areas of highlighted interest in the Preliminary Design Document.

Area to Review Evidence Expected Notes

Attachment K Engineering Design Review- Technical Stage Gates 1A and 1B Process DescriptionTechnical Quality Control (TQC) Assessment

Version: 2.0 15 6/27/2016

Area to Review Evidence Expected Notes

Architectural Decisions Listed all applicable Architectural Decisions

1. All decisions or agreed upon design directions should be documented fully as an

Architectural Decision

2. Architectural Decisions should be fully documented with alternatives with a risk weight of choosing the agreed upon path.

Non-Functional Design All areas listed in the Non- Functional Requirements are addressed.

1. Calculations for the feasibility of the Non-

Functional Requirements with a recommend architecture to support the design

2. Documented risks of not meeting the Non-

Functional Requirements if designed architecture cannot meet the requirements.

Architecture Overview Design contains a clear view of the proposed architecture

1. Overall architecture is shown as a model and with descriptive text of the entire architecture at a high-level

2. Integration points are identified internally and externally

3. Architecture is shown in logical components that map to the requirement set categories defined in the System Requirements baseline

Functional Component Model

Design has logical groupings of the architecture

1. Design has logical groupings of like functionality into components

2. Components are mapped to the requirement set categories defined…