Exhibit_3a_TFITS_SEP.pdf

i

TOTAL FORCE INFORMATION

TECHNOLOGY SYSTEMS (TFITS)

SYSTEMS ENGINEERING PLAN

VERSION 1

30 September 2013 ii

SUBMITTED BY

Andrew Mittura

TFITS APdM-Engineer

Date

Mr. William Sandlin

Product Manager

Date iii

Table of Contents

1. Introduction – Purpose and Update Plan

1.1 Approach for SEP Updates

2. Program Technical Requirements

2.1 Requirements Management

2.2 Architectures and Interface Control

2.2.1 Interface Control

2.2.2 Technical Certifications

3. Engineering Resources and Management

3.1 Technical Schedule and Schedule Risk Assessment

3.2 Engineering Resources and Cost/Schedule Reporting

3.3 Engineering and Integration Risk Management

3.4 Technical Organization

3.4.1 Government Program Office Organization

3.4.2 Program Office Technical Staffing Levels

3.4.3 Engineering Team Organization and Staffing

3.5 Relationships with External Technical Organizations

3.6 Technical Performance Measures and Metrics

3.6.1 Usability

3.6.2 Sustainment Assessment

3.6.3 Metrics

4. Technical Activities and Products

4.1 Planned SE Activities

4.1.1 Development Efforts

4.1.2 Life Cycle Engineering

iv

4.2 Requirements Development and Change Process

4.2.1 Requirements Management

4.2.2 Change Procedure

4.3 Technical Reviews

4.3.1 Technical Review Action Plan

4.3.2 Tailored Review Check Lists

4.4 Configuration and Change Management

4.5 Design Considerations

4.6 Engineering Tools

4.6.1 ECP Tracker

4.6.2 Chronotecture

4.6.3 Monthly Health Assessment

5. Annex A – Acronyms

NOTE: All sections above are driven by Section 139b of title 10 United States Code and DoDI 5000.02 policy;

additional content is optional at the discretion of the Component.

Table of Tables

TABLE 1-1: TFITS PROJECTS

TABLE 1-2: SEP UPDATE RECORD (MANDATED) (SAMPLE)

TABLE 2-1: REQUIRED INTERFACE AGREEMENTS

TABLE 3-2: ENGINEERING WORK PACKAGES (SAMPLE FORMAT)

TABLE 3-3: GRADING SCHEME FOR SUSTAINMENT ASSESSMENT

TABLE 3-4: METRIC DEFINITIONS

TABLE 4-1: 4-OCTET VERSIONING SYSTEM

TABLE 4-2: REQUIREMENTS DOCUMENTS

v

Table of Figures

FIGURE 3-1: ENGINEERING IPT RESOURCES

FIGURE 3-2: PROGRAM OFFICE ORGANIZATION

FIGURE 3-3: PMM110, ISI ORGANIZATION

FIGURE 3-4: TFITS TECHNICAL ORGANIZATION

FIGURE 3-5: INFORMATION ASSURANCE PROJECT ALLOCATION

FIGURE 4-1: TAILORED PROCESS DESIGN FLOW

FIGURE 4-2: CHRONOTECTURE DIAGRAM

1. INTRODUCTION – PURPOSE AND UPDATE PLAN

This Systems Engineering Plan (SEP) defines the systems engineering approach for the Total Force Information Technology Systems (TFITS) Product Office projects. The TFITS Product Office provides acquisition oversight for over 24 projects. This SEP is intended to define the systems engineering (SE) policies and processes implemented within the TFITS product office. This SEP applies to all TFITS projects listed in Table 1-1.

Table 1-1: TFITS Projects

Project Acronym Project Acronym Automated Manifest System - Tactical AMS-TAC

Manpower Mobilization Assignment System MMAS

Automated Performance Evaluation System APES Manpower Models MODELS Automated Tool Inventory Control and Tracking System ATICTS

Optical Digital Imaging – Records Management System ODI-RMS

Drill Manager Model DMM Performance Evaluation System PES Defense Messaging System DMS PRBuilder PRB Defense Readiness Reporting System DRRS Radio Frequency Identification RFID Joint Configuration Management Information System JCMIS

Secure Personnel Accountability Module SPA

Joint Engineering Data Management Information and Control System JEDMICS Standard Procurement System SPS Joint Force Requirements Generator JFRG Total Force Data Warehouse TFDW Marine Corps Recruiting Information Support System MCRISS Total Force Retention System TFRS Marine Corps Training Information Management System MCTIMS

Theatre Medical Information Program TMIP

MAGTF Deployment Support System MDSS II WEB Manpower Assignment Support System WEBMASS

As a Systems Engineering (SE) roadmap, the TFITS SEP demonstrates integration with the key project management products, processes and documents to include the overall TFITS Acquisition Strategy Plan (ASAP), the Integrated Master Schedule (IMS), the Configuration Management Plan (CMP), the Software Test and Evaluation Strategy (ST&ES), and the Requirements Management Plan (REQMP). This SEP frames the product group’s SE approach and strategy to provide a firm and well-documented technical foundation for the product commensurate with the level of programmatic maturity at each Milestone (MS). This plan explains how each TFITS project accommodates cost, schedule, performance, and sustainment trades. Also, this document describes the expected products of the SE effort and how these products contribute to project decision-making.

The highlights of this SEP include the following areas:

• Showing traceability to the fulfillment of Capability Development Document (CDD) and Capability

Production Document (CPD) requirements, Engineering Change Proposals, Concepts of Operations, Business Case Analysis, and other approved source requirements documentation

• Describing the TFITS Integrated Product Team (IPT) structures

• Detailing processes for managing project baselines and conducting TFITS Systems Engineering Technical

Reviews (SETRs)

• Describing the methods for implementing all system requirements

The intended audience for this document includes the TFITS Product Management Office (PdMO) and its Project Managers (PjM), the project contractor (hereafter referred to as the Systems Integrator (SI)), the TFITS IPTSs, and other personnel who oversee the TFITS projects. The TFITS PdMO has the responsibility for executing the requirements in this SEP under the direction of the TFITS Assistant Product Manager-Engineering (APDM-E) or designated Project Lead Engineer and applying them throughout the lifecycle of each TFITS projects.

Additionally, the SI shall develop a Software Development Plan (SDP) and execute a SE process that addresses and coordinates with the processes, products, functions, and objectives described in this SEP. The SI shall provide an initial SDP with the technical proposal and delivered an updated version as required, after contract award. The SI shall continue to update the SDP to maintain alignment with this SEP. This SEP is prepared for TFITS projects that are past MS C in the Production & Deployment/Operations & Support phase. This SEP is considered a living document that will be updated as necessary to support evolving requirements and maturing products and processes. These updates capture the evolving SE implementation and its relationship with the overall product/project management effort.

1.1 Approach for SEP Updates

This SEP follows the DoD Systems Engineering Plan (SEP) Outline, 20 April 2011, for format and content. The TFITS SEP applies to projects that are fully managed by the TFITS Product Manager.

The TFITS Assistant Product Manager-Engineering (APdM-E) will update this SEP annually. All updates will be coordinated with the PM 110 Assistant Program Manager-Engineering (APM-E) for concurrence and signing by the PM.

This document is maintained under the TFITS Product Office Plans Configuration Management Board. Changes to this plan shall be submitted in the form of an Engineering Change Proposal (ECP).

Refer to the Record of Changes, Table 1-2, for all changes incorporated into the updated version of this document after initial approval. The SEP is a configuration item and will be managed in accordance with the Configuration Management (CM) process established for the Product Office. Updates to this document will go through the CM process for approval. The APdM-E may make minor updates, such as newer versions of figures and tables, as they become available.

Table 1-2: SEP Update Record (mandated) (sample)

Revision Number Date Log of Changes Made and Description of

Reason/Changes Approved By

2. PROGRAM TECHNICAL REQUIREMENTS

Specific technical requirements for each of the projects are defined in the project requirement traceability matrix (RTM). Many of TFITS projects are legacy projects developed as urgent needs, or some other category, and turned over to Marine Corps System Command (MCSC) to sustain. As such there are limited governing or defining requirements. Through the course of maintaining these systems, requirements have been defined and documented in formal RTMs for some of the systems. It is the intention of the product office to document the current capability for all systems and to institute a configuration management system to control those requirements.

2.1 Requirements Management

TFITS has developed a requirements management plan which provides the Project Managers (PjM) and the systems engineer with the policies and procedures implemented in TFITS regarding requirements management.

Traceability between the initiating requirement and the design requirements is the responsibility of the project systems engineer. Initiating requirements may be in the form of an official requirements document such as a CPD, or simply the ECP TFITS intends to develop System Design Specifications (SDS) to support all new capability or functional upgrades. If a SDS exists for the project, it will be updated to support the change. If an SDS does not exist, a determination will be made to the amount of SDS documentation that is required to be generated.

At a minimum, the new capability or functionality will be described along with the interfaces to the system necessary to insure accurate implementation of design requirements. The systems engineer is responsible for the development of the SDS. The SDS will be decomposed and integrated into the project RTM. The TFITS PdMO is responsible for the maintenance of the RTM and SDS to the system level.

2.2 Architectures and Interface Control

The level of architecture diagrams that exist for each of the projects is defined in the respective appendix. As the systems have a need for reaccreditation of Information Assurance (IA) certifications, necessary architecture models will be developed based on guidance from the IA decision authority. The minimum architecture needed for managed systems is Operational View (OV 1, OV 2) and System View (SV 2, SV 6). Additional architecture diagrams will be developed as necessary to support ISP development or enhancements.

TFITS has an architecture and interface management Working-Level In Product Team (WIPT) chartered to develop the minimum set of DODAF Models for each system and to maintain current interface agreements and manage the interfaces and architecture baselines.

2.2.1 Interface Control

Projects’ dependencies on external interfaces must be defined and tracked. Interfaces can be physical such as the connection to a radio or soft such as data messages. How essential the data is to the systems on each side of the interface will drive whether a formal agreement is required between the system managers. Formal Memorandums of Agreement (MOAs), Memorandums of Understanding (MOUs), Interface Control Agreements (ICAs), and System Interface Agreements (SIAs) help to establish what is being provided and how notifications will occur for interface changes.

Interfaces that provide or carry general/non-critical information that does not affect either system’s operations do not necessarily require formal agreements. If either system is dependent on the information provided by the interface for system operation then it is good practice to establish an agreement laying out the working relationships to include points of contact, change control, change notification and any financial agreements in the development or changes of the interface. TFITS will establish ICAs when either system is dependent on information from each other. Any agreement that has financial liability requires an MOA and approval by MCSC.

Interface engineering details do not need to be part of the agreement. Such details should be documented and placed under CM in an Interface Control Document (ICD) or Interface Design Description (IDD) and may be referenced or included as an appendix.

Each project will assess the need to develop interface agreements with the external system owners, based upon each system’s architectures. Each individual system must maintain separate agreements with each interfacing partner, even if multiple systems interface with the same partner. Where possible, projects should maintain a common interface between multiple partners. Each of these SIAs shall be reviewed annually to ensure applicability. All already signed or future interface agreements will be documented in Table 2-1. All interfaces managed for each project are defined in respective appendices.

Table 2-1: Required Interface Agreements

Interface Cooperating

Agency Controlling

MOA Type Required By

Date Impact if Not Completed

Each project will submit interface Table 2-1to the APdM-E each September. The APdM-E may request select documents for further review.

2.2.2 Technical Certifications

A Clinger-Cohen confirmation is needed for systems undergoing milestone review or major contract award. This requirement exists for both Acquisition Category (ACAT) and Accelerated Acquisition Program (AAP) designated systems.

JITC certification is required for systems with joint external interfaces. This may require an Information Support Plan (ISP) to be developed.

DoD information Assurance Certification and Accreditation is required for all systems or applications which receives, processes, stores, displays, or transmits DoD information must be certified and accredited.

3. ENGINEERING RESOURCES AND MANAGEMENT

The TFITS APdM-E is the Lead Systems Engineer, and together with the PdM, ensures projects are provided adequate engineering resources. Engineering resources include IA personnel, CM, test and evaluation, and systems engineers. Engineering resources consist of organic PdM personnel, external government personnel from either within MCSC or from external agencies, and contractor support personnel.

3.1 Technical Schedule and Schedule Risk Assessment

Engineering resources are applied to projects based on demand. It is critical that projects identify the need for engineering resources. This identification is based on work packages integrated into the projects over all IMS. It begins with the vision for the project identified in the ASAP and expressed in the Chronotecture. The Chronotecture reflects the major and minor baseline changes to the system over the 7-year planning cycle. The work packages align with these changes and also include continuous efforts.

The versioning work packages for TFITS projects are capability change, functionality change, major or minor change, and maintenance changes. Each of these upgrades has a work package associated with it. The work package defines the hours and months needed for implementation. The work package will include the time needed to develop the required system documentation appropriate for each change.

There is also a change management work package that defines the work associated with implementing the TFITS configuration management plan for a project. It defines the demand for a Functional Review Board (FRB), Engineering Review Board (ERB) and a Configuration Control Board (CCB) and the execution of the Engineering Change Proposals.

Additionally there are engineering work packages for project sustainment including In Process Reviews (IPRs), project health monitoring, certification and accreditation monitoring.

3.2 Engineering Resources and Cost/Schedule Reporting

TFITS engineering IPT is responsible for providing the resources for projects as shown in Figure 3-1. Engineering resources are allocated by the APdM-E based on the requirements identified by the project IPT. This IPT planning effort defines the Integrated Master Plan (IMP) for the project and the resources needed to perform each task. TFITS APdM-E intends to provide consistent organic systems engineering and IA support to projects going through development. This staff will be supplemented with CM, test and evaluation, additional SE and IA support as required.

Figure 3-1: Engineering IPT Resources

At the weekly SE IPT, the engineering competency persons will discuss the status of all the projects in development. During this meeting, adjustments will be made to the augmentation staff to meet the need of upcoming system reviews, CCBs, test events and documentation requirements.

3.3 Engineering and Integration Risk Management

One of the greatest risks to legacy systems in sustainment is from technical obsolescence and from interfacing partners implementing new functionality, which require change to TFITS systems. TFITS has developed a Risk Management Plan (RMP) to identify risks at the Tier 3 level that are briefed up to the Tier 1 level. Through the constant monitoring of the project health, the project engineer identifies degradation in system performance and reports status and risks to the PjM at the Tier 3 IPT. The metrics identified in this SEP provide the areas of performance monitoring expected for each project.

3.4 Technical Organization

3.4.1 Government Program Office Organization

PdM TFITS is organized in a three-tier IPT structure. Tier 1 is comprised of the Product Manager (PdM); the Deputy PdM and the Competencies’ Assistant Product Managers for Logistics, Engineering and IA, Contracts, and Finance. Tier 2 consists of nine portfolio-level IPTs. Tier 3 consists of project/product group IPTs. This structure is shown in Figure 3-2. This organization is comprised of organic MCSC personnel, external government personnel from the warfare centers and support contractor personnel.

Figure 3-2: Program Office Organization

3.4.2 Program Office Technical Staffing Levels

Program Office technical staffing levels are defined in Project Work Packages. The nominal work packages are defined in Table 3-2. These work packages are provided for the PjM and the project IPT to use when developing the project IMS and must be adjusted to accommodate the specific needs of each project. The table provides the nominal hours per sub-competency and the expected duration or frequency of the event.

Table 3-1: Engineering Work Packages (Sample Format)

Task System Engineering

Configuration Management

Test and Evaluation

Information Assurance

Frequency/ duration

FRB 5 4 4 4 1 per Qtr.

ERB 12 10 6 6 1 per Qtr.

CCB 12 10 6 6 1 per Qtr.

IPT 6 6 6 6 1 per week

TIM 7 6 6 6 1 per week

Capability Change 496 60 448 202

235 days

Major Change 397 36 281 142 150 days

Minor Change 112 35 107 52 99 days

Maintenance Change 11 25 7 6

30 days

Dip 3 12

IPR 6 6 6 6 1 per month

Recertification 485

3.4.3 Engineering Team Organization and Staffing

The Information Systems and Infrastructure (ISI) engineering organization is shown in Figure 3-3. It consists of product groups and functional groups. TFITS is a product group and contains engineering personnel to directly support the PdM and APdM-E in the management of TFITS products. The functional groups consist of IA, enterprise architecture, test and evaluation, and modernization. Only the IA group contains resources, and its resources are allocated to the product groups through the IA group manager based on demand submitted by the product group APdM-E. Resources are requested from the Functional managers by the APdM-E to support the projects as needed.

Figure 3-3: PMM110, ISI Organization

Figure 3-4 depicts the distribution of TFITS technical personnel as allocated to TFITS projects.

Figure 3-4: TFITS Technical Organization

Figure 3-5 depicts the IA organization and its allocation to projects. The IA allocation is made primarily at the Tier 2 level but there is an IAM assigned to each project in the Tier 2.

Figure 3-5: Information Assurance Project Allocation

3.5 Relationships with External Technical Organizations

The primary interface to external organizations is the PjM. However, the assigned engineer will provide the primary technical interface to the vendor, the customer, and interfacing partners. The FRB and the Technical Interchange Meetings (TIMs) are the primary method for interfacing with external organizations. Projects going through a capability upgrade can also expect to interface with SIAT and much more with the PMO. The engineer is responsible for engaging SIAT and external test organizations including Joint Integrated Test Center (JITC) and Marine Corps Operational Test Agency (MCOTEA).

3.6 Technical Performance Measures and Metrics

Technical Performance Measurement (TPM) categories have been established. Since these systems are deployed and in a sustainment status, TPMs are used to measure system performance to ensure continued performance when upgrades are made. The TPMs are assessed as part of each Government Acceptance Test (GAT) performed. While not required during functional user evaluations, assessments will be performed whenever possible. TPMs will also be used to gauge system obsolescence, thereby providing insight as to feasibility of system upgrades to meet TPMs.

3.6.1 Usability

Usability metrics, collected for each system, are all supported by help desks which provide monthly metrics reflecting user problems. Each system integrator is also tasked with providing monthly metrics for number of transactions generated. These metrics are used to compute system failure rate, site usage and system suitability.

3.6.1.1 System Operational Percentage

System operational percentage is computed by dividing operational time by possible operational time on a monthly basis.

3.6.1.2 Site Usage

Site usage is determined for each installation and deployed application. Site usage is computed as the sum of user log-ins per month. Site usage may also be use the total number of transactions processed by each site.

3.6.1.3 System Suitability

System suitability, an indication of how well the system is performing, is based on evaluation of help desk calls received.

3.6.2 Sustainment Assessment

Assessment of sustainability of the system is based on the POPS 6.5 assessment. These categories have been tailored for PM TFITS. A grading scheme has been defined for assessment of each of the criteria and is shown in Table 3-3.

3.6.2.1 Sustainment

• Is the program performing as designed and supporting the users’ needs?

• The Project is monitoring help desk tickets to support the user community with timely corrective actions.

• Hardware and software items are receiving support from the vendor via warranty or software licensing agreements.

• The program has a training system or materials that are updated with each new version or enhancement.

• The program has facilities in place that are ready to support the fielded system? Mark as green if the program does not have a facility requirement.

3.6.2.2 Software

The program is adequately supported. The software support provider retains experienced personnel to support the system.

Software patches and upgrades are applied per vendor specifications. Software components are upgraded to maintain currency.

Software sustainment facilities and infrastructure have been established and are supporting system operations.

Facilities provide a development environment, including servers, developmental software, and testing models and simulations. Current licenses are maintained; software vendors maintain support for their software; the test environment is up to date with interface test capabilities, models, and simulations.

3.6.2.3 Threat and Capability Review

PM TFITS operates logistic and non-combat systems. There remain threats to these systems in terms of information assurance. As part of the re-accreditation process, a threat assessment may be conducted.

Throughout system life, the ability of the system to satisfy requirements and provide intended capabilities shall be monitored. This shall be done through user feedback, help desk ticket evaluation and an annual in-service review.

3.6.2.4 Technology Protection

All software and hardware components are monitored and tracked for obsolescence. Equipment and software refreshes and upgrades are planned and executed to avoid obsolescence issues.

3.6.2.5 Technical Maturity

Assessment of technical maturity includes determination of availability of elements needed to manufacture or maintain the end item to specification and there are no major design deficiencies not covered by waiver.

Technical maturity identifies its functional obsolescence when the requirements are obsolete due to business process change or advancements in technology. Logistical obsolescence includes digital media obsolescence, formatting or degradation limits.

3.6.2.6 Interfacing Programs

Interdependence with other programs to use or provide data shall be assessed during upgrades.

Table 3-2: Grading Scheme for Sustainment Assessment

Category Criteria\ grading, points

Good

Acceptable

Marginal

Unsatisfactory

Weighting

Usability System failure rate

>5% >25% > 50% < 50% 50%

Usage < 1Sigma Average >1 Sigma >2 Sigma 20 Suitability No user issues Pri 4 Pri 3 Pri 2 or Pri 1 30

Threat assessment ATO Renewable IATO? Pending ATO? Not renewable *

Technology protection

Dev vendor license

< 4yr Supported Extended support

No support 50

Deployment licenses

< 4yr Supported Extended support

No support 50

Technical Maturity Maintainable In production >3 yrs. old <6 yrs. old >6 yrs. old

Cannot maintain

Design Category 4 Category 3 Category 2 Category 1 20 deficiencies

Functional obsolescence

Current process Meets minimum requirements?

Supported with workarounds

Capability Gaps 50

Logistical obsolescence new Within expected life

Extensions Cannot be supported

Interfacing Programs

Interface adaptability?

Standard interfaces

Intermediary interfaces?

Unique interfaces

Cannot be updated

* Viability of system and need for replacement/upgrade has to be accessed.

3.6.3 Metrics

In addition to the sustainment assessment criteria described above, the following project-specific metrics will be collected at regular intervals throughout the project, Table 3-4.

Table 3-3: Metric Definitions

Category Metric Description Usage # of Users number of unique accounts applications

# Active Users number of unique account log ins for the month #Transactions per month number of logons or transactions for the month

Availability Unscheduled down time events number of unscheduled down time events

Scheduled down time events number of scheduled down time events

Time off line total down time Usability Help Desk Trouble

Tickets Tier 1 number of help desk calls

Response Time total time from submission of trouble ticket to time to initiate action divided by number of tickets

Resolve Time total time from initiate action to time of resolution divided by number of resolutions

Help Desk Tickets Tier II number of help desk calls Help Desk Tickets Tier III number of help desk calls

Technical Maturity

# ECP new number of new changes submitted into PMO ECP tracker # ECP closed number of closed ECPs for month # ECP Priority 1 Open number of Priority 1 ECP in system not closed, includes new ECPs # ECP Priority 2 Open number of Priority 2 ECP in system not closed, includes new ECPs # ECP Priority 3 Open number of Priority 3 ECP in system not closed, includes new ECPs # ECP Priority 4 Open number of Priority 4 ECP in system not closed, includes new ECPs # ECP Priority 5 Open number of Priority 5 ECP in system not closed, includes new ECPs # ECP Ageing > 90 Days Number of open ECPS older than 90 days. Includes deferred.

Developer Metrics

# defects new developer defects (bugs) found by developer during development # defects closed number of defects closed during the reporting period # defects open number of defects opened during the reporting period # defects ageing>15 days number of defects open for more than 15 days

Category Metric Description Information Assurance

FISMA Date

ATO Expiration Date

Retina Scan Date of last Retina Scan

SCAP Date of last SCAP Scan with Manual Controls

POA&M Submission Date of last POA&M submission

CAT 1 open # of CAT 1s discovered or open

CAT 1s aging # of CAT1s open > 30 days

CAT 2s Aging # of CAT 2s open > 90 days

Post IAVA/OpDirs met # of applicable OPDIRS/IAVAs

Code review Date of last automated Code Review or MWRAC Assessment

Engineering Maintenance Release GAT Passed

Date Maintenance Release GAT was passed

Design Review date Date of Design Review

Design Review Findings Number of RFAs per Design Review

CCB Meeting Date number of CCBs held during the reporting period

When assessing the suitability, usability, technical maturity and similar factors the data collected needs to be for both the hardware and software elements of the system.

4. TECHNICAL ACTIVITIES AND PRODUCTS

4.1 Planned SE Activities

TFITS projects are in sustainment. All changes to these programs are either an enhancement or correction of a defect. All changes are implemented through ECPs as defined in the TFITS configuration management plan (CMP). As stated in the CMP, enhancement changes originate from multiple sources including Functional Advocates, system integrators, and users.

There are two types of engineering efforts associated with TFITS projects. Development efforts that incorporate enhancements and fixes, and life cycle engineering that provides technical oversight of the fielded system.

4.1.1 Development Efforts

TFITS has implemented a 4-octet versioning system that applies a System Engineering Technical Review (SETR) process as defined in the Marine Corps Technical Review Handbook to varying degrees based on the level of change. The development effort building blocks from an IMS and resource perspective are: Capability Change, Major Change, Minor Change, and Maintenance Change. Table 4-1 defines the octets and the tailoring provided.

Table 4-1: 4-Octet Versioning System

In addition to the SETR reviews, the amount of work associated with the effort is also tailored.

4.1.1.1 Capability Change

A Capability Change describes a system change driven by changes in business process and/or technology capabilities. These are documented with formal requirements documents from the capabilities sponsor. If a change is considered high risk, it may also result in a first-octet change. A Capability Change is documented by a change in the first octet (e.g., 01.xx.xx.xx). There is minimal tailoring at this level. It is expected that the PdM will most likely chair the technical reviews along with the APdM-E.

4.1.1.2 Major Release

A Major Release reflects an increase in functionality driven by significant user enhancements or multiple hardware and/or software upgrades. A Major Release is documented by a change in the second octet (e.g., xx.01.xx.xx). The Major Release is the most tailored. Depending on the depth and breadth of the change, tailoring will occur in the number of design reviews and in the extent of the artifacts produced.

4.1.1.3 Minor Release

A Minor Release is a change driven by minor enhancements to functionality or a hardware/software refresh, which causes no significant change to form, fit, or function. A Minor Release is documented by a change in the third octet (e.g., xx.xx.01.xx). Minor releases are always tailored. There is only one design review prior to entering the development phase. Documentation is also tailored to change pages for many of the documents.

Testing is also scaled back to focus only on the enhancement and regression testing.

4.1.1.4 Maintenance Release

A Maintenance Release is a change driven by a defect/bug or underlying minor vulnerability. A Maintenance Release is documented by a change in the fourth octet (e.g., xx.xx.xx.01). There are no scheduled reviews here.

The engineering oversight here is by awareness. It is envisioned that the normal technical interface meetings and integrated product team meetings will suffice.

The design flow for the tailored process is shown in Figure 4-1. The process supports agile development as well as the traditional waterfall approach.

TFITS ECP Workflow

Change Initiator/Help

Desk

Test & Evaluation

Systems Engineering

Release Management

Project Management

Configuration Management

Change Management

CLASS II

COMPLIANT

NOT COMPLIANT

Notification

Update Baseline

ECP Form Submitted via

SharePoint

WG/

IPTs

Production Validation

Build

ECP Closure

CITDB

Forward Schedule of

ChangesRelease

Trouble TicketsIAVAs

CCB

USERs Defects

NOT COMPLIANT

YES

CLASS I

NOT APPROVED

(CLOSED NOTIFICATION)

ROM

Contracting

Action

YES

Production ReviewCOMPLIANT

NOT COMPLIANT

Contract Mod

ROM

FRB

ERB

FUNCTIONAL ECP

(Prioritize)

APPROVED

PENDING

FUNDS

DEFFERED

NO

Test ReviewCOMPLIANT

SI TEST Design Review DESIGN

NOT COMPLIANT

COLOR KEY

SI PjMO

ALL SETR

Functional

SI & PjMO

Requirements Review COMPLIANT

NOT

COMPLIANT

GAT

Requirements Elicitation

Figure 4-1: Tailored Process Design Flow

4.1.2 Life Cycle Engineering

There are several engineering tasks which must be continuously accomplished independent of the development process. The tasks are: Engineering Review Board (ERB), Configuration Control Board (CCB), Technical Interchange Meeting (TIM), Monthly Health Assessment (MHA), Annual In-service Review (AIR), and Project Management Review (PMR).

4.1.2.1 Engineering Review Boards

ERBs are led by the project engineer to ensure that proposed changes are understood and defined sufficiently to develop a set of requirements to provide to the developer for design and implementation. The ERB is a project IPT with some developer input.

4.1.2.2 Configuration Control Board

The CCB is defined in the CMP and is chaired by the PjM to review and approve changes to the system.

4.1.2.3 Technical Interchange Meeting

The TIM is used by the engineering community to discuss with the developers the design of the changes. It is typically held only during developmental efforts. It is particularly useful when there are multiple developmental efforts on going or there are larger versions being developed.

4.1.2.4 Monthly Health Assessment

The monthly health assessment consists of assimilating metrics and assessing them to determine system performance.

4.1.2.5 Annual In-service Review

The Annual In-Service Review is an opportunity to take a more in-depth look at how well a system is performing and how well it is continuing to meet the user’s requirements. In addition to personnel from the Project IPT, personnel taking part in the annual review will include representatives from the user, personnel from field sites and from system operations.

4.2 Requirements Development and Change Process

Changes made to the TFITS projects are implemented through requirements development and the change process. TFITS manages this development by controlling the requirements documents. The state of the document is defined based on the development phase. Table 4-2 identifies the documentation required based on the magnitude of the change.

Table 4-2: Requirements Documents

4.2.1 Requirements Management

TFITS has developed a requirements management plan which defines the process for requirements development, analysis, traceability, storage, the relationship to the RTM. The project engineer is responsible for requirements management. This includes requirements elicitation to support enhancements being implemented.

4.2.2 Change Procedure

TFITS uses a SharePoint ECP tracker to manage ECPs. It contains an example of ECPs against TFITS systems. As ECPs are received, it is the responsibility of the PjM to push them through the ECP Review Process. This process is documented in the TFITS CMP.

4.3 Technical Reviews

PdM TFITS conducts technical reviews in accordance with MCSC SETR Handbook. A Technical Review Action Plan (TRAP) will be developed for each SETR review to define the expectation for the review. When using the tailored SETR, a TRAP is not required however; a TRAP may still be used to set expectations. When a TRAP is not used, TFITS has developed a checklist to define the entrance and exit criteria used to evaluate the readiness to proceed to the next phase of development. A database of documentation required for each technical review will be established in SharePoint and made accessible to each member of the review board (read only).

4.3.1 Technical Review Action Plan

The TRAP will include the following sections:

1. Introduction – description of the review

2. Program Background – details of the program and release history

3. Purpose – Purpose of the TRAP and intended completion result from the review

4. Specific Tailoring – Statement outlining any tailoring adjustments made

5. Schedule – provide both the project development schedule and the detailed inch stones for the review

6. Technical Review Board Membership

7. Technical Review Participants

8. Entrance Criteria

9. Documentation Products (and link to SharePoint site where the documents are available for review)

10. Agenda – Describe time, location, and date as well as specific items to be presented

11. Exit Criteria – Expected results following completion of all Exit criteria

12. Request for Action Process – Official requests and tracking of pre/during event action items

13. Technical Review Closeout

4.3.2 Tailored Review Check Lists

TFITS has developed tailored technical review checklists to define the entrance and exit criteria for tailored reviews. On the following pages are checklists for the Requirements Review, Design Review, Test Review, and the Production Review.

Requirements Review Description

The Requirements Review (RR), or System Functional Review (SFR), is a multi-disciplined product and process assessment to ensure that the system under review can proceed into design* and can meet the stated requirements within cost (program budget), schedule (program schedule), risk, and other system constraints.

Generally, this review assesses the requirements and ensures traceability, completeness, and accuracy of the requirement to the established baseline. Configuration items may consist of documentation and requirement elements.

* 3rd and 4th octet changes will typically proceed to final design (i.e., Design Review). 1st and 2nd octet changes will typically proceed to preliminary design (i.e., PDR).

Entrance Criteria: Requirements Review (RR) or System Functional Review (SFR)

ID Comp Entrance Criteria Status

(Not Started/ In Progress/ Complete/NA)

E-R-01 SE A preliminary agenda has been coordinated (nominally) to an agreed upon ___ days prior to the event.

E-R-02 SE

SRD and RTM made available prior to the review. Preliminary system/software functional requirements made available prior to the review.

E-R-03 SE System/software functionality description made available prior to the review.

E-R-04 SE Preferred system solution definition made available prior to the review.

E-R-05 SE Updated risk assessment made available prior to the review.

E-R-06 SE System Engineering Plan (SEP) made available prior to the review.

E-R-07 SE Updated schedule data made available prior to the review.

E-R-08 SE HIS related documentation made available prior to the review, if applicable.

E-R-09 SE

SDP is complete and made available prior to the review. (May be a standard organizational or company document tailored to the program)

E-R-10 SE

System/Software architecture (hardware, software, human, material as necessary) made available prior to the review. SAD made available prior to the review.

E-R-11 SE Integrated system architecture and supporting views (System Views and Technical Views) made available prior to the review.

E-R-12 Log Logistics documentation (e.g., LRFS, an IUID Plan, etc.) should be made available prior to the scheduled review.

E-R-13 IA

Information Assurance documentation made available prior to the review:

(E.g. RTM includes security requirements in accordance with applicable Security Technical Implementation Guidelines (i.e.

Application & Development, Database STIGs, etc.))

E-R-14 PM All applicable CDRLs are reviewed.

Exit Criteria: Requirements Review (RR) or System Functional Review (SFR)

ID Comp Exit Criteria Status (Met/Not Met)

Comment

X-R-01 SE

Requirements, as disclosed, are traceable back to the approved requirements in the ECP, SDS, CDD, CPD, or

CONOPS.

X-R-02 SE

Requirements are sufficiently detailed and understood to enable system functional definition and functional decomposition.

X-R-03 SE Requirements are complete, concise, traceable, and unambiguous.

X-R-04 SE The system functional requirements, as disclosed, satisfy the ECP, SDS, CDD, CPD, or CONOPS.

X-R-05 SE

The functional baseline (FBL) has been established to enable preliminary design to proceed with proper Configuration Management.

X-R-06 SE The RTM is complete and ready to be placed under configuration management.

X-R-07 SE

Software functionality in the approved functional baseline is consistent with the updated metrics and resource loaded schedule.

X-R-08 SE Processes and metrics are adequate and in place for the program to succeed. (CM, SE, T&E, RM, and REQM)

X-R-09 SE All appropriate documents have been updated and put under Configuration Management control.

X-R-10 SE, IA

Software, HW, HSI, and interface requirements to be implemented in each software build and/or release have been identified.

X-R-11 IA

Information Assurance requirements to be implemented in each software build and/or release have been identified.

X-R-12 PM Risks are known and manageable for design and development.

X-R-13 PM Program schedule is executable (technical/cost risks).

X-R-14 PM Program is properly staffed.

X-R-15 PM Program is executable within the existing budget.

X-R-16 PM Proper independent subject matter experts were at the review.

X-R-17 PM All applicable CDRLs are accepted.

X-R-18 LOG Logistics requirements to be implemented in each software build and/or release have been identified.

Design Review Description The Design Review (DR), or Critical Design Review (CDR), is a multi-disciplined product and process assessment to ensure that the system under review can proceed into development, code, and can meet the stated requirements within cost (program budget), schedule (program schedule), risk, and other system constraints. Generally this review assesses the design of the requirement or ECP and ensures traceability of the design to established baselines. Configuration items may consist of both hardware and software elements.

Entrance Criteria: Design Review or Critical Design Review (CDR)

ID Comp Entrance Criteria Status (Not

Started/ In Progress/ Complete/NA)

E-CDR-01 SE A SRR/SFR/RR has been successfully conducted.

E-CDR-02 SE All exit criteria have been satisfied, and all critical RFAs have been closed.

E-CDR-03 SE A preliminary agenda has been coordinated (nominally) at an agreed to ___ days prior to the event.

E-CDR-04 SE

Technical Products have been made available prior to the review:

• SE: SRD, RTM, SSDD, STP, TEMP/STES, Metrics, CSCI, SAD, analyses, diagrams, database designs, System Views (SVs) and Standards Views (StdVs), HSI related documentation, and other items as applicable

• SE/IA/LOG: Configuration Item Technical Database (CITDB)

• IA: Security Impact Assessment Form, Network Diagrams, Disaster Recovery Plan (DRP), Access Control Matrix, POA&Ms, Risk/Vulnerability Assessments, Hardware Software Inventory, System Security Plan

E-CDR-05 PM, IA,

LOG, SE

Programmatic products have been made available prior to the review or will be reviewed at the PDR.

• Schedule

• Risk assessments,

• supportability plan,

• Logistics documentation, CITDB or Hardware/ Software

Inventory

• Fielding/deployment plan

E-CDR-06 PM All applicable Contract Data Requirements Lists (CDRLs) have been reviewed.

Exit Criteria: Design Review or Critical Design Review (CDR)

ID Comp Exit Criteria Status (Met/Not

Met) Comment

X-CDR-01 SE The detailed design, as disclosed, satisfies the baseline requirement

X-CDR-02 SE The status of the technical effort and design indicate T&E success and operationally suitable, survivable, and effective.

X-CDR-03 SE

The integrated architecture System and Technical Views (SVs and TVs) support, and are consistent with, the appropriate operational architecture, the CDD/CPD/CONOPS/ECP, the Information Support Plan (ISP) and Net-Ready Key Performance Parameter (NR-KPP).

X-CDR-04 SE

The product baseline has been established and documented to enable software coding to proceed with proper configuration management.

X-CDR-05 SE Adequate processes and metrics in place for the program to succeed.

X-CDR-06 SE HSI requirements are included, where needed, in the overall design.

X-CDR-07 SE

The software functionality in the approved product baseline is consistent with the updated software metrics and resource-loaded schedule.

X-CDR-08 SE

The key product characteristics having the most impact on system performance, integration, cost, reliability, or safety have been identified.

X-CDR-09 PM, IA,

LOG, SE

Impact to the following products/items have been identified:

• User manuals or any training material

• TFSMS AAO for either the Principle End Item or an associated component

• CITDB

• GFP listing or a Consolidated Memorandum Receipt

(CMR)

• Hardware or software disposal actions

• Life Cycle Cost Estimate, Logistics Requirements and

Funding Summary or hardware procurement plan, or program office cost estimate

• Required specialty tools, software or support equipment

X-CDR-10 PM, IA,

LOG, SE

New or any changes to PSA’s, e.g., SLA, ICD or MOA is identified and in progress.

X-CDR-11 SE, IA

Sufficient test planning has been accomplished to allow system and sub-system level testing to begin (code review, re-testing of affected IA controls, scans, testing of STIGS, unit and system level tests, etc.).

X-CDR-12 IA

IA requirements are included, where needed, in the overall design package:

• modification to firewalls/ports and protocols

• updated hardware or software

• security issues

X-CDR-13 IA Certification and accreditation required actions have been identified and started w/ sufficient time.

X-CDR-14 IA/PM

Hardware and Software approved for use and obtained through authorized sources (i.e. DADMS, MCHS, and

MSELMS).

X-CDR-15 PM The risks are known and manageable.

X-CDR-16 PM The program schedule is executable (technical/cost risks).

X-CDR-17 PM The program is properly staffed.

X-CDR-18 PM The detailed design is producible within the budget.

X-CDR-19 PM The proper independent subject matter experts were at the review.

X-CDR-20 PM All applicable CDRLs have been accepted.

X-CDR-21 Log Impact to training environment requiring change or updates are identified.

X-CDR-22 Log Will this change result in the return of unused software licenses to MCSELMS?

X-CDR-23 Log Will this change utilize MCHS and MCSELMS to the maximum extent as possible?

X-CDR-24 Log Will this change impact existing government or contractor personnel (e.g., contract language, new access, security)?

X-CDR-25 Log Will this change increase or decrease the electrical, air conditioning, or facility footprint?

Test Review Description The Test Review (TR), or Test Readiness Review (TRR), is a multi-disciplined product and process assessment to ensure that the system/subsystem/system of systems under review is ready to proceed into government acceptance test (GAT). The TR assesses test objectives, test methods and procedures, scope of tests, and determines if required test resources have been properly identified and coordinated to support planned tests.

The TR verifies the traceability of planned tests to baseline requirements. The TR also determines the completeness of test procedures and their compliance with test plans and descriptions. Finally, the TR assesses the system under review for development maturity, cost/schedule effectiveness, and risk to determine readiness to proceed to GAT.

Entrance Criteria: Test Review (TR) or Test Readiness Review (TRR)

Status

(Not Started/ In Progress/

E-TRR-01 SE A DR/CDR has been successfully conducted.

E-TRR-02 SE All exit criteria have been satisfied, and all critical RFAs have been

E-TRR-03 SE A preliminary agenda has been coordinated (nominally) at an agreed to ___ days prior to the event.

E-TRR-04 SE Test requirements have been documented and are traceable to system, engineering, operational, IA, or program requirements.

E-TRR-05 SE

Configuration of the SUT, including software, has been defined and agreed to, and all interfaces have been placed under configuration management.

E-TRR-06 SE All known discrepancies have been documented and handled in accordance with an agreed to plan.

E-TRR-07 SE

Updated technical products have been made available prior to the review:

• SE: SRD, RTM, SSDD, STP, TEMP/STES, Metrics, CSCI, SAD, analyses, diagrams, database designs, System Views (SVs) and Standards Views (StdVs), HSI related documentation, and other items as applicable

• SE/IA/LOG: Configuration Item Technical Database (CITDB)

• IA: Security Impact Assessment Form,IA Diagrams, Disaster

Recovery Plan (DRP), Access Control Matrix, POA&Ms, Risk/Vulnerability Assessments, Hardware Software Inventory

E-TRR-08

PM,

IA,

LOG,

SE

Updated programmatic products have been made available prior to the review or will be reviewed at the PDR.

• Schedule

• Risk assessment,

• supportability plan,

• Logistics documentation, training materials, CITDB, Hardware

Software Inventory

• Fielding plan

E-TRR-09 SE, IA All applicable functional, security, unit level, subsystem, system, and qualification tests (if applicable) have been conducted successfully.

E-TRR-10 IA At a minimum, an ATO with Caveats or IATO has been received.

E-TRR-11 PM Roles and responsibilities of all test participants are defined and agreed to.

E-TRR-12 PM All applicable Contract Data Requirements Lists (CDRLs) have been reviewed.

E-TRR-13 PM,

Log

All required test personnel have been trained, are available, and have the proper system access to support the required tests.

E-TRR-14 Log All required test resources (people, facilities, test articles, test instrumentation, updated user manuals) have been identified and are available to support the required tests.

Exit Criteria: Test Review (TR) or Test Readiness Review (TRR)

(Met/Not Met) Comment

X-TRR-01 SE Test requirements are traceable, documented, and approved.

X-TRR-02 SE Test plan is based on traceable requirements, complete, and approved for the SUT.

X-TRR-03 SE Software test descriptions and procedures are defined, verified, and base-lined.

X-TRR-04 SE

Requirements that cannot be tested at the Computer Software Configuration Item (CSCI) level are identified, with an agreed to plan.

X-TRR-05 SE Software metrics show readiness for testing.

X-TRR-06 SE Software (and hardware) problem report system is defined and implemented.

X-TRR-07 SE

Testers have a high degree of confidence that the SUT will…