Exhibit_5G_TFITS_Systems_Engineering_Plan.pdf

TOTAL FORCE INFORMATION TECHNOLOGY

SYSTEMS (TFITS)

SYSTEMS ENGINEERING PLAN (SEP)

Version 3.0 November 2015

Prepared by:

Prepared by:

MARINE CORPS SYSTEMS COMMAND (MARCORSYSCOM)

TFITS

Information Systems & Infrastructure (IS&I)

Quantico, Virginia 22134 ii

FOR OFFICIAL USE ONLY

SIGNATURE PAGE

SEP

TECHNICAL REVIEW:

Mr. Robert Lovejoy Assistant Product Manager – Engineer (APdM-E)

PMM 110 IS&I, TFITS

MARCORSYSCOM

CONCURRENCE:

Mr. Tony Brill Deputy Product Manager (DPdM)

PMM110 IS&I, TFITS

APPROVED BY:

Mr. Lou Peradotto Product Manager (PdM)

PMM 110 IS&I, TFITS

iii

Table 1: SEP Update Record

Revision Number Date

Log of Changes Made and Description of

Reason/Changes Approved By

1.0 • Initial Submission Andy Mittura/Pete Sandlin

2.0 10/28/2014

• Added project level SEP update and References section.

• Updated Section 2.0 to annotate where Engineering Change Proposal (ECP)/Requirements are generated.

• Section 2.0, 2.1, 2.3, updated to reflect current TFITS Engineering Integrated Product Team (IPT) structure.

• Added Human Systems Integration (HSI) section 2.4.

• Added Figure 2.1.

• Updated Figure 3-1, 3-2.

• Added table 3-3 & 3-4.

• Updated section 3.6.2 to remove old assessment criteria and identify new Probability of Program Success (PoPS) criteria.

• Updated section 4.1.2 to include Table 4-1 updates & Figure 4-1.

• Updated content on section 4.3.4 to address tailoring of Systems Engineering Reviews.

• Updated section 4.2 to define Technical Review Action Plan (TRAP) requirements.

• Updated section 4.3.3 to reflect proper approving authority for technical reviews.

• Updated section 4.5 to include the TFITS Systems Engineering Technical Review (SETR) & ECP tracker toolsets.

• Added sections 4.3.4.3 & 4.3.4.4 to address SEP’s lack of information on System Verification Review (SVR)/Production Readiness Review (PRR)/Post Implementation Review (PIR).

• Added all appendices, pulled current staffing, and

Lou Peradotto/Robert Lovejoy iv

3.0 11/1/2015

• Added Acquisition Program Baseline (APB) as key document (where applicable)

• Outlined approving authorities in Section 1.2

• Updated Architecture Section 2.2

• Removed Requirements Management Work

Group as IPT/Working IPT

• Major Update to HSI Section 2.4

• Removed requirement for Annual In-Service

Review, Section 3.6.2

• Defined where Technical Performance

Measurements (TPMs) are captured, Section 3.6

• Updated verbiage for where project requirements are maintained, Section 3.6.1

• Changed 3rd Octet dates in Table 4-1 to 10 days due prior to SETR event due to Contract Data Requirements List (CDRL) review and remediation process

• Updated Figure 4-1 in accordance with Configuration Management Plan (CMP)

• Updated Section 4.3 regarding TRAP requirements

• Removed reference to TFITS Event Tracker in Section 4.5

• Updated Appendix A & C for grammar, content and readability

• Added Table of Figures

• Added section 4.3.4.3 to address System

Integrator Developmental Testing (SIDT)

• Updated Table 4-2 regarding artifact delivery requirements

• Updated all references of System Integrator

Testing, to SIDT v

Table of Contents 1 Introduction – Purpose and Update Plan

1.1 Audience

1.2 Approach for SEP Updates

1.3 Project Level SEP Documentation

1.4 References

2 Program Technical Requirements

2.1 Requirements Management

2.2 Architecture

2.2.1 Interface Control

2.2.2 Technical Certifications

2.3 Test & Evaluation

2.4 Human Systems Integration

2.4.1 Introduction

2.4.2 Directives and Guidance

2.4.3 HSI Strategy

2.4.4 TFITS HSI Team & Support Activities

2.4.5 Project HSI Plan

2.4.6 Programmatic Environment, Safety and Health Evaluation (PESHE)

3 Engineering Resources and Management

3.1 Technical Schedule and Schedule Risk Assessment

3.2 Engineering Resourcing and Cost/Schedule Reporting

3.3 Engineering and Integration Risk Management

3.4 Technical Organization

3.4.1 Government Program Office Organization

3.4.2 Engineering Team Organization and Staffing

3.4.3 Engineering Resource Prioritization

3.5 Relationships with External Technical Organizations

3.6 Technical Performance Measures and Metrics

3.6.1 Technical Metrics

3.6.2 Sustainment Assessment

3.6.3 Other Measures and Considerations

4 Technical Activities and Products

4.1 Planned Systems Engineering (SE) Activities

4.1.1 Lifecycle Sustainment Engineering

vi

4.1.2 Developmental Engineering Efforts

4.1.3 ECP Management

4.2 Requirements Development and Change Process

4.2.1 Requirements Management

4.2.2 Change Procedures

4.3 Technical Reviews

4.3.1 Technical Review Action Plan Criteria

4.3.2 Review Checklists

4.3.3 Technical Review Approving Authority

4.3.4 Tailored SETR Reviews

4.4 Design Considerations

4.5 Engineering Tools

4.5.1 ECP Tracker

4.5.2 Engineering Health Assessments

Appendix A - Tailored TFITS SETR Review Criteria A-1 Requirements Review (Combined System Functional Review (SFR)/ System Requirements Review (SRR) or SRR1/SRR2) Event

A-1.1 Entrance Criteria: Requirements Review (RR) or SFR A-1.2 Exit Criteria: RR or SFR

A-2 Design Review Event A-2.1 Entrance Criteria: Design Review (DR) or Critical Design Review (CDR) A-2.2 Exit Criteria: DR or CDR

A-3 Test Readiness Review Event A-3.1 Entrance Criteria: Test Readiness Review (TRR) A-3.2 Exit Criteria: TRR

A-4 Production Readiness Review Event A-4.1 Entrance Criteria: PRR A-4.2 Exit Criteria: Production Review (PR) or PRR

A-5 Post Implementation Review Event A-5.1 Entrance Criteria: PIR A-5.2 Exit Criteria: PIR

Appendix B - Acronyms Appendix C – PoPS Template vii

Table of Tables

Table 2-1 Interface Information Requirements Table 2-2 HSI Assessment Criteria Table 3-1 TFITS Managed Systems Table 3-2 Supported Systems Table 3-3 TFITS Monitored Systems Table 4-1 TFITS SETR & System Version Mapping

Table of Figures Figure 2-1 Engineering IPT Structure Figure 3-1 PdM TFITS Structure Overview Figure 3-2 Engineering Support Structure Figure 4-1 Sustainment Engineering & ECP Workflow Process

1 Introduction – Purpose and Update Plan This SEP defines the systems engineering approach for the TFITS Product Office projects. The TFITS Product Office provides acquisition oversight for over 47 United States Marine Corps (USMC) business systems. This SEP is intended to define the SE policies and processes implemented within the TFITS product office to apply to all projects and portfolios. Any variance of this plan shall be outlined within a project specific SEP approved by the TFITS PdM, Deputy PdM, and APdM-E.

As a SE roadmap, the TFITS SEP demonstrates integration with the key project management products, processes and documents to include the overall TFITS Acquisition Strategy Plan (AS/AP), APB, the Integrated Master Schedule (IMS), the Lifecycle Sustainment Plan (LCSP), the CMP, the System Test and Evaluation Strategy (STE&S), the Performance Measurement Plan (PMP), and the Requirements Management Plan. This SEP frames the product group’s SE approach and strategy to provide a firm and well-documented technical foundation for the product commensurate with the level of programmatic maturity at each milestone. This plan explains how each TFITS project accommodates cost, schedule, performance, and sustainment trade-offs. In addition, this document describes the expected products of the SE effort and how these products contribute to project decision-making.

The highlights of this SEP include the following areas:

• Describing the TFITS IPT structures

• Detailing processes for managing project baselines and conducting TFITS SETRs

• Define the support structure of the TFITS organization and identify support teams and Points of Contact (POCs) for subject matter expertise.

Additionally, the Systems Integration contractor, hereafter referred to as Systems Integrator (SI), shall develop a Systems Engineering Management Plan (SEMP) or execute a project level SE process that addresses and coordinates with the processes, products, functions, and objectives described in this SEP.

The SI shall provide an initial Software Development Plan (SDP) with the technical proposal and deliver an updated version as required, after contract award. The SI shall continue to update the SEMP and SDP to maintain alignment with this SEP. This SEP is considered a living document that will be updated as necessary to support evolving requirements and maturing products and processes. These updates capture the evolving SE implementation and its relationship with the overall product/project management effort.

1.1 Audience

The intended audience for this document includes the TFITS Product Management Office (PdMO) and its Project Managers (PjMs), the SI, the TFITS IPTs, and other personnel who oversee the TFITS projects.

The TFITS PdMO has the responsibility for executing the requirements in this SEP under the direction of the TFITS APdM-E or designated Project Lead Engineer and applying them throughout the lifecycle of each TFITS projects.

1.2 Approach for SEP Updates

This SEP follows the DoD SEP Outline, 20 April 2011, for format and content. The TFITS SEP applies to projects that are fully managed by the TFITS Product Manager.

The TFITS APdM-E will conduct an in-depth review of this SEP annually with all Government project engineers. All updates will be coordinated with the PM 110 APdM-E for concurrence and signature by the PdM.

This document is maintained under the TFITS configuration management policies. Changes to this plan shall be submitted through the TFITS document staffing process. The approving authority for this document is the TFITS PdM with concurrence from the Deputy PdM.

Refer to the Record of Changes, Table 1, for all changes incorporated into the updated version of this document after initial approval. The SEP is a configuration item and will be managed in accordance with the Configuration Management (CM) process established for the Product Office. Updates to this document will go through the CM process for approval. The APdM-E may make minor updates, such as newer versions of figures and tables, as they become available.

1.3 Project Level SEP Documentation

It is the intention of the APdM-E to have all projects provide an individual project level SEP. This requirement will ensure all projects have a proper engineering strategy defined, and agreed upon by the system PjM, and program PdM. The intention for project engineers is to leverage this TFITS SEP document as a template and define any deviations from the program SEP within the project SEP.

The project engineer will work closely with the PjM to complete and sign this document. Finalization of the project SEP and any updates will require TFITS PdM and APdM-E approval and signature.

1.4 References

• Marine Corps Systems Command SETR Handbook

• Guidebook for Acquisition of Naval Software Intensive Systems

• Updated Software Guidance Software Criteria for Systems Engineering Technical Reviews

• Department of Defense Directive (DoDD) 5000.1, The Defense Acquisition System

• Department of Defense Instruction (DoDI) 5000.02, Operation of the Defense Acquisition System

• Defense Acquisition Guidebook

• MIL-STD-881C: Work Breakdown Structures for Defense Materiel Items

• TFITS CMP

• TFITS Requirements Management Plan

• TFITS PMP

• TFITS STE&S

• Naval Systems Engineering Technical Review Handbook v1.0

2 Program Technical Requirements Specific technical requirements for each of the projects are defined in the project requirement traceability matrix (RTM), project Software Requirements Description/Specification (SRD/SRS), and System Design Specification (SDS). Many TFITS projects are legacy systems developed to meet an immediate business need, and turned over to TFITS in a post Milestone-B status with limited acquisition and engineering documentation, and little standardization across all projects. Through the course of maintaining these systems, requirements have been defined for these projects and are contained in the project specific SRD, SDS, and RTM. It is the intention of the product office to document the current capability for all systems and to integrate the current configuration management system (to include processes and tools) to control those requirements through policy defined in plans such as this program level or the project level SEP.

To maintain consistency across all projects, and technical documentation, PdM TFITS has organized integrated project teams around core engineering skillsets to ensure a multi-disciplined center of excellence supporting the successful implementation of technical requirements; these IPTs are depicted in Figure 2-1 as:

• Systems Architecture

• Test & Evaluation

• Information Assurance

• Configuration Management

2.1 Requirements Management

TFITS has developed a Requirements Management Plan, which provides the PjM, and the systems engineer with the policies and procedures implemented within TFITS in managing system requirements.

The goal of TFITS requirements management is to ensure accuracy in generating new requirements, traceability of the requirement through design, and subsequent testability ensuring overall success of requirements change. Initiation of a new requirement must be delivered in one of the following forms:

• An official requirements document such as a Capability Production Document (CPD) / Capability Development Document (CDD)

• A Statement of Need

• A Letter of Clarification

• An Advocate or Program Office submitted, and Engineering Review Board (ERB) / Configuration

Control Board (CCB) approved ECP

TFITS intends to develop a SRD, or update existing SRD documentation to support all new capability or functional requirements upgrades. If a SRD exists for the project, it will be updated to support each change. If an SRD does not exist, a determination will be made to the amount of SRD documentation that is required to be generated. It is the responsibility of the lead project engineer to review the current project artifacts and work with the PjM and APdM-E to determine the appropriate level of documentation necessary. At a minimum, the new capability or functionality must be described along with the interfaces changes necessary to the system. This must be accomplished prior to delivery to a potential contractor/SI award. The systems engineer is responsible for the development of the project SRD. The SRD will be decomposed by the system integrator into a RTM and SDS. The TFITS PdMO and assigned project

Tier 1 IPT

Tier 2 IPTs

Figure 2-1 Engineering IPT Structure

SE IPT

(Robert Lovejoy)

T&E IPT

(Allen Home)

CM IPT

(Todd Supernaugh)

IA IPT

(Liz Hendrick)

AIPT

(Tim French) engineer is responsible for the maintenance and traceability of the RTM and SDS to the system level SRD.

The TFITS Requirements Management Plan identifies the appropriate supporting documentation and formats to ensure standardization across all managed projects. TFITS has established Tier 1 requirements management resources under the direction of the SE IPT to assist project engineers with requirements management by providing best practices, standards, and when necessary, resources to address individual project needs. These resources will assist projects and provide guidance for standardization of the following documentation:

• System Design Specification

• Software Requirements Description

• Requirements Traceability Matrix

• System Design Documentation

2.2 Architecture

The level of architecture diagrams that exist for each of the projects is defined in the project SEP. As the systems have a need for certification of program funding and interoperability testing, necessary architecture models will be developed as required and defined in the USMC Architecture Development Guide v2.0. This guide provides an overview of what models are required for each system and under what conditions architectures will be developed. Examples include Business Enterprise Architecture compliance for USMC business systems or for joint interfaces, Joint Interoperability Test Command (JITC) in order to fully evaluate the system for system interoperability across a family of systems that exchange key business or warfighting data. The project level SEP shall outline the specific architecture requirements necessary to meet compliance, or those architecture desired for as Fit-For-Purpose.

The Project Manager, Systems Engineer, and Architecture Integrated Product Team (AIPT) Lead must assess architecture needs and coordinate to align resources required to develop the required products as approved by the TFITS APdM-E.

The TFITS AIPT is chartered to provide overarching guidance, management, and performance objectives needed to successfully develop compliant architecture models, Marine Corps and Joint Interoperability Certification Conformance documentation, and Information Support Plans (ISPs). Please note - It is the responsibility of the project engineer to address all architecture needs with the TFITS AIPT, prior to addressing resource needs with the APdM-E. The AIPT will work with that project engineer to determine a Level of Effort to address critical deficiencies and present at the SE IPT once the necessary plan of action has been determined.

2.2.1 Interface Control

Projects’ dependencies on external interfaces must be defined and tracked. Interfaces can be physical such as the connection to a radio or soft such as data messages. How essential the data is to the systems on each side of the interface will drive whether a formal agreement is required between the system managers. Formal Memorandums of Agreement (MOAs), Memorandums of Understanding, Interface Control Documents (ICDs), and System Interface Agreements (SIAs) help to establish what is being provided and how notifications will occur for interface changes.

Interfaces that provide or carry general/non-critical information that do not affect either system’s operations do not necessarily require formal agreements. If either system is dependent on the information provided by the interface for system operation then it is good practice to establish an agreement laying out the working relationships to include points of contact, change control, change notification and any financial agreements in the development or changes of the interface. TFITS will establish ICDs when either system is dependent on information from each other. Any agreement that has financial liability requires an MOA and approval by MARCORSYSCOM. Interface engineering details do not need to be part of the agreement. Such details should be documented and placed under CM control in an ICD and stored in the TFITS approved SharePoint document library.

Each project will assess the need to develop interface agreements with the external system owners, based upon each system’s architectures. Each individual system must maintain separate agreements with each interfacing partner, even if multiple systems interface with the same partner. Where possible, projects should maintain a common interface between multiple partners. Each of these SIAs shall be reviewed annually to ensure applicability. All signed or future interface agreements must be documented in the project Configuration Item Technical Database (CITDB), with updated links to associated documents in the TFITS SharePoint portal. All interfaces shall be documented with the minimum information in Table 2-1, as identified in the TFITS CM Plan and CITDB.

Table 2-1 Interface Information Requirements

The APdM-E will conduct an interface audit annually starting in September. Each project CITDB will be reviewed for current interface information and where necessary, random spot checks of documentation may be conducted to ensure all documentation coincides with information stored in the CITDB.

2.2.2 Technical Certifications

Clinger-Cohen confirmation is needed for systems undergoing milestone review or major contract award.

This requirement exists for both Acquisition Category and Accelerated Acquisition Program designated systems. JITC certification is required for systems with joint external interfaces. This may require an ISP to be developed. All project level engineering plans must indicate the necessary documentation requirements; documentation updates approved in the project engineering plan will be stored and managed in the TFITS SharePoint portal and CITDB. It is the responsibility of the project engineer to identify technical certification requirements and assist the PjM with planning for re-accreditation efforts by ensuring the necessary activities and tasks are identified on the project IMS.

2.3 Test & Evaluation

TFITS maintains a STE&S as its overarching plan for managing the technical test requirements for each system release. TFITS test resources are those independent of engineering duties on any one specific project. Instead, Test and Evaluation (T&E) resources represent an independent pool of resources allocated to projects to ensure the testability of the system design meets the approved requirement. Test and Evaluation maintains a series of project artifacts that document the test scenarios, procedures, and results of each test event; these documents are defined in the STE&S and include:

Name Inbound / Outboun d Delivery Schedule Status Requirement POC Interface

Type Comman d Agreement

Type

System Inbound Daily Active List of USMC equipment with assigned

IDN

Project Lead: Jane

Doe (jane.doe@ usmc.mi

System Maintained --

Auto SFT LogCom ICD

• Software Test Plan

• Test Scripts/Test Cases

• Requirements Traceability Matrix (updates)

• GAT Report

A TFITS Test & Evaluation IPT has been established to provide support and guidance to project engineers and PjMs for test and evaluation requirements. This IPT is tasked with managing test resources and working directly with the TFITS APdM-E to address resource deficiencies or issues specific to a project release. Test resources report directly to the T&E Lead, who in turn reports directly to the TFITS APdM-E.

All T&E technical documentation requirements will be managed at the project level, documented in the project Test & Evaluation Strategy, and stored in a configuration managed SharePoint document library on the TFITS portal:

(https://mcscviper.usmc.mil/sites/isi_tfits/SEIPT/Systems%20Engineering%20Wiki/TnE%20IPT.aspx).

2.4 Human Systems Integration

2.4.1 Introduction

HSI is a systematic, organization-wide approach to implementing new technologies and modernizing existing systems. It is a combination of managerial philosophy, methods, techniques, and tools designed to emphasize the central role and importance of end-users in organizational processes or technologies.

This approach optimizes the safety and efficiency of these systems through the consideration of all the system’s elements. HSI techniques and processes are used to address the acquisition and modernization of systems with the human in mind during all acquisition phases, making HSI the means for integrating technology with the users.

HSI consists of seven specialty areas of concentration that are integrated into a single interdisciplinary team approach. The areas of concentration are: Human Factors Engineering (HFE), Manpower (M), Personnel (P), and Training (T), Environment, Safety and Occupational Health (ESOH), Habitability (H), and Force Protection/Survivability. The scope of the HSI effort for TFITS focuses on HFE and Training, areas related to performance measures assessed for both technology and the end user.

In addition to traditional HFE requirements, TFITS programs are also assessed for usage by persons with disabilities as outlined in the Section 508 of the Rehabilitation Act of 1973. The requirements generated from Section 508 focus on the development efforts of applications and operating systems to follow development standards to be readable or usable with assistive technology that supports persons with disabilities.

2.4.2 Directives and Guidance

Policies for incorporating HSI as part of the system life cycle can be found in:

• DoDI 5000.02 Enclosure 7, January 2015

• Secretary of the Navy Instructions (SECNAVINST) 5000.2E, September 2011

These polices reflect the importance of HSI efforts with all programs to improve the usability, maintainability, and the effectiveness of training. In addition to the HSI principles instilled with TFITS programs, Section 508 compliancy standards are also addressed as a subset to HFE usability criteria.

Section 508 of the Rehabilitation Act, as amended in 1998, requires that when Federal Agencies, including DoD, develop, procure, maintain, or use Electronic and Information Technology (E&IT), they shall ensure that the E&IT allows Federal employees and members of the public with disabilities have access to and use of information and data that is comparable to that of persons without disabilities, unless an undue burden would be imposed on the agency. TFITS complies with Marine Corps Order (MCO) 5230.22, which provides policies for Section 508 E&IT compliance.

MCO 5230.22 (December 2013) provides the policies for Section 508 E&IT compliance. This includes coordinating instructions to address Section 508 compliance, describes the exceptions to Section 508, how to handle 508 complaints, how to address undue burdens, reasonable accommodations for the disabled, and the legal/regulatory scope. Additional guidance is provided to assist Product Managers, System Engineers, and HSI practitioners in developing and maintaining an effective HSI program.

Definitions, strategies, and instructions to developing an HSI program that satisfies a total system approach and assists in identifying risks and mitigation strategies are provided in:

• Defense Acquisition Guidebook, May 2013

• Secretary of the Navy Manual (SECNAV M) 5000.2, May 2012

• MIL-STD-46855A

• MIL-HDBK-46855A

2.4.3 HSI Strategy

The TFITS strategy is to employ HSI principles and assessment procedures proactively throughout all phases of program development and modernization: requirements development, design, development and testing. This HSI plan includes the support provided by the TFITS HSI team in the effort to address HSI risks, impact and mitigation strategies, HSI requirements assessment, and software usability assessments.

This plan also includes the support provided at the program level for addressing HSI issues and documenting the HSI efforts.

2.4.4 TFITS HSI Team & Support Activities

Naval Surface Warfare Center Dahlgren Division supports TFITS System Engineers and their programs as independent HSI assessors. The following sections describe how the HSI team is integrated into TFITS programs through: participation in Government and vendor Working Group (WG) meetings, support of requirements solicitation, reviews of Government and vendor documents, reviews of training materials, and the development and implementation of HSI program evaluations and assessments.

2.4.4.1 Participation in Government and SI Working Group Meetings Much of the HSI team’s influence on TFITS is achieved through active participation in the various program IPT meetings and WG meetings. Active participation allows HSI to provide domain expertise and design support to program level IPTs and WGs. This support is provided on an as-needed basis during the earliest stages of modernization efforts. HSI actively participates in the following IPT and WG meetings:

• Weekly TFITS SE IPT meetings

• Monthly TFITS Logistics IPT meetings

• Program specific WG and IPT meetings

• Program specific Technical Interchange Meetings (TIM)

• Test & Evaluation WG meetings

• Training WGs

• Interface Control Working Groups

2.4.4.2 Requirements Elicitations and SETR Reviews

HSI is an active participant in the review and assessment of system requirements, design, and test reviews. Requirements provide the foundation of the system where HSI ensures these requirements reflect the needs and capabilities of the users. HSI practitioners support program meetings for the development of requirements to be added to the ECP process for further review with program advocates, systems engineers, system integrators, and PjMs. The HSI team supports the following change process and SETR meetings:

• Requirements elicitations

• Functional Review Board (FRB)

• ERB/CCB

• SFR/SRR

• Preliminary Design Review (PDR)

• CDR

• PRR

• In-Process Design Review

HSI also supports various program reviews to ensure HSI requirements are successfully integrated prior to system testing and release. These reviews include:

• Technical Review Board

• TRR

• In-Process Program Review

• Post Implementation Review

2.4.4.3 Training

The HSI team also provides support to the TFITS Integrated Logistics Support and Training leads through the development and implementation of training, support of the Training Working Group, and review of training and logistics documentation. As needed, a Training Working Group is established by the Training Lead to provide oversight across the development, implementation, and maintenance of training materials. HSI works with the Training Leads and Subject Matter Experts to review the mission essential tasks that need to be trained. They also assist in the determination of the training course structure, learning objectives, and training delivery method (ex. instructor-led vs. web-based) as well as review training plans and strategies.

HSI ensures all training incorporates HSI principles and Section 508 guidelines. HSI reviews the initial storyboards and the draft course materials or Web-Based Training modules and user manuals throughout the development process. After deployment, HSI and the Training Leads develop training surveys to examine users’ experiences with the courses and recommendations for future adjustments. HSI also supports the semi-annual to annual reviews of training materials. With each maintenance release, HSI reviews differences training materials and any requisite changes to the user manuals. HSI also reviews and provides input into logistics and training program-level documents including the LCSP, Integrated Logistics Assessment Checklist and Logistics Sustainment Assessment Processes. The TFITS HSI team also reviews help desk surveys that are designed to solicit user feedback on product usability and make recommendations as applicable.

2.4.4.4 HSI Evaluations and Assessments

The main focus areas of HSI efforts is the verification of the human performance requirements identified in relevant sections of MIL-STD-1472G and Section 508 of the Rehabilitation Act of 1973 as system design and program performance specifications during the evaluation of applications and interfaces. The goal is to assess the impact of the specifications on human performance with new or updated program designs and processes.

HFE requirements compliance assessments will be conducted as an integral part of the HSI design review for all user interface changes. HFE requirements assessment during design will save the time and cost associated with software refactoring and will provide the opportunity to mitigate HSI risks prior to software implementation. The HSI design reviews will be conducted prior to software implementation to ensure HSI principles are incorporated early in the design process to avoid costly redesign.

The TFITS HSI team also supports system testing from SIDT, User Assessment Tests, and Government Acceptance Tests. HSI reviews SIDT results, test plans, and test scripts to ensure HSI requirements are being evaluated. HSI observes test events and, as necessary, independently completes test cases to confirm HSI principles have been accurately addressed in the system design. HSI will develop user surveys to obtain user feedback from general users, administrators and maintainers. User feedback collection will be recommended by HSI throughout the software development cycle to identify potential HSI risks including potential human performance issues at the discretion of the SE, PdMO, PjM. HSI will review, as necessary, training and help desk surveys that are designed to solicit user feedback on product usability making recommendations as applicable.

The key focus areas for HSI assessments are derived from MIL-STD-1472G and Section 508 and are provided as an overview in Table 2-2 below.

Table 2-2 HSI Assessment Criteria

Reference Paragraph Title MIL-STD-1472G 5.1.2 Control / Display Integration

5.1.2.1.4 Feedback

5.1.2.1.5 Error Management

5.1.2.2 Position Relationship

5.1.2.2.1 Functional Grouping

5.1.3 Computer Controls

5.1.3.4 Data Entry

5.2 Visual Displays

5.2.2 Displays - Content

5.2.2.2 Information Architecture

5.2.2.3 Coding

5.2.2.4 Text Presentation

5.2.2.5 Graphic and Representational Displays

5.2.2.6 Dynamic Displays

5.2.2.7 Help

5.2.2.8 Printing Display

5.2.2.9 Data and Message Transmission

5.4 Labeling

5.7.2 Display of warnings and hazards

5.7.3 Visual displays

Section 508 § 1194.21 Software applications and operating systems § 1194.22 Web-based intranet and internet information and applications § 1194.24 Video and multimedia products § 1194.25 Self-contained, closed products § 1194.26 Desktop and portable computers

Additional resources are available for general guidance in HSI and HFE. The industry standards for HFE are developed and managed by the American National Standards Institute (ANSI) and Human Factors and Ergonomic Society (HFES). Two documents that may be referenced for guidance are ANSI/HFES 100 (Human Factors Engineering of Computer Workstations) and ANSI/HFES 200 (Human Factors Engineering of Software User Interfaces). Other resources can include:

• ASTM 1166

• W3C.COM

• Usability.gov

2.4.5 Project HSI Plan

According to SECNAVINST 5000.02E, each program should document the HSI strategy in an HSI Plan, and be included in the project level SEP or as a stand-alone document. Other HSI documentation provided by vendors, such as HSI Project Plan, Human Engineering System Analysis Report, or Human Engineering Design Approach Document may be requested and will be included in Performance Work

Statements released to industry.

HSI risk identification and mitigation is addressed as an integral part of the Technical Review Action Plans. HSI risks will be identified, documented and provided to the PdMO, PjM and SE as soon as they have been identified by HSI. The documented HSI risk will include its description, potential impacts and risk mitigation strategies. HSI will track all risks and document progress made toward mitigation of the risks. The HSI risk tracking documentation will be provided to the PdMO, PjM and SE quarterly and as requested.

2.4.6 PESHE

Potential ESOH concerns associated with the development, manufacture, use and demilitarization of TFITS will be identified and summarized in the project PESHE. ESOH considerations have a direct impact on the design. ESOH and HFE work to ensure the system design meets all applicable safety standards. ESOH also interfaces with training to ensure the safety design and manual procedures are included in the training curriculum and training aides.

DoD acquisition policy requires program offices to address ESOH considerations in an integrated fashion.

DoD policy directs that the approach to and result of the integrated treatment of ESOH considerations in the SE process be accomplished using the system safety methodology in MIL-STD-882E and be documented in the PESHE. Per DoDI 5000.02, a PESHE is not required for software programs that do not contain hardware components. Therefore, a PESHE shall not be required for PM TFITS systems operating on hardware provided by the hosting datacenter, such as PM MCEITS or Marine Corps Recruiting Command. However, these projects shall still adhere to ESOH analysis and compliance with the National Environment Protection Act standards and will be analyzed in accordance with MIL- STD882E upon each system change.

3 Engineering Resources and Management The TFITS APdM-E is the Engineering Competency Manager for TFITS, and together with the PdM, ensures managed projects are provided adequate engineering resources. Engineering resources include IA personnel, CM, test and evaluation, and systems engineers. Engineering resources consist of organic PdM personnel, external government personnel from either within MARCORSYSCOM or from external agencies, and contractor support personnel.

3.1 Technical Schedule and Schedule Risk Assessment

Engineering resources are applied to projects based on demand. It is critical that projects identify the need for engineering resources. This identification is based on work packages integrated into the projects’ overall IMS. It begins with the vision for the project identified in the AS/AP and expressed in the project roadmap or forward schedule of change. These planning tools reflect the major and minor baseline changes to the system over the 5-year planning cycle. The work packages associated with each system release should align to these planned changes in order to effectively plan and budget for out-year costs.

The versioning work packages for TFITS projects are capability change, functionality change, major or minor change, and maintenance changes (as defined in section 4.1 of this document). Each of these upgrades has a work package associated with it. The work package defines the hours and months needed for implementation. The work package will include the time needed to develop the required system documentation appropriate for each change.

There is also a change management work package that defines the work associated with implementing the TFITS configuration management plan for a project. It defines the demand for a FRB, ERB and a CCB and the execution of the Engineering Change Proposals. Additionally there are engineering work packages for project sustainment including In-Process Reviews, project health monitoring, certification and accreditation monitoring.

It is the responsibility of the project engineer to assess these work packages and ensure they are properly reflected in TFITS program planning tools, such as the project IMS.

3.2 Engineering Resourcing and Cost/Schedule Reporting

TFITS SE IPT is responsible for providing the PdM TFITS portfolio resources for projects as defined in Figure 3-1. Engineering resources are allocated by the APdM-E based on the requirements identified by the system engineer and project manager leads. This IPT planning effort defines the IMS for the project and the resources needed to perform each task. TFITS APdM-E intends to provide consistent organic systems engineering and IA support to projects going through development. This staff will be supplemented with CM, test and evaluation, additional SE and IA support as required.

Cost planning per project is based upon planned system changes. It is the responsibility of the Project Engineer to formulate (at minimum) a 3 year planned roadmap of system changes. Once changes are identified, a work package is associated to provide Program Management staff a methodology to estimate Government resource needs.

At the weekly SE IPT, the project engineers will discuss the status of all projects under support. During this meeting, adjustments will be made to the augmentation of staff to meet the need of upcoming system reviews, CCBs, test events and documentation requirements.

3.3 Engineering and Integration Risk Management

One of the greatest risks to legacy systems in sustainment is from technical obsolescence and from interfacing partners implementing new functionality, which require change to TFITS systems. TFITS has developed a Risk Management Plan (RMP) to identify risks at the Tier 3 level that are briefed up to the Tier 1 level. Through the constant monitoring of the project health, the project engineer identifies degradation in system performance and reports status and risks to the PjM at the Tier 3 IPT. The metrics identified in the TFITS PMP provide the areas of performance monitoring expected for each project.

3.4 Technical Organization

3.4.1 Government Program Office Organization

PdM TFITS is organized in a three-tier IPT structure comprised of organic MARCORSYSCOM personnel, external government personnel from the warfare centers and support contractor personnel. The Tier 1 IPT consists of the Product Manager (PdM); the Deputy PdM and the Competencies’ Assistant Product Managers for Logistics, Engineering, Contracts and Finance. The Tier 2 IPTs consists of three portfolio-level IPTs that correspond to the Competencies’ Assistant Product Managers. The Tier 3 IPTs correspond to specific projects. This structure, to include organic MARCORSYSCOM engineering support resources, is shown in Figure 3-1.

Figure 3-1 PdM TFITS Structure Overview

3.4.2 Engineering Team Organization and Staffing

The APdM-E manages a pool of engineering resources that directly support the PdM in the management of TFITS products. The engineering organization is broken down into several functional groups supporting the overall TFITS product group. The functional groups consist of CM, Enterprise Architecture (EA), IA, T&E and HSI. Figure 3-2 depicts the functional engineering groups and their relationship to PdM TFITS Tier 2 and Tier 3 IPTs.

The distribution of TFITS engineering resources in support of the PdM’s objectives is the responsibility of the APdM-E. The TFITS APdM-E assigns engineers to support projects and other efforts, as requested

Figure 3-2 Engineering Support Structure by the PdM, from a pool of engineering personnel assigned to the PdM. These assignments are based upon project needs, and take into consideration personnel’s technical and subject matter expertise as they relate to the specific project needs.

3.4.3 Engineering Resource Prioritization

PdM TFITS has created a formal method of classifying projects in a fashion that designates demand for Government personnel to be assigned from one or more competency areas. Engineering resources will be allocated based upon the following project designation.

• Managed – managed projects will be assigned engineering resources including project systems

Engineer, CM, T&E, Architecture, IA, HSI, and will execute appropriate Technical Reviews (TR) in accordance with this plan. It is the responsibility of that Project Engineer to ensure the TFITS SETR process, planning, and risk management is conducted and reported at the regularly scheduled SE IPT.

• Supported – Supported systems receive Program Management and Information Assurance support, however no dedicated engineering resources are assigned unless requested by the PdM. Engineering resources will be provided to Supported systems on an “as needed” basis, as requested by the TFITS PdM.

• Monitored – Monitored systems receive no Information Assurance or engineering support.

Table 3-1 lists PdM TFITS portfolio of Managed systems, Table 3-2 lists Supported TFITS systems, and Table 3-3 lists Monitored TFITS systems.

Table 3-1 TFITS Managed Systems

Project Acronym Project Acronym

Active Radio Frequency Identification aRFID Performance Evaluation System PES

Automated Manifest System - Tactical AMS-TAC PR Builder PRB

Automated Performance Evaluation System

APES

Requirements Transition and Manpower

Management Systems

RTAMMS

Defense / Organizational Messaging Service DMS / OMS Radio Frequency Identification RFID

Defense Readiness Reporting System – Marine Corps

DRRS-MC Secure Personnel Accountability

SPA

Joint Force Requirements Generator JFRG Standard Procurement System SPS Marine Corps Recruiting Information Support

System MCRISS Total Force Data Warehouse TFDW

Marine Corps Training Information Management System

MCTIMS Total Force Retention System TFRS

MAGTF Deployment Support Systems II MDSS II Total Force Structure Management System TFSMS

Manpower Mobilization Assignment System MMAS Theatre Medical Information Program TMIP

Manpower MODELS MODELS WEB Manpower Assignment Support System WEBMASS

Optical Digital Imaging – Records Management

ODI-RMS

Table 3-2 lists PdM TFITS portfolio of Supported systems.

Table 3-2 Supported Systems

Project Acronym Project Acronym

Cargo Movement Operation System Cargo Movement Operation System

Last Tactical Mile – In Transit Visibility LTMITV

Defense Casualty Information Processing System DCIPS LOGCOM PDSS PDSS

Defense Personal Property System / Transportation Operational Personal Properly

System

DPPS/TOPPSS

Marine Corps Certification & Accreditation Support Tool MCCAST

Integrated Computerized Deployment System ICODES

Table 3-3 lists PdM TFITS portfolio of Monitored systems.

Table 3-3 TFITS Monitored Systems

Project Acronym Project Acronym Army Container Management System ACAMS GCSS-MC LIS PDSS Support Contract PDSS

Automated Export System/Shipper Export Declaration Form

AES/SED Global Air Transportation Execution System GATES

Automated Governments Transportation Request

AGTR Global Freight Management GFM

Cargo and Billing System CBS Group Passenger GOPAX

Common Naval Packaging Website CNP Integrated Booking System IBS

Defense Enterprise Accounting & Management System

DEAMS Integrated Data Environment Global Transportation Network

IGC

Defense Table of Official Distances DTOD Intelligent Road Rail Information Systems IRRIS

Defense Transportation Tracking System DTTS My Sabre MySabre

Defense Travel System DTS SYNCADA SYNCADA

Discrepancy Identification System DIS Synchronized Pre-deployment and Operational Tracker

SPOT BID

DoD Wood Packing Materials Website WPM Tracker TRACKER

Electronic Document Access EDA Tracker Lite/TGET Tracker Lt

Federal Procurement Data Systems FPDS Transportation Facility Guide TFG

Financial and Air Clearance Transportation

FACTS Wide Area Workflow WAWF

Freight Acquisition Shipping Tool FAST

3.5 Relationships with External Technical Organizations

The primary interface to external organizations is the PjM. However, the assigned engineer will provide the primary technical interface to the vendor, the customer, and interfacing partners. The FRB and the TIMs are the primary method for interfacing with external organizations. The engineer is responsible for engaging SIAT and external test organizations including JITC and Marine Corps Operational Test and Evaluation Agency, where applicable. The engineer will report at the weekly SE IPT all activities engaging external organizations.

3.6 Technical Performance Measures and Metrics

TPM categories have been established and are defined as a representative set for all TFITS projects to use.

Since most TFITS systems are in the sustainment phase of the system lifecycle, and leverage similar web/computer based technologies, TPMs are used to measure system performance to ensure usability during operations and when changes are applied. The TPMs are assessed as part of each test performed, this includes: SIDT in the development and pre-production environments, Government Acceptance Testing (GAT) in the pre-production and production environments, and Production Verification Testing

(PVT). While not required during functional user evaluations, assessments should be performed on (at minimum) a quarterly basis or upon each system baseline change. TPMs will also be used to gauge system obsolescence, thereby providing insight as to feasibility of system upgrades to meet user performance requirements.

3.6.1 Technical Metrics

Technical performance measures are identified in the TFITS PMP and will serve as the guideline for project requirements. It is required that all projects tailor the PMP to establish a performance baseline for their system; these metrics shall be documented in the project level SEP. Project specific metrics shall be reported on in a format capable of comparative analysis to assess performance upon each system change.

Performance shall be addressed through all phases of testing to include: SIDT, User Acceptance Testing, GAT, and PVT.

The Project Engineer shall identify technical metrics from the PMP during requirements development to ensure a proper operational baseline is established and user performance can be measured. Categories of metrics defined in the PMP include:

• Application specific performance measures

• System profile metrics

• Service operations metrics

• Engineering / Integration metrics

• System technical performance measures

• System compliancy metrics

3.6.2 Sustainment Assessment

Assessment of sustainability of the system is based on the PoPS 6.5 assessment. It is the responsibility of the project engineer to work with each PjM on a quarterly basis to update the PoPS 6.5 assessment.

These updates will be supplied for TFITS quarterly reviews and bi-annual briefs provided to the PMM- 110 Tier 0 lead.

The following categories represent the project engineer’s responsible sections to the PoPS 6.5 questions; a sample PoPS template is supplied in Appendix C of this document:

• Test & Evaluation : Gate 6 Sustainment 9.1

Criteria Statement: Major deficiencies identified in Initial Operational Test and Evaluation (IOT&E) and/or Follow-on Operational Test and Evaluation (FOT&E) reports are available for review.

• Test & Evaluation : Gate 6 Sustainment 9.2

Criteria Statement: Approved test plans for correction of deficiencies and/or system modifications are aligned with deployment and sustainment strategies and schedules.

• Technical Maturity : Gate 6 Sustainment 10.1

Criteria Statement: Required elements to manufacture/maintain the end item to specification are in place and are available.

• Technical Maturity : Gate 6 Sustainment 10.2 Criteria Statement: There are currently no major design deficiencies that have not been addressed via implementation of a Class I Engineering Change Proposal (ECPs) or waiver(s).

The project engineer is responsible for working with the PjM to determine an appropriate response to the following PoPS 6.5 metrics questions:

• Software : Gate 6 Sustainment 12.1

Criteria Statement: Software sustainment capability, facilities, and infrastructure have been established and are supporting systems operations. Experienced software maintenance staff is in place and are being retained.

• Software : Gate 6 Sustainment 12.2

Criteria Statement: Software sustainment is being conducted, as detailed in the LCSP, and is being tracked, analyzed, and reported per established software metrics.

3.6.3 Other Measures and Considerations

• Program Protection – All software and hardware components are monitored and tracked for obsolescence. Equipment and software refreshes and upgrades are planned and executed to avoid obsolescence issues.