MS_Intended_Uses_Description_Document_20_Apr_2015_DE_Signed.pdf

MISSILE DEFENSE AGENCY (MDA)

BALLISTIC MISSILE DEFENSE SYSTEM (BMDS)

MODELING AND SIMULATION (M&S)

MDA M&S INTENDED USES DESCRIPTION

DOCUMENT

20 APRIL 2015

DISTRIBUTION STATEMENT D:

Distribution authorized to the Department of Defense (DoD) and U.S. DoD contractors only (Reason, Software Documentation) (15 March 2015). Other requests shall be referred to MDA/DE/S, Building 730, Schriever Air Force Base.

i

Document History

DATE AUTHORIZATION REVISION/CHANGE

DESCRIPTION

PAGES

AFFECTED

20 Apr 2015 See Signature Page Initial Release - Baseline All ii

Table of Contents

1.0 INTRODUCTION

1.1 Intended Uses and the M&S Systems Engineering Process

1.2 Document Overview

1.3 Change Control

2.0 REQUIREMENTS ENGINEERING

2.1 Identify Stakeholders

2.2 Classify Stakeholders

2.3 Capture Stakeholder Needs

2.4 Develop Stakeholder Intended Uses

3.0 INTENDED USE DESCRIPTION

3.1 Intended Use Definition and Content

3.2 Intended Use Category Description

3.3 MDA M&S Intended Use Key Attributes Description

Non-hierarchical Key Attributes ................................................................................. 14 3.3.1 Hierarchical Key Attributes ......................................................................................... 15 3.3.2 Additional M&S Attribute Information ....................................................................... 15 3.3.3

4.0 COMMON M&S FUNCTIONALITY

4.1 Scenario Processing

4.2 Execution Control

4.3 Data Logging and Storage

4.4 Data Analysis

4.5 Information Presentation

5.0 MDA M&S INTENDED USES

5.1 BMDS Concept Evaluation Intended Use Category

Concept Evaluation - Future Concept Analysis .......................................................... 27 5.1.1 Concept Evaluation - System-Level Trade Studies ..................................................... 29 5.1.2

5.2 BMDS Developmental Engineering Intended Use Category

Developmental Engineering - Evaluate Algorithms and Processing Techniques ....... 32 5.2.1 Developmental Engineering - Support Element Engineering ..................................... 34 5.2.2 Developmental Engineering – System Integration Testing ......................................... 36 5.2.3

5.3 BMDS Testing and Verification Intended Use Category

Testing and Verification – Requirements Verification ............................................... 40 5.3.1 Testing and Verification – Scenario Design and Certification .................................... 42 5.3.2 Testing and Verification – Flight Test Analysis.......................................................... 44 5.3.3 Testing and Verification - Flight Test Training .......................................................... 46 5.3.4 Testing and Verification - Developmental and Operational Test ................................ 48 5.3.5

5.4 BMDS Assessment Intended Use Category

Assessment – Characterize BMDS Capabilities and Limitations ............................... 52 5.4.1 Assessment – Evaluate BMDS Effectiveness ............................................................. 54 5.4.2

5.5 BMDS Training Intended Use Category

Training – National-Level and CCMD Training (Tier 1) ........................................... 58 5.5.1 Training – Joint Task Force (JTF) Training (Tier 2) ................................................... 60 5.5.2 Training – Functional and Service Component Training (Tier 3) ............................... 62 5.5.3 Training – Individual Organization Training (Tier 4) ................................................. 64 5.5.4 iii

5.6 BMDS Operational Support Intended Use Category

Operational Support - BMDS Planning ....................................................................... 67 5.6.1 Operational Support - Quick Response Analysis ........................................................ 69 5.6.2

6.0 STAKEHOLDER CLASS BY MDA M&S INTENDED USE

7.0 MDA M&S INTENDED USE KEY ATTRIBUTE SUMMARY

8.0 BMD SYSTEM-LEVEL M&S - ROSETTA STONE

9.0 COMPARISON OF SIMULATION CONCEPTUAL MODEL (SCM)

ATTRIBUTES AND MDA M&S INTENDED USE KEY ATTRIBUTES

10.0 ADDITIONAL ATTRIBUTES

11.0 DEFINITION OF TERMS

12.0 REFERENCES

13.0 ACRONYM LIST

Table of Figures FIGURE 2-1: Requirements Engineering Concept [7] FIGURE 2-2: Elicit Stakeholder Needs Step Details [7] FIGURE 2-3: MDA M&S Intended Use Stakeholder Classes FIGURE 3-1: MDA M&S Intended Use Categories Relationship to MDA BMDS

Systems Engineering Plan (SEP) Systems Engineering “V” FIGURE 3-2: MDA M&S Intended Use Categories Relationship to Defense Acquisition

Guidebook (DAG) Systems Engineering Technical Processes FIGURE 4-1: Common Functional Requirements FIGURE 5-0: MDA M&S Intended Uses by Category and Common M&S Functionality FIGURE 5-1: BMDS Concept Evaluation M&S Intended Uses FIGURE 5-2: BMDS Developmental Engineering M&S Intended Uses FIGURE 5-3: BMDS Testing and Verification M&S Intended Uses FIGURE 5-4: BMDS Assessment M&S Intended Uses FIGURE 5-5: BMDS Training M&S Intended Uses FIGURE 5-6: BMDS Operational Support M&S Intended Uses FIGURE 8-1: Example of the BMD System-Level M&S - Rosetta Stone Product

Table of Tables TABLE 3-1: MDA M&S Intended Use Key Attribute Enumerations Summary TABLE 3-2: MDA M&S Intended Use Key Attribute Definitions and Enumeration

Descriptions iv

1.0 INTRODUCTION

Purpose: This document defines and describes the Missile Defense Agency (MDA) Modeling and Simulation (M&S) Intended Uses, identifies the importance of Intended Uses in a M&S systems engineering process, and establishes a lexicon for the MDA Directorate for Engineering (MDA/DE) that will assist in the execution of its responsibilities.

Scope: This document applies to staff in the MDA/DE organizations that fund, develop, support, and/or use Ballistic Missile Defense System (BMDS) models, simulations, or the data supplied by them.

Additionally, this document applies to MDA/DE staff that fund external development of non-MDA Element M&S.

The MDA is developing an M&S program that will support:

• BMD system concept evaluation.

• Engineering development of the BMD system and its components.

• BMD system verification and validation.

• Credible assessment of system-level effectiveness and functionality.

• Required Warfighter (WF)/Operator training.

• Real-world operational analysis.

The MDA M&S program supports a large stakeholder community that uses a variety of models and simulations to support their work. These stakeholders have ‘needs’1 that must be well-understood to develop and provide suitable models and simulations. The process of capturing stakeholder ‘needs’ results in a set of MDA M&S Intended Uses. These MDA M&S Intended Uses apply to the software representations of BMD Components, Elements, and Systems needed to support the MDA program.

Non-BMD system representations by software (e.g., man-made and natural environment, lethality, threat models, etc.) are not in the scope of this document.

The definition of Intended Use is: A description of the problem to be addressed by a model or simulation and its associated data, including the system or process being represented and the role it plays in the overall program. [1] [2].

1.1 Intended Uses and the M&S Systems Engineering Process

Intended Uses are developed because they influence all phases of an M&S systems engineering process.

Intended Uses are important because they bound the scope of models and simulations to be represented, guide model and simulation requirements development, and support model and simulation accreditation [1]. Intended Uses can also drive high-level model-based systems engineering (MBSE) use cases and assist in determining whether a ‘legacy M&S’2 application3 or product4 is appropriate for an Intended

1 A ‘need’ is an operational capability description that describes what the system will accomplish and how well.

2 Any model or simulation that was developed either in the past or for a different purpose. [55]

Use different from its original designed or developed Intended Use.

The Intended Use influences the verification, validation, and accreditation (VV&A) process that an M&S application or product undergoes to determine its credibility to solve the problem5. The need for VV&A is derived from a model’s or simulation’s Intended Use and the risk associated with decisions being made using the M&S’s data. The greater the consequence of a decision using the M&S data the more diligent, rigorous, and thorough the VV&A needs to be. The VV&A process is tied to the importance of the activity being conducted (i.e., how important is the problem being solved with respect to the overall program6?) and the degree of data accuracy required (i.e., what problem is the M&S associated data7 addressing?). [3]

The Intended Use provides the M&S community a starting point for the selection of an approach to a solution. A legacy M&S Intended Use helps the M&S community understand the original purpose and scope of legacy M&S applications and products and how it relates to the problem in need of a solution.

The typical approaches to solve the problem are [4]:

• Use of non-M&S methods.

• Use of an available M&S, as-is.

• Modification of an existing M&S.

• Creation of a new M&S.

The approach to solve the problem is important because it directly affects cost, schedule, manpower, and technical performance of an M&S program (i.e., M&S programmatics). The ‘Use of an available M&S, as-is’ solution will require no M&S development and existing VV&A material should suffice to accredit the ‘M&S, as-is’ that will be utilized to solve the problem in need of a solution. This approach typically will have lower cost, shorter schedule, and can be technically achieved. In contrast, ‘Creation of a new M&S’ solution will require the execution of a complete software development life cycle process to create the new M&S to solve the problem in need of a solution. In this case, there is no M&S that exists to solve the problem. This approach will have higher cost, longer schedule duration, probable manpower impacts, and need to be technically assessed. The ‘Modification of an existing M&S’ approach is somewhere in-between the ‘M&S, as-is’ and ‘new M&S’ approaches, and the ‘Use of non- M&S methods’ approach is outside the scope of an M&S systems engineering process.

The Intended Use assists in developing associated M&S documents. The M&S documents are typically:

requirements, framework architectures, software conceptual models, MBSE artifacts, VV&A plans, and VV&A reports. These documents should have the same context and relate to the problem being solved;

3 Executing software on a host computer that models or simulates one or more BMDS entities. The M&S Application represents real-world entities for the purpose of training, analysis, or experimentation and requires an accreditation assessment for a specific intended use.

4 A composition of M&S Applications into a configuration that represents the BMDS suitable or tailored to meet objectives of a venue or event. This configuration requires an accreditation for a specific intended use.

5 Refer to the Intended Use definition in Section 1.0.

6 Ibid.

7 Ibid.

include what M&S data is needed to solve the problem; describe what ‘needs’ to be represented and how well, and how the representation contributes to the overall program. Since these M&S documents are usually created by different groups and individuals, an Intended Use provides a common understanding of the problem to ensure interconnection among the various documents. To ensure this common understanding, the M&S documents can be written from the perspective of an Intended Use and describe such things as [5]:

• Program risks and impacts associated with developing and/or using a model or simulation.

• Program risks resulting from identified constraints and limitations.

• Program risks associated with doing or not doing various VV&A tasks.

• Software development and VV&A impediments.

• The extent that a model or simulation is an accurate representation of the real world.

1.2 Document Overview

This document starts with a brief overview of the M&S requirements engineering process (Section 2.0) to provide the context of how MDA M&S Intended Uses are developed. The MDA M&S Intended Uses are organized into six categories: BMDS Concept Evaluation (Section 5.1), BMDS Developmental Engineering (Section 5.2), BMDS Testing and Verification (Section 5.3), BMDS Assessment (Section 5.4), BMDS Training (Section 5.5), and BMDS Operational Support (Section 5.6). An overview description of each category is provided at the start of each category in Section 5.0 MDA M&S Intended Uses, followed by the MDA M&S Intended Uses in that category.

The contents of an MDA M&S Intended Use description are defined in Section 3.0. In particular, Section 3.1 describes the information contained in an Intended Use description; Section 3.2 describes the Intended Use Categories; Section 3.3 describes the Intended Use Key Attributes. The reader should review and understand the Key Attribute definitions and their enumerations described in Section 3.3 before reading the MDA M&S Intended Uses described in Section 5.0. Each Intended Use description starts at the top of a new page to ensure a “fact sheet” format for each Intended Use with acronyms always spelled out.

Section 4.0 describes common M&S functionalities required across all MDA M&S Intended Uses.

These functionalities are necessary to ensure the successful use of M&S to satisfy a particular Intended Use. The information is presented for completeness and does not influence the descriptions in Section

5.0 MDA M&S Intended Uses.

Section 6.0 contains the list of MDA M&S Intended Use stakeholders and their classifications as cited in the Requirements Engineering Process, Section 2.2 Classify Stakeholders.

Section 7.0 contains a summary of the MDA M&S Intended Use Key Attributes Enumerations as cited in the Intended Use Description, Section 3.3 MDA M&S Intended Use Key Attributes Description.

Section 8.0 contains an example of the M&S Program Directorate (MDA/DE/S) maintained product called the BMD System-Level M&S - Rosetta Stone. The product is a diagram depicting the complex nature of the BMD System-Level M&S by showing the product and process interdependencies starting with the requirements chain and ending with an event architecture. Updates to the BMD System-Level M&S - Rosetta Stone are independent of this document and occur as necessary.

The remaining document sections are provided as referent and standard content:

Section 9.0 Comparison of Simulation Conceptual Model (SCM) Attributes and MDA M&S Intended Use Key Attributes.

Section 10.0 Additional Attributes.

Section 11.0 Definition Of Terms.

Section 12.0 References.

Section 13.0 Acronym List.

1.3 Change Control

This document serves as the source of MDA M&S Intended Use information to support the MDA/DE systems engineering process. This document was developed under the authority of the MDA Directorate for Engineering – Modeling and Simulation Program (MDA/DE/S) [6]. Recommendations for changes should be submitted to the Director of MDA/DE/S. This document will be updated annually and when necessary.

2.0 REQUIREMENTS ENGINEERING

Based on the information provided by the BMDS Architect (MDA/DE/B) and the BMDS Systems Engineer (MDA/DE/E) as described in the BMDS Systems Engineering Plan (SEP), the DE/S Enterprise Team will follow a model-based systems engineering (MBSE) approach to develop and modify models and simulations.

MBSE produces a system model that includes M&S requirements and specifications, design, analysis, and verification artifacts. The MBSE approach provides a more comprehensive, reliable, and traceable model or simulation design. DE/S will use System Modeling Language (SysML) as the visual modeling language. SysML not only supports rigorous modeling techniques among Systems Engineers, it also serves as a ‘lingua franca’8 among various types of engineers (Software Engineers, Electrical Engineers, Mechanical Engineers, etc.) and stakeholders who need to understand the MBSE work products. These work products include: visual requirements analysis and verification, Concept of Operations (CONOPS), functional analysis and allocations, performance analysis, trade studies, and system architectures.

An M&S requirements engineering process follows an Elicit–Extract– Report sequence embedded within a requirements management

8 “bridge language” or “serves to bridge communication”

Requirements Management

Requirements Baseline

Requirements Engineering

Report Requirements

Elicit Stakeholder

Needs

Extract M&S Requirements

FIGURE 2-1:

Requirements Engineering

Concept [7] function as depicted in FIGURE 2-1: Requirements Engineering Concept [7]. The development of Intended Uses occurs during the Requirements Engineering - Elicit Stakeholder Needs step in FIGURE 2-1 and is detailed in FIGURE 2-2: Elicit Stakeholder Needs Step Details [7]. FIGURE 2-2 and the associated paragraphs describe the process followed to develop MDA M&S Intended Uses. The paragraphs in Sections 2.1 thru 2.4 describe the process depicted in FIGURE 2-2.

FIGURE 2-2: Elicit Stakeholder Needs Step Details [7]

2.1 Identify Stakeholders

To develop M&S ‘needs’, an engineer must identify who requires M&S (i.e., who are the stakeholders).

(Step 1, FIGURE 2-2). A stakeholder has an interest, share, or claim in the model or simulation; can be affected by the model or simulation; or is in possession of model or simulation characteristics that meet their ‘needs’. This step identifies the M&S stakeholders for an MDA M&S Intended Use.

2.2 Classify Stakeholders

The Classify Stakeholders step places stakeholders into classes (Step 2, FIGURE 2-2). These classes provide the engineer with context regarding the stakeholder’s point of view, relationship to the system, rank order of importance, and motivations. The classes also provide insight into where the system will be deployed, operated, and sustained. This supplies the engineer with background that lends perspective and understanding to resolve conflicting or competing ‘needs’.

Each M&S stakeholder class levies the ‘needs’ the M&S must fulfill to support the Intended Use. The MDA M&S Intended Use stakeholder classes are depicted in FIGURE 2-3 and are described in the following paragraphs. These identified classes are sufficient for the purpose of this document.

FIGURE 2-3: MDA M&S Intended Use Stakeholder Classes

M&S User – The term used to represent the organization, group, or person responsible for the overall use of the model or simulation. The M&S User needs to solve the problem or make a decision, and wants to use models or simulations to do so. The M&S User defines the requirements, establishes the criteria by which model or simulation fitness will be assessed [validation], determines what method or methods to use [for the model or simulation assessment], makes the accreditation decision, and ultimately accepts the results. [1] The M&S User is interested in the M&S associated data generated by the model or simulation.

M&S Developer – The agency that develops a model or simulation or the agency that is overseeing the model or simulation development by a contractor. [1] The M&S Developer performs verification of the model or simulation.

Operator – Represents personnel who are the consumers (typically those who will actually operate the [BMD] system the M&S represents) of the model or simulation and includes a normal operator, maintenance operator, operational support, planner, analyst, trainer, trainee, and integrator. [8] The Warfighter is the Operator of a BMD system.

Others – This class is for personnel, organizations or agencies that are not identified in one of the above classes.

The results of identifying (step 1) and classifying (step 2) the stakeholders are in Section 6.0 Stakeholder Class By MDA M&S Intended Use. Each MDA M&S Intended Use described also contains a list of stakeholders for that particular Intended Use.

2.3 Capture Stakeholder Needs

After identifying and classifying the stakeholders, an engineer must define stakeholder ‘needs’ for the M&S (Step 3, FIGURE 2-2). A ‘need’ is an operational capability description that describes what the system will accomplish and how well. [9] There are many techniques available to an engineer to capture ‘needs’ including interviews, surveys, focus groups, stakeholder feedback, rapid prototyping, and field reports. M&S ‘needs’ are expressed from a stakeholder’s perspective as either a new M&S capability or improvements to existing M&S capabilities. With the captured ‘needs’, the engineer will have the context to develop the M&S to solve the problem.

2.4 Develop Stakeholder Intended Uses

The stakeholder ‘needs’ should be expressed in a simple, concise, and understandable format to ensure proper development of capabilities and outcomes of the systems engineering and accreditation processes (Step 4, FIGURE 2-2). The engineer collaborates amongst stakeholders to focus the ‘needs’ into M&S Intended Uses; not all ‘needs’ correlate to a problem with an M&S solution, so not all ‘needs’ contribute to the Intended Uses. The Intended Uses will assist M&S engineers, developers, and accreditors in performing their M&S tasks because a clear understanding of the problem being solved is established.

Intended Uses are important because they bound the scope of the problem being solved, assist in determining of how well the M&S solution to the problem must be represented, and support the determination of the credibility of the M&S solution to the problem; i.e., Intended Uses ensure that the proper M&S applications and products are developed to meet the ‘needs’ of the stakeholders.

The results of capturing ‘needs’ (step 3) and developing Intended Uses (step 4) are in Section 5.0 MDA M&S Intended Uses.

3.0 INTENDED USE DESCRIPTION

In June 2010, The Johns Hopkins University Applied Physics Laboratory (JHU/APL) was tasked, by the Office of the Director – Defense Research and Engineering - Systems Engineering9, to define a “systems engineering framework” focused on the development of models and simulations, to identify practices for the efficient development and evolution of credible models and simulations, and to integrate these practices into a best practices “systems engineering framework”. The JHU/APL study team performed a literature search to identify major “systems engineering frameworks”. Each framework was assessed to identify its applicability to the M&S domain, along with its relative strengths and weaknesses [10]. Of the seven major “systems engineering frameworks” surveyed, four identified the importance of defining the Intended Use for a model or simulation. The examination of the Best Practices for the Development of Models and Simulations; Technical Report NSAD-R-2010-037, generated by JHU/APL, led to the development of the MDA M&S Intended Uses.

3.1 Intended Use Definition and Content

The definition of Intended Use (IU) is:

A description of the problem to be addressed by a model or simulation and its associated data, including the system or process being represented and the role it plays in the overall program [1] [2].

The above compound sentence can be broken down in the context of MDA as follows:

9 now known as the Office of the Deputy Assistant of Defense – Systems Engineering (ODASD (SE))

1. What problem is the BMD model or simulation and M&S associated data addressing?

2. What portion of the BMD system (i.e. Component, Single Element, Multiple Elements, or an entire System (e.g., CCMD Area of Responsibility (AOR))) needs to be represented to address the problem?

3. How does the system or process being represented by the M&S contribute to the MDA missions?

An Intended Use is not a venue10, capability, solution, or implementation; an Intended Use is a problem statement. The problem statement serves as the foundation for the definition of requirements, acceptability criteria, and ultimately the accreditation assessment of a model or simulation. The problem statement documents (1) the question(s) to be answered and the particular aspects of the problem that the M&S will be used to help address; (2) the decisions that will be made based on the M&S data; and (3) the consequences resulting from erroneous M&S outputs [1].

An MDA M&S Intended Use contain these five components (various examples are in italics):

1. IU Category – IU Name – An IU Category allows for the grouping of similar Intended Uses.

The MDA M&S Intended Use categories are described in Section 3.2. An IU Name is for ease of communication and reference.

Testing and Verification – Requirements Verification

2. Description – An Intended Use Description (a.k.a. the problem statement) contains five elements.

The elements may not be grammatically correct because of the nature of the item being documented.

a. The first element is a narrative to identify the problem.

This Intended Use employs M&S to examine potentially fielded BMD system configurations or deployments to explore innovative methods of operation, assess feasibility concepts, evaluate utility, or determine limitations that may improve BMD system effectiveness and/or functionality.

b. The second element describes what ‘tasks or processes’ the Intended Use supports. This provides some insight into the inputs.

1) Evaluation of new capabilities and requirements identified in specification documents.

2) Satisfying Assessment Requirements (ARs) that support requirements verification.

3) Satisfying Hardware-in-the-loop (HWIL) test objectives that support requirements verification.

10 A Live, Virtual, and/or Constructive environment that employs representations of the Blue Force, Red Force, communication infrastructure, and man-made and natural effects to produce data that services one or more Intended Uses.

These representations may include a combination of tactical hardware, tactical software, physical models, simulations, and supporting infrastructure.

c. The third element is the ‘controls or governances’ that identify documents, procedures or processes to which the Intended Use conforms. This also provides some insight into the inputs.

1) MDA Initial Requirement Documents (IRDs).

2) MDA Technical Objectives and Goals (TOG).

3) Warfighter Concept Plans (CONPLAN).

4) Warfighter Doctrine.

5) Warfighter Operation Plans (OPLAN).

d. The fourth element identifies the ‘venues or M&S computing environments11’ under which the model or simulation should run to produce the needed outputs & data.

1) HWIL – Distributed.

2) HWIL – Focused.

3) HWIL – Integrated.

4) Integration Exercises.

e. The fifth element identifies model and simulation ‘outputs’ used to produce the products or documents by an analyst. Information contained in { } identifies an output owner.

1) Inputs to BMD System Verification and Validation (V&V) reports.

2) Inputs to BMD System Verification Closure Notices (VCN) {EEV}.

3) Inputs to Element-level Verification Closure Notices {MDA Elements}.

4) Inputs to M&S V&V documentation.

5) Inputs to the BMD System Operations Readiness and Acceptance (OR&A) process.

6) Inputs to the Integrate Master Assessment Plan (IMAP) Verification Assessment products {EEA}.

3. Key Attributes – The MDA M&S Intended Use Key Attributes provide a starting point for discussions with stakeholders to develop and refine requirements for a model or simulation. The MDA M&S Intended Use Key Attributes also provide a method to identify commonalities and differences among MDA M&S Intended Uses. The MDA M&S Intended Use Key Attributes are described in Section 3.3. The following is an example of key attributes for an MDA M&S Intended Use.

Focus Epoch Simulation Type Fidelity Single Element, Multiple Elements, BMD System

Current (N), Next (N+1)

Live, Constructive

Medium

Uncertainty Quantification

Tactical Interface Stimulation Interface

Operator Screens

Non-Reproducible AU Emulation Medium Detail Approximation

11 An on-demand computing environment from which various tasks are performed using M&S applications and products.

The environment can be a standalone PC, a Unix-based workstation, or a client-server system.

4. Stakeholders – The list of M&S stakeholders for the MDA M&S Intended Use. The Primary stakeholders, identified in a list of stakeholders, have a responsibility to coordinate with other affected stakeholders.

Primary:

• MDA/DE/B – BMD System Architect.

• MDA/DTWX – Wargames and Exercises.

• Warfighter.

MDA Element Program Offices.

MDA/DE/S – Modeling and Simulations.

MDA/DV - Advanced Technology.

MDA/DVR – Advanced Research.

MDA/DVS – Advanced Concepts.

MDA/ESO – Non-MDA Elements.

5. References – Additional information sources concerning the MDA M&S Intended Use.

3.2 Intended Use Category Description

The Intended Use categories provide a method to group problem statements. The grouping assists stakeholders to understand which MDA M&S Intended Uses are relevant to them. The grouping philosophy is best described when the MDA M&S Intended Use categories are compared against the MDA BMDS Systems Engineering Plan (SEP) Systems Engineering “V” (FIGURE 3-1) and the Defense Acquisition Guidebook (DAG) Systems Engineering Processes (FIGURE 3-2) diagrams. These “V” diagrams represent the systems engineering processes followed by the MDA community to design and develop the BMD system12. The placement of an Intended Use into a category depends on where the problem occurs in the overall BMD systems engineering process. The left side of a “V” represents concept development and the decomposition of the ‘needs’ and requirements into a functional BMD system that can be architected, designed, and developed. The MDA M&S Intended Uses that support the left side of the “V” are in the BMDS Concept Evaluation and BMDS Developmental Engineering categories. The right side of a “V” represents the integration, requirements verification, system evaluation, and operational use of a BMD system. The MDA M&S Intended Uses that support the right side of the “V” are in the BMDS Testing and Verification, BMDS Assessment, BMDS Training, and BMDS Operational Support categories. The mapping of Intended Uses to the BMD systems engineering “V” provides a natural and logical grouping of the Intended Uses. In addition, this “V” related grouping also provides the following information:

• How the Intended Use contributes to the MDA missions (i.e., the role). The preamble of each category describes the contribution each category, and the associated Intended Use, make to the MDA missions.

• The degree of data accuracy required.

12 While the design and development of a representation of a BMD Component, Element, or System (i.e., a model or simulation) will follow similar “V’s”, the “V’s” shown here are for the design and development of the BMD system.

• The impact and consequence of making a decision based on the M&S associated data. As an example: The M&S data generated for a BMDS Concept Evaluation Intended Use Category will have less overall MDA mission impact than the M&S data generated for a BMDS Operational Support Intended Use Category.

The placement of an Intended Use into a category also depends on the M&S associated data importance to the MDA missions. As an example, in the BMDS Testing and Verification Intended Use Category, there are two Intended Uses: ‘Scenario Design and Certification’ and ‘Requirements Verification’.

Scenarios are designed and developed for any of the Intended Use categories; however, the consequence of a decision, based on the scenario data, becomes important to the MDA missions when decisions on BMD system verification and validation are being made. The scenario data becomes important in a critical decision-making process. This is the first time the scenario data could lead to consequences of a decision affecting the MDA missions; therefore, the ‘Scenario Design and Certification’ Intended Use is in the BMDS Testing and Verification Intended Use Category. The same concept applies to the ‘Requirements Verification’ Intended Use; in any category, verification of BMD system requirements could occur; however, the Intended Use becomes significant to the MDA missions during the BMD system testing and verification decision process.

Based on the problem and M&S associated data, the Intended Use categories provide a method to organize thoughts; group things that have common functionality, attributes, or purpose; and provide a way to measure a degree of importance to the MDA missions. Therefore, it is possible that a given M&S application or product could be accredited for more than one Intended Use and/or for one or multiple Intended Use categories (e.g., the BMD International Simulation (I-Sim) product supports the BMDS Concept Evaluation Intended Use Category - Future Concept Analysis Intended Use and the BMDS Training Intended Use Category).

The six MDA M&S Intended Use categories are:

5.1 BMDS Concept Evaluation Intended Use Category.

5.2 BMDS Developmental Engineering Intended Use Category.

5.3 BMDS Testing and Verification Intended Use Category.

5.4 BMDS Assessment Intended Use Category.

5.5 BMDS Training Intended Use Category.

5.6 BMDS Operational Support Intended Use Category.

FIGURE 3-1: MDA M&S Intended Use Categories Relationship to MDA BMDS Systems

Engineering Plan (SEP) Systems Engineering “V”

FIGURE 3-2: MDA M&S Intended Use Categories Relationship to Defense Acquisition Guidebook

(DAG) Systems Engineering Technical Processes13

13 Per the Technical Processes box, the Product would undergo Verification before a Validated Solution is reached as depicted in the diagram.

3.3 MDA M&S Intended Use Key Attributes Description

Attributes are those model and simulation features and characteristics that come up repeatedly in discussions with stakeholders. The model and simulation attributes provide a starting point for discussions with stakeholders to develop and refine requirements for a model or simulation. The MDA M&S Intended Use Key Attributes, and their enumerations, also provide a method to identify commonalities and differences among MDA M&S Intended Uses. These key attributes are also important in the identification of legacy M&S that can be re-used for a different Intended Use.

Eight Key Attributes have been identified to focus the M&S Intended Uses for MDA. The TABLE 3-1 MDA M&S Intended Use Key Attribute Enumerations Summary shows these Key Attributes and a summary of their enumerations. The detailed Key Attribute definition and enumeration descriptions are in TABLE 3-2 MDA M&S Intended Use Key Attribute Definitions and Enumeration Descriptions.

Readers should read and understand the attribute enumerations to completely comprehend an MDA M&S Intended Use. Each MDA M&S Intended Use description has its own Key Attribute section. The selection of enumerations occurred during the requirements engineering process as described in Sections

2.3 Capture Stakeholder Needs and 2.4 Develop Stakeholder Intended Uses. A Key Attribute may have an enumeration of “---” which means that the Key Attribute does not apply or is simply not important for an MDA M&S Intended Use.

The MDA M&S Intended Use Key Attributes focus on the M&S associated data needed to address the problem. There are two types of MDA M&S Intended Use Key Attributes: non-hierarchal and hierarchal.

TABLE 3-1: MDA M&S Intended Use Key Attribute Enumerations Summary

Focus Epoch Simulation Type Fidelity

Component

Single Element

Multiple Elements

BMD System

Current (N)

Next (N+1)

Future I (N+2 to 5)

Future II (> N + 5)

Live

Virtual

Constructive

High

Medium

Low

Uncertainty Quantification

Tactical Interface Stimulation Interface

Operator Screens

Full

Reproducible AU

Non-Reproducible AU

Sensitivity Analysis Only

Replication

Emulation

Faithful

Approximation

None

High Detail

Medium Detail

Low Detail

Replication

Faithful

Approximation

None

Non-hierarchical Key Attributes 3.3.1 The enumerations for non-hierarchical Key Attributes are independent of each other and therefore they capture different aspects of the problem based on the M&S associated data needed. The non-hierarchical attributes can lead to many possible combinations of data ‘needs’ (i.e., multiple Key Attribute sections for a single Intended Use); however, the combinations are presented as a single Key Attribute section because the multiple enumerations situations identify discussions that need to occur with stakeholders to refine model and simulation requirements. The non-hierarchical attributes can imply there are multiple data emphasis areas needed by the stakeholders, and models and simulations will need to generate data for these various data emphasis areas. It should be noted that the various combinations of Key Attributes do not imply that the same model or simulation meet all the combinations of ‘needs’; however, it is plausible that a single model or simulation could be accredited to cover multiple data emphasis areas of an MDA M&S Intended Use.

The Focus and Epoch attributes are non-hierarchical and can have multiple selections. A Focus enumeration of ‘Multiple Elements, BMD System’ means there are two data emphasis areas. As an example, in the case of 5.1.1 Future Concept Analysis, which says ‘…examine developmental or conceptual BMD system configurations…’, the data of interest may be between a THAAD and an Aegis (i.e., Multiple Elements), or the data of interest may involve a CCMD AOR (i.e., BMD System); the problem is the same, just a different data emphasis. The same applies to the Epoch attribute only with respect to the timeframe of a data emphasis. The Focus and Epoch attributes address the ‘what portion of the BMD system needs to be represented to address the problem’ part of the Intended Use definition.

The Simulation Type attribute is non-hierarchical and can also have multiple selections. The need for multiple selection stems from the definitions of live14, virtual15, and constructive16. A Simulation Type enumeration of ‘Virtual, Constructive’ means there are two environments where the M&S ‘needs’ to generate the M&S associated data; however, determining which environment is used to generate the M&S associated data, depends on the unique aspects of the problem being solved. Discussions with stakeholders to refine M&S requirements will scope the M&S operational environment. As an example, in the case of 5.1.1 Future Concept Analysis, which says ‘…examine developmental or conceptual BMD system configurations…’, real-world assets (a.k.a. Live) will not be used (i.e., not an enumeration); however, it is very likely that a human (a.k.a. virtual) or a computer (a.k.a. constructive) will operate the BMD system and make decisions during the execution of the M&S.

14 Live simulation involves real people operating real systems. Military training events using real equipment are live simulations. They are considered simulations because they are not conducted against a live enemy. [11] 15 A simulation involving real people operating simulated systems. Virtual simulations inject human-in-the-loop in a central role by exercising motor control skills (i.e., flying an airplane), decision skills (i.e., committing fire control resources to action), or communication skills (i.e., as members of a C4I team). [11] 16 A constructive simulation includes simulated people operating simulated systems. Real people stimulate (make inputs) to such simulations, but are not involved in determining the outcomes. A constructive simulation is a computer program. For example, a military user may input data instructing a unit to move and to engage an enemy target. The constructive simulation determines the speed of movement, the effect of the engagement with the enemy and any battle damage that may occur. [11]

Hierarchical Key Attributes 3.3.2 The hierarchical Key Attributes have a single enumeration because they describe the “level of detail” needed for the data. The minimum “level of detail” needed to generate the data is captured from the stakeholder(s) with the minimum ‘need’ because a model or simulation with characteristics above a minimum attribute enumeration will satisfy the ‘need’. While all of the Key Attributes support the comparison of Intended Uses, the hierarchical attributes provide the greatest level of contrast.

The Fidelity, Uncertainty Quantification, Tactical Interface, Stimulation Interface, and Operator Screens attributes are hierarchical in nature. These five attributes address the “level of detail” needed to generate the M&S associated data to address the problem. The enumerations for these key attributes represent the minimum attribute (i.e., minimum “level of detail”) to solve the problem. For example; if an MDA M&S Intended Use has a Fidelity enumeration of Medium and a model or simulation has High fidelity characteristics, then that model or simulation would meet the Intended Use; a Low fidelity model or simulation would not meet the Intended Use. A model or simulation with characteristics above a minimum attribute enumeration will satisfy a particular attribute. These five attributes also need to be considered as a whole; the selection of one enumeration may influence the selection of another enumeration. For example; a Fidelity - High, Uncertainty Quantification - Full model or simulation is beyond MDA’s available computer environments because of the amount of time required to reasonably generate the data. A Fidelity – Medium, Uncertainty Quantification - Full model or simulation is likely to be achievable.

Additional M&S Attribute Information 3.3.3 An in-development or legacy model or simulation could have additional attributes than the ones shown in TABLE 3-2: MDA M&S Intended Use Key Attribute Definitions and Enumeration Descriptions, or greater detail, or an expansion of the attributes shown in TABLE 3-2. An example of this is described in Section 9.0 Comparison of Simulation Conceptual Model (SCM) Attributes and MDA M&S Intended Use Key Attributes. This attribute comparison does not influence the contents of the MDA M&S Intended Uses.

Also, during the development of this document, other M&S attributes were identified and are captured in Section 10.0 Additional Attributes for reference. These additional attributes are suitable for the refinement of stakeholder software and hardware ‘needs’. These additional attributes do not influence the contents of the MDA M&S Intended Uses.

TABLE 3-2: MDA M&S Intended Use Key Attribute Definitions and Enumeration Descriptions

# IU Key Attribute Definition Enumerations 1 Focus This attribute refers to the focus of the M&S

Intended Use, whether it is at a System-level or at the Element or Component -level.

1. Component: individual piece(s). (e.g., Element’s radar, launcher, or sensor; a communication network.)

2. Single Element: one Element. (e.g., THAAD, Aegis, or

PATRIOT).

3. Multiple Elements: includes two or more Elements, but does not include system- or mission-level support.

4. BMD System: collection of Elements integrated to support a specific BMD mission. May vary by Combatant Commands (CCMD’s) area of responsibility (AOR), may also include multiple CCMDs AOR’s.

2 Epoch This attribute refers to the planned time period of capability deployment following the engineering development of a system.

1. Current (N): Capabilities commensurate with the fielded operational system.

2. Next (N+1): Capabilities commensurate with the next fielded operational system.

3. Future I (N+2 to 5): Capabilities representing systems potentially fielded in the next two to five fiscal years.

4. Future II (> N + 5): Capabilities representing systems potentially fielded in five or more fiscal years; beyond the Future Years Defense Program (FYDP). May be exploratory in nature.

3 Simulation Type This attribute explains the level of simulation needed to perform the M&S Intended Use.

1. Live Simulation: involves real people operating real systems. Military training events using real equipment are live simulations. They are considered simulations because they are not conducted against a live enemy.

2. Virtual Simulation: involves real people operating simulated systems. A video game or cockpit mockups used to train pilots are examples of virtual simulation.

A simulation involving real people operating simulated systems. Virtual simulations inject human-in-the-loop in a central role by exercising motor control skills (i.e., flying an airplane), decision skills (i.e., committing fire control resources to action), or communication skills (i.e., as members of a Command, Control, Communications, Computers, and Intelligence (C4I) team).

3. Constructive Simulation: involves simulated people operating simulated systems. A constructive simulation is a computer program. For example, a military user may input data instructing a unit to move and to engage an enemy target. The constructive simulation determines the speed of movement, the effect of the engagement with the enemy, and any battle damage that may occur.

[11]

4 Fidelity 1. The degree to which a model or simulation reproduces the state and behavior of a real world object or the perception of a real world object, feature, condition, or chosen standard in a measurable or perceivable manner; a measure of the realism of a model or simulation;

faithfulness. Fidelity should generally be described with respect to the measures, standards or perceptions used in assessing or stating it. 2. The methods, metrics, and descriptions of models or simulations used to compare those models or simulations to their real world referents or to other simulations in terms of measures, standards, or perceptions. Note: This requires the M&S User to present clearly-defined characteristics to be assessed, along with the quantifiable parameters that are required to support the assessment.

Fidelity measures, standards or perceptions include (see glossary for definitions):

- Accuracy

- Precision

- Repeatability

- Resolution

- Scope

- Sensitivity [12]

1. High: Subjective judgment of measures, standards or perceptions which concludes that the aggregation of measures, standards or perceptions is very similar to a real world object’s features.

2. Medium: Subjective judgment of measures, standards or perceptions which concludes that the aggregation of measures, standards or perceptions is somewhat similar to a real world object’s features.

3. Low: Subjective judgment of measures, standards or perceptions which concludes that the aggregation of measures, standards or perceptions is marginally similar to a real world object’s features.

5 Uncertainty

Quantification (UQ) The capability of a representation (i.e., model, simulation, HWIL, or real-world asset) to provide a quantitative estimate of the uncertainties in simulation outputs. The purposes of the estimate are

1) to bound the uncertainty (precision) of the M&S-based estimate of the performance

2) to provide information to reduce the uncertainty in the M&S-based performance estimate.

Aleatoric Uncertainties (AUs) are “known unknowns” representing aspects of the BMDS or core truth that are not controllable (by anyone) or perfectly predictable;

terrestrial weather is a good example of AU.

Epistemic Uncertainties (EUs) are “unknown unknowns” representing aspects of the BMDS or core truth that are ideally or notionally controllable or predictable, but perhaps not in practice. (i.e., Red order-of-battle; Red knows perfectly but Blue does not.)

See Glossary, Uncertainty Quantification, for more details.

[13], [14]

1. Full: representation characterizes all AUs and EUs of inputs, models and software.

• Representation execution is fully reproducible.

• Representation propagates AUs by Monte Carlo sampling or other appropriate method.

• Representation provides automated sensitivity analysis or simulation optimization for EU propagation.

• Representation enables joint propagation of both AUs and EUs to simulation output uncertainties.

2. Reproducible AU: representation provides Monte Carlo sampling or other appropriate method for AU propagation.

• Representation explicitly characterizes uncertainties only by probability distributions, but user may undertake UQ of EUs through non-automated sensitivity analysis.

• Representation execution is fully reproducible.

3. Non-Reproducible AU: as with Reproducible AU, but representation cannot guarantee reproducible execution (e.g., operator- or hardware-in-the-loop).

4. Sensitivity Analysis Only: representation has no AU probability characterization or propagation.

• User may undertake informal, proximate UQ of EUs through non-automated sensitivity analysis.

• Representation may or may not be reproducible.

[13]

6 Tactical Interface This attribute refers to the accuracy to which tactical Component, Element, or System external interfaces are recreated in the M&S environment. (e.g., communication networks, message type and format, etc.)

1. Replication: perfect recreation of as-built tactical interface; conforms to interface in fielded system, regardless of whether or not the fielded system conforms to the operational Interface Control Documents.

2. Emulation: perfect recreation of as-specified tactical interface; conforms to complete operational Interface Control Documents.

3. Faithful: provides only the information within the tactical interface; conforms to message format in operational Interface Control Documents; does not conform to complete operational Interface Control Documents.

4. Approximation: provides information contained within tactical interface and simulation information; does not conform to operational system Interface Control Documents.

5. None: Component, Element, or System external interfaces not required.

7 Stimulation Interface This attribute refers to the level of detail contained within stimulation data distributed through the system. Information includes everything within Common Referent Stimulation (CRS) grouping (e.g., threat, communication networks, man-made and natural environments, phenomenology, lethality, etc.).

1. High Detail: data provided contains all detailed information and interactions.

2. Medium Detail: data provided contains detailed information regarding threat(s) (6DOF);

communications, man-made and natural environments are modeled as simple systems with few states;

phenomenology and lethality are modeled basically (binary; on/off) or not at all.

3. Low…