APPENDIX_A_-_Definitions.pdf

APPENDIX A – DEFINITIONS

The following definitions are applicable to this Performance Work Statement.

Agile – A project management methodology which features flexibility and evolutionary approach in providing capability to the warfighter in a timely and cost effective manner.

Acquisition Category (ACAT) – Categories established to facilitate decentralized decision making and execution and compliance with statutorily imposed requirements. The categories determine the level of review, decision authority, and applicable procedures.

Analysis of variance (ANOVA) – A collection of statistical models used to analyze the differences among group means and their associated procedures (such as "variation" among and between groups).

Analysis Structure – The combination of evaluation framework and supporting data source matrix information assembled to define and support the accomplishment of test and evaluation objectives.

Approved Product List (APL) – A single consolidated list of products that have completed Interoperability and Information Assurance certification. The APL is the only listing of equipment by DoD to be fielded in DoD networks. DoD components are required to fulfill their system needs by only purchasing APL listed products, providing one of the listed products meets their needs.

Architecture – The structure of components, their relationships, and the principles and guidelines governing their design and evolution over time.

Business Capability Lifecycle (BCL) – The overarching framework for the planning, design, acquisition, deployment, operations, maintenance, and modernization of defense business systems (DBS). It promotes rapid delivery of business capability by facilitating a process tailored to the unique requirements of DBS and is based on industry best practices, studies, and emerging legislation.

Combined – Description of any military operation, organization, or system involving elements of two or more allied nations.

Combined Enterprise Regional Information Exchange System (CENTRIXS) – A common set of networks built on a set of standard hardware, software, and services for U.S. and Coalition partner forces to share classified operational and intelligence information at the SECRET/REL level. Each of these CENTRIXS networks operates at a single security classification level and operates globally, regionally, and locally.

Command, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance (C4ISR) – Systems, procedures, and techniques used to collect and disseminate information.

Cybersecurity – Prevention of damage to, protection of, and restoration of computers, electronic communications systems, electronic communications services, wire communication, and electronic communication, including information contained therein, to ensure its availability, integrity, authentication, confidentiality, and nonrepudiation.

Data Authentication Group (DAG) – Representatives from stakeholder organizations/commands who assist with test data reduction, data analysis, the investigation of problems surfaced in test data, and determine the impact of system issues on the overall system’s capabilities.

Data Source Matrix (DSM) – The component of a test and evaluation analysis structure that defines the data requirements, data sources, and data collection methods supporting an evaluation framework. An evaluation framework and supporting DSM collectively service as a test and evaluation analysis structure.

Defense Business System (DBS) – An information system, other than a national security system (NSS), operated by, for, or on behalf of DoD, including financial systems, mixed systems, financial feeder systems, and Information Technology and Information Assurance infrastructure. DBS support business activities such as acquisition, financial management, logistics, strategic planning and budgeting, installations and environment, and human resource management.

Defense Cyberspace Operations (DCO) – Coordinated efforts taken to direct and synchronize actions to detect, analyze, counter, and mitigate cyber threats and vulnerabilities; to outmaneuver adversaries taking or about to take offensive actions; and to otherwise protect critical missions that enable U.S. freedom of action in cyberspace. This line of operation can trigger offensive cyberspace operations or other response actions necessary to defend DOD networks in response to hostile acts, or demonstrated hostile intent.

Demilitarized Zone (DMZ) – A physical or logical subnetwork that contains and exposes an organization's external services to a larger untrusted network, usually the Internet.

Department of Defense Information Networks (DoDIN) – The globally interconnected, end-to-end set of information capabilities, associated process and personnel for collecting, processing, storing, disseminating, and managing information on demand to Warfighters, policy makers, and support personnel. The DoDIN includes owned and leased communications and computing systems and services, software (including applications), data, security services, other associated services, and National Security Systems.

Design of Experiments (DOE) – A structured process that provides the scientific and statistical methods needed to rigorously plan and execute testing.

Developmental Test and Evaluation (DT&E) – The process of testing concurrent with product development to verify the status of technical progress, verify that design risks are minimized, to substantiate achievement of contract technical performance, and to certify readiness for initial operational testing.

ENCORE III – The follow-on contract to DISA’s ENCORE II contract, which expires in May 2018. ENCORE III will provide information technology solutions, including network engineering, analysis and support for the acquisition, installation, fielding, training, operation and life-cycle management of components and systems in the operational environments of Combatant Commands and their subordinate components, the military services, Defense agencies, Office of the Secretary of Defense and other Federal agencies.

Enterprise Services – A standards-based way of encapsulating enterprise functionality and exposing it as a reusable business service that can be combined with other services to meet new requirements.

Evaluation Framework – Establishes the scope, objectives, and specific measures and criteria that form the basis for the test and evaluation of a particular program, system, or component. An evaluation framework and supporting data source matrix collectively service as a test and evaluation analysis structure.

Global Information Grid (GIG) – The GIG is the globally interconnected, end-to-end set of capabilities, processes, and resources for collecting, processing, storing, managing, and disseminating on-demand information to the warfighter.

Information Assurance (IA) – Measures taken to protect and defend DoD, Federal Agency and/or warfighter information within a system, process, or network (strategic or tactical). This is performed by ensuring the availability, integrity, authentication, confidentiality, and non-repudiation of the information, to included providing for restoration of information systems by incorporating protection, detection, and reaction capabilities.

Information technology (IT) – The use of any computers, storage, networking and other physical devices, infrastructure and processes to create, process, store, secure and exchange all forms of electronic data.

Information Technology Service Management (ITSM) – A strategic approach for designing, delivering, managing and improving the way information technology is used within an organization. The goal of every ITSM framework is to ensure that the right processes, people, and technology are in place so that the organization can meet its business goals.

Initial Operational Test and Evaluation (IOT&E) – An operational test conducted on production, or production representative articles, to determine whether systems are operationally effective and suitable for intended use by representative users to support the decision to proceed beyond Low-Rate Initial Production.

Integrated Architecture Traceability Matrix (IATM) – A methodology providing a standardized approach to Solution Architecture requirements analysis that facilitates test planning and risk reduction. The IATM provides a crosswalk between operational, systems, and technical views and provides a tool to identify disconnects and high-risk areas.

Interoperability (IOP) – The ability of systems to provide services to, and accept services from, other systems and to use the services exchanged to enable them to operate effectively together.

Interoperability Certification – The process of ensuring that a system meets the joint interoperability requirements of its users. It includes the collection of the data necessary to determine whether or not the system conforms to applicable interoperability standards and can effectively exchange all required information with all pertinent systems.

Interoperability Testing – The process of assessing the ability of a system to exchange usable electronic information with systems of other services or nations as specified in its requirements documents. Specialized test tools are used to monitor performance of products to determine if the proper actions and reactions are produced. A system is certified as interoperable at the completion of successful interoperability testing.

JITC Hotline – The warfighter hotline provides technical support from JITC via phone, fax, or e-mail. The hotline is operational 24/7/365 at no cost to the customer in resolving technical joint and combined interoperability issues.

JITC Risk Assessment Database (J-RAD) – A tool that aids the tester in defining and documenting their standards and related risk analysis. All forms of testing at JITC require the tester to conceptualize their systems through Technical Views and System Views.

Joint – Description of any military operation, organization, or system involving elements of two or more armed services or agencies.

Joint Enterprise Test Infrastructure (JETI) – A single operational governance structure to support robust, effective, suitable, interoperable, and secure solutions for interconnectivity of JITC laboratories.

Joint Information Environment (JIE) – A framework to synchronize and integrate C4ISR capabilities and services delivered through a shared, secure global information grid using standardized guidance designed to achieve the decisive information advantage for the Warfighter. It is comprised of shared information technology infrastructure, enterprise services, and single security architecture to achieve full spectrum superiority, improve mission effectiveness, increase security and realize IT efficiencies.

Joint Interoperability Evaluation Plan (JIEP) – New reference used in place of Joint Interoperability Evaluation Framework, referenced from the Joint Interoperability Test Command Instruction 308-50-02 Interim. The JIEP should contain some basic program information, e.g. system identification, stakeholders, etc. It should also contain the final Integrated Architecture Traceability Matrix, Evaluation Methodology, and Data Management and Analysis Plan or, at least, links to those documents.

Joint Worldwide Intelligence Communications System (JWICS) – A Top Secret/Sensitive Compartmented Information network run by the United States' Defense Intelligence Agency and used across the Department of Defense, Department of State, Department of Homeland Security and Department of Justice to transmit especially sensitive classified information.

Milestone Decision Authority (MDA) – The overall executive sponsor responsible of any Major Defense Acquisition Program. The MDA formally initiates each increment of an evolutionary acquisition program as required by DoD Instruction 5000.02 “Operation of the Defense Acquisition System” – 26 Nov 2013. They determine if a program has met its phase exit requirement and can proceed into the next phase during a Milestone review in terms of cost, schedule, and performance.

Major Range and Test Facility Base (MRTFB) – The designated core set of DoD Test and Evaluation infrastructure and associated workforce that must be preserved as a national asset to provide T&E capabilities to support the DoD acquisition system.

Mission Partner – Those with whom Department of Defense cooperates to achieve national goals, such as other departments and agencies of the U.S. Government; state and local governments, allies, coalition members, host nations and other nations; multinational organizations; non-governmental organizations; and the private sector.

Mission Partner Environment (MPE) – An operating environment that leverages U.S. and mission partner information technology infrastructures with integrating capabilities to realize the DoD Joint Information Environment framework.

National Security System (NSS) – Any information system (including any telecommunications system) used or operated by an agency or a contractor of an agency, or other organization on behalf of an agency, the function, operation, or use of which:

(1) involves intelligence activities;

(2) involves cryptologic activities related to national security;

(3) involves the command and control of military forces;

(4) involves equipment that is an integral part of a weapons or weapons system; or

(5) is critical to the direct fulfillment of military or intelligence missions. This does not include a system that is to be used for routine administrative and business applications (including payroll, finance, logistics, and personnel management applications).

NetOps Readiness Review Process (NRRP) – A process during the entire acquisition lifecycle to ensure operational considerations are integrated into planning, design, and development.

Network Readiness Review Board (NRRB) – A governance body that verifies completion of the NetOps Readiness Review Process and assesses residual operational risk prior to operational fielding.

Net Ready-Key Performance Parameters (NR-KPP) – The NR-KPP has been developed to assess net-ready attributes required for both the technical exchange of information and the end-to-end operational effectiveness of that exchange. The NR-KPP replaces the Interoperability KPP, and incorporates net-centric concepts for achieving IT (including NSS) interoperability and supportability. The NR-KPP assesses information needs, information timeliness, Information Assurance, and net-ready attributes required for both the technical exchange of information and the end-to-end operational effectiveness of that exchange. The NR-KPP consists of verifiable performance measures and associated metrics required to evaluate the timely, accurate, and complete exchange and use of information to satisfy information needs for a given capability.

Operational Test Agency (OTA) – Within the DoD, an organizational entity approved by the Director, Operational Test and Evaluation to carry out operational test and evaluation.

Operational Test and Evaluation (OT&E) – As performed by JITC, OT&E is field testing, performed under operationally representative conditions on predominately IT-based systems, conducted to determine operational effectiveness, suitability, and interoperability.

Operational Test Readiness Review (OTRR) – Assesses if a system should proceed into Operational Test and Evaluation (OT&E). The review addresses and verifies system reliability, maintainability, and supportability performance and determines if the hazards and Environmental, Safety, Occupational and Health residual risks are manageable within the planned testing operations. This assessment determines if changes are required in planning, resources, or testing necessary to proceed with OT&E. Of critical importance to this review is the understanding of available system performance to meet the Capability Production Document performance threshold values. The OTRR may be conducted in multiple steps to ensure that the “production configuration” system (usually the Low- Rate Initial Production system) can proceed into OT&E with a high probability of success.

Quick Look Report – A preliminary status of testing results provided before the test report is finalized.

Regression Analysis – A statistical process for estimating the relationships among variables. It includes many techniques for modeling and analyzing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables (or 'predictors').

Regression Testing – Testing repeated to determine whether the findings of previously conducted testing are unaffected by the introduction of configuration changes. Following a system configuration change, regression testing can be conducted to re-verify the results of previously conducted performance, functionality, security, and/or other tests performed to satisfy DT&E objectives.

Requirements Traceability Matrix (RTM) – A document that links requirements throughout the validation process.

The purpose of the RTM is to ensure that all requirements defined for a system are tested in the test protocols.

Representational State Transfer (REST) – The software architectural style of the World Wide Web. REST is an architectural style consisting of a coordinated set of architectural constraints applied to components, connectors, and data elements, within a distributed hypermedia system. REST ignores the details of component implementation and protocol syntax in order to focus on the roles of components, the constraints upon their interaction with other components, and their interpretation of significant data elements. Through the application of REST architectural constraints certain architectural properties are induced: Performance, Scalability, Simplicity, Modifiability, Visibility, Portability, and Reliability.

Risk Management – The process of managing risks to organizational operations (including mission, functions, image, or reputation), organizational assets, individuals, other organizations, or the nation resulting from the operation or use of an information system, and includes: 1) the conduct of a risk assessment; 2) the implementation of a risk mitigation strategy; 3) employment of techniques and procedures for the continuous monitoring of the security state of the information system; and 4) documenting the overall risk management program.

Risk Management Framework (RMF) – The unified information security framework for the entire federal government using a structured approach used to oversee and manage risk for an enterprise.

Supply Chain Risk Management (SCRM) – The implementation of strategies to manage both everyday and exceptional risks along the supply chain based on continuous risk assessment with the objective of reducing vulnerability and ensuring continuity.

Science Based Test Design (SBTD) – The use of Scientific Test and Analysis Techniques and other scientific, statistical, and analytical methods as means of achieving T&E objectives through rigorous, systematic, and defensible approaches.

Scientific Test and Analysis Techniques (STAT) – Implements the use of scientific and statistical methods in developing rigorous, defensible test plans and the evaluation of their results. STAT assists the test community in improving the planning, execution, analysis, and reporting of integrated testing. See: http://www.afit.edu/STAT/.

Service Oriented Architecture (SOA) – A paradigm for organizing and utilizing distributed capabilities that may be under the control of different ownership domains.

Simple Object Access Protocol (SOAP) – A protocol specification for exchanging structured information in the implementation of web services in computer networks. It uses XML Information Set for its message format, and relies on application layer protocols, most notably Hypertext Transfer Protocol or Simple Mail Transfer Protocol, for message negotiation and transmission.

Standards Compliance and Conformance Testing – The process of assessing compliance of a product to the defining specification or standard. Specialized test tools are used to exercise a product to determine if the proper actions and reactions are produced. Successful completion of a conformance test will enhance the probability of interoperability with other products that have been successfully conformance tested. Standards conformance testing serves as a foundation for overall joint interoperability evaluation and should be conducted prior to joint interoperability testing

Systems Engineering Technology and Innovation (SETI) – A suite of engineering products, services, expertise, and support in the planning, research, development, production, deployment, implementation, operations and sustainment activities for future, proposed, current, and legacy DISA capabilities, services, and systems.

System Tracking Program (STP) – An on-line database that tracks a system’s progress toward joint interoperability certification. The STP monitors the complete life-cycle of IT and NSS from requirements/capabilities document, to ICTO, through T&E, and culminating with joint interoperability certification status.

Teleport – Provides interoperability between multiple satellite communications (SATCOM) systems and deployed networks and is the gateway to the Global Information Grid in support of tactical communication systems users. The DoD Teleport is a global system of SATCOM facilities that are being developed in a spiral acquisition process, designed to provide the Warfighter with access to the Defense Information Systems Network from any geographical location via military or commercial satellites and frequency bands.

Test and Evaluation (T&E) – Process by which a system or components are exercised and results analyzed to provide performance-related information. The information has many uses including risk identification and risk mitigation and empirical data to validate models and simulations. T&E enables an assessment of the attainment of technical performance, specifications, and system maturity to determine whether systems are operationally effective, suitable and survivable for intended use, and/or lethal.

Test and Evaluation Master Plan (TEMP) – Describes an acquisition program’s planned Test and Evaluation (T&E) activities over a program’s life-cycle and identifies evaluation criteria for the testers. It serves as an executive summary and provides a framework within which to generate detailed T&E plans and documents schedule and resource implications associated with the T&E program.

Test and Evaluation Scorecard – A standardized template for reporting program test and evaluation status. The test and evaluation scorecard establishes the high-level scope for program test and evaluation.

Test Concept Brief (TCB) – Required by Director, Operational Test and Evaluation before milestone B to define the system’s integrated test strategy to ensure test adequacy prior to dedicated operational testing. The briefing identifies which office will perform which type of testing, what information is needed to information critical decisions and when the information is needed, and where and how the system will be evaluated.

Test Readiness Review (TRR) – A multi-disciplined technical review designed to ensure that the subsystem or system under review is ready to proceed into formal test. The TRR assesses test objectives, test methods and procedures, scope of tests, and safety and confirms that required test resources have been properly identified and coordinated to support planned tests.

Unified Capabilities (UC) – The seamless integration of voice, video, and data applications services delivered ubiquitously across a secure and highly available Internet Protocol infrastructure to provide increased mission effectiveness to the warfighter and business communities. UC integrate standards-based communication and collaboration services including, but not limited to, the following: messaging; voice, video and Web conferencing;

Presence; and UC clients.

Validation Testing – The process of ensuring: (1) proper requirements coverage by the proposed standards or specifications and (2) correct standards or specifications are available as the basis for developing products. In the context of validation, correct standards would be those demonstrated to be self-consistent, complete and feasible.

Validation testing consists of two general phases: static analysis which satisfies item (1) above and dynamic analysis which satisfies item (2) above.