IMS_AVE_RFI_W911W6-19-R-0004.pdf

W911W6-19-R-0004

1 Introduction and Background:

AMRDEC is looking for Industry, Academia, and other entities who can provide information and experience about methods and approaches for evaluating architectures against its requirements.

1.1 Integrated Mission Equipment (IME) Program Overview:

The Integrated Mission Equipment (IME) for Vertical Lift Systems project is tackling challenges with current architectural approaches and integration methods that cannot sustain the rapid growth of system complexity nor the rate of change required to keep Army aviation systems ahead of emerging threats. IME is a 6.3 Science & Technology (S&T) program managed by the Aviation Development

Directorate (ADD) that builds on the Joint Multi-Role Technology Demonstrator (JMR TD) Mission

Systems Architecture Demonstration (MSAD) architectural approach and model-based engineering

(MBE) tools and methods. This effort will explore methods for independent verification and validation of architectural requirements and concepts for an agile and resilient digital backbone approach for agile mission systems that can outpace emerging physical and cyber threats. IME will develop the infrastructure for a representative mission system that will undergo laboratory demonstrations and a flight demonstration in FY24. The effort will also serve as the future flying testbed for various ADD S&T programs to include Advanced Teaming, Air Launched Effects, and Holistic Situational Awareness with

Decision Making.

IME supports capability advancement for Future Vertical Lift through the transition of multiple knowledge products and demonstration materials. The lessons learned will be applied to: refine MBE methods and tools; update existing architecture approaches and standards (i.e. JMR Comprehensive

Architecture Strategy (JCAS) and the Future Airborne Capability Environment (FACE™) Technical

Standard); implement guides for MBE, airworthiness, cybersecurity, and product lines; improve the verification and validation of architectural requirements; and mature representative mission system architectures, designs, components and system implementations.

1.2 IME Architecture Approach

IME will utilize the Comprehensive Architecture Strategy (CAS) described in Attachment 2. CAS governs a holistic and strategic architecture decision making process to satisfy the technical and business requirements at all levels of the system. Included in CAS is the selection of the tactics, patterns, and methods to guide and inform design solutions that align to stakeholder technical objectives and key business drivers. CAS describes three levels of architectural specificity (mirroring typical government acquisition organizations) where the organizational focus and control varies. The three levels are referred to as the Reference Architecture (RefArch), Objective Architecture (ObjArch), and the System

Architecture (SysArch). Following the CAS approach enables IME to enforce an architecture that meets the government business and technical drivers, such as the intent to foster competition for components which will retain the Army’s independence from vendor-lock during acquisition, enabling industry to provide competitive innovative system-level designs. The three levels of architectural specificity result in traceable architectural requirements.

1.3 Architecture Verification Environment (AVE) Overview

IME is exploring the ability to verify architecture requirements such as modifiability and interoperability.

In practice architecture requirements are not expressed with the precision and concreteness that allows for objective testing nor the traceability to higher level goals, resulting in these being waived or removed during execution. To address this challenge IME will establish an Architecture Verification

Environment (AVE). The goal is to provide processes, methods, and tools to verify a variety of types of architecture requirements, enabling the government to measure progress towards satisfying architecture requirements throughout the development lifecycle. AVE will be applied to architectures at different conceptual levels (e.g., reference, product family, and system architectures), at different stages of maturity or completeness (e.g., early sketches, Preliminary Design Review (PDR), Critical Design

Review (CDR), or incremental evolutions) all from various organizations (e.g. acquiring organization, system developer, component provider). The AVE will be used multiple times throughout the lifecycle to improve government visibility, improve communications between government and suppliers, and allow issues to be found and corrected earlier.

AVE is anticipated to be a laboratory-based environment that combines automated and manual analyses that will evolve over time through the addition of new and improved analyses. AVE will qualitatively and, where feasible, quantitatively analyze and verify architectures, designs, and implementations for satisfaction of architecture requirements. This enables assessment of the impacts between quality attribute requirements and architectural development decisions. The analysis data and assessments will be collected and stored to create a broad body of architectural knowledge. The verification activities will support maturation and transition to an engineering process supported by the body of knowledge for tradeoff and procurement decisions.

1.4 Audience

This RFI targets the following communities:

The first community includes consultants, industrial and academic researchers, and tool developers involved in architecture analysis, evaluation methods and tools. Responses from this community identify current and planned industrial capabilities, technology gaps and challenges, and less mature or emerging approaches. For this community, AVE is an environment to systematically apply architecture analysis and evaluation across the system lifecycle. AVE success in the avionics domain will provide evidence to apply this technology to other domains.

The second community includes developers and integrators of aviation technology. Responses from this community identify how the AVE vision aligns with current and emerging architecture development, analysis, and evaluation practices and to understand the most effective ways to apply AVE to improve the quality and capabilities of products delivered to warfighters. For this community AVE provides a systematic, evidence-based approach to support architecture and design tradeoffs throughout the system lifecycle.

2 RFI Objectives This RFI requests feedback from industry on the AVE approach and specific questions on current technologies, practices, and methods. Responses will help inform and shape the use cases and AVE requirements.

2.1 AVE Use Cases

The government’s overarching goal is to acquire and field systems that meet its business goals and technical requirements in a timely, cost-effective manner. Architecture is an enabler of key decisions regarding business and technical goals. Architecture verification will provide the government with early and ongoing insight into the ability of an implemented system to meet these goals and requirements.

2.1.1 AVE Concepts

As depicted in the following figure, a simple view of the AVE is that it consumes inputs, performs analysis of those inputs, and generates findings. Though the specific inputs vary across use cases, they generally consist of a set of artifacts that represent the architecture being analyzed and the specification of a set of goals or requirements. Sections 2.1.2 – 2.1.6 represent some of use cases that illustrate AVE capabilities.

Architecture artifacts represent an architecture and are provided by its designer. The range of architecture artifacts seen in practice today varies widely. The AVE will have to establish some minimum level of information content, clarity of semantics, and acceptable range of notations for artifacts to allow AVE analyses to produce desired types of findings. Artifact content may include:

structural descriptions in terms of elements (e.g., components, modules, or hosts), relations among elements (e.g., message-based communications or inheritance relations), and their properties, often organized in terms of architecture views constraints data models describing the semantics of information identification of architecture patterns and tactics rationale for architecture decisions

Notations may consist of a mix of textual and graphical syntax, though a machine-readable form is preferable (e.g. SysML model). Notations vary in terms of the clarity or formality of their semantics, with more formal semantics usually providing greater support for rigorous analyses. The set of architecture artifacts used to represent a single system may include multiple notations, particularly where one notation is not expressive enough to capture a sufficient breadth of architecture views.

Architecture requirements refer to a range of requirements that influence and can be analyzed in the context of an architecture. Many architecture requirements can be expressed as quality attribute requirements (sometimes referred to as non-functional requirements). Examples of quality attributes that are important to IME include interoperability, reliability, portability, security, and modifiability.

Quality attribute requirements clarify goals in the context of a specific scope, usually with accompanying measures (see Section 2.1.2 for examples). Other architecture requirements include adherence to technical, regulatory, or policy constraints, such as compliance with standards or use of legacy platforms.

As with architecture artifacts, in practice, architecture requirements are expressed using a variety of notations and with differing degrees of formality. Prose text is a common representation. Some architecture requirements provide clear target measures, while others do not.

Each finding generated by the AVE will include a confidence level, based on the underlying analysis (e.g., ranging from expert judgment to mathematically proven analytic theories) and the characteristics of the architecture artifacts (e.g., less conclusive findings will be derivable from an early architecture model in which few decisions have been made). The government will use AVE findings at multiple points in the acquisition lifecycle (e.g., requirements analysis, System Requirements Review (SRR), PDR, and CDR) to inform their decisions and improve their interactions with vendors. The government will also use AVE findings in S&T activities to improve government use of architecture, improve the capabilities of the

AVE, and enhance the body of architecture knowledge and practices.

2.1.2 Use Case 1 – Validating architecture artifacts against an architecture requirement Input Artifacts Analysis Outputs

Architecture artifacts for a system

An architecture requirement for the system

Check whether the architecture satisfies the requirement

Yes, no, or insufficient data to determine

Evidence supporting the finding and characterization of confidence in that finding

Conceptually, this is the simplest and perhaps most common use case – to validate whether the architecture represented by a set of artifacts satisfies an architecture requirement. In practice, this use case has many refinements, the specifics of which vary with the nature of the artifacts and the requirement. The AVE should address a range of both, which implies a range of analytic capabilities.

The following are examples illustrating some of the potential refinements of this use case:

Given a mix of informal sketches and unstructured prose description of an architecture, determine if architecture enables portability across different computing hardware.

Given Architecture Decision Records for an architecture, determine if the architecture provides interoperability with a specified class of related systems.

Given a mix of Unified Modeling Language (UML) and structured prose description of an architecture, determine if the architecture supports the Army Aviation Safety Process.

Given a UML model of an architecture, determine if component X can be replaced with component Y within a given number of months.

Given an Architecture Analysis & Design Language (AADL) model of an architecture, determine if a typical workload and deployment environment (both specified in some detail) allows capabilities to execute with X% CPU margin remaining.

This use case may be performed at various points in the lifecycle of a single system and may need to support different inputs at different points (e.g., the analysis may be repeated using more formal or more precise artifacts as an architecture matures). Findings may be quantitative or qualitative, depending on the inputs provided and the quality attributes for which requirements are provided, and will include confidence levels as discussed in Section 2.1.1.

The AVE should support many refinements or variations of this use case. Systems must satisfy architecture requirements across many quality attributes, and the AVE should provide the capability to validate as many such requirements as feasible. To realize this, it is expected that the AVE will use multiple technologies and analytic theories to drive its analyses.

2.1.3 Use Case 2 – Reasoning across requirements for multiple quality attributes

Architecture artifacts for a system

Architecture requirements for multiple quality attributes

Assertion by an architect that the architecture cannot simultaneously satisfy two different quality attribute requirements (i.e., a conflict exists)

Determine whether the design decisions embodied in the architecture can satisfy both requirements

Explore modifications (e.g., alternate mechanisms) that could feasibly satisfy both

Evidence confirming or refuting the inability to satisfy two quality attribute requirements with the provided architecture decisions

This use case involves analyzing requirements satisfaction across multiple quality attributes to support tradeoffs, particularly when determining if the mechanism selected to achieve one quality attribute requirement impacts the satisfaction of requirements related to other quality attributes.

The following examples illustrate some of the potential refinements of this use case:

Given a set of Architecture Decision Records is available for a system with quality attribute requirements for interoperability and to support the Army Aviation Safety Process. The system designer asserts that a self-describing interface data format is needed to satisfy the interoperability requirement; however, the exception handling that this mechanism uses does not support the Army Aviation Safety Process. Other interface formats would satisfy the Safety

Process but will not provide the required interoperability. The analysis confirms or refutes this assertion.

Given a SysML model is available for an architecture with quality attribute requirements for latency and safety. The designer asserts that distributing the processing across multiple computers is needed to satisfy the latency requirement; however, this impacts the safety requirement. Using a single computer would satisfy the safety requirement; however, it will not satisfy the latency requirement. The analysis provides evidence that one approach or the other is more closely aligned with desired objectives.

Given an AADL model is available for an architecture with quality attribute requirements to maintain X% CPU margin and to be able to replace sensor X with a new sensor. The architect asserts that hard-coding sensor configuration is needed to achieve the required CPU margin, however this results in unacceptable effort to replace the sensor. Runtime discovery and dynamic configuration would satisfy the modifiability requirement; however, this will require more processing and will not satisfy the CPU margin requirement. AVE provides evidence that one approach or the other is more closely aligned with desired objectives.

The possible tradeoffs in an architecture are unbounded. However, there are pairs of quality attributes that are frequently hard to jointly satisfy, e.g., security requirements and usability requirements, or throughput requirements and modifiability or portability requirements. Variations of this use case cover both the well-understood sets of tradeoffs and also tradeoffs driven by system-specific requirements and constraints.

2.1.4 Use Case 3 – Impact analysis for a proposed architecture change

Architecture artifacts for a system

A proposed change to the architecture

Architecture requirements for the system

Perform an impact analysis on the proposed change

List of architecture requirements affected

Elements of the architecture that would be affected

This use case is to determine the impact of a proposed architecture change. In this use case, the impact analysis includes identification of the scope of the architecture that is affected by the change (e.g., how many elements and how substantially each is affected) and identification of which architecture requirements are affected by the change. Note that effects on individual architecture requirements could be positive (e.g., moving from unsatisfied to satisfied), negative (e.g., moving from satisfied to unsatisfied), or neutral.

Realizations of this use case will rely in part on the capabilities from Use Case 1 to support architecture requirement analysis but will require additional analysis capabilities to determine the scope of the architecture that is affected. The specification of an architecture change can take many forms, ranging from a fully re-designed architecture to statements like “replace component X with component Y”, but in any case, the proposed change will have to be documented consistent with the representation of the architecture itself to support analysis.

For example, consider an architecture represented in a model that satisfies its architecture requirements prior to a proposed change to replace component X with component Y. Findings from applying this use case to this example could include:

because component Y does not recover from failures as quickly as component X, an availability requirement will not be satisfied after the replacement because component Y provides a different interface than component X, components A, B, and C will also be affected by the replacement

2.1.5 Use Case 4 – Verifying conformance of one architecture to another Input Artifacts Analysis Outputs

Architecture artifacts representing two related architectures

A mapping between the two architectures

Check that no decision in the concrete architecture contradicts a decision in the abstract architecture

A list of decisions in the concrete architecture that do not conform to the abstract architecture

This use case is to verify whether one architecture conforms to another. In practice, this is performed when a more concrete architecture (e.g., the architecture of a specific system) is developed that is meant to conform to a more abstract architecture (e.g., an architecture applied to many systems).

Conformance means that no decisions in the abstract architecture are violated in the concrete architecture.

This use case is not about whether either architecture satisfies a set of architecture requirements (see

Use Case 1), thus the omission of requirements from inputs is intentional.

Specific conformance checks will depend on the content of the architecture artifacts and what is considered a decision. For example, given two architectures represented in the same notation, checks could include:

whether all elements and relations present in the abstract architecture trace to the concrete architecture (these could represent software, data, or hardware elements and relations)1 whether all properties of elements and relations in the concrete architecture are compatible with corresponding properties in the abstract architecture (properties could address a variety of topics such as resources required, protocol versions, expected execution time, or units of measure on data elements) whether elements, relations, and their properties in the concrete architecture fall within allowable bounds or variations expressed in the abstract architecture

1 Note that in some architecture views, such as a layered view, absence of a relation between elements is significant and notes a prohibition. In such a case, a concrete architecture that includes a relation that is not allowed in the abstract architecture would not pass a conformance check. This is a good example of the nuances involved in precisely defining the meaning of architecture conformance.

When comparing two architectures represented in different notations, the checks are more complex, and must address issues such as:

a mapping between the two sets of artifacts must be provided or discovered semantic differences must be reconciled (e.g., between SysML and AADL semantics) differences in the level of detail must be addressed (e.g., comparing structural choices in a concrete architecture to decisions captured in Architecture Decision Records for an abstract architecture) whether relations and decisions in the architecture under evaluation are consistent with another architecture’s business drivers, architectural drivers, and/or quality attributes (this could be abstract to abstract, abstract to concrete, concrete to concrete, etc.)

One architecture may comply to another to varying degrees. A fully conformant architecture would pass all checks. A partially compliant architecture may pass most, but not all checks. The findings of this capability should consist of a list of specific decisions (in whatever form) in the concrete architecture that violates one or more decisions in the abstract architecture.

2.1.6 Use Case 5 – Verifying that an implementation satisfies an architecture requirement

Implementation of an element of an architecture

An architecture requirement for the system

Check that the implementation satisfies the architecture requirement

Yes, no, or insufficient data to determine

Evidence supporting the finding and characterization of confidence in that finding

This use case verifies whether the implementation of an element or elements of an architecture satisfies architecture requirements. Similar to Use Case 1, this use case has many refinements, the specifics of which vary with the set of quality attributes addressed by the architecture requirements.

Implementations will need to be verified against many such requirements. This includes both functional and non-functional (quality attribute) requirements.

The AVE should have the capability to verify an implementations delivered in one of following form:

software source code software compiled code processing card assembly processing device

The AVE should be able to verify these implementations against the same range of architecture requirements described in Use Case 1 (e.g., addressing requirements for quality attributes such as interoperability, performance, modifiability, safety, and portability), which implies a range of analytic capabilities that may be realized by multiple technologies and analytic theories.

3 Submission Content AMRDEC is looking for information and experience about methods and approaches for evaluating architectures against their requirements. The following questions cover various aspects of the AVE;

respondents are not expected to address every item.

3.1 Approach:

3.1.1 Are there any general comments on the AVE approach and use cases?

3.1.2 What technologies, practices, and methods do you see as applicable?

3.1.3 What are their major capabilities and limitations?

3.2 Best Practices

3.2.1 How do you represent, analyze, and verify an architecture?

3.2.2 What aspects of an architecture can you analyze and what measures do you produce?

3.2.3 How do you verify that a system implementing an architecture satisfies architecture requirements, especially non-runtime quality attribute requirements such as modifiability or portability?

3.3 Architecture Verification Tools

3.3.1 What tools do you use to perform architecture analysis?

3.3.2 To what degree do these tools require human expertise, intervention, and interpretation of results?

3.3.3 Identify any tools or methods that you have stopped using or did not adopt after evaluation? If any, why?

3.4 Quality Attributes

3.4.1 Which quality attributes can (and cannot) be effectively analyzed with available technologies, practices, and methods?

3.5 Architecture Requirements

3.5.1 How do you represent architecture requirements, such as key business goals and quality attribute (or non-functional) requirements?

3.5.2 Are these requirements stated in such a way that an architecture can be analyzed to validate satisfaction of the requirements?

3.6 Architecture Representation

3.6.1 Which architecture description languages, notations, or frameworks do you use?

3.6.2 Are there any representations that you have stopped using or did not adopt after evaluation? If any, why?

3.7 Architecture Trade Analysis

3.7.1 Which tools and methods do you use to perform architecture tradeoff or trade space analysis?

3.7.2 What is your level of confidence in the outputs of these tools and methods?

3.7.3 What special skills, training, and knowledge are required to use these tools and methods?

3.8 Work Force

3.8.1 What special skills, training, and knowledge are required to analyze architectures?

3.9 Conformance

3.9.1 Which technologies, practices, and methods do you use to verify architecture conformance and what are the limitations of this analysis?

3.9.2 How do these technologies, practices, and methods define conformance?

3.9.3 How are hard they to learn and use?

3.10 Infrastructure

3.10.1 What infrastructure is required to support verification of implementations as proposed?

3.11 Vendor Information and Capabilities

3.11.1 What is your relevant experience in performing architecture verification and/or developing technologies, practices, and methods to perform architecture verification?

Provide a description.

3.11.2 Are there any other relevant comments, information, or experience that you wish to include? If so, please provide.

4 Format Requirements Responses should be submitted to: Scott Wigginton via e-mail at scott.a.wigginton.civ@mail.mil.

The responses should be submitted via email, in a password protected Adobe PDF. The password should be submitted in a separate e-mail. All responses should be submitted no later than 4:00pm EST on 15 February 2019.

All responses to this notice should include the following: Company/Organization Name, Point of

Contact, Address, Telephone and Fax Numbers, and Email Address.

A corporate capability brief may be included as an attachment but should not exceed five (5) typed pages. Please structure responses so information is in the same order as requested in

Section 3. Additional sections can be included as needed, but the section title and introduction should clearly describe what additional information is being provided as well as its value and relevance to the efforts described by the government in this RFI. Please do not send brochures for specific products.

Any and all information provided should be limited to unclassified data.

mailto:scott.a.wigginton.civ@mail.mil

5 DISCLAIMERS AND IMPORTANT NOTES:

This is an RFI issued solely for information and planning purposes and this RFI does not constitute a formal solicitation for proposals or proposal abstracts. In accordance with FAR 15.201(e), responses to this notice are not offers and cannot be accepted by the Government to form a binding contract.

Submission of a response is strictly voluntary and is not required to propose to subsequent Broad

Agency Announcements (if any) or research solicitations (if any) on this topic. No solicitation exists;

therefore, do not request a copy of the solicitation. If a solicitation is released, it will be synopsized on the Federal Business Opportunities website. It is the responsibility of any potential offerors/bidders to monitor FedBizOpps.gov (www.fbo.gov) for release of any solicitation.

AMRDEC will NOT provide reimbursement for costs incurred in responding to this RFI or participating in any subsequent workshop. Any and all costs incurred in responding to this RFI, including conducting a capability presentation, shall be borne by the respective respondents.

AMRDEC will consider scheduling an industry day to facilitate additional information exchanges about the program. AMRDEC may reach out to companies to set up one-on-one meetings to further discuss responses.

Any information contained in responses to this RFI that Respondents consider to be confidential, privileged, proprietary or a trade secret will not be distributed beyond U.S. Government employees and

Systems Engineering and Technical Assistance (SETA) support contractors.

To the maximum extent possible, please submit non-proprietary information. If proprietary information is submitted, it must be appropriately and specifically marked. It is the submitter's responsibility to clearly identify to the Government what is considered proprietary data. Any proprietary information should be clearly labeled as “proprietary.” AMRDEC will not publicly disclose proprietary information obtained as a result of the RFI. To the full extent that it is protected pursuant to the Freedom of

Information Act and other laws and regulations, information identified by a respondent as “Proprietary” will be appropriately controlled. Submissions may be reviewed by Government personnel and support contractors bound by appropriate non-disclosure agreements. Responses to this RFI will not be returned.

Respondents are advised that AMRDEC is under no obligation to acknowledge receipt of the information received or provide feedback to respondents with respect to any information submitted under this RFI.

Responses to this RFI do not bind AMRDEC to any further actions.