QA_DVT_Industry_Engagement_Event_Final.pdf

Live Chat Questions during DVT Industry Engagement Event

Question 1: Will the FAA address any of the questions previously submitted in writing per the FAA's instructions?

• Answer 1: The FAA provided answers to the questions sent in advance of Industry Day as part of the morning presentation session. Thus, the FAA did not plan to go through these questions. If additional discussion is desired regarding a particular question, participants can ask follow-up questions.

Question 2: Is there a reason that you selected 4754 vs 4754 (A)?

• Answer 2: When the FAA references SAE ARP4754, the latest version of the document is used.

For a contract, the applicable revision is referenced so the contract requirements do not change over time.

Question 3: Is there any plan to have this OSA results rolled into the appropriate international standards?

• Answer 3: The FAA maintains a cooperative and collaborative partnership with international regulatory organizations. There are not any current plans to have the DVT OSA results incorporated into any international standards.

Question 4: Can the Government provide the planning scenarios for GPS outages that were used in developing the OSA in terms of duration, probability/frequency, and type (jamming, spoofing, space weather)?

• Answer 4: The scenarios considered in assessing the severities of the hazard effects were formed as credible system states. GPS outages are considered credible events. The duration, probability/frequency, and type of GPS outages are irrelevant at the OSA level of safety assessment.

Question 5: The DVT OSA is inconsistent with the previously published DME OSA. The DME OSA is more consistent with other certification authorities’ assessments. Would the FAA consider reevaluation of the hazards with a broader audience that would include other cert authorities and a larger group of DER pilots?

• Answer 5: The DVT OSA was developed and vetted by FAA and DOD experts, having a broad range of knowledge and experience. While the DME OSA was used as a starting point, the development of the DVT OSA included working meetings with ATC operations, aircraft certification, and flight standards. Table 3-1 in the OSA lists the 40+ participants that formed the SRM panel that conducted an extensive review of the assumptions made, methodology used, hazard analysis and evaluation performed, and the resulting outcomes. An independent peer review was performed by Air Traffic Organization’s Office of Safety and Technical Training.

It was this broader stakeholder involvement in the development of the DVT OSA that produced a more informed and vetted understanding regarding the extent to which the NAS Safety Net mitigates hazard effects for the six flight applications analyzed. Furthermore, the FAA certification authorities are the most relevant certification authority to FAA systems serving NAS users. Given the above, the FAA is not considering expanding the audience for input and review.

Question 6: The FAA's WAAS Dual Frequency Operations Segment Draft SIR and FAA-E-32892e Draft specify that the probability of HMI must be less than 1E-7/hour and software development in accordance with DO-178B Level B, Consistent with a Design Assurance Level (DAL)-B. It seems inconsistent with the primary and more accurate means of navigation has a lower Development Assurance Level than that required for the secondary means

• Answer 6: The WAAS Dual Frequency Operations Draft SIR asserts software development/assurance in accordance with DO-178B, Level B. The DVT OSA determines a ground system level requirement of Design Assurance Level A. Note, a system can be verified to DAL-A which has components with lower assurance levels. Such is the case when partitioning is used to separate and isolate components having the most safety critical functionality from those components with lower or no safety criticality. Navigation at an aircraft level considers capability for the planned route of flight, available avionics, and additional requirements if the operation is commercial. Secondary means of navigation is not a recognized term. NavAids have different functionality and requirements for use on a given flight segment.

Question 7: EUROCAE Draft ED-57A is currently being prepared by WG-107. The previous DME OSA was used as input into this process in developing DAL-B safety requirements. Would the FAA consider presenting this safety analysis at the next working group meeting?

• Answer 7: The FAA participates in RTCA SC-227 and made an offer to discuss the DVT OSA. SC- 227 actively collaborates with WG-107 and has joint participation. The FAA would consider a request from WG-107.

Question 8: Will the government provide industry the WAAS OSA?

• Answer 8: An OSA does not exist for the WAAS. WAAS was commissioned prior to the implementation of the Safety Management System.

Question 9: Considering the architecture and functionality of the TACAN Antenna, and the broad definition of responsibility for TACAN Azimuth Service within an entire TACAN System, what parts and functions specifically applicable to the TACAN Antenna and ACU will be required to be certified to Level A of the ARP4754A?

• Answer 9: The FAA cannot determine what parts of a system must be assigned as Level A until an analysis of the design has been performed. However, functionality of the TACAN antenna system that must be Level A is identified in the FAA TACAN Antenna Functional Fault Hazard Analysis document. The TACAN Antenna FFHA is planned to be made available when approved.

Question 10: Is there an expectation that DoD and/or NATO would adopt the TACAN specification (Beacon and Antenna)?

• Answer 10: The FAA has no expectations of decisions the DoD or NATO may or may not make.

Question 11: Is the FAA planning separate installation contracts for DVT components or will contracts include product provision and installation services?

• Answer 11: The DVT Sustainment Contract will include contractor installation services. We are currently developing the acquisition strategy for the DVT Sustainment Contract. The current strategy does not consist of separate contracts. The TACAN Antenna Contract will include contractor installation services. As the planning (site survey) for each site is accomplished, the determination will be made as to the installation approach that is best for the site.

Question 12: Why is the FAA setting the safety level for TACAN azimuth when the OSA implies that civil aircraft do not use the azimuth function of TACAN?

• Answer 12: US Code, Title 49 establishes the FAA as the Air Navigation Service Provider in the United States. The FAA provides the same level of safety for civilian and military users of the National Airspace System.

Question 13: Will this new safety assessment allow the continued installation of new non-fed NAVAIDS that were certified to a lower safety level

• Answer 13: The OSA is currently being used to understand requirements for planned and new system procurement and qualification.

Question 14: Can the FAA provide an outline of how Level A design and implementation occurs with respect to the anticipated award schedule? When are the first TACAN Antennas anticipated to be installed?

• Answer 14: The anticipated contract award for TACAN antennas is in Fiscal Year 2022. The FAA plans antenna installations starting no later than FY24.

Question 15: Assuming there are new specs coming for DME and VOR, when should we anticipate seeing them for industry comment? And will the VOR include Doppler?

• Answer 15: The FAA is developing new specifications for DME and VOR (to include Doppler).

The FAA intends to continue releasing specifications for industry comment as they reach the 80% development level. The FAA’s goal is to complete all specification development for the DVT program early in 2022.

Question 16: If the VOR/DME transmits a valid signal to an aircraft, could the aircraft receiver corrupt that information thereby presenting HMI to the pilot? If so, won’t the aircraft receiver need to be assured the DAL-A? If that aircraft flew off course and hit terrain or another aircraft, wouldn’t the hazard be catastrophic?

• Answer 16: The DVT program can only comment on what transpired as part of its safety process. The OSA began with the DVT service (rho-theta navigation) so the severity of the effects caused by loss or malfunction of the service could be assessed. This top-level assessment was necessary to determine the safety objectives and requirements for ensuring that an acceptable level of safety risk is achieved at the service level. Then, the assessment focused on allocating those objectives and requirements appropriately to the DVT ground equipment.

Question 17: Would a TACAN MON include the need for additional “Man-Portable Aircraft Navigation Aids” desired by the Air Force where open and active solicitations exist?

• Answer 17: The TACAN MON assumes a rationalization of the current TACANs. The FAA is unaware of any need for Man Portable Aircraft Navigation Aids. The FAA is unable to answer questions on any Air Force solicitations.

Question 18: As DAL-A is inconsistent with COTS/NDI NAVAIDS, will the DVT program be treated as a development program?

• Answer 18: The FAA is aware of the existing NDI NavAids. In order to achieve full compliance with DVT program and specification requirements, there may be a need to update or develop some changes to existing NDI NavAids. While the Program requirement for DAL-A is one such requirement that may likely drive the need for changes, this situation does not make the program a major development program.

Question 19: How will the DVT Program Office monitor the status of the resilient navigation infrastructure for the >900 SDPs during the execution of the DVT Contracts?

Follow up Question: Antenna performance and service availability with facility information via FSEP or available inventory via LIS?

• Answer 19: The DVT Program office is coordinating closely with the NextGen DME and VOR MON programs that are implementing the resilient navigation infrastructure. DVT will sustain those assets. The FAA will monitor the systems in accordance with the monitoring requirement for each system. The FAA Facility Service Equipment Profile (FSEP) and Logistics Information System (LIS) are not monitoring systems.

Question 20: What are the Small Business Goals for the DVT Sustainment Acquisition Contract? Have they been identified? How has the FAA prioritized the SDPs for implementation?

• Answer 20: Small business goals will be set forth in the final SIR. Please note any goals will align with the agency’s annual small business goals. The FAA has not prioritized SDPs at this time. We are currently working to identify SDPs for the Antenna Contract and that initially depends on the plans for the NextGen DME and VOR MON programs.

Question 21: Can you clarify why the program office sometimes use 2045 as end date for Phases 2-4 and sometimes 2062?

• Answer 21: Program life cycles are typically assumed for 20 years. Initially the FAA was using the end date for VOR MON as 2025 so identified the DVT end 2045 as (2025+20). As we went through the development of estimates for the Initial Investment Analysis Decision we calculated what it would take to replace all DMEs, VORs, and TACANs. The end date of 2062 is based on the assumptions of how fast we could complete the program if we have to replace every DVT. Assumptions affecting pacing include funding, operational resources (e.g., availability of flight inspections and engineering resources), and priorities for the program given everything else in the NAS. We will continue to refine the program with each phase.

Question 22: What NAICS code will the FAA assign to the small business set aside for the TACAN Antenna solicitation?

• Answer 22: Small business goals will be set forth in the final SIR. Please note any goals will align with the agency’s annual small business goals. NAICS codes below.

NAICS

Code

Description

Size Standards in millions of dollars or number of employees

423610 Electrical Apparatus and Equipment, Wiring Supplies, and Related Equipment Merchant Wholesalers 200

334511 Search, Detection, Navigation, Guidance, Aeronautical, and Nautical System and Instrument Manufacturing 1,250

541330 Engineering Services $16.5 488190 Other Support Activities for Air Transportation $35.0

Question 23: Is the FAA able to make this PowerPoint available for industry to download?

• Answer 23: Yes

Questions Submitted Prior to the Event

Question 1: The FAA’s WAAS Dual Frequency Operations Segment 2 Draft SIR and FAA-E-32892e Draft specify that the probability of HMI must be less than 1E-7/hour and software development in accordance with DO-178B Level B, consistent with a DAL-B. It seems inconsistent that the primary and more accurate means of navigation has a lower Development Assurance Level than that required for the secondary means.

• Answer 1: See answer 6 from Live Chat Questions during DVT Industry Engagement Event section.

Question 2: The OSA assumes a loss of GNSS for all users, a single NavAid in line of site, and a busy airspace. One would assume a region wide loss of GNSS would be an infrequent event, would result in a natural reduction in operational tempo, and that multiple NavAids are within line of site in busy airspace. Has the FAA numerically evaluated these mitigations or the probability of all three occurring simultaneously and does the current OSA reflect the results of this analysis?

• Answer 2: The FAA’s guidance for an OSA does not allow for the consideration of likelihood of effects, only credibility. The OSA is an analysis tool for identifying hazards, determining the credible worst-case severity of their effects, and developing safety requirements early in the lifecycle of a system. Primarily, it determines the likelihood bound required to achieve the safety objective – that is, to achieve an acceptable risk.

Question 3: Page 88 – States: “Decades of operational experience indicate that existing controls and requirements are sufficient to ensure that safety objectives will be met for the flight applications addressed in this OSA. However, new/proposed equipment designs will be assessed throughout the program lifecycle to ensure they provide existing controls and meet these requirements.” Can the FAA describe the program they have in place, that can detect faults via service history to demonstrate the in-service equipment meets this requirement, particularly those operated Non-Federally? Has this program been audited by an external entity (e.g., OIG)?

• Answer 3: The FAA has a ground and flight test program to monitor NavAid performance. This program applies to Federal and non-Federal facilities. Additionally, we are not aware of any NTSB reports concluding DVT systems were causal to an incident/accident due to misleading navigation information.

Question 4: Page 88 – States: “Decades of operational experience indicate that existing controls and requirements are sufficient to ensure that safety objectives will be met for the flight applications addressed in this OSA. However, new/proposed equipment designs will be assessed throughout the program lifecycle to ensure they provide existing controls and meet these requirements.” As Gen 2

VORs have a collective ~3E8 hours of operation and newer build equipment far less, it would seem the FAA cannot claim decades of experience is sufficient to demonstrate the equipment meets a failure rate of 1E-09. Will the FAA detail it plans to bring these systems into compliance? Similar to this, can the FAA demonstrate Software and Complex Hardware (FPGAs/CPLDs) is free of latent faults in legacy NAVAIDS without conducting a commensurate AL1 DO-278 software process and/or Level A DO-254 complex hardware process for those systems that use Software and Complex Hardware in monitoring and control systems?

• Answer 4: The use of the term “safety objectives” from the OSA is a high-level statement meaning that the FAA wants a low likelihood of realizing the occurrence of a catastrophic effect from misleading navigation on the flight applications discussed. Techniques to verify the objectives include development assurance determinations for complex hardware and software or exhaustive failure mode analysis for simple architectures.

~3E8 hours, although a large number, is not sufficient to demonstrate an integrity fault (not a failure rate) of less than 1E-09. The safety objective for 2nd Gen DVT was most likely to be failsafe. 3E4 hours with no misleading information events does provide significant confidence of the monitor providing failsafe operation.

2nd Gen is certainly from a different time and enjoyed the ability to exhaustively test failure modes.

Question 5: If the FAA cannot demonstrate the legacy equipment meets the newest safety requirements, will the agency ask Congress for a significant increase in funding to accelerate the DVT program to ensure a safe and resilient navigation infrastructure?

• Answer 5: See answer to previous question.

Question 6: Will the FAA provide safety meeting minutes (OSA references [6], [7], [16], and [17])?

• Answer 6: The OSA is the completed and approved work of the safety working group.

Question 7: As DVT NavAids are installed worldwide, has the FAA coordinated this change and achieved consensus with the ICAO Nav Systems Panel and EUROCAE WG-107?

• Answer 7: See answer 3 from Live Chat Questions during DVT Industry Engagement Event

Question 8: From the OSA Section “As Table 4-1 indicates, both the airborne and ground equipment can individually cause the three hazards to occur. Thus, satisfying the service-level safety objective requires assurance that newly acquired ground and airborne equipment designed meet requisite safety objectives and requirements.

While safety objectives and requirements for airborne equipment are not addressed herein, coordination meetings occurred with FAA Aircraft Certification (AIR) and Flight Standards (AFS) during the development of this OSA [16][17]. The DVT SRM Panel included representatives from these two organizations. It was confirmed that Aircraft Certification and Flight Standards are aware that the airborne equipment faults or flight crew error can cause the hazards listed in Table 4-1 and that equipment and crew training requirements are in place to address the potential causes [8][9].”

As airborne equipment is not required to be certified to DAL-A, the airborne community must believe there is sufficient mitigation to drop the assurance level or that the Worst-case severity is not catastrophic. Will the FAA provide this data and justification as why it does not apply to the ground component or a draft to AC 20-138 and proposed date of implementation reflecting this change?

• Answer 8: This OSA was developed to understand and address safety requirements necessary for new DVT ground equipment.

Question 9: The DVT OSA is inconsistent with the previously published DME OSA. The DME OSA is more consistent with other certification authorities’ assessments. Would the FAA consider reevaluation of the hazards with a broader audience that would include other cert authorities and a larger group of DER pilots?

• Answer 9: See answer 5 from Live Chat Questions during DVT Industry Engagement Event

Question 10: As multipath produces undetected out of tolerance conditions in the form of azimuth bends and ranging errors, is it even feasible for the SiS to meet 1E-9/hour integrity (particularly as the impact of multipath changes due to season and new structure installation)? Assuming it is not, rather than levying integrity requirements on the ground system well above what can be achieved in the radiated signal, has the FAA considered the role of airborne mitigation to achieve the desired integrity?

• Answer 10: Effects from multi-path are procedurally controlled and mitigated. Certainly, the ground equipment is the start of a robust signal-in-space.

Post Event Questions

Question 1: It is reasonable to conclude that any existing NavAids – which have been developed to meet ICAO standards - will require design changes to meet DAL-A, up to and including potential changes in product architecture. In light of the DVT OSA, does the FAA now consider DVT to be a development contract? If not, please clarify the rationale.

• Answer 1: The DVT Sustainment Contract is considered a modified commercial products contract. The DMEs, VORs, and TACANs provided will need to meet the FAA requirements for

FDAL-A.

Question 2: It is our understanding that the TACAN Antenna will be installed with the FA-9996 TACAN beacon, once the new TACAN antenna is certified for use in the NAS. Given that the FAA operates and fully maintains the FA-9996 (i.e. there is no OEM vendor support), and that signal integrity is a function of the beacon and the TACAN antenna, will the FAA be responsible for the overall system performance and signal integrity?

• Answer 2: The new TACAN Antenna is planned to be installed with the FA-9996. The FAA is responsible for the performance of all of its NavAids.

Question 3: As part of the Industry Day presentation, the FAA reaffirmed that the TACAN Antenna procurement will be conducted as a competitive Small Business Set-Aside, despite the fact that DAL-A certification of the TACAN Antenna is now required. Given the fact that no OEM vendor, regardless of size, has ever certified a DVT system to meet DAL-A, how does the FAA plan to mitigate the risk of contract performance failure, and its potential ramifications on the DVT Sustain acquisition?

• Answer 3: The acquisition strategy for the DVT Sustainment Program Contract has not been finalized. In the process of developing the strategy, the FAA will consider the TACAN antenna contract status and other risks involved in the DAL-A requirement.

Question 4: The FAA intends to procure a new TACAN Beacon as part of the DVT Sustain Contract. That new TACAN system, once certified, will be installed in the NAS to replace the aging FA-9996. The timing of the two contract activities creates some challenges with respect to assessing overall system integrity and safety risk. Given that signal integrity is a function of the TACAN Antenna and the TACAN beacon, will OEMs TACAN bidders have access to the necessary drawings and ICD’s to conduct an interface hazards analysis at the time of the DVT SIR? Further, how should OEM TACAN bidders assess the risk of integrating with a TACAN Antenna that has yet to be certified as DAL-A? Does the FAA expect the DVT Sustain Prime contractor to assume the risk of TACAN signal integrity performance if the TACAN Antenna is provided GFE?

• Answer 4: The TACAN service integrity was partitioned to the ground equipment and the antenna since the two equipments are not described in the same specification/statement of work. This should allow one vendor to not be dependent on another vendor’s work. The interface between the ground equipment and antenna are simple and fully described in the specifications.

Question 5: During the Q&A session of the DVT Industry Day, the comment was made by the FAA SME that it should be relatively straight forward for “COTS” NavAids to be modified in order to achieve DAL- A. This perspective is inconsistent with recent program experience. The FAA has certified two new NavAids for use in the NAS since 2010 – the ILS 420 and the NextGen DME. The required design assurance level for both of these systems was less than DAL-A, yet the certification process took much longer than industry anticipated. In the case of ILS-420, it took 9 years and 9 months from contract award to achieve In Service Decision. While industry can reasonably and accurately estimate the effort to develop, modify and test (through DQT) a COTS NavAids to achieve DAL-A, recent procurement history clearly demonstrates that industry cannot estimate the length of time that it will take the FAA to certify approve the system. Similarly, industry cannot accurately estimate the potential cost impact of any changes that the FAA might mandate resulting from its internal certification process and procedures, unless the certification review process, timelines and approval requirements are constrained and completely detailed in Section C of the SIR.

• Answer 5: The FAA SME’s intent was to suggest that techniques exist to achieve FDAL-A without every LRU being designed and assured to Level A.

Question 6: In structuring the DVT acquisition, the FAA should reflect on recent acquisition history objectively. It’s recommended, the period of performance associated with system certification as well as any product modifications resulting from FAA mandated certification reviews be governed by Cost Reimbursable CLINS. The AMS recommends that the selection of contract type should “result in reasonable contractor risk and ensure efficient and economical contractor performance.” Further, the following guidance is referenced from the AMS (Section T3.2.4) with respect to Cost Reimbursable contracts, specifically Cost Plus Fixed Fee:

(2) Use When:

(a) The uncertainties involved in contract performance do not permit costs to be estimated with sufficient accuracy to use a fixed-price contract.

(b) The total value of the contract is high enough to justify the higher administrative costs when compared to another contract type.

(c) The contractor’s accounting system is adequate for determining costs applicable to the contract.

(d) The level of effort required is unknown such as for the performance of research or preliminary exploration or study.

(e) The extra incentive of a cost plus award fee is not necessary, but payment of profit is still appropriate.

Given that no DME, VOR or TACAN system has ever been developed to meet DAL-A, all five of these guiding principles apply.

• Answer 6: The acquisition strategy for the DVT Sustainment Contract has not been finalized.

The contract will utilize numerous contract types and CLIN structures based on the requirements.

Question 7: Will the FAA please provide the WAAS OSA, which included the dual frequency WAAS?

• Answer 7: See answer 8 from Live Chat Questions during DVT Industry Engagement Event

Question 8: Will the FAA provide answers to the questions submitted in writing prior to the event, as per FAA instructions?

• Answer 8: Yes

Question 9: Will the FAA please provide written answers to the questions that were posted in the live chat during the DVT Industry Event?

• Answer 9: Yes

Question 10: Since we submitted questions to the TACAN spec for the transponder for version.7; do we need to resubmit those questions for the TACAN spec version.8 or will they already be considered?

• Answer 10: As the FAA shares the draft versions of the specifications with vendors, the comments are considered and addressed in follow-on versions of the documents. We may or may not incorporate all the comments.