UXM RFI Questions and Answers.pdf

User Experience Monitoring (UXM) RFI Questions and Answers

AFLCMC/HNIK

User Experience Monitoring (UXM)

RFI Q&A

# Industry Question Government Response 1 Can you provide more details on the level of granularity required for service dependency mapping (e.g., application layer, network layer)?

The level of granularity required for service dependency mapping should be as detailed as possible in order to identify and address the root cause of any issues. We are seeking industry best practices in this regard. Therefore, we would like the mapping to include the application layer, network layer, endpoints, network devices and flows, as well as any other components within the transaction path.

Additionally, it would be beneficial to include the web application server and other relevant services such as DNS and OCSP.

2 Are there specific types of dependencies or interdependencies that are particularly critical for your operations?

While we cannot adopt a "green-field" approach and must work with existing network, endpoint, and cloud services, it is crucial for our operations to have visibility into the dependencies and interdependencies within our environment. Although we may not have visibility into DISA services, we aim to achieve as much end-to-end visibility of the AFNET (Air Force Network) as possible. This will enable us to effectively manage and address critical dependencies that impact our operations.

3 What is the frequency of changes in service dependencies, and how often should the mapping be updated?

The frequency of changes in service dependencies may vary, but it is important to note that the RFI specifically requests the ability to automatically map out interdependencies between applications. In order to ensure accurate and up-to-date mapping, it is recommended that updates to the mapping occur at least on a weekly basis, with a preference for daily updates. This will enable us to have timely and accurate visibility into the evolving service dependencies within our environment.

4 How many devices and locations do you anticipate will need to be included in the network map?

For the NIPR network, we anticipate the inclusion of 150+ locations, with an estimated range of 50 to 1000 devices (routers/switches) per location. Similarly, for the SIPR network, we anticipate 150+ locations, with an estimated range of 20 to 500 devices (routers/switches) per location. As for the JWICS network, the specific number of devices and locations is yet to be determined.

5 Can you provide examples of network changes that you typically simulate?

We typically simulate a variety of network changes to ensure the effectiveness and resilience of our network infrastructure. Some examples of network changes that we commonly simulate include:

1. Changes in firewalls: Testing the impact of firewall rule modifications, additions, or removals on network traffic and security.

2. Base boundaries: Assessing the network performance and security implications of adjusting or redefining base boundaries within our network architecture.

3. SD-WAN alternatives: Evaluating the performance and reliability of different SD-WAN solutions and their impact on network connectivity and traffic routing.

4. Zero Trust (ZT) implementations: Testing the implementation and functionality of Zero Trust frameworks and their impact on network access control and security.

5. Different endpoints: Assessing the compatibility and performance of various endpoint devices (e.g., laptops, mobile devices) within our network environment.

These simulations help us identify potential issues, optimize network performance, and ensure the successful implementation of network changes while minimizing disruptions to our operations.

6 What are the acceptable thresholds for deviations when comparing running configurations to "known good" configurations?

In order to meet the specific needs of Air Force Operators, it is essential to have the flexibility to set custom deviations for alert thresholds when comparing running configurations to "known good" configurations. The ability to customize these thresholds allows operators to define acceptable ranges based on their specific requirements and operational context.

Additionally, it is important to have the capability to perform derivative calculations such as standard deviation (STD) and moving averages. These calculations provide valuable insights into the stability and trends of the configurations, enabling operators to make informed decisions and identify potential deviations from the "known good" configurations.

By providing customizable alert thresholds and incorporating derivative calculations, we can ensure that the monitoring system meets the unique needs of Air Force Operators and enables effective configuration management and anomaly detection.

7 What specific network events and metrics are crucial for your current logging and monitoring activities?

The Government seeks to implement industry best practices for UX (User Experience) monitoring. In line with this objective, specific network events and metrics that we would be interested in include, but are not limited to, packet drops, jitter, connection failures, application crashes, and hangs. These metrics are crucial for our current logging and monitoring activities as they provide valuable insights into the performance and stability of our network infrastructure.

By monitoring these events and metrics, we can proactively identify and address issues that may impact user experience and overall network performance. Implementing industry best practices for UX monitoring will enable us to ensure optimal network performance, minimize disruptions, and deliver a seamless user experience for our operators and end-users.

8 How do you currently correlate network events with performance metrics, and what improvements are you looking for?

The Government currently relies on manual processes to correlate network events with performance metrics. However, we recognize the need to improve and implement industry best practices for UX monitoring. By adopting modern approaches such as OpenTelemetry (OTEL) and other industry-proven methods, we aim to enhance our ability to correlate network events with performance metrics in close to real-time.

Implementing industry best practices for UX monitoring will enable us to automate the correlation process, reducing manual effort and improving efficiency. This will provide us with timely insights into the relationship between network events and performance metrics, allowing us to identify and address issues more effectively. By leveraging advanced monitoring techniques, we can enhance our overall network performance management and deliver a seamless user experience.

Therefore, we are seeking improvements in our current correlation capabilities through the adoption of industry best practices, such as OTEL, to achieve close to real-time correlation ability and streamline our monitoring processes.

9 Are there any third-party tools or existing systems that the distributed tracing needs to integrate with?

We currently utilize several third-party tools and existing systems within our network environment. These tools and systems may require integration with the vendor's solution to ensure seamless functionality and interoperability. While the specific list of third-party tools and integrations may vary based on our unique requirements, some examples of commonly used tools and systems include:

1. Network monitoring and management tools: These tools provide insights into network performance, traffic analysis, and device management.

2. Security information and event management (SIEM) systems: SIEM systems collect and analyze security event data from various sources to detect and respond to security incidents.

3. Configuration management databases (CMDB): CMDBs store and manage configuration information for network devices and services.

4. Incident management systems: These systems track and manage incidents, facilitating timely resolution and minimizing service disruptions.

5. Log management and analysis tools: These tools collect, store, and analyze log data from various network devices and systems.

6. Ticketing and workflow management systems: These systems facilitate the tracking and resolution of network-related issues through structured workflows.

It is important for the vendor's solution to be compatible and integrate seamlessly with these existing tools and systems to ensure effective distributed tracing and monitoring capabilities. Further details regarding specific third-party tools and integrations can be provided upon request to facilitate a more comprehensive assessment of compatibility and integration requirements.

10 Can you specify the types and number of applications (both custom and commercial-off-the-shelf) that need monitoring?

The types and number of applications that require monitoring within our environment include:

1. Web activity: This encompasses all types of browsers and both DOD and non-DOD websites accessed by our users.

2. Office 365 applications: Monitoring the performance and availability of Office 365 applications, such as Outlook, SharePoint, and Teams, is crucial for ensuring productivity and user experience.

3. Custom DOD desktop applications: We have over 500 custom DOD desktop applications that need to be monitored for performance, stability, and security.

4. Cloud services: Monitoring the performance and availability of cloud services, such as AWS, Azure, or Google Cloud, is essential for ensuring optimal operation and user experience.

5. Network services: This includes monitoring critical network services such as DNS (Domain Name System), Active Directory, OCSP (Online Certificate Status Protocol), and other infrastructure components that are vital for network operations.

While this is not an exhaustive list, it provides an overview of the types of applications, both custom and commercial-off-the-shelf, that require monitoring within our environment. The specific number of applications may vary, but it is important to have comprehensive monitoring coverage across these application types to ensure optimal performance, security, and user experience.

11 What are the unique challenges you face with your on-premises virtual environments like VMware?

When it comes to on-premises virtual environments like VMware, the unique challenges we face primarily revolve around the management and monitoring of the Domain Controller services hosted within these environments. Specifically, the challenges are focused on the DNS (Domain Name System), DHCP (Dynamic Host Configuration Protocol), Active Directory, and other related services.

Some of the key challenges we encounter include:

1. Log management: Effectively collecting, storing, and analyzing logs generated by the Domain Controller services to identify errors, security events, and performance issues.

2. Resource allocation: Ensuring proper resource allocation, such as CPU, memory, and storage, to support the optimal performance of the Domain Controller services.

3. Performance monitoring: Monitoring the performance of the Domain Controller services to identify bottlenecks, latency issues, or resource constraints that may impact their functionality.

4. Event management: Tracking and managing events related to the Domain Controller services, such as user authentication events, group policy changes, and replication status.

5. Security monitoring: Detecting and responding to security events and vulnerabilities within the Domain Controller services, including unauthorized access attempts, account lockouts, and potential security breaches.

To address these challenges, it is crucial to have robust monitoring and management tools specifically designed for virtual environments like VMware. These tools should provide comprehensive visibility into the performance, logs, errors, and resource utilization of the Domain Controller services, enabling proactive management and timely resolution of any issues that may arise.

12 How often are synthetic tests conducted, and what performance metrics are most important to you?

Currently, synthetic network tests are conducted at over 80 bases within our network environment. These tests are performed at intervals ranging from every 10 to 30 minutes. The purpose of these tests is to evaluate the performance and functionality of various network components, including DNS, OCSP, FileDownload, TCP connectivity, and more.

In terms of performance metrics, the following are considered most important:

1. Throughput: Measuring the amount of data transferred per unit of time, indicating the network's capacity and efficiency.

2. Latency: Assessing the time it takes for data to travel from the source to the destination, indicating the responsiveness and speed of the network.

3. Errors: Monitoring and tracking any errors or failures encountered during the synthetic tests, such as connection failures, timeouts, or other network-related issues.

4. Custom information: Gathering additional custom information, such as cache misses or other specific metrics relevant to our network environment, to gain deeper insights into performance and potential areas for improvement.

By collecting and analyzing these performance metrics, we can identify any bottlenecks, latency issues, or errors that may impact the overall network performance. This information allows us to proactively address any issues, optimize network performance, and ensure a seamless user experience for our operators and end-users.

13 How frequently do non-expert programmers need to create custom tests?

The frequency at which non-expert programmers need to create custom tests can vary depending on the specific requirements and changes within the network environment. However, it is common for non-expert programmers to create custom tests on a regular basis to address evolving needs and ensure comprehensive testing coverage.

Based on the information provided, the system should support the ability for a single non-programmer to create new scripts within a few hours. This indicates a need for a user-friendly interface or tool that simplifies the process of creating custom tests, allowing non-programmers to quickly generate scripts without extensive coding knowledge.

Additionally, the system should be flexible enough to accommodate the addition of ten new scripts each week. This suggests a need for scalability and ease of script management, enabling non-programmers to efficiently add new tests to the system on a regular basis.

The specific timing and level of effort that the industry can provide may vary based on the capabilities of the monitoring system and the expertise of the non-programmers involved. It would be beneficial to engage with potential vendors to understand their proposed solutions and determine the feasibility of meeting the desired requirements within realistic timeframes.

14 Can you provide examples of third-party data sources that need to be ingested and correlated with the monitoring solution?

While specific vendors cannot be disclosed, understanding the general types of data sources can help in scoping the problem and identifying the integration requirements. Here are some examples:

1. Application Performance Monitoring (APM) tools: These tools collect data on application performance metrics, such as response times, throughput, error rates, and resource utilization. Integration with APM tools allows for comprehensive monitoring of application performance and user experience.

2. Synthetic network test tools: These tools simulate network traffic and test various network components, such as DNS, HTTP, TCP connectivity, and more. Integration with synthetic network test tools enables proactive monitoring of network performance and availability.

3. Endpoint monitoring tools: These tools monitor the performance and health of endpoints, such as desktops, laptops, or mobile devices. Integration with endpoint monitoring tools provides insights into device performance, resource utilization, and potential issues affecting end-user experience.

4. Distributed tracing solutions: These solutions capture and trace requests as they flow through complex distributed systems, providing visibility into the performance and dependencies of microservices or other components. Integration with distributed tracing solutions allows for end-to-end monitoring and troubleshooting of distributed applications.

5. Unified monitoring solutions (e.g., OpenTelemetry): These solutions provide a standardized approach to collecting and correlating telemetry data from various sources, including applications, infrastructure, and network components. Integration with unified monitoring solutions enables a holistic view of the entire system and facilitates correlation and analysis of data from multiple sources.

15 Are there any existing custom scripts or tools that need to be integrated?

The Government has not yet determined if there are any existing custom scripts or tools that will need to be integrated with the UX monitoring system. The decision regarding the integration of existing custom scripts or tools will be further refined during the market research phase, and more specifics will be included within the Government's formal Request for Quote (RFQ) at a later date.

16 What specific industry standards (other than OpenTelemetry) are critical to your operations?

While it is not possible to provide an exhaustive list of all the DISA (Defense Information Systems Agency) and Air Force standards critical to operations, here are some examples and guidance on where vendors can find more information.

Some examples of DISA and Air Force standards that may be relevant include:

1. Security Technical Implementation Guides (STIGs): These guides provide security configuration recommendations for various software, hardware, and network devices to ensure compliance with DoD security requirements.

2. Defense Information System for Security (DISS) standards: These standards outline the requirements and processes for managing personnel security clearances and access to classified information.

3. Air Force Instructions (AFIs): AFIs are policy directives that provide guidance and procedures for various aspects of Air Force operations, including cybersecurity, acquisition, personnel management, and more.

4. DoD Information Assurance Certification and Accreditation Process (DIACAP): This process outlines the requirements and procedures for certifying and accrediting DoD information systems to ensure they meet security standards.

To help vendors familiarize themselves with these standards, they can refer to publicly available resources such as the DISA website (https://www.disa.mil/) and the Air Force e-Publishing website (https://www.e-publishing.af.mil/). These websites provide access to various publications, instructions, and guidelines that outline the standards and requirements for DoD operations.

17 How do you currently ensure compliance with Semantic Conventions for attributes across your systems?

See #14. Ensuring compliance with Semantic conventions is the current failing of our legacy system, and the thrust of the RFI is to find solutions that make compliance and integration across multiple telemetry gathering systems manageable and not-labor intensive.

18 Are there any specific certifications or compliance requirements that the solution must meet?

See #16.

19 Can you list the existing network, systems, and application management tools currently in use that the solution must integrate with?

See #14. But in general we have a mix of Windows Servers, Clients, Unix machines, Macintosh, some tablets. And network gear (firewalls and routers) from CISCO, Palo Alto, Juniper, etc.

20 Are there any legacy systems that might present integration challenges?

There is a legacy system currently in place. However, the intention is to replace this legacy system entirely, if possible. As a result, the integration challenges associated with the legacy system may not be a significant concern, as the goal is to transition to a new system that can replace it entirely.

21 What are your preferred methods for integration (e.g., APIs, direct connectors)?

The preferred methods for integration with the monitoring solution include a range of options to ensure flexibility and compatibility.

These methods include:

1. APIs (Application Programming Interfaces): APIs provide a standard way for different systems to communicate and exchange data.

Integration through APIs allows for seamless data transfer and interaction between the monitoring solution and other systems or data sources.

2. Direct connectors: Direct connectors are specific integrations or connectors built to establish a direct connection between the monitoring solution and other systems or data sources. These connectors facilitate efficient and reliable data transfer, enabling real-time monitoring and analysis.

3. Streaming/Messaging Queues: Integration through streaming or messaging queues allows for asynchronous data transfer and processing. This method is particularly useful for handling large volumes of data or for scenarios where real-time processing is not required.

By supporting these integration methods, the monitoring solution can accommodate a variety of systems, applications, and data sources, ensuring comprehensive data ingestion and correlation. This flexibility enables seamless integration with existing infrastructure and systems, allowing for a holistic view of the environment and effective monitoring and analysis capabilities.

22 How do you currently handle vulnerability notifications and patches?

The current approach to handling vulnerability notifications and patches involves multiple processes, including vulnerability scans, configuration management, change management, and patch management.

1. Vulnerability Scans: Regular vulnerability scans are conducted to identify potential vulnerabilities in the system. These scans may use automated tools or services to assess the system's security posture and identify any known vulnerabilities.

2. Configuration Management: Configuration management involves maintaining a centralized repository of system configurations and ensuring that systems are configured securely. This process helps to reduce the attack surface and minimize the risk of vulnerabilities resulting from misconfigurations.

3. Change Management: Change management processes are implemented to manage and control changes to the system. This includes assessing the impact of proposed changes, evaluating potential security risks, and implementing appropriate controls to mitigate vulnerabilities introduced by changes.

4. Patch Management: Patch management involves the identification, testing, and deployment of software patches and updates to address known vulnerabilities. This process ensures that systems are up to date with the latest security patches and helps to mitigate the risk of exploitation.

23 What are your key requirements for the test plan prior to applying patches?

The key requirements for the test plan prior to applying patches include:

1. Declaration of Third-Party Software: If the patches involve third-party software, it is important to declare and document the involvement of these third-party components. This helps in understanding the dependencies and potential impacts of the patches on the overall system.

2. Thorough Testing: Patches must undergo thorough testing before their application. This includes various types of testing, such as functional testing, regression testing, performance testing, and security testing. The testing should cover different scenarios and use cases to ensure that the patches do not introduce new issues or negatively impact the system's functionality, performance, or security.

3. Test Environment: A dedicated test environment should be set up to replicate the production environment as closely as possible.

This allows for realistic testing of the patches and helps identify any potential conflicts or issues that may arise during the patch application process.

4. Test Coverage: The test plan should outline the specific areas and components of the system that will be tested. This includes identifying critical functionalities, interfaces, and dependencies that may be affected by the patches. The goal is to ensure comprehensive test coverage to validate the patches' effectiveness and minimize the risk of unintended consequences.

5. Test Documentation: The test plan should include clear documentation of the testing process, including test cases, test scripts, and expected results. This documentation helps in tracking the progress of testing, identifying any issues or deviations, and providing a reference for future patch management activities.

24 Are there specific security standards or protocols that must be followed during vulnerability management?

DoD/Air Force follows the NIST (National Institute of Standards and Technology) Special Publication 800-53 Revision 5 standards and protocols for vulnerability management. The specific security standards and protocols that must be followed during vulnerability management include:

1. Vulnerability Scans: Conducting regular vulnerability scans to identify potential vulnerabilities in the system. These scans help in assessing the security posture of the system and identifying any known vulnerabilities.

2. Security Technical Implementation Guides (STIGs): Implementing STIGs, which provide security configuration recommendations for various software, hardware, and network devices. Following STIGs helps ensure compliance with DoD security requirements and best practices.

3. Security Requirements Guides (SRGs): Adhering to SRGs, which provide specific security requirements for different types of systems or applications. SRGs help in implementing necessary security controls and mitigating vulnerabilities.

4. Configuration Management: Implementing configuration management processes to maintain secure configurations for systems.

This includes managing and controlling changes to configurations, ensuring adherence to security standards, and minimizing the risk of vulnerabilities resulting from misconfigurations.

5. Change Management: Following change management processes to manage and control changes to the system. This includes assessing the impact of proposed changes, evaluating potential security risks, and implementing appropriate controls to mitigate vulnerabilities introduced by changes.

6. Patch Management: Implementing patch management processes to identify, test, and deploy software patches and updates to address known vulnerabilities. This helps ensure that systems are up to date with the latest security patches and minimize the risk of exploitation.

25 What are the primary roles and responsibilities of your team members who will be using this system?

Some of the primary roles and responsibilities of team members who will be using this system include communication squadrons at DAF bases managing and maintaining the base level comms infrastructure; AFNET troubleshooters to resolve complex network and comms issues within the AFNET; and the acquisitions team to troubleshoot, analyze, and validate system performance. These team members will leverage the system to monitor, analyze, and troubleshoot the communication infrastructure, ensuring optimal performance, security, and availability of the network services.

26 How often do you anticipate needing refresher or advanced training sessions?

The frequency of these sessions can vary depending on factors such as the complexity of the vendor system, the rate of system updates, and the evolving needs of the users. The Government will provide more information specific to the training needs within a future RFQ.

27 What are your minimum acceptable response times for technical support?

This will be determined within the service level agreements (SLAs) that outline the expected response times for technical support. The response times will vary depending on the severity of the issue and the level of support required. The Government seeks to implement industry best practices for UX monitoring, and is looking for industry to provide information to support standard response times for technical support.

28 Is on-line training and certification at the associate and engineer level acceptable?

On-line training and certification at the associate and engineer level are considered acceptable options for training. However, the Government has not defined its training requirements yet and may consider other training options as well which will be specified in a future RFQ.

29 What challenges have you encountered during previous implementations of similar solutions?

See #17.

30 Are there any upcoming projects or deadlines that might impact the implementation timeline?

There are no specific upcoming projects or deadlines that might impact the implementation timeline, except for potential budgetary constraints related to the fiscal year spend allowances.

31 What are the critical success factors for the deployment and configuration phases from your perspective?

The specific technical and programmatic factors will be identified within a future RFQ. Some potential success factors for the deployment and configuration phases include:

1. Time-to-rollout: The efficiency and timeliness of the deployment process are crucial for successful implementation. Minimizing deployment time can help ensure that the system is up and running quickly, reducing any potential disruptions or delays.

2. Adaptability to existing telemetry sources: The system's ability to seamlessly integrate and adapt to existing telemetry sources is critical. Compatibility with existing infrastructure and data sources allows for a smooth transition and ensures that valuable data can be effectively utilized within the system.

3. Level of standardization of unified and consumed data: Standardization of data formats and structures is essential for efficient data processing and analysis. A high level of standardization ensures that data can be easily consumed and utilized by the system, enabling accurate and meaningful insights.

4. Automated catalogs and search and discovery of curated data: The availability of automated catalogs and robust search and discovery capabilities can greatly assist analysts in conducting forensic and forecasting studies. These features enable efficient access to curated data, facilitating comprehensive analysis and decision-making.

32 What budget constraints or timelines should we be aware of when proposing pricing models?

At this time the Government does not have specific budget constraints or timelines to share. The Government looks for vendors to provide responses to this RFI that identify its technical solution and responses to the RFI questions to support further market research for a future requirement.

33 Are there preferences for licensing models (e.g., perpetual vs.

subscription-based)?

There are no preferences on the licensing structure; however, the Government typically received funding on an annual basis for software licensing.

34 What is your process for evaluating the total cost of ownership, including implementation, training, and ongoing support?

At this time the Government is focused on market research to identify potential technical solutions in response to the RFI questions.

35 Are there specific types of organizations or industries whose implementations you would like us to highlight in our references?

DoD implementations would be preferable due to the unique nature of working within a secure environment and incorporating the system into geographically separated classified networks; however, the Government is open to receiving any references that provide the best examples of implementation of similar requirements (commercial or non-commercial).

36 What aspects of previous solutions' implementations have you found most beneficial?

The Government seeks to identify current industry best practices for UX monitoring, and is interested in what vendors can provide that meet the questions within the RFI.

37 How do you measure success and return on investment for these types of solutions?

Specific measures of success will be included within the future RFQ. Some potential measures include baseline performance, user experience, problem resolution time, network anomaly detection, and infrastructure performance monitoring.

38 What compute, storage and networking Infrastructures are being used, i.e. HPE/ HPE Aruba

The Government's current compute and store and networking solution is primarily through DoD cloud based tools that are not integrated into a single monitoring system.

39 What VDI client solutions are being used, i.e. Vmware, Citrix, HP Anyware

The Government currently utilizes VMWare and Citrix as VDI client solutions. Additionally, there is an ongoing pilot of Azure Virtual Desktop (AVD).

40 What Data solutions are being used, i.e. Microsoft SQL Server, Oracle

The Government currently utilizes various data solutions across the base area processing centers. However, there is an ongoing migration to cloud solutions, including AWS (S3, RDS, Elastic), Azure Data Lakes, Palantir Foundry, and others.

41 What Analytics solutions are being used, i.e. TigerGraph, Elastic Search, SAS

The Government's current analytics solution includes No-Code ETL, No-code SPARK based analytics, and HDF. However, the government is open to considering other solutions that can effectively support the requirements of the UXM project. (for example, see: #14, #17, #31).

42 What AI solutions are being used, i.e. Artisight, Jupyter, TensorFlow, NVIDIA Riva, Waittime

The Government does not currently have a user community with expertise in AI solutions like Artisight, Jupyter, TensorFlow, NVIDIA Riva, or Waittime. However, the Government is interested in considering automated AI-ML solutions that align with the requirements of the UXM project. For example, solutions like Co-Pilot for Data Fabric & Analytics from Microsoft could provide a frame of reference.

The Government is open to receiving RFI responses that highlight automated AI-ML solutions.

43 What Edge solutions are being used, i.e. Vmware vSphere, RedHat OpenShift, SuSe Rancher

The Government currently doesn't have a enough information to adequately respond to this question.

44 What Hybrid cloud solutions are being used, i.e. Vmware VCF, vSAN ReadyNodes, Azure Stack HCI

The Government currently doesn't have a enough information to adequately respond to this question.

45 What Cloud-ready container solutions are being used, i.e. HPE Exmeral, Red Hat OpenShift

The Government currently doesn't have a enough information to adequately respond to this question.

46 What Virtualization solutions are being used, i.e. Vmware vSphere, Microsft Hyper-V

The Government currently utilizes VMWare vSphere and Microsoft Hyper-V as virtualization solutions.

47 How many applications are expected to be monitored? See #10.

48 How many end users are hitting the applications? The Government anticipates around 750,000 end users.

49 How many servers/assets/network devices will be monitored? See #4.

50 What is the incident management ticketing tool used? The Government uses an incident management ticketing tool called ServiceNow.

51 Can USAF confirm that the proposed monitoring tool needs to be compliant with IL4 standards?

The Government requirement is for a minimum of IL5 and IL6 standards. Specifics about the requirement will be included in a future

RFQ.

52 Can you share a list of other systems/technologies that will need to be integrated? (e.g., Ticketing, reporting, other monitoring, etc.).

ServiceNow is one of the systems/technologies that will need to be integrated. However, the list of other systems/technologies that need to be integrated may vary depending on the comprehensiveness of the approach or the need for supplementation.

53 Can you share a list of example systems and applications for which user experience is desired? (e.g, Web based GOTS, Thick client COTS, Cloud SaaS, other?)

The Government will include more specifics within a future RFQ; however, some examples of user experience metrics desired for various systems and applications, include:

1. Web-based GOTS (Government Off-The-Shelf) applications: These are custom-built applications specifically developed for government use.

2. Thick client COTS (Commercial Off-The-Shelf) applications: These are commercially available software applications that are installed and run on individual client machines.

3. Cloud SaaS (Software-as-a-Service) applications: These are cloud-based applications that are accessed and used over the internet without the need for local installation.

54 What is the total number of security enclaves being supported i.e.

CUI/Unclass, SIPR, NATO SECRET, Coalition Classified, Top Secret.

While the specific number of security enclaves being supported is not firm, the main emphasis is on NIPR, with the possibility of expanding to SIPR and Top Secret enclaves in the future.

55 Estimated end users supported for each security enclave? The Government anticipates 750,000 end users supported for NIPR.

56 Estimated total routers, switches, firewalls across enclave enterprise (Layer 2 and Layer 3).

NIPR = ~40,000; SIPR = ~20,000; JWICS = unknown

57 Estimated physical servers per security enclave across enclave enterprise.

NIPR = 15,000; SIPR = 5,500; JWICS = unknown

58 Are there any requirements for IPV6? Firm requirements will be included in a future RFQ, but by ensuring IPv6 support, the solution will align with the current and future needs of the DAF and enable the Government to address the IPv4 address exhaustion.

59 Will requirements support on-premises, edge-cloud computing, cloud computing?

Firm requirements will be included in a future RFQ, but by supporting on-premises, edge-cloud computing, and cloud computing, the solution will provide the government with the flexibility to choose the most suitable deployment model based on their specific requirements and constraints.

60 PKI/CAC/SSO requirements? The requirement for PKI/CAC/SSO is not applicable in this case.

61 Estimated number of Data Center nodes at each level i.e.

Enterprise, Regional, Local, Deployable Data Centers.

The Government currently doesn't have a enough information to adequately respond to this question.

62 What types of devices will be monitored in the environment and what are the main manufacturers of those devices?

In the environment, the monitoring will cover network and server infrastructure, as well as user endpoints. The devices from various manufacturers will be monitored, including but not limited to Dell, Cisco, HP, and others.

63 It is assumed that all of the applications being monitored is COTS software. What applications will be monitored in the environment?

In the environment, the monitoring will cover a range of applications, including GOTS (Government Off-The-Shelf), COTS (Commercial Off-The-Shelf), and web endpoint applications. This encompasses both thick and thin client applications.

Additionally, the monitoring will extend to cloud services, domain controller services such as DNS (Domain Name System) and Active Directory, as well as both cloud-based and on-premises servers.

64 How many Maps do you anticipate utilizing? The Government currently doesn't have a enough information to adequately respond to this question.

65 What types of NoSQL Database will be in the envitroment? In the environment, there may be various types of NoSQL databases present, depending on the groups who own those services and their preferences. Some of the commonly used NoSQL databases in the DAF environment include:

1. Data Lakes: Data lakes are storage repositories that store large amounts of raw data in its native format. They provide a flexible and scalable solution for storing and analyzing vast amounts of structured, semi-structured, and unstructured data.

2. Delta Lakes: Delta Lake is an open-source storage layer that brings reliability and performance optimizations to data lakes. It provides ACID (Atomicity, Consistency, Isolation, Durability) transactions, schema enforcement, and data versioning capabilities on top of existing data lakes.

3. Elastic: Elastic is a distributed, RESTful search and analytics engine built on Apache Lucene. It is commonly used for real-time data exploration, log analytics, and full-text search capabilities.

4. Other Cloud Storage: Various cloud storage solutions, such as Amazon S3, Microsoft Azure Blob Storage, or Google Cloud Storage, may also be used in the DAF environment for storing and managing data.

66 Is there a requirement for Modeling / Lab Licenses? Firm requirements will be included in a future RFQ; but, the Government anticipates a requirement for modeling/lab licenses for a small-scale pilot testing prior to the initial implementation. These licenses would be necessary to set up and conduct the pilot testing in a controlled environment to simulate and evaluate the proposed solution or system before its full-scale implementation.

67 Will the modeling be performed in a LAB instance of the software, or in the actual production environment?

Anticipated modeling will primarily be performed in a LAB instance of the software, which provides a controlled and isolated environment for testing and experimentation. This allows for the modeling to be conducted without impacting the actual production environment.

There is a preference to ingest near-live data into the modeling environment if available. This means that the modeling process can utilize data that closely resembles the actual production data, providing more accurate and realistic results. While the preference is to have near-live data, more information regarding this requirement will be included within a future RFQ.

68 Is it a critical requirement or enhancement capabillty for ability to monitor COTS/GOTS/proprietary software? Integration of emerging technologies/software and rapid response for integration solutions?

The ability to monitor COTS, GOTS, proprietary software, and integration of emerging technologies/software is a critical requirement for the environment. This broad coverage ensures comprehensive monitoring across various types of software and technologies. The focus of this monitoring effort is primarily on detection rather than automated fixes. The system should be capable of detecting and alerting on potential security incidents, anomalies, or other relevant events within the monitored software and technologies.

Additionally, if the system can integrate with workflow management systems like ServiceNow and other collaboration/ticketing systems, it can help streamline incident response and facilitate efficient resolution of detected issues. This integration capability would be considered an enhancement capability.

The coverage of COTS, GOTS, web, cloud, DNS, Active Directory, and OCSP (Online Certificate Status Protocol) systems is important to ensure comprehensive monitoring across critical components of the environment.

69 Federal or other software certification requirements? If "yes" what are those requirements and what security enclave are they required for?

The Government currently doesn't have a enough information to adequately respond to this question.

70 Estimated project award (by FQ/FY). The Government anticipates contract award by 4th Quarter of Fiscal Year 2025.

71 Estimated project start (by FQ/FY). The Government anticipates contract award by 4th Quarter of Fiscal Year 2025.

72 Estimated project Initial Operations Capable (IOC), Full Operations

Capability (by FQ/FY).

The Government will define the timeline to IOC/FOC within the future RFQ.

73 Is there a pending/existing service contract requirement to support installation phase(s)?

There is no current or pending contract in place.

74 Is there a pending/existing service contract requirement to support systems configuration phase(s)?

There is no current or pending contract in place.

75 On site support requirement throughout the engineering, installation and sustainment phase/through duration of software license term?

The Government will further define this requirement within the future RFQ, but there is potential for on-site support for some period of time.

76 Contact team requirements through the duration of the license term?

This will be based on the agreed SLA on support for different priority issues.

77 Federal Support Services requirements i.e. expection or mandate for CONUS based engineering/product/customer success support that is familiar with DoD/processes, procedured, NIST and service/joint specific DODIN policy?

The Government currently doesn't have a enough information to adequately respond to this question.

78 Initial and enduring training requirement methods and setting i.e. production environment, lab environment, combination, virtual or in-person, combination?

The Government currently doesn't have a enough information to adequately respond to this question.

79 Will the prospective or awarded software publisher be provided with details regarding proprietary software details to enable network monitoring/observability functions.

The Government currently doesn't have a enough information to adequately respond to this question.

80 Please provide details regarding clearance requirements for publisher pre-sale/post sale?

The Government currently doesn't have a enough information to adequately respond to this question.

81 SAP/STO read-in requirement for awarded personnel? The Government currently doesn't have a enough information to adequately respond to this question.

82 Can the customer clarify the scope, and DAF customer sets, that comprise the proposed network monitoring?

The Government currently doesn't have a enough information to adequately respond to this question.

83 Does the government intend to award to a single awardee across on-prem and cloud? Or is the government open to multiple awardees to support multiple infrastructure and networking constructs?

The Government is currently in market research and an acquisition strategy has not yet been determined.

84 Is the customer seeking visualization or event notification tooling, such as a dashboard tool?

Yes.

85 What is the deployment model for the requested solution? On-premise and locally hosted or Software as a Service?

The Government currently doesn't have a enough information to adequately respond to this question.

86 Does the data reside in NIPR, SIPR, or higher? The data resides in both NIPR and SIPR.

87 What 3rd party tools and applications will be leveraged for data ingestion?This will be further defined within a future RFQ, but it is anticipated that the Government will prefer vendors to provide the tools.

88 Are there any current tools being utilized by the HNI UXM PMO? See #20.

89 What sensors are being utilized to monitor the various networks, infrastructure and applications?

The Government currently doesn't have a enough information to adequately respond to this question.

90 Is there an incumbent contract doing some or most of this work today?

There is no current or pending contract in place.

91 Are you looking for both large and small business participation? This RFI is open for any sized business to submit a response.