Cyber Analytic Platform Requirements.pdf

Capability Overview:

Cyber analytic platform that ingests, normalizes, correlates, and aggregates multiple data types from multiple sources. The platform indexes data by type and enables users to input relevant infrastructure information and retain and enrich that information. The platform enriches and correlates data in the different indexes. Users conduct searches against the data in the platform and the data is presented in an intuitive interface, enabling analysts to identify cyber threats and vulnerabilities.

Requirements:

Ingest multiple data types from multiple data sources.

• Data Source 1 ingest is on average between 1.1– 4.5 billion records per day

• Data source 1 ingest should occur no less than once per every 24 hours

• Data Source 2 ingested no less than once per week

• Data Source 3 ingested no less than once per week

• Data Source 4 ingested no less than once per week

• Average of 3.5 million queries per day to enrich ingested data sources

• Data enrichment of 60 – 300 million records per day

• Average ingest amount ~2TB per day

• Storage capacity ~800TB

The capability must be scalable to support up to 1,000 total users

The capability must be scalable to support up to 500 concurrent users

Platform as a Service: Implement a secured platform-as-a-service system based in hardware, software, cloud, or a combination therein. Capable of ingesting and exporting data in multiple formats (e.g. JSON, XML, SQL, CSV, etc.) and presenting that data in a manner that enables analysts to identify cyber threats and vulnerabilities.

User Interface: The capability should have an intuitive graphical user interface (UI) that does not require knowledge in a specific programming language. The UI should enable visual depictions of all indexes and allow access to those indexes to users based on role. The UI should enable users to easily view and query the data.

Functionality:

• The capability must be able to ingest, normalize, correlate and aggregate multiple indexes.

o The capability must be able to scope/limit ingest of specific data types to only pre-defined relevant data within certain data sets.

• The capability must be able to enrich ingested data types.

• The capability must have an intuitive graphical user interface.

• The capability must allow advanced query against all data fields.

• The capability must allow for export of results in multiple formats.

• The platform must be capable of implementing and maintaining business rules in compliance with established and developing policies and laws.

• The platform must provide reliable access based on admin and user needs. Scheduled maintenance shall not significantly impact availability and will be conducted during off peak hours. The system shall be capable of technology and software refresh. The capability must deliver 99.9% availability.

Hierarchy and Role Based Security: The platform must support role-based security / access control model. The platform must support a role-based security approach that grants and restricts access to data and capabilities based upon user role.

IL5: Capability must be hardware, software, cloud based, or a combination therein, hosted and maintained by the vendor. Solution should be a Commercial-off-the-Shelf (COTS) or Government-off-the-

Shelf (GOTS) capability to reduce the overall cost. The capability may be cloud based to enable the processing and storage of large amounts of data without the dependance and cost of purchasing and maintaining hardware. Data ingestion and access must be scalable and flexible depending on mission requirements. The capability must have the ability to back-up enriched data in an alternate location for data security. The capability must be Impact Level 5 certified in order to store and maintain Controlled

Unclassified Information (CUI).

• Data availability: 12 months of data.

o Hot Storage: Up to 6 months of data must be readily available to the users

▪ Hot storage definition: Low latency: Hot storage is optimized for low-latency access, making it ideal for applications and data that require rapid response times.

▪ Scalability: Hot storage services often offer easy scalability to accommodate data growth or changing usage patterns.

▪ Data access speed: Hot storage is designed for frequent read and write operations, making it appropriate for data that requires rapid access.

o Warm storage: Data ranging from 6-12 months can be stored in warm storage

▪ Warm storage definition: Warm storage is data that is accessed less frequently.

This could be data that is used for reporting or analytics. It doesn’t need to be accessed as quickly as hot data, so it can be stored on slightly slower, capacity-optimized storage.

Administration: The platform must have centralized administration capability based on hierarchy and role-based security to manage security, automation, and business rules.

Data Tagging: The capability must include a data/record tagging capability.

Auditing: The capability must include logging of activities (e.g., accessing, creation, modification, or deletion), records, and artifacts in accordance with applicable laws and National, DoD, and DCSA policies.

Advanced Analytics: The capability may include advanced analytic capabilities such as Machine Learning and Artificial Intelligence algorithms. Advanced analytics should analyze data and provide a result based on a user defined need.

Collaborative Environment: The capability may include functionality for collaboration, workflow management, data sharing, persistent searches, document/record/query sharing, etc.