AISAP_Consolidated_QA_final_10 Feb 25.pdf

# Question Answer

Is this a new requirement or is there an incumbent performing these services? If so, who is the incumbent and what are the contract numbers? Yes, this is a new requirement.

Why is PEO IEW&S not establishing any set aside tracks and/pools for Small Businesses that specialize in AI/ML services and solutions ?

AI onboarding (1.1) is set-aside for small businesses. Additionally, market research at the Task Order level will dictate if an individual Task Order should be set-aside for small business.

We had submitted detailed response to the prior RFI released as Fast Track MATOC - W56KGY-24-R-MATOC. Did the government evaluate the responses and leverage those to update the draft PWS ? Please confirm.

Yes, the Government evaluated the responses and will leverage those to update the draft PWS as applicable.

The PWS does not have any mention of “ Platform or Technology evaluation for AI/ML solutions” , does the government anticipate doing any evaluation of platforms as a part of the services procured through the MATOC?

At present, our strategy does not involve the acquisition of platforms; however, we will permit selected vendors to utilize their own proprietary tools and systems to deliver services in support of the MATOC, thereby enabling them to leverage their existing investments and expertise to meet the requirements of the contract.

It's worth noting that our Project Linchpin (PL) is not a single platform, but rather a complex ecosystem comprised of various interconnected components, tools, and services, designed to support a wide range of customer use cases and requirements. Ultimately, the procurement of specific products or platforms will be driven by customer use cases and requirements, and we will adapt our procurement approach as needed to ensure that the solutions provided meet the evolving needs of our customers and effectively integrate with the broader PL ecosystem.

Will PEO IEW&S provide technology platforms for development of AI/ML solutions, or does it expect vendors to bring forth their own platforms ?

PL is providing the environment w/ basic AI/ML tools, but will allow the vendors to utilize their own tools.

What specific outcomes or deliverables are you expecting for the initial task orders under this RFP?

The initial task orders will focus on onboarding customer use cases, as well as managing, collecting, and labeling data to prepare it for artificial intelligence (AI) processing, thereby ensuring that the data is AI-ready.

Are there any preferred approaches or methodologies for addressing AI/ML and software development challenges? For example, do you prioritize COTS solutions over custom-built products for certain use cases?

Project Linchpin is outcome-focused, prioritizing achievable objectives over specific methodologies. For AI/ML, we seek modular, interoperable components that can be easily integrated. Success will be measured by delivery of tangible results within established cost and schedule parameters, with a focus on timely, cost-effective solutions that minimize lifecycle costs and maximize return on investment for the customer. We do not prefer any particular approaches, we are more outcome focused. For AI/ML we are not mandating a specific methodology, but we do want modular components that are interoperable.

What systems would you like us to integrate with, and would APIs be better than another screen?

Project Linchpin intends to deploy Artificial Intelligence (AI) models within standard containers, complete with exposed Application Programming Interface (API) interfaces. These containerized AI models will be designed to seamlessly integrate with Project Linchpin's customers' existing systems and user interfaces, enabling efficient and scalable deployment of AI capabilities.

Potential programs were described in the Industry Day slides (morning).

What level of customization, if any, is anticipated for COTS software to meet mission needs (APIs, Reporting, Customized views, etc.)?

Project Linchpin's core capabilities and use cases with customers will inform the level of customization required for each solution. To ensure maximum interoperability and reuse, Project Linchpin is establishing standards, APIs, and best practices that will be enforced across all solutions, thereby promoting a high degree of consistency and compatibility among disparate systems and components.

Can you elaborate on the role AI/ML tools will play in supporting decision-making and situational awareness for PEO IEW&S?

Project Linchpin is collaborating with customers to gather and refine use cases, and to identify the specific services they wish to consume from the project. Ultimately, it is the responsibility of the functional customer to determine how Artificial Intelligence/Machine Learning (AI/ML) capabilities will be utilized to support their decision-making processes and situational awareness requirements, ensuring that AI/ML solutions are tailored to meet their unique needs and operational contexts.

What datasets or types of data (e.g., social media, dark web, other (multimedia, Online Print?) are the highest priority for AI/ML applications?

Project Linchpin will support a wide range of data types across the Army, including imagery, Radio Frequency (RF) data, and text. The datasets themselves are produced and owned by the respective Army customers, who will provide subsets of the data to Project Linchpin for the purpose of developing and training AI capabilities. This approach ensures that data ownership and control remain with the functional customers, while Project Linchpin is able to leverage the data to deliver advanced AI capabilities.

Are there preferred metrics or benchmarks for evaluating the performance of AI/ML solutions (e.g., accuracy, timeliness, scalability)?

Project Linchpin is focused on supporting Army use cases, each of which will have its own set of desired metrics that must be met in order to effectively support the use case. The performance of the AI solutions will be directly tied to the capabilities they provide in support of each use case, ensuring that the solutions are measured and evaluated based on their ability to deliver tangible value and support to the Army's operational stakeholders.

What specific security requirements apply to the AI/ML data pipelines and model training environments?

Project Linchpin is focused on supporting Army use cases, with associated data ranging from unclassified to Top Secret. The security requirements for the project will be driven by the specific data associated with each use case, taking into account its sensitivity and classification, as well as the operational context in which it will be used.

14 Will this be a Small Business Set-Aside (SBA) or what are the SB considerations?

This will be a multiple award IDIQ contract. Subpool 1.1 will be set-aside for small business.

Additionally, market research at the Task Order level will dictate if an individual Task Order should be set-aside for small business.

We have noted a significant focus on Data Management & Labeling (sRP1.2), which addresses key aspects such as onboarding data, ensuring security, designing data pipelines, hiring labelers, and maintaining labeling standards. However, we observe that there is no dedicated section addressing the Production and Utilization of Synthetic Data—a solution that could significantly enhance the RFP’s objectives. See pdf

Project Linchpin intends to utilize synthetic data in instances where real data is unavailable or insufficient. Currently, PL is actively exploring the applications and potential benefits of synthetic data through various initiatives, including Small Business Innovation Research (SBIR) programs and customer-sponsored activities. While the strategic integration of synthetic data is a key component of Project Linchpin's overall approach, we do not anticipate acquiring this capability within the scope of this specific subpool.

Clarification on Requirement Pools (RPs) and Sub-Requirement Pools (sRPs): • Comment: While the draft PWS introduces the concept of RPs and sRPs, the criteria for dividing requirements into sub-pools and restricting contractors’ participation are not clear.

Question: Could the Government provide specific criteria for how RPs will be further broken into sRPs and any rationale for restricting participation across RPs/sRPs?

The Government is still in the process of finalizing the RFP and further guidance will be provided.

Please also see the Industry Day slides posted. See final RFP.

Comment: The PWS emphasizes rapid delivery and innovation; however, the current language does not address potential challenges associated with rapid onboarding and performance evaluation.

• Question: Can the Government clarify its expectations for balancing rapid delivery with quality assurance and risk mitigation? Will key metrics or performance thresholds be included in future task orders?

All metrics and risk mitigation will be taken into consideration at the task order level. The project's quality and risk management approach will be tailored to ensure that the delivered capabilities meet the customer's needs and expectations, while minimizing potential risks and impacts to cost and schedule.

• Comment: The PWS states that TOs will not extend more than one year beyond the ordering period. For complex AI/ML solutions, a longer PoP may be required for operationalization and sustainment.

• Question: Will the Government consider allowing multi-year PoPs for certain types of requirements under the IDIQ?

The draft PWS states "Performance of an individual TO/DO awarded during the IDIQ period will not extend more than one year beyond the ordering period." This statement is in reference to the base IDIQ ordering period which is a base period of five years plus a five year option period. Per DFARS 217.204(e)(iii), approval from the Senior Procurement Executive (SPE) is required before issuing an order an order against a task or delivery order contract if performance under the order is expected to extend more than 1 year beyond the 10-year limit. Task Orders are not limited to one year unless they are placed on the last day of the base IDIQ ordering period.

Open Architecture Requirements (Section 1.11):

• Comment: The Government encourages open architecture but notes that proprietary solutions may increase risk. While this is valid, not all proprietary solutions inherently hinder modularity or adaptability.

• Question: Will the Government consider a risk-based evaluation approach that allows proprietary solutions if they meet modularity and interoperability requirements?

Project Linchpin will assess proprietary capabilities prior to leveraging them. This will ensure that any potential risks or vulnerabilities associated with the use of proprietary capabilities are identified, mitigated, and managed in accordance with established protocols and best practices.

Artificial Intelligence/Machine Learning (RP1):

• Comment: The scope for AI/ML includes data engineering, cloud infrastructure, and governance frameworks but lacks specific details on tools, performance standards, and success criteria.

• Question: Will the Government define specific performance metrics for AI/ML solutions (e.g., model accuracy, latency, data throughput) in Task Order solicitations?

The subpools established under Project Linchpin are specifically designed to support and address the diverse needs of our customers through tailored use cases. Each use case will be accompanied by a distinct set of conditions and associated metrics, which will serve as the foundation for measuring success and progress. The specific metrics will be formally defined and articulated at the task order level

Software Development (RP2):

• Comment: While RP2 references modern software practices, it does not specify whether continuous integration/continuous deployment (CI/CD) pipelines and DevSecOps are mandatory.

• Question: Can the Government clarify whether CI/CD pipelines, automated testing, and DevSecOps will be standard requirements for software development task orders?

It is not mandatory for Base award. Task Orders may need CI/CD pipelines, automated testing, and DevSecOps as standard requirements and will be determined at that time.

Conflict Between Pools:

• Comment: The draft RFP mentions that contractors may not participate in certain RPs/sRPs concurrently, but it does not explain the rationale.

• Question: Will the Government provide specific examples of conflicts between pools and their impact on contractor eligibility?

The Government is still in the process of finalizing the RFP and further guidance will be provided.

Please also see the Industry Day slides posted.

Contract Types and Cost Management:

• Comment: The inclusion of various contract types (FFP, CPFF, T&M) adds flexibility but may lead to inconsistent cost proposals.

• Question: Will the Government provide templates or guidance for cost proposals to ensure consistency across Task Orders?

Each TO will have a specified contract type. Contract types will vary by TO and not by vendor.

Guidance will be provided in the task order RFP as to contract type and required support for cost/price.

Ceiling Value and Minimum Guarantee:

• Comment: The contract minimum of $500 does not align with the overall IDIQ ceiling value of $999M and may not incentivize contractors to invest in innovative solutions.

• Question: Does the Government anticipate larger minimum guarantees for specific Requirement Pools or Task Orders to encourage contractor investment? No.

Rectify and Roll (R&R) Strategy (H.8.1): Comment: The R&R strategy allows for recompeting work after unresolved performance issues but could create significant operational disruption for inprogress AI/ML or software solutions.

• Question: Will the Government provide clearer criteria for determining “performance concerns” that trigger R&R? How will transition risks be mitigated when shifting to a new vendor mid-performance?

To the best of the Government’s ability, R&R will occur at the end of the TO base PoP when possible.

Small Business Participation Commitment (H.10):

• Comment: While small business participation is addressed at the task order level, there is no set goal or requirement for small business involvement.

• Question: Will the Government set a baseline small business participation percentage for each task order or the overall IDIQ?

Each TO will specify the small business participation/subcontracting goals, this is not anticipated to be set at the base level. Additionally, market research will be conducted at the TO level to determine whether the effort is a small business set aside.

Perpetual Onboarding (H.9):

• Comment: The perpetual onboarding process is innovative but may lack clarity on evaluation timelines and thresholds for acceptance.

• Question: Will the Government provide more specific criteria for evaluating White Paper submissions (e.g., technical depth, cost-benefit analysis)?

Yes, the Government is developing both timelines and processess for the perpetual onboarding process. The Government does not anticipate perpetual onboarding to begin sooner than 6 months post award of the base contract. The Government will provide additional details prior to onboarding.

Sections L&M - Technical Approach and Risk Mitigation:

• Comment: For complex AI/ML and software development requirements, it is critical that proposals include risk mitigation plans.

• Question: Will the Government require offerors to submit risk management plans as part of their technical proposals?

Sections L & M are being updated and appropriate changes will be reflected in the next version of the RFP.

Definition of Success Metrics: Incorporating specific success metrics (e.g., AI model accuracy, software delivery timelines) in Task Orders will ensure consistent contractor performance. Comment acknowledged.

Pilot Programs and Prototyping: The Government could allow pilot programs or prototype deliveries as part of initial task orders to assess contractor capabilities before full-scale execution. Comment acknowledged. Comment acknowledged.

Cybersecurity Requirements: While DFARS clauses are referenced, the draft PWS does not emphasize cybersecurity compliance for AI/ML and software development solutions. Will the Government include specific cybersecurity frameworks (e.g., NIST SP 800-171, CMMC) as requirements for all Task Orders?

Project Linchpin is incorporating comprehensive cyber controls and protections throughout each subpool. To ensure security and integrity, cyber controls will be obligated through secure hosting environments and task orders, which will outline required security measures and standards

Clear Alignment with Mission Objectives: We commend the focus on innovation and modular, open systems approaches in both RP1 and RP2. This aligns with our capabilities in agile software development and secure infrastructure. The structure of sub-requirement pools (sRPs) ensures focused expertise, which we believe is critical for mission success.

N/A The government acknowledges and appreciates the emphasis on innovation and modular, open systems approaches

Innovation Emphasis: The Government's call for innovative solutions across all RPs is well-aligned with industry trends. However, specific evaluation criteria for "innovation" are not explicitly defined in the Draft RFP or PWS. Clearer metrics to evaluate innovation (e.g., percentage reduction in development cycles, demonstrated cost efficiencies, or technology advancements) would help offerors craft more targeted proposals.

Sections L & M are being updated and appropriate changes will be reflected in the next version of the RFP. Sections L & M are being updated and appropriate changes will be reflected in the next version of the RFP.

Rapid Onboarding Process: The perpetual onboarding approach ensures a dynamic vendor pool but may cause operational challenges due to continuous integration of new contractors. Guidance on how task orders (TOs) will be allocated between base contract holders and onboarded vendors would ensure clarity for planning and resource allocation.

There is no predetermined allocation of Task Orders. TOs will generally be awarded through full and open competition unless approved for sole source. Acknowledged as comment

Data Rights Strategy: The Government’s emphasis on Government Purpose Rights (GPR) is reasonable. However, the requirement to share modified and transformed data exclusively with the Government and third-party contractors introduces concerns about intellectual property (IP) protections for vendor-developed proprietary processes. A mechanism to balance IP retention with GPR would be beneficial. Comment acknowledged.

Proposal Process (Draft RFP Section L & M)

• Feedback: The phased proposal process is logical, but specific criteria for advancing from Phase One to Phase Two could be clarified. For example, what constitutes a "significant strength" versus a "strength" in the evaluation?

o The scoring for technical experience (Factor 2) relies heavily on recent performance within 24 months. Allowing examples of up to 36 months would better capture long-term, impactful projects, especially for rapidly evolving technologies like AI/ML.

similar efforts (e.g., IL5 hosting, EMS techniques, modular software) will be weighted against innovation and technical solutions in the evaluation?

2. Will draft TOs or sample TOs be released before the final RFP to allow vendors to understand task-specific expectations? Sections L&M are being updated. Final RFP will be updated to reflect further guidance sRP2.2: Electromagnetic Spectrum (EMS) Techniques Support Feedback: This pool’s emphasis on advanced RF effects and signal exploitation is excellent.

However, the requirement for multi-platform RF effects, such as geolocation or coherent jamming, could benefit from more specific use case examples.

technical capabilities during the proposal phase?

2. Will software tools developed for EMS techniques (e.g., custom RF processing algorithms) need to comply with open standards like Modular Open Systems Approach (MOSA) or Open Radio Architecture?

3. Are there specific latency or throughput requirements for EMS capabilities in contested environments?

Specific case examples can be provided at a higher classification level.

1. Yes, Government will provide data sets to validate technical capabilites.

2. Not required for Base Award. Requirements will be further defined at the Task Order level and will be required when appropriate.

3. Not required for Base Award. Requirements will be further defined at the Task Order level and will be required when appropriate.

sRP2.3: Infrastructure as a Service (IaaS) o Feedback: Hosting requirements, particularly for Impact Level 5 (IL5) compliance, are well-defined. However, the expected frequency and scale of updates to hosted environments (e.g., software patches, security updates) are not mentioned, which may affect resource planning.

GovCloud, Azure Government), or are all providers equally acceptable if compliant?

2. How will the Government evaluate the scalability of IaaS environments to support surge requirements (e.g., expanded CI/CD pipelines)?

3. Can vendors propose hybrid IaaS solutions (mixing on-premises and cloud hosting) for cost optimization?

1. There are no preferred providers, however proof of IL5 authorization is required.

2. Not required for Base Award. Requirements will be further defined at the Task Order level and will be required when appropriate.

3. Yes, vendors can propose a hybrid IaaS solution. However, restrictions may be applicable at the Task Order level.

Evaluation Factors and Innovation (RFP Section M)

• Feedback: While the scoring matrix is helpful, the lack of specific weights for technical subfactors (e.g., data onboarding, CI/CD pipeline setup) creates ambiguity. This could lead to uneven proposal emphasis among offerors. We recommend explicitly defining "innovation" metrics for each subfactor to ensure proposals focus on deliverables that exceed baseline requirements.

example, will exceeding IL5 compliance requirements (e.g., meeting IL6 standards) count as a significant strength?

2. Will technical demonstrations (e.g., sRP2.2’s RF signal analysis report) require independent third-party validation during the proposal phase?

Sections L & M are being updated and appropriate changes will be reflected in the next version of the RFP. Sections L & M are being updated and appropriate changes will be reflected in the next version of the RFP.

Security and Compliance (PWS Section 6.0 & Appendix A)

• Feedback: The inclusion of specific security requirements (e.g., TS/SCI clearances) is critical, but the timeline for vendors to meet these clearances (e.g., for subcontractors) is not addressed. Vendors might need clarification to ensure compliance.

o The requirement for DevSecOps pipelines is forward-thinking, but more details about preferred tools or platforms would align industry solutions with government expectations.

additional guidance for handling classified data securely in remote environments?

2. Will the Government support TS/SCI clearance sponsorships for subcontractors, or must vendors ensure clearances are in place before award?

Specific classification requirements/guidance will be addressed at the TO level. The RFP is being updated and vendors are advised to await the next version. 1. Yes the Government will provide additional guidance and security requirements for handling classified data in remote environments at the task order level. Vendors should assume that all classified data will be handled in accordance with applicable laws, regulations, and Government policies. For remote work, vendors will be required to ensure that all locations where classified information will be accessed or stored are approved through a DD Form 254.

2. Due to the rapid nature of the MATOC, the Government requires that vendors have clearances in place to facilitate timely task order award and execution. It's important to note that not every task order will require a Top Secret/Sensitive Compartmented Information (TS/SCI) clearance.

Vendors should be prepared to provide evidence of existing clearances for key personnel and be able to obtain clearances for additional personnel as needed to support specific task orders that require such access.

Performance Metrics and Off-Ramping (PWS Section 3.3 & 4.0)

• Feedback: Off-ramping for non-performance is reasonable but may discourage participation without a clear remediation process. Including remediation steps before off-ramping would encourage vendors to proactively address issues.

performance expectations and metrics?

2. Will off-ramping decisions consider external factors (e.g., delays caused by government-furnished property or resources)?

1. A base level QASP will be included in the final RFP. More specific QASPs will be determined at the task order level to support specific requirements. 2. The Government intends for industry to propose a milestone performance plan with each TO proposal submission. Within the milestone plan the contractor will identify where they will be at the end of the TO base period of performance, which will then be used in the Government evaluation of performance for off-ramping decisions. There will be open dialog throughout each TO on contractor performance, we do not anticipate recompetes being a surprise to industry.

Prioritize Firm Fixed Price Task Orders/Delivery Orders. In our experience, delivering trusted AI solutions to both commercial and US Federal Government clients, we have observed that contracts with awards made on a Firm Fixed Price (FFP) basis are often the best approach for the Government to reduce cost and schedule risk. In addition, FFP contracts encourage an “all-in” staffing approach where vendors bring their best resources and whatever time and expertise are needed to deliver a particular outcome. For this reason, we recommend that PEO IEW&S award individual TO/DOs on a FFP basis whenever possible. Acknowledged as comment

Create separate orders for each task area or phase of development to target the most relevant qualifications. In our experience with large, complex AI/ML and SW development contracts, we have found that an approach that awards separate contracts for each task area or phase of development allows the government to access the most relevant qualifications for each order. If PEO IEW&S awards contracts that cover a broad range of services (e.g., EMS techniques support and AI/ML model development and training) or end-to-end phases of work from solution design and prototyping through operations and maintenance, there is a significant risk that the government will go “all in” on one vendor instead of fostering a competitive vendor ecosystem that takes advantage of the unique strengths of each vendor. The government benefits most when separating task areas and phases of development because these tasks/phases require different experience and expertise to ensure delivery of desired outcomes. Separating task areas also prevents long-term vendor lock-in, thereby minimizing long-term cost risk. Comment acknowledged.

Include change management support requirements within the scope of each RP. The greatest risk during AI/ML development and deployment efforts is often not technical but largely exhibited in organizational and change management aspects of the journey.

Organizations often fail to upskill and train employees or update objectives, creating misalignment with the transformational goals and causing setbacks. Programs may fail to fully address the need of a cultural shift or reduce people’s resistance to change, resulting in challenges in adoption and implementation. To proactively mitigate these risks, we recommend including change management support requirements within the scope of each RP and including evaluation criteria that assess the vendor’s qualifications to meet these requirements. Managing the change process, enabling a smooth transition to new systems and processes, and ensuring that stakeholders are prepared for new ways of working will ensure PEO IEW&S is equipped to drive adoption of AI/ML solutions. Effective change management should include training programs, communication strategies, and support structures to assist personnel in adapting to the changes, which are critical for improving the overall success of technology initiatives. Comment acknowledged.

Include Responsible AI requirements within the scope of each RP. It will be important for PEO IEW&S to protect the organization with clear governance addressing limits and risks of AI. The government can prepare for risk by adopting clear policies and training, developing AI guardrails to avoid leaks of proprietary data, and including a human-in-the-loop review process to validate model outputs. We recommend that the government consider incorporating Responsible AI (RAI) requirements into the scope of each RP to ensure that vendors deploy technologies and tools that are aligned with ethical AI principles.

Additionally, when evaluating task order/delivery order responses, the government should consider each vendor’s RAI credentials and approach. For example, all vendors should have a plan for identifying and mitigating risks, and they should be able to share prior experience implementing RAI safeguards when developing, deploying, and scaling AI/ML solutions. Comment acknowledged.

g g g p y While system integrators (SI) play an important role in providing infrastructure support, deploying and integrating new IT systems, and scaling and maintaining tech solutions, delivery by a single, large SI can lead to “vendor lock” which can hamper realization of desired long-term mission benefits, while driving up costs. In historical AI and SW development programs, we have observed that the SI model is vulnerable to vendor scope, schedule, and cost overruns, which results in programs that are over budget, behind schedule, and with long value realization timelines.

To address these SI model issues and support the organizational and change management needs of an AI/ML development effort, the government should include requirements for a “Navigator” to enable PEO IEW&S to more effectively put the government in control of its agenda, ensure value delivery on time and on budget and coordinate across vendors. A Navigator is an unbiased vendor who helps develop requirements, shapes the case for change, designs & prototypes solutions, and reviews approaches, with the best interests of the government in mind. For example, the government can use a Navigator to provide architectural pressure testing to ensure that proposed solutions are not only technically feasible but also align with broader strategic goals. We recommend establishing a new RP with requirements for Navigator support, including helping to navigate through complex technical and organizational challenges, managing change effectively, and supporting adoption of new technologies and tools. Suggested requirements include:

• Design target state capability sets and functional/technical requirements

• Drive architecture, requirements development, key design decisions, and selection of AI/ML use cases

• Shape comprehensive roadmap balancing budgets & funding

• Enable and accelerate standup of AI governance and RAI guardrails

• Navigate stakeholders (e.g., Board, Procurement, CIO), enforce governance, and serve as thought partner to generate buy-in for a coordinated approach and proactively anticipate roadblocks Comment acknowledged.

Consider partners with experience across commercial and Federal Government clients. In our experience, the government benefits from partnering with vendors who bring a combination of commercial and Federal Government experience. By applying cutting edge commercial best practices, vendors can decrease delivery risk and minimize costs. For this reason, PEO IEW&S should partner with a firm that has:

• Industry-leading, customizable tools designed to deliver step change performance results in commercial transformations

• A proven track record of building and fielding scalable advanced analytics and GenAI capabilities for DoD and Fortune 500 companies

• A proven track record of success in the private sector, with 10+ transformations executed with Fortune 100 companies in the past 5 years

• A proven track record of applying the commercial transformation toolkit in an Air Force & DoD Context in the past 5 years Comment acknowledged.

We believe the draft RFP and PWS provides the flexibility necessary to support AI/ML and Software (SW) Development for diverse programs and requirements. The inclusion of multiple initial requirement pools (RP) – with optionality for the government to introduce additional RPs – promotes maximal industry participation in innovative solution development, ensuring a competitive and comprehensive approach to addressing varied and evolving needs at the IDIQ and TO levels. This approach not only fosters an environment where a broad spectrum of solutions can be proposed but also encourages a diverse range of vendors to participate, driving innovation and quality.

By adopting this flexible approach, the Government incentivizes vendors to invest in both commercial and dual-use capabilities. This ensures that the most advanced technologies are available to meet mission requirements. Vendors are more likely to allocate resources towards innovation when they see opportunities for their solutions to be considered and potentially adopted by the Government. This dynamic encourages continuous investment in cutting-edge technologies, benefiting both the Government and industry by promoting a robust and innovative technological landscape. Comment acknowledged.

Remove bidding restrictions from RP 1. Currently, the IDIQ is structured such that bidders who propose to sRP 1.3 may not propose to sRP 1.1, 1.2, or 1.4. Including non-OCI based restrictions, such as this one, artificially limits the Government from acquiring capability from vendors who may be broadly qualified to provide solutions for sRP 1.2, 1.3, and 1.4. If the Government desires to acquire capability that can feasibly be restricted to one vendor (e.g., a vendor’s specific T&E solution for measuring model performance), this type or restriction can be denoted at the TO level; and therefore, only preclude that single vendor, rather than precluding the entire awarded vendor pool. Comment acknowledged.

Account for product, service, and hybrid solutions for each sRP. Many of the sRPs within the draft PWS imply that vendors should provide a service or labor-based solution to individual task orders released under the sRP. For example, sRPs 1.1 and 1.2 explicitly state “the contractor shall be responsible for providing services…” and sRP1.2 goes further by describing the breakdown of personnel with the data management team. The Government’s focus on services for the majority of the sRPs may inadvertently exclude vendors who offer diverse product-, solution-, or hybrid-capabilities that could be crucial for meeting future needs. To mitigate this risk, we recommend that the Government revise the language to explicitly include service, product, and hybrid-based solutions. For instance, Section 1.3 Scope currently states “The Contractor shall provide all personnel, equipment, supplies, facilities, transportation, tools, materials, supervision, and other items necessary to perform services as described in this PWS as well as associated Task Order/Delivery Order (TO/DO) PWSs…” We suggest that the Government amend the scope of the PWS and review each sRP to incorporate product and hybrid solutions where applicable . Many of the sRP requirements can be met through existing commercial products. These products can often provide more efficient, scalable, and cost-effective solutions. By including product-based and hybrid approaches, the Government can leverage advanced technology and innovation present in the market. Comment acknowledged.

Distinguish between Commercial and COTS products. We believe the Government should emphasize the use of commercial item solutions, which encompass both COTS but also configurable offerings for the Government. For example, RP2, SW Development, includes a bullet for “Commercial Off the Shelf (COTS) SW solutions, to include modified COTS to meet Government requirements.” This bullet can be amended to include “COTS and commercial item solutions”. Commercial items are more flexible and adaptable than COTS products, allowing them to be tailored to address specific requirements or use cases, and have been stress-tested by capital markets. Commercial solutions have repeatedly proven their ability to deliver functional solutions without cost or schedule overruns.

Because of these benefits and the Government’s guidance to acquire commercial items to the maximum extent practicable, we recommend the Government conduct additional market research at the task order level to determine the availability of commercial items to meet the needs of the requirements. Pending that market research, the Government can then design the task order contract structure accordingly (see Contract Types bullet below). Acknowledged as comment

Contract Types. We agree with the Government’s decision to include multiple contract types at the TO level based on the requirements of the sRP. In particular, we commend “Rectify and Roll (R&R)” ordering procedure that prioritizes FFP, short period of performance TOs to incentivize contract performance and encourage participation from all of industry, including non-traditional defense contractors and commercial software vendors, who can be prevented from bidding when TOs are solicited using cost-type contracts. Acknowledged as comment

Section 3.1.2, sRP 1.2 – Data Management and Labeling. Successful model development relies on the curation of high-quality datasets, which is often bottlenecked at the speed at which humans can manually create labels. Multiple commercial solutions exist that transform a traditionally expensive and highly manual enrichment process by automating the process of data selection, task submission, and label retrieval and employing third-party AI/ML models to pre-label images. Given this sRP’s intent to include automated methods for data management and labeling, we believe the Government should amend the requirement to indicate “The contractor shall be responsible for providing services and/or product-based solutions within the following scope”. Comment acknowledged.

Section 3.1.4, sRP 1.4 – Test & Evaluation (T&E). The first sub-bullet in this section is specific to personnel experience and qualifications. However, if the solution is product-based, the emphasis on personnel qualifications becomes less critical. Product-based solutions typically come with built-in functionalities, support, and documentation that allow for seamless integration and operation with minimal dependence on personnel. This shifts the focus from the qualifications of the people implementing the solution to the capabilities and reliability of the product itself. We do believe that model T&E would benefit from human-in-the-loop where possible for observation and approval of workflows and for subject matter experts (SMEs) to probe system limitations. Comment acknowledged.

Section 3.2, RP2 – SW Development. As previously stated, we believe that emphasizing commercial item solutions over strictly COTS for SW development would offer significant advantages. Commercial item solutions encompass not only COTS products but also configurable items, providing a greater degree of flexibility and customization to meet specific needs. This dual capability allows PEO IEW&S and other customer portfolios to leverage the immediate availability and proven reliability of COTS products while also enabling the tailoring of solutions to fit unique requirements without extensive development time. Furthermore, commercial item solutions often come with robust support and maintenance options, ensuring that the Government can benefit from ongoing updates and enhancements. This approach can lead to more efficient and effective implementations, reduced total cost of ownership, and the ability to adapt more swiftly to evolving technological and mission demands. Comment acknowledged.

Section 3.2.3, sRP 2.3 – Infrastructure as a Service. Consistent with our previous recommendations, we believe the Government should revise this sRP to include product-and hybrid-based solution.

Vendors can propose a hybrid IaaS solution. However, restrictions may be applicable at the Task Order level.

Regarding the draft RFP, we recommend that the Government include NAICS Code 513210 in addition to the stated NAICS Code of 541511. Acknowledged as comment

Section H.4, Data/Software Rights on Orders. We agree that the goal of the AIS@P IDIQ should be to foster innovation and competition; however, we disagree that “proposing proprietary solutions under Order RFPs may be considered an element of an Offerors proposal that increases risk of unsuccessful contract performance.” Computer Software provided by a vendor with commercial data rights to meet Government requirements can provide “open software architecture” that aligns with future adaptability, modularity, and competitiveness. Proprietary software solutions, while developed and maintained by specific companies, often incorporate open standards and APIs that facilitate integration with other systems and technologies. These solutions are designed to be flexible and customizable, allowing users to adapt them to meet specific needs and requirements.

Furthermore, many commercial software products are built with a modular architecture, enabling users to add or remove functionalities as needed without disrupting the core system.

Additionally, proprietary software vendors frequently release updates and enhancements, ensuring that their solutions remain cutting-edge and capable of supporting emerging technologies and standards. This continuous improvement cycle not only fosters innovation but also provides users with a reliable and secure platform for their operations. We recommend that the Government reconsider the thesis that proprietary solutions increase risk of unsuccessful contract performance. Rather, we encourage the Government to require all vendors to demonstrate the openness and interoperability of their proposed systems. We have found indicators of openness and modularity include open, documented APIs, ability to easily import and export data in common formats, and component swapability and extensibility. Acknowledged as comment

1.11 - Data Rights - AIOps+ Data - Synthetic data that is NOT derived from Government data or parameters Unlimited Rights. Question - is the government requiring industry to provide Unlimited Data rights on synthetic data used to train industry models?

The RFP is being updated and Project Linchpin will consider the implications of GPR versus unlimited rights with respect to synthetic data as part of the final RFP.

2.2 Contract Deliverables - "Requests for data under the AIS@P MATOC will be specified on a DD Form 1423, Contract Data Requirements List (CDRL) and will be specified at the TO/DO level. All data shall be prepared and submitted in accordance with the applicable CDRL at the TO/DO level. A list of common CDRLs has been provided below, however this list is not all inclusive and additional CDRLs may be required. Additionally, associated DIDs may vary at the TO/DO level for required CDRLs. This below list is provided as an example only."

Comment - recommend tailoring CDRL formats at the TO/DO level to accommodate leveraging digital model outputs rather than legacy DID formats to take full advantage of modern toolsets Acknowledged as comment

3.0 - Contract Structure - "Restrictions and set-asides are TBD for this draft of the RFP/PWS."

Comment - recommend the government publish these restrictions and set-asides ahead of RFP to facilitate industry teaming and business case optimization Acknowledged as comment

3.0 - Contract Structure - Question - will industry select the sRPs they are bidding into at the time of RP bid, or will the USG make that determination based on industry RP bid and corresponding offeror capabilities? Industry will identify the sRP they are proposing to.

3.1 RP1 - AI/ML - Question - there appear to be time-phased dependencies between the various sRP outputs and inputs. An example - sRP 1.3 tasking depends on sRP 1.2 data and feeds sRP 1.4 T&E strategy. Does the government intend to control those dependencies as GFI with each TO/DO?

Project Linchpin will manage the data and environment, which will be dependencies for performers. This means that performers will rely on Project Linchpin for access to the necessary data and environment and will be required to operate within the parameters and constraints established by the task.

3.1.2.4 sRP1.2 Data Management & Labeling - "Maintain Government provided and Government hosted data Application Programming Interfaces (API) to enable data access controls and integration with other systems." Question - does the government intend to define these APIs with the base MATOC RFP?

Project Linchpin is actively developing API standards and will make them available when they are ready

3.1.2.16 sRP1.2 Data Management & Labeling -"Leverage the Army’s environment for handling and storing the data, and ensure that all data is kept confidential, secure, and does not leave the Army’s environments". Question - does the government intend to define this environment, with corresponding compute performance parameters, with the release of the base MATOC RFP?

Project Linchpin has environments established on both unclassified and classified networks. The specific details and requirements for accessing and utilizing these environments will be defined at the task order level.

3.2 "RP2 shall support SW Development which shall include design, development, fabrication, and testing of developmental systems covering the complete range of the Electronic Warfare and Cyber, PEO IEW&S or other customer portfolios as required."

Question - can the government clarify what is meant by "fabrication" in the context of Software Development? The word "fabrication" will be removed from the RFP for clarification purposes.

3.2 "Support shall include but not be limited to system engineering and integration, test and evaluation services, logistics support, as well as operation and maintenance support."

Question - can the government clarify what is meant by "logistics support, as well as operation and maintenance support" in the context of Software Development?

Logistics Support- In accordance with PWS paragraph 3.2.8 Maintenance Support- In accordance with PWS paragraph 3.2.1.7

There appear to be time-phased dependencies between the various sRP outputs and inputs.

An example - the output of sRP 2.3 and 2.4 would likely be needed for any contractor performing sRP 2.2. Does the government intend to provide those dependencies as GFI which each TO/DO?

Requirements will be further defined at the Task Order level and will be required when appropriate. GFI will also be determined at the TO/DO level and be provided when deemed appropriate.

3.2.4-3.2.11 various - DevSecOps, Integration, CM, Metrics, ILS, T&E, Cybersecurity, O&M Question - Are each of these envisioned to be separate sRPs?

No- these are encompassed in the overall 3 main subpools (PWS 3.2, 3.3, and 3.4) and will not be evaluated as separate subpools.

3.2.5 Integration - "The contractor shall have thorough knowledge of the existing EWC systems operations, components and subsystems; and the interoperability of proposed systems to existing networks. Question - this tasking reads as general integration expertise of integrated EWC solutions. Does the government intend to use this sRP to award general integration scope or would it be used to integrate the capabilities being developed under sRP 2.2? It is both. Requirements will be further defined at the Task Order level.

3.2.8 Integrated Logistics Support - "The Contractor shall provide Integrated Logistics Support (ILS) services and data in all areas of Life Cycle Management within the EWC portfolio as defined at the DO/TO level."

Question - this tasking reads as general ILS expertise. Can the government provide the rationale why this is included in the SW Development RP?

Requirements will be further defined at the Task Order level and will be required when appropriate.

3.2.9.6 Test & Evaluation - Integration and verification testing. "The Contractor shall ensure that all components and subsystems work together seamlessly and verify overall system functionality."

Comment - this tasking appears to overlap the tasking in 3.2.5. Suggest combining Acknowledged, however, the Government will be keeping the PWS paragraphs the same.

3.2.11 Operation & Maintenance- "The contractor shall support operations and maintenance activities, including the acquisition of spare and repair parts, ancillary items, support equipment, depot support, technical support, repair services, overhaul, provisioning, leasing, storage, training, extended field support operation and maintenance activities, reporting, record keeping, engineering changes, logistic material updates, software maintenance, engineering services and data preparation, studies and analysis to sustain and enhance fielded EWC systems and…