Industry Day Q and A - 11272024.pdf

All information presented here and associated with the ODOS IV Industry Day is subject to change. The content herein is for information and market research purposes only and is subject to change. If there is a conflict between any later issued Request for

Quote (RFQ) and any information provided herein, that RFQ takes precedence.

Question Number Question Response

For the Labor Hour portion of the contract, there is a mention that the four (4) FTE resources are to be determined by the vendor (and that the mix must only include priced labor categories existing on the pricing schedule). Please share the desired list of labor categories in its entirety.

The draft PWS provided to industry on October 28, 2024, included a list of labor categories under Attachment 4 – Key Personnel Qualifications and ODOS IV Labor Category Descriptions. The pricing schedule refers to the Task Order pricing schedule. This will be made clear in the RFQ.

Please confirm that USCIS’ intent is to integrate the functionality and data of Global, CLAIMS 3, and Benefits Hub into ELIS. If so, please confirm that the result will be that – either during ODOS IV’s period of performance or later – Global, CLAIMS 3, and Benefits Hub will cease to exist as separate systems

USCIS has been migrating benefits from CLAIMS-3 into ELIS for several years. In order to decommission CLAIMS-3, USCIS will continue migrating CLAIMS-3 functionality into ELIS. The integration with Global is not new and will continue with ODOS-IV.

In Section 5, it is stated that "The Contractor’s responsibilities will be to maintain ELIS, Global, CLAIMS 3, and Benefits Hub code repository of deployed releases and to deliver innovative and secure business solutions as required". In the PWS for RIVER, C3 and Benefits Hub were specified as the systems to be supported by the RIVER contract.

Would the Government clarify if C3 and Benefits Hub are in scope for ODOS IV despite the same systems being in scope for RIVER?

In Section 5 of the PWS, only decommission activities of CLAIMS 3 and Benefits Hub will be supported by ODOS IV. The continuation of support for C3 and Benefits is in scope for RIVER.

In Section 7.2, it is stated that "may receive additional UXD/IxD resources on an optional basis". Furthermore, in Section 5.3, it is stated that "The UXD/IxD multi-vendor team will comprise of four teams will have one UXD/IxD lead and two UXD/IxD team members".

Would the Government clarify how many UXD/IxD resources would be awarded to every vendor and how many could be optionally awarded?

Initially every vendor will get the following Key Personnel: PM, Technical Lead, BA lead and UX Lead. Every two teams may get one one UX person and they will be imbedded within the teams. The number of optional resources assigned will depend on how many teams the vendor will be awarded. (Please see section 7.3 of the PWS for an example, but these numbers may be revised in the final PWS)

Detailed Transition Expectations: Could you elaborate on the expectations for the Transition- In and Transition-Out phases, including timelines, knowledge transfer requirements, and collaboration with incumbent contractors?

Transition In is a 3 month PoP. During this time the Government is looking for the vendors to staff up per the PWS. The government will lead the process of the knowledge transfer between all stakeholders.

Unfortunately, we are unable to share the time-line at this point, but this will be a joint effort.

Across your multiple cloud environments, what is your overall Infrastructure as code strategy and vision?

Infrastructure as code (IaC) is a strategy that uses code to define and manage infrastructure, instead of manual processes. The goal of IaC is to make infrastructure management more efficient and consistent.

[Question related to PWS p. 5, "Overview] The expectation is there will be a singular entity that will be responsible for program governance between the CO, COR Government Liaison, stakeholders and integrating contractors. Has this organizational element been established and what will the contractor responsibilities be to that governance body?

Yes. Contractor's responsibilities will be as specified in the PWS.

Are there specific tools and practices used presently or does the client want innovative approaches?

Please refer to the technical landscape in the PWS

Is the work on ODOS III currently divided between DevSecOps/UXD/IxD tasks and Infrastructure/Integration tasks, or will this be a new feature of ODOS IV? If it is new, why is the work being divided in this manner?

The current work being performed on ODOS-III includes:

DevSecOps/UXD/IxD tasks and Infrastructure/Integration tasks. In ODOS IV, the Infrastructure/Integration task is being split out and will be a separate award to a single vendor. The decision to split out the Infrastructure/Integration task is based on lessons learned from ODOS

III.

Are any incumbent ODOS III contractors already working on shifting and integrating CLAIMS 3 and Benefits Hub into ELIS? If so, which incumbents are doing this work?

USCIS has been migrating benefits from CLAIMS-3 into ELIS for several years and the work is being supported by all ODOS-III contractors.

Given that ODOS IV will acquire a large number of DevSecOps teams and that the awardees must performance at the highest level, we recommend that the Government select Time and Materials versus Firm Fixed Price/Labor Hours CLINs.

Thank you for your feedback/suggestion.

To ensure that ODOS IV goals are achieved, we encourage the Government to allow the Infrastructure and Integration Services task awardee to perform both ODOS IV tasks.

Thank you for your feedback/suggestion.

To provide flexibility, we recommend that the mandatory AWS cloud professional path certification for two Infrastructure and Integration Services task team members are required at time of award.

Thank you for your feedback/suggestion.

What are the key responsibilities and expectations for the ODOS IV contractor?

The draft PWS provided to industry on October 28, 2024 provides a description of the responsibilities and expectations.

Section 7.1 states: “The composition for the DevSecOps teams will consist of nine (9) Full Time Equivalents (FTEs) resources per DevSecOps team.

The Contractor shall provide five (5) FTE resources whose labor categories are determined by the Government, and four (4) FTE resources whose labor category mix is determined by the vendor (mix can only include priced labor categories existing on the pricing schedule).”

The "ODOS IV Labor Categories - DevSecOps" table lists 3 different experience levels for Application Developers, and 2 different experience levels for BA's. For the 5 FTEs comprising the FFP spots on each DevSecOps team, which levels of Application Developers and Business Analysts are required?

USCIS will clarify in the RFQ which labor categories will be required in the FPP CLINs.

Could the government clarify whether the ODOS acquisition will be structured similarly to RAPID/RIVER and FALCON/EYAS, with separate awards for Other Than Small Business (OTSB) and Small Business (SB) tracks? Will this work remain exclusively for small businesses, or is the government also considering an OTSB track?

ODOS IV will be 100% set aside for Small Business.

Workload Metrics Clarification: The PWS mentions that ELIS currently handles over 80% of the agency's immigration case management workload, and there's also a target to process approximately 75% through combined IT systems.

Could you clarify if these metrics refer to the same measure, and what constitutes 'combined IT systems'?

ELIS currently handles 85% of the agency workload. This will be updated in the PWS. The target for the combined workload will be updated as well.

Release Process Automation: Is there always a manual approval step before production deployment, or have any teams implemented fully automated deployments with controls such as automated release notes and test suite validations?

Releases are fully automated but a manual approval step is required.

Runtime Environments Beyond Java: For specialized use cases like AI/ML or data analytics, would USCIS consider runtime environments beyond Java (e.g., Python) if they meet all operational and security requirements?

No

Preference for Native AWS Services: Should solutions prioritize native AWS services over third-party or open-source alternatives when designing and implementing systems?

The Government expects contractors to use the tools listed in the technical landscape. Any other solutions needed will be evaluated and approved by the Government for use

Flexibility in Technical Innovation: In our technical proposals, is USCIS interested in receiving suggestions for innovative solutions, including newer capabilities within the AWS ecosystem or advanced DevSecOps practices, even if they are not currently listed in the technical landscape (Attachment 1)?

The Government is open to innovative ideas. The RFQ will provide instructions and the requirements for quote submittal.

Use of AI-Assisted Development Tools: What is USCIS's position on using AI-assisted development tools (e.g., code generators, intelligent IDEs) that meet FedRAMP High requirements to enhance development productivity?

At this point we do not use AI Development Tools. The process of using AI Development Tools is still be worked out.

Incorporation of AI/ML Capabilities: Is USCIS interested in incorporating AI/ML capabilities within ELIS or other systems, such as for adjudication support or data analysis, provided they meet all security and compliance requirements?

At this point we do not use AI/ML Development Tools. The process of using AI/ML Development Tools is still be worked out within USCIS.

Preference for Kubernetes Certified Service Providers: Given the emphasis on microservices architecture and Kubernetes platforms like Amazon EKS in the PWS, does USCIS prefer or require vendors to be Kubernetes Certified Service Providers (KCSP) or have team members with Kubernetes certifications?

ODOS-IV does not require any vendor certifications for Kubernetes.

There are individual certification requirements, please see Section 7.4 for certification requirements.

AWS Certification Requirements: The PWS states that at least four (4) of the nine (9) FTEs on each Infrastructure and Integration team must have AWS certifications. Could you specify which AWS certifications are preferred or required (e.g., AWS Certified Solutions Architect Professional, AWS Certified DevOps Engineer Professional)?

There are individual certification requirements, please see Section 7.4 for certification requirements. Additional certifications are listed in each labor category in attachment 4.

Facilitation of Collaboration Across Vendors: Will USCIS require or facilitate formal agreements, such as Agreements to Collaborate/Associate (ACAs), to ensure incumbent vendors cooperate in knowledge sharing and smooth transitions between contractors

The Government requires knowledge sharing during transition period between vendors.

Balancing AWS Cost Optimization: How should teams balance AWS cost optimization against other factors like performance, scalability, and security when designing solutions? Are there specific expectations for incorporating cost visibility and FinOps (Financial Operations) practices into daily operations?

The system is continuously monitored for cost savings. Post award, the contractor(s) will work with the government to develop and implement cost optimization strategies.

Cloud Cost Management in Performance Metrics:

Under the 'Business Value and User Satisfaction' performance criteria, will teams be evaluated on cloud cost management and optimization? If so, what specific cost-related outcomes or KPIs will be measured?

Yes, teams are expected to collaborate when necessary to achieve cost savings across the program. Specific measures will be shared post award.

Implementation of Zero-Trust Framework: Could you provide more details or guidelines regarding the implementation of the zero-trust security framework mentioned in the PWS? Are there specific standards or models USCIS expects vendors to follow?

USCIS follows CISA Zero Trust Maturity Model. Section 5.2.1 bullet L and section 6.2.1 bullet L references the zero trust framework requirements.

Use of Generative AI Tools: Is USCIS open to the use of private generative AI tools for development productivity, assuming they are self-hosted or meet FedRAMP High requirements, and comply with all DHS security policies?

See Question # 23 regarding AI.

Existing Operational Metrics: Can USCIS provide current operational metrics such as support ticket volumes, system outage frequencies, existing SLOs and SLAs, and code quality benchmarks to help us accurately estimate the level of effort required?

Metrics will be provided upon award.

It appears that there is text missing. Can the government clarify the requirement to "enforce Application Program Interface (API) task orders on how to interact, set up/maintain a CI/CD environment and assist in continuous delivery of the software."

This section is accurate and complete.

f) The Contractor shall review and participate in redesigning monolith application functionalities into manageable microservices. Which applications are currently considered to be monolith applications?

ELIS monolith.

f) The Contractor shall review and participate in redesigning monolith application functionalities into manageable microservices. Does this redesign involve any applications that are currently hosted on-premises?

No. All ODOS-IV case management systems and microservices are hosted in the AWS cloud.

o) The Contractor shall ensure that the applications and Infrastructure key performance parameters (KPPs) are maintained to DHS/USCIS standards. Can the government provide the current KPPs for these systems?

Information will be provided upon award.

t) The Contractor shall be responsible for performing reverse engineering of physical data models from databases Does this mean that the Contractor would not have direct access to certain databases?

Access to databases is limited to qualified database administrators.

eel) The Contractor shall ensure interoperability between different platforms during integration.

Can the government provide an example of the different platforms referred to here?

Refer to the technical landscape in the PWS.

Can the Government provide more detail about what aspects of the CLAIMS 3 functionality are implemented in modernized frameworks and how the legacy CLAIMS 3 system is currently used?

Please reference the PWS and the recording.

Can the Government provide architectural diagrams of the systems for which the Contractor will be responsible for providing DevOps support?

Architectural diagrams will be available to contractors after awards.

Is there flexibility in bringing in new technologies, or is the expectation that the new contractor will utilize the existing technology in use?

While experience in the existing technology is necessary, the program is open to the exploration of new technologies

What is the requirements for the migration of Benefits Hub and CLAIMS 3 into ELIS? Will feature parity need to met, or only a subset of features from CLAIMS and Benefits Hub be added or reused within ELIS?

Please see answer to question 2.

Is there a timeline provided for when the deprecation should be completed? Is there a period where these services would need to be supported in parallel?

1. No. 2. To be determined.

Is support for Ruby on Rails and C# tech stacks necessary after the migration is complete?

The Government expects support for all platforms and software listed in the technical landscape.

Is there consideration to maintaining an infrastructure (ICS) technical lead? Refer to Teams composition in the PWS.

Would the government consider multiple vendors for the infrastructure task to enhance cross-team collaboration and expanded skillset or is this task single award?

This a single award.

Axure is no longer a program UX/UI tool on ODOS and Figma has since replaced it. Will the final PWS reflect this change in enterprise tool?

Yes- The software section of the PWS will be updated.

What are the key responsibilities and expectations for the ODOS IV contractor? Please reference PWS.

48 What are the specific challenges? Please reference PWS.

49 What are the specific challenges? Please reference PWS.

In prior USCIS solicitations, we have seen requirements for key personnel that allow flexibility for substitutions under specific circumstances, without imposing tenure restrictions or financial penalties, fostering competitive participation from small businesses.

We recommend that the government avoid implementing a one-year substitution restriction for the ODOS IV contract, as such conditions could limit the operational flexibility of small businesses. Instead, we suggest allowing reasonable substitutions, provided the replacement personnel meet or exceed the qualifications of the originally proposed key personnel. This approach would help maintain a level playing field, enabling small businesses to participate competitively while ensuring the ODOS IV contract is staffed with highly qualified, dedicated personnel.

Thank you for your feedback/suggestion.

Under Section 5.5, Key Personnel, it is stated that there are four key personnel (KP) roles, and for each role, except the Project Manager (PM) and UX/IX Designer, a KP will be assigned to oversee every four teams. Therefore, if an offeror is awarded 20 teams, the prime contractor would need five KPs for each applicable role, excluding the PM and UX/IX Designer. In this scenario, the offeror would require a total of 12 KPs to remain compliant. Could the government please clarify if this interpretation is correct?

Reference Section 5.5. Please see RFQ when issued.

52 How many Key positions will there be? See the PWS.

53 Will there be remote work? Yes. See Section 10.1 "Place of Performance"

Is having $30-$40M in app dev across all contracts as a team crucial? Corporate experience requirements will be provided in the final RFQ.

Can you please clarify what is meant in the first paragraph in reference to multi-vendor teams, individual teams, and project teams?

ODOS-IV will be awarded to multiple vendors. Initially each vendor will be awarded 5 teams.

Has the government made any decisions for the contract vehicle and the set aside status for this procurement?

The Government expects to issue an RFQ on GSA MAS as a Small Business set-aside. Please see the APFS record F2024066162 for up-to-date information.

Section 7.4 of the Draft PWS states: “The Government plans to award the contract to only one (1) Infrastructure and Integration vendor.”

Has the government made any decision on how many vendors will be selected for the “DevSecOps and UXD/IxD Services”?

There will be up to four task orders (TO) awarded for the DevSecOps/Design requirement and one TO awarded for the Integration/Infrastructure requirement.

Section 4 of the Draft PWS states: “The scope of ODOS IV has two separate requirements that will be awarded as separate task orders.”

Will the procurement be done in a manner where a vendor can select which one of the task areas they want to bid (i.e. “DevSecOps UXD/IxD” or “Infrastructure and Integration Svcs”?

Yes, contractors may select the task area.

Will there be more than one vendor selected for each of the above two task areas?

There will be up to four task orders (TO) awarded for the

How does USCIS intend to differentiate bidders on the two tracks? Companies will need to submit separate Quotes for each task.

61 Can one company/team bid on both tracks? Yes, but separate Quotes will be required for each task.

Is the winner of the Integration and Infrastructure task prevented from performing on both tracks?

Contractors may quote on both task areas and could be awarded separate task orders for each requirement if they are the selected.

Does government intend to have an FTE/ # of Agile teams requirement for the corporate experience? What is it?

Details associated with the evaluation criteria are still being developed and will be made available when the RFQ is released.

Will USCIS require companies to recertify size status? Information will be posted to APFS once finalized.

Given the specific labor category qualifications, will USCIS allow vendors to submit open market rates that match those requirements and are being added to the vendor’s schedule?

Open market items will not be accepted.

Given the difference in the scope/focus on the two tracks, does the solicitation intend to have different evaluation criteria for the two?

Yes. Evaluation factors and phases will be the same for both task areas. The specific evaluation criteria for each is still being developed.

Will corporate experience from a JV require work completed by the managing partner? The specific evaluation criteria is still being developed.

Can a company participate in more than one technical challenge? The evaluation criteria is still being developed.

Could you please confirm if this will be a multi award IDIQ? If yes, are there ceilings to the individual Task orders?

This will not be a multiple award IDIQ. There will be up to four task orders (TO) awarded for the DevSecOps/Design requirement and one TO awarded for the Integration/Infrastructure requirement.

It is our understanding that this RFP is scheduled to drop at the end of January 2025. Is that still the case?

Yes, any changes will be published under ODOS IV APFS record F2024066162.

The scope of ODOS IV has two separate requirements that will be awarded as separate task orders. Will a contractor be able to bid on one of the two task orders or are contractors expected to submit proposals that cover all PWS areas?

Contractors may choose which to quote on or both.

What will be the number of awards? There will be up to 4 TOs awarded for DevSecOps/Design requirement and 1 TO award for the Integration/Infrastructure requirement.

Which contract vehicle will this effort be released on? GSA MAS

Please confirm that ODOS IV will be 100% set-aside for small business. This will be a 100% small business set aside.

In evaluating Corporate Experience and/or Past Performance, will USCIS require that examples only come from a task order or a single stand-alone contract? For example, for the current USCIS RIVER solicitation, multiple task orders for the same customer on a BPA/IDIQ could not be combined into a single example for evaluation.

Given that such BPAs/IDIQs often express how a given federal agency wants to manage the execution IT services and do not relate to an offeror’s capabilities or experience, per se, we recommend that USCIS allow offers to submit multiple task orders for the same customer on a single BPA/IDIQ for evaluation. Doing so will allow otherwise fully qualified offerors to compete to potentially support USCIS with its ODOS IV mission.

The evaluation criteria is still being developed.

What will be the number of awards? There will be up to 4 TOs awarded for DevSecOps/Design requirement and 1 TO award for the Integration/Infrastructure requirement.

Which contract vehicle will this effort be released on? GSA MAS.

In Section 7.4, it is stated that the Government will make 1 award for the Infrastructure and Integration workstream. How many awards are anticipated for DevSecOps? We suggest that the Government maintain flexibility in this area to make the appropriate number of awards based on quality of proposals received (e.g., not instituting a hard cap of maximum or minimum number of awards).

There will be up to 4 TOs awarded for the DevSecOps/Design requirement and 1 TO award for the Integration/Infrastructure requirement.

In Section 4, it is stated that: "Each vendor may be eligible to receive up to twenty (20) teams.

Vendors may receive additional teams based on their demonstrated ability to secure qualified staff with appropriate skill sets." Can the Government confirm the intent of this statement is that vendors may receive beyond 20 teams? We do recommend the Government give themselves maximum flexibility in this area in order to leverage the highest performing vendors when adding new teams.

DevSecOps Teams, which will include Team CLINS and optional Team CLINS available for exercise, are expected to be limited to 20 for each TO, however number of teams and limitations may change. Optional CLINS may or may not all be exercised.

In Section 7.1, it is stated that “four (4) FTE resources whose labor category mix is determined by the vendor (mix can only include priced labor categories existing on the pricing schedule).” Would the Government clarify if the term “pricing schedule” means the vendor’s GSA price list and that vendors can use any labor categories from their GSA price list or would the vendors be required to propose the mix only based on the labor categories listed in Attachment 4 (ODOS IV Labor Categories Descriptions).

The PWS is referring to the Labor Categories that would be priced in the TO pricing schedule for that Team CLIN. This will be clarified in the RFQ.

Section 6.4 states: "The Contractor shall provide resumes for individuals identified as key personnel within ten (10) business days after the date of the award." Consistent with other recent DevSecOps focused solicitations, we suggest that USCIS require all offerors to identify their key personnel at the proposal stage and the key personnel be employees of a Prime, CTA member, or major subcontractor.

Thank you for the suggestion. Feedback will be considered.

Please confirm that all small business offerors will be required to recertify their small business status at the time of proposal submission. Without this requirement, companies that have grown substantially beyond small business size standards can continue to compete using their original GSA MAS small business status, potentially years after outgrowing these thresholds. By requiring recertification, the Government ensures that contract awards support actual small businesses, fostering a more competitive environment.

Thank you for your suggestion. Once finalized, information will be posted to APFS.

In Section 5.5, it is stated that "The Project Management Lead shall be a full-time employee of the prime contractor or lead CTA member."

Would the Government consider allowing the Project Management Lead to be a full-time employee of any of the CTA members or a major subcontractor?

Thank you for the suggestion. Feedback will be considered.

In Section 4, it is stated that "The scope of ODOS IV has two separate requirements that will be awarded as separate task orders." Would the Government require separate proposals for each of the task orders and would the proposals be evaluated differently for each of the task orders?

Yes, separate quotes would be required and each requirement will be evaluated separately.

Could the government clarify how corporate experience reference requirements will apply to joint venture (JV) members? If a Mentor-Protégé Joint Venture (MPJV) is bidding, we recommend requiring at least one corporate experience reference from the protégé.

The evaluation criteria is still being developed.

Could the government clarify whether USCIS will continue to set a specific threshold for corporate experience relevancy (in terms of size) for the ODOS acquisition? If so, could the government specify the value required for references to meet or exceed?

The evaluation criteria is still being developed.

USCIS has a history of requiring Small Business re-representation at the time of proposal submission to ensure that offerors are still considered small under the procuring NAICS code. This includes ODOS III, SPEED, FINCH, ESIS, EYAS, and most recently RIVER. Given this significant precedence, will USCIS continue to up hold it's commitment to awarding to true Small Businesses and include a requirement for Small Business representation in the new ODOS IV procurement?

By not requiring re-representation USCIS would be setting back years of progress and precedence that has been made by Small Business advocates to ensure programs such as ODOS provide fair and reasonable opportunity for true Small Businesses to compete for awards deemed as Small Business set asides. Without re-representation, Small Business are forced to compete against Large Businesses who have significantly larger past performances, budgets, and access to resources creating an unfair advantage against Small Businesses who are truly small as designated by the 541512 NAICS code. This includes ODOS III incumbents who were required to re-represent for the previous ODOS III requirement but have now fully outgrown the SB size standard and should no longer be eligible to compete for ODOS IV as prime contractors. By continuing to uphold the re-representation requirement for ODOS IV, USCIS demonstrates it's commitment to affording

Thank you for your suggestion and feedback. Once finalized, information will be posted to APFS.

the same level playing field it afforded the Small Business offerors for ODOS III.

We recommend that USCIS set the corporate experience threshold at $34 million or below, in line with the NAICS code 541512 size standard.

Keeping the experience threshold within this limit will ensure that the contract remains accessible to small businesses, consistent with the intent of this small business set-aside.

By allowing offerors to provide a single corporate experience example within this $34 million threshold, USCIS can encourage greater small business participation and avoid unintentionally favoring firms with larger contract histories. This approach supports a competitive field of qualified small businesses capable of meeting the contract requirements while adhering to SBA standards.

The evaluation criteria is still being developed.

Ability to Bid on Both Requirements: Can vendors submit proposals for both the primary requirement (DevSecOps and UXD/IxD Services) and the secondary requirement (Infrastructure and Integration Services)?

Yes, separate quotes would be required and each requirement will be evaluated separately.

Each vendor may be eligible to receive up to twenty (20) teams. Is the expectation for up to 20 teams meant for each Task Order?

DevSecOps teams will include Team CLINS and optional Team CLINS available for exercise. Teams are expected to be limited to 20 overall for each DevSecOps/Design TO. Optional CLINS may or may not be exercised.

During the Base Transition-In period and following 3-month period, all FTE will be awarded on a Labor Hour (LH) basis. This statement conflicts with the table beneath where the Base Transition- In period and the 3-month period have FFP hours listed. Can the government please clarify?

No final decision has been made. Any conflicting language will be resolved.

The Government plans to award the contract to only one (1) Infrastructure and Integration vendor. Initially, the vendor will be assigned three

(3) Infrastructure and Integration teams. The maximum number of teams for entire contract is approximately six (6). How does the maximum of six teams relate to the earlier statement that a vendor may be awarded up to twenty teams?

These are 2 separate requirements. The 20 team reference is related to the DevSecOps/Design requirement.

93 What is the value/ size of the opportunity? See APFS.

94 What set-aside is planned on being used? Small Business set-aside.

Would the government consider staffing TDD with a federally employed designer to ensure PWS items relating to UXD/IXD meet quality standards?

Thank you for your question/feedback.

Is the max 20 teams or additional teams based on performance beyond the 20 teams?

DevSecOps teams, which include Team CLINS and optional Team CLINS, are expected to be limited to 20 for each TO. Optional CLINS may or may not be exercised.

97 Can a contractor compete on both Task orders? Yes.

For the ODOS IV RFP, will subs to the prime be allowed to use their past performance in response to the RFP?

The evaluation criteria is still being developed.

For the ODOS IV RFP, will all participants in the challenge be expected to be full time employees working on the contract after the award?

The evaluation criteria is still being developed.

Or are only 2 awards anticipated based on the first sentence of Section 4 – Scope?

There are 2 separate requirements. There will be up to four task orders (TO) awarded for the DevSecOps/Design requirement and one TO awarded for the Integration/ Infrastructure requirement.

Will the number of teams per awardee be allocated based on PWS task areas? If not, what will the method be?

There are 2 separate requirements (tasks). There will be up to four task orders (TO) awarded for the DevSecOps/Design requirement and one TO awarded for the Integration/Infrastructure requirement.

Will all teams be turned on immediately or will some be optional teams?

Please see Industry Day presentation. There will be optional team

CLINS.

Is the same number of teams (45) and awards (4) contemplated for ODOS IV?

There are 2 separate requirements. There will be up to four task orders (TO) awarded for the DevSecOps/Design requirement and one TO awarded for the Integration/Infrastructure requirement. The number of DevSecOps teams initially awarded is expected to be 5 for each TO. Optional CLINS will be included for additional teams. The Government may or may not exercise all Optional CLINS. See also #79, 90, 96. USCIS currently has 45 teams working across four ODOS III task orders. It is anticipated that during ODOS IV the number of teams at any time, across all TO’s, would not exceed approximately 60.

Will USCIS require a SB recertification as part of the task order solicitation process? We recommend that USCIS not require such a task order-specific recertification, given that all offerors have the ability to team with larger firms to mitigate any personnel and/or capability shortfalls

Thank you for your suggestion and feedback. Once finalized, information will be posted to APFS.

Consideration of Non-USCIS Past Performance:

Will the evaluation of past performance consider relevant experience from non-USCIS programs that demonstrate similar scale, complexity, and technical scope, especially when key personnel have direct USCIS experience?

The evaluation criteria is still being developed.

How many total prime awards does USCIS anticipate making for ODOS IV, and how many of these will be for DevSecOps/UX work (as opposed to infrastructure work)?

There will be up to four task orders (TO) awarded for the DevSecOps/Design requirement and one TO awarded for the Integration/Infrastructure requirement.

In evaluating Corporate Experience and/or Past Performance, will USCIS require Prime offers to satisfy specific criteria for size? For example, for the current USCIS RIVER solicitation, size was defined as a minimum of $30M and 34 FTEs. Will a similar size standard be part of the ODOS IV solicitation?

The specific evaluation criteria is still being developed.

In evaluating Corporate Experience and/or Past Performance, will USCIS accept examples that satisfy two out of three criteria for size, scope, and complexity? For example, for the current USCIS RIVER solicitation, the evaluation criteria stated that, “When assessing the overall relevance of a quoter’s experience, the Government will accept a combination of two or more areas (i.e. Size, Scope, and Complexity) as Relevant. Each area is equally important.” If a specific size standard is used, we recommend USCIS retain this evaluation language so as not to unduly disadvantage small businesses, and – if needed – strengthen the evaluation criteria concerning an offeror’s hiring and staffing practices.

The specific evaluation criteria is still being developed.

Will offerors bid – or be allowed to bid for – both requirements – (1) DevSecOps & UXD/IxD for ELIS and Global, and (2) Infrastructure/Integration to support ELIS, Global, CLAIMS 3 and Benefits Hub – as part of the solicitation process, or will Offerors only be allowed to bid for one requirement? How will USCIS determine the awardees for the separate task orders? What evaluation factors will be used to determine the awardees for either of the requirements?

Contractors may quote for one DevSecOps and UXD/IxD Services, the Infrastructure and Integration Services, or both. A contractor could possibly be awarded both.

Contractors will be required to submit separate quotes for each requirement if interested in quoting for both.

The specific evaluation criteria is still being developed.

In Section 4, it is stated that "The scope of ODOS IV has two separate requirements that will be awarded as separate task orders." Would the Government clarify if multiple task orders could be awarded to the same vendor given that there is potential for OCI if the same vendor would develop code and support the infrastructure and integration - for example, an Infrastructure and Integration Services vendor could treat the tasks differently if the sane vendor also has DevSecOps/UXD/IXD task order.

This was answered live during the Industry Day. Please see the recording. The Government sees no conflict as both requirements are in the current ODOS III TOs.

Recommend the Government consider requiring Offerors be appraised at CMMI for Services Level 3 or higher. The scope of the ODOS IV requirement is ideally suited for a company that not only has comprehensive software development experience but also has taken the time and made the investments necessary to demonstrate for the Government the maturity of their processes, standards, and rigors. Vendors certified at Levels 1 and 2 do not have the process maturity to gain the most efficiency and effectiveness under programs such as the ODOS program. Level 3 certified vendors are mature enough to realize the maximum benefits, while vendors appraised at Levels 4 and 5 have demonstrated their ability to drive down costs, increase staff retention, and continuously improve support outcomes without increasing program risk.

Thank you for your suggestion/feedback.

What metrics are monitored by the executive team - are these Agile metrics or are there also DORA metrics or some modification of that being monitored? Not using DORA metrics. Agile metrics will be used.

so this will only be Small Business set a side under MAS for both awardees? Yes.

Marilyn slides stated that USCIS is looking for contractors that have "knowledge of USCIS business operations". Heavily weighting this requirement will heavily favor incumbent contractors. How will the RFQ requirements be structured to ensure an equitable evaluation process to provide real opportunities for non-incumbent contractors to compete.

Incumbent experience is not required. Evaluation criteria/instructions are still being developed.

Is there a reason why DevSecOps isn't marked as key personnel in ODOS IV where it was in ODOS III? Please Reference PWS Section 5.

Does USCIS intend to expand the number of awardees under this next iteration? Is there consideration to moving to a specific socio-economic set-aside given the number of SB concerns able to meet the work requirements (WOSB, SDVOSB, etc.)? 1. No 2. No

Will this be open to all small businesses or just 8a? This will be 100% set aside for small businesses.

Hi, Thanks for the time today, Where will the Corporate experience be accepted from in Phase 1?

The specific evaluation criteria are still being developed.

Can you clarify the number of awards?

The GSA MAS RFQ will be for up to four task order (TO) awards for the DevSecOps/Design requirement and one TO award for the

What will the corporate experience questionnaire criteria entail (This info will be helpful to ensure we're building the best, most compliant teams)?

Will a draft RFP (vs. PWS) be shared?

The specific evaluation criteria is still being developed. No draft RFQ is anticipated.

Can you elaborate if DevSecOps and UX Services will be separate TO’s or combined together? Please reference Section 5 in the PWS.

Will the multiple awards for the DevSecOps area be awarded off the initial solicitation, or will you make multiple BPA awards and then compete the four task orders among the awardees?

USCIS will not be awarding an IDIQ or BPA. USCIS anticipates awarding up to four DevSecOps/Design TOs and a single Infrastructure/Integration TO through GSA MAS.

Is there a standard Agile set of practices to be used in KANBAN method of Agile? We, at USCIS use DevSecOps and Agile principles and practices.

What metrics are monitored by the executive team - are these Agile metrics or are there also DORA metrics or some modification of that being monitored?

The metrics will be available upon award. Yes Agile metrics are monitored. No DORA metrics

The recent SEVIS procurement required experience with SEVIS programs. Will ODOS require experience with ODOS or USCIS programs? This is restrictive and prevents new players from entering the USCIS space. USCIS will be looking for relevant experience as outlined in the PWS.

If the prime is to describe their immigration experience (if such exists) how will that be evaluated? If it doesn’t directly preclude, how does it weigh in next to other factors, specifically?

I should say how will* it weigh in, if at all?

The specific evaluation criteria is still being developed.

Does subcontractor's Immigration experience will be considered? The specific evaluation criteria is still being developed.

Will the link to this recording be sent to the registered attendees for this call?

It will be posted to the SAM.gov Industry Day notice location when available.

Where can we find the publications for this Q&A session?

Information from the Industry Day and questions and answers can be found at the SAM.gov Industry Day notice location.

How many agile teams for Software effort and how many for infrastructure team and will they participate in PI planning or Releases planning and participate during daily huddle or standup meetings? 1. Reference Section 5 and Section 6 in the PWS 2. Yes

Would appreciate if the govt can share the reasons/lessons learned for embedding the UI/UX into the DEV teams. thanks!!!

The government has used lessons learned in updating the ODOS-IV PWS and will provide additional information upon award.

about DevSecOps request, what do you believe should be the primary focus: enhancing existing processes or developing technologies that provide a competitive edge in leveraging AI for application security and operations?

Maintaining, Enhancing and improving with innovative solutions, ideas and tools. Using any AI tools and models is evaluated, adjudicated and approved on case by case, adhering to DHS guidelines.

Will we get a copy of the recording? Information from the Industry Day and questions and answers can be found at the SAM.gov Industry Day notice location.

134 Is there any size limitation for past performance? The specific evaluation criteria is still being developed.

Is it mandatory for contractors to provide experience in all three major case management systems - ELIS, Global, and CLAIMS3? Please clarify. No. USCIS is looking for relevant experiences as outlined in PWS.

136 Will past performances be evaluated? The specific evaluation criteria is still being developed.

Will Subcontractor Immigration experience be considered, In case prime contractors doesn't have?

The specific evaluation criteria is still being developed.

138 Do you have any specific Proposal requirements? The specific evaluation criteria is still being developed.

Can teams submit solely for one task and not the other? For example, can we submit only for the “DevSecOps and UXD/IxD Services” task and not “Infrastructure and Integration Services”?

Yes.

Our understanding is that within the “DevSecOps and UXD/IxD Services” task, organizations cannot submit solely for DevSecOps or UXD/IxD services.

Capability for both should be provided. Is this understanding correct?

That is correct. The GSA MAS RFQ will be for up to four task orders (TO) awarded for the DevSecOps/Design requirement and one TO awarded for the Integration Infrastructure requirement.

Can you share which incumbents are providing UXD services for ODOS III? The services are optional on all ODOS III TO's.

Can you share the main lessons learned/issues faced in ODOS III from having separate UXD and DevSecOps teams?

The government has used lessons learned in updating the ODOS-IV

PWS.

Will developers be allocated to support the design system in addition to designers? Yes.

As we have seen from previous solicitations from USCIS, will there be a dollar threshold that the prime must meet for past performance?

The specific evaluation criteria is still being developed.

Does USCIS anticipate competing additional Task Orders under the vehicle IN ADDITION to the initial task orders mentioned by Patricia?

The GSA MAS RFQ will be for up to four task orders (TO) awarded for the DevSecOps/Design requirement and one TO awarded for the Integration/Infrastructure requirement. No additional requirements are anticipated.

Are you planning to have type of small business commitments - SDVOSB, WOSB, 8a etc.? The RFQ will be a Small Business set-aside.

Will the TCV change commensurate with the change in POP to 5 years or will annual spending decrease?

No information is available on value other than what has been posted on APFS. .

What is the anticipated size requirement for Past Performances and Corporate Experience? The specific evaluation criteria is still being developed.

Who are the incumbent contractors?

Also see APFS: ADG-REI TECHNOLOGY VENTURES LLC

(70SBUR22F000000112), ALPHA OMEGA INTEGRATION LLC

(70SBUR22F000000197), DV UNITED LLC (70SBUR22F000000113),

SOFTRAMS LLC (70SBUR22F000000114)

Will solicitation result in TWO separate IDIQ/BPAs -- one for DevSecOps and one for Infrastructure/Platform Support Services?

USCIS will not be awarding an IDIQ or BPA. USCIS anticipates awarding

Integration/Infrastructure TO through GSA MAS.

Does USCIS anticipate that the solicitation will result in ONE MA IDIQ/BPA that covers both DevSecOps and Infrastructure Teams?

USCIS will not be awarding an IDIQ or BPA. USCIS anticipates awarding

Integration/Infrastructure TO through GSA MAS.

Since this is anticipated to be solicited on GSA will it be a requirement that all contractors competing already have a MAS or FSS contract and have SIN 54151S listed? Yes, also USCIS will not be accepting open market items.

USCIS has already released the draft PWS. Will USCIS also be releasing draft RFP documents, including the solicitation requirements, before the final RFP release planned for January?

USCIS is not planning on releasing a draft RFQ.

How many total awards does USCIS intend to make?

There will be up to four task orders (TO) awarded for the

Is this a total SBSA or will you narrow it to 8a only? This will be a total Small Business set-aside.

Can you clarify the number of awards?

There will be up to four task orders (TO) awarded for the

What is the anticipated number of awards for

ODOS IV?

There will be up to four task orders (TO) awarded for the

Recommend the Government require vendors to recertify as small at the Task Order level and at the time of proposal submission to ensure only those vendors who are currently still small under their GSA Schedule and the appropriate NAICS are able to bid.

Thank you for your suggestion/feedback. Once available, information will be posted to APFS.

Interested in teaming with a small business to support the requirements on this contract, does USCIS plan to releasing a list of attendees at the I- Day to support teaming discussions?

Please see the Industry Day notice on SAM.gov.

All information presented here and associated with the ODOS IV Industry Day is subject to change. The content herein is for information and market research purposes only. If there is a conflict between any later issued Request for Quote (RFQ) and any information provided herein, that RFQ takes precedence.