20220602 Responses to Ques for SIR 693KA9-R-00013.pdf

Responses to Questions Posed under FAA SIR 693KA9-22-R00013

1. Is there a small business participation or subcontracting requirement? No. If not a specific participation requirement, is small business participation encouraged for this effort? All capable businesses are encouraged to submit a response.

2. The RFP states: “Data gathered for this project shall be used for the FAA’s research purposes only and will become the property of FAA/AHR/AHF-200 once the project is complete. Any and all data and products created in this project will become the sole property of the FAA.” To clarify, is this referring to the assessment strategy recommendation product, or to the assessment battery itself? Data collected during the project (e.g., survey data, validation analyses and results) and the assessment strategy will be owned by the FAA and may not be used for any other non-FAA purpose. This restriction also applies to the assessment battery if a custom test is created. However, COTS tests will remain the property of the test publisher with the FAA purchasing licenses for a specific contract period. If an assessment is proven to have adequate validity in the course of this project, is the FAA open to licensing assessments for existing assessment content for which the vendor owns the IP, that are then validated or customized in the scope of this effort? Yes. Or other engagements that deliver the actual assessment product other than full government purpose rights? This would depend on the specifics of the situation.

3. A related question, is the FAA open to licensing vendor assessments that are customized using data from this effort, i.e., FAA maintains ownership of the content, vendor licenses and operates the assessment delivery and scoring service to the FAA for the life of the assessment after it is validated? Yes.

4. To confirm, how soon after contract initiation will the vendor have access to FAA ATC

SMEs? The RFP mentions at least 120 days….can we assume this amount of time for in-depth SME interviews, job observations etc.? Yes.

5. Will controller SMEs have open internet access to support job analysis and validation activities? Surveys can be administered to controllers electronically by embedding a hyperlink to the web-based survey in an email communication.

6. Will the vendor be provided on-site access to select FAA tower facilities for SME engagement and observations? Yes, pending any COVID-19 restrictions or other restrictions required by NATCA.

7. Will the vendor have access to controller SMEs in the TRACON and ARTCC facilities?

Yes, pending any COVID-19 restrictions or other restrictions required by NATCA.

8. What is the most current source document that includes information on recent ATC applicant and hiring rates, including number of applicants and hires per year? The number of candidates the FAA tests each year depends on workforce demand. Prior to

COVID, FAA’s target was to test approximately 8,000 candidates per year. During

COVID, the number of candidates decreased significantly. However, we anticipate administering the ATSA to approximately 8,000 candidates in 2022. Additional data will be provided to the selected vendor after contract award.

9. Please describe the FAA’s existing full selection process for your Air Traffic Control

Specialist position. Candidates submit applications on USAJobs.gov in response to a vacancy announcement for controllers. Applications are screened by an HR representative to determine if the candidate meets minimum qualifications. Candidates who are deemed qualified are invited to take the ATSA. Candidates who complete the

ATSA successfully are screened more extensively and may be offered a tentative offer of employment depending on agency need. At that time, candidates must pass additional assessments including security, psychological, and medical screenings. Successful candidates are selected for employment, assigned to a facility type, and issued a firm offer letter.

10. Where does FAA envision this new pre-employment controller selection test fitting into the current selection process? The FAA’s intent is to replace ATSA. However, we acknowledge that the proposed assessment strategy may result in steps being added and/or removed from our current selection process.

11. What does FAA like about the current pre-employment controller selection test? What specifically do you most want to see changed or improved? The FAA is replacing ATSA because the contracts for its content are expiring. With regard to a replacement, the FAA seeks to maximize validity evidence and minimize adverse impact.

12. If FAA seeks an enhanced candidate experience, please define what a great candidate experience looks like to FAA. The FAA is always interested in a pleasant candidate experience. Generally, we seek to minimize the time required to complete assessments, protect privacy, and ensure that the test-taking process is transparent and smooth.

13. We understand the FAA requires updated job analysis that will drive the new assessment, however, is the FAA looking for any specific test/item types (e.g., personality, cognitive, simulations, situational judgment, etc.)? The FAA does not have a preference regarding item type(s).

14. Does the FAA have a preference for how long the new test should take candidates to complete? The FAA requests that the vendor make recommendations for our consideration. However, generally, the FAA would prefer that any proctored test not take more than 3.5 hours. We acknowledge that the proposed assessment strategy created in

Task Order 001 may result in steps being added and/or removed from our current selection process.

15. Does the FAA prefer commercial off-the-shelf (COTS) or a custom test? A combination of both COTS and custom for the new test? The choice of COTS versus custom has implications for the FAA. However, we do not have a preference at this time; we are seeking recommendations. The RFP says: “Any and all data and products created in this project will become the sole property of the FAA.” Is the FAA open to licensing a contractor test/Intellectual Property? The details of the agreement will depend on whether a COTS or custom solution is selected. COTS content would be owned by the test publisher (the licensor) with the FAA purchasing rights to administer it during the contract period. Custom content would be owned by the FAA.

16. Currently FAA proctors the tests at testing centers. For the new test, does the FAA prefer unproctored or proctored assessment? The FAA anticipates that the replacement for

ATSA will be proctored. However, we acknowledge that the proposed assessment strategy created in Task Order 001 may result in steps being added and/or removed from our current selection process.

17. Is the FAA looking for mobile test delivery? No.

18. What type of reports is the FAA looking for? Please reference Section C for details regarding the technical reports that the FAA requires the selected vendor to provide.

19. Are any other languages than English needed for the new test? Reports? No. All deliverables are to be created and delivered in English only.

20. Does the FAA have an existing, updated, and valid common success profile or competency model for the controller position? If so, may the FAA provide a copy?

Recent controller job analyses and other supporting information will be provided to the selected vendor after contract award.

21. The RFP says the new test will be launched by your test administration contractor on their platform. Please describe. The current testing platform uses APIs to call web-based

COTS test content from the test publishers and presents that content through a customized graphical user interface (GUI) to applicants sitting in one of our test administration vendor’s testing centers. This process allows the FAA to create a seamless experience even though the test content comes from two different sources. Is it possible for the new test to integrate with their platform? Yes. A new COTS or custom test can be integrated by programming new APIs to call the new content and by revising the GUI. Is an Applicant Tracking System (ATS) involved? No.

22. Is the plan for the contractor to develop the test only or to host as well? The selected contractor will be responsible for identifying or developing the best test content for selecting controllers irrespective of whether that content is owned by or could be hosted by the contractor.

23. Are there any requirements for where data is hosted/sitting (e.g., United States)? The selected vendor will be required to adhere to the requirements in the current version of

FAA Order 1370.121, FAA Information Security and Privacy: Policy and any associated directives as stated in Section I of the SIR. Contact Stephanie McKnight-Bailey

(Stephanie.McKnight-Bailey@faa.gov) for this document but note that a signed NDA is required.

24. The RFP says: “Each year, the FAA typically administers the ATSA to thousands of controller job applicants.” Please further describe the test candidate volume (e.g., 5,000, 25,000, 100,000 candidates for testing)? The number of candidates the FAA tests each year depends on workforce demand. Prior to COVID, FAA’s target was to test approximately 8,000 candidates per year. During COVID, the number of candidates decreased significantly. However, we are on target to test approximately 8,000 candidates in 2022.

25. FAA states in the RFP that the services are provided by two vendors. Who are the incumbents? SHL US LLC and Airways International Limited provide test content. Our test administration vendor is Pearson Vue.

26. Table L-1: In the solicitation, it states that the offeror must provide a proposal that contains four volumes in accordance with Table L-1. Table L-1 has each volume broken down into different sections, each with different file naming conventions. For confirmation purposes, is this asking for a different PDF for the individual sections of the volume such as Vol IA, Vol IB, Vol IC, and so forth? Yes. Bidders should submit a total of nine (9) separate pdf files.

27. RE: L.18. VOLUME II (PAST PERFORMANCE) INSTRUCTIONS ..."The Offeror must select three (3) contracts/subcontracts... At least two (2) of the Past Performances must be for prime contract work. An Offeror has relevant past performance where the work was performed in its entirety -- 100% -- by the Offeror on a "pass-through" role, where for the government's convenience the work was contracted through a prime mailto:Stephanie.McKnight-Bailey@faa.gov contractor's BPA or similar umbrella purchasing vehicle, but the prime contractor played no part in performing or delivering the work. May this Offeror present this past performance as "prime contract work"? Yes. Please explain this arrangement in your response.

28. Re: Attachment L003 -- Labor Rates. Are Offerors required to provide labor rates for all listed labor categories? Yes. Labor rates are being requested for all IDIQ periods.

29. Does the development and validation of the Air Traffic Controller assessments prevent a contractor from subsequently administering these assessments on an assessment platform or in physical locations? No. The experimental test battery will be administered by our current test administration vendor. Details of how the final test battery will be administered will be developed as part of the respective subsequent Task Order.

30. Re: Attachment L002 and SIR. What is the PoP for Task 001? Attachment L002, page 5 lists the PoP as 14 months. The SIR, page 2 indicates the PoP for the base year is one year. The base period for the IDIQ contract is 12 months. The PoP for Task Order 001 under the IDIQ contract is 14 months.

31. Re: Attachment J002. The LCAT descriptions in Attachment J002 include two versions of the same LCATs, one starting on page 1 and slightly different versions starting on page 5. Is one version preferred over the other? A revised version of Attachment J002 has been posted to www.Sam.gov under Notice ID: 693KA9-22-R-00013.

32. RE: Test length. Over the last 10 years, the length of the test session has changed - is there a desired length of testing the FAA has for the candidate? No. The FAA requests that the vendor make recommendations for our consideration. However, generally, the

FAA would prefer that any proctored test not take more than 3.5 hours. We acknowledge that the proposed assessment strategy created in Task Order 001 may result in steps being added and/or removed from our current selection process.

33. Re: OTS Content. Is the FAA open to incorporating off-the-shelf content in addition to the development of new content? Yes.

34. Re: Job requirements. Has the FAA seen much, if any, change in job requirements / critical KSAOs over the past few years and/or anticipate many changes in the next few years? If so, what are the major ways in which the job has changed / is anticipated to change? The job analysis described in Task Order 001 is designed to answer these questions.

http://www.sam.gov/