ATSA Draft SOW .pdf

P a g e 1 | 13

DRAFT SECTION C - STATEMENT OF WORK

C. 1. Introduction and Overview

C. 1.1 Background

The Federal Aviation Administration (FAA) is seeking test administration services for its pre-employment selection test battery for air traffic control specialists (hereafter, controllers). The

FAA currently administers the Air Traffic Skills Assessment (ATSA), a computer-based entry-level employment test battery designed to assess the majority of skills and attributes necessary for success on the job. Applicants must pass the ATSA to obtain entry-level positions as controllers at the FAA. An applicant’s score on the ATSA is one of the factors used in the applicant selection process.

The ATSA consists of multiple commercial-off-the-shelf subtests from third-party vendors and includes a combination of multiple-choice items and dynamic graphic-based problems. The assessment is administered by computer in proctored testing centers nationwide. During each testing session, the test administration vendor accesses ATSA content via Application Program

Interfaces (APIs) from the test content vendors’ websites and displays this content to test takers through a graphical user interface (GUI), which calls content in a pre-set sequence. Applicants are allowed nearly 3.5 hours to complete the assessment.1

FAA intends to replace the current test battery in late 2026. For convenience, this document describes test administration requirements for the ATSA, but note that the process may change when the test battery is updated in FY27. At that time, the Contractor may have to build a new

GUI to accommodate the new testing material. The cost and schedule for GUI development would be handled in a separate modification to the task order.

C.1.2 Scope of Work

The FAA requires a Contractor with a vast network of proctored testing facilities (multiple facilities in each of the 50 United States and its territories) to administer the ATSA. The opportunity to test internationally is also welcomed, but not required. The number of applicants to be tested each year varies based on the FAA' s hiring needs. During the past five years, FAA has administered an average of nearly 7,000 tests per year, with a mean of 5,700 tests; however, testing rates fell during the national health pandemic. The FAA estimates administering between

5,000 and 12,000 tests per year over the next five years.

The FAA typically opens at least one job announcement for controllers annually.2 Each announcement kicks off a “testing cycle” that follows the schedule below:

• Upon release of the controller vacancy announcement, applicants have 3-5 days to apply

• FAA rates and qualifies the applicants in 60 days or less, depending on the number of applications received, then provides applicant information to the Contractor

1 The current test battery is 2 hours and 49 minutes, and applicants may take up to 30 minutes for breaks.

2 The FAA is considering moving to a continuous application and testing model in the future. Should this requirement come to fruition, it will be addressed in the final SOW upon release of the solicitation.

P a g e 2 | 13

• The Contractor notifies applicants of their authorization to test

• Applicants schedule the ATSA during a 30-day scheduling window

• Applicants take the ATSA during a 30-day testing period. The testing period may be extended by one or two weeks to accommodate special situations.

During this process, the Contractor will be responsible for the following:

• Scheduling applicants via a web-based application

• Conducting load assessments (by applicant zip code) to predict geographical areas where scheduling may be challenging

• Administering the ATSA in a proctored test center during the testing cycle established by the FAA. Specifically, this includes providing:

o physical testing facilities o trained personnel (receptionists and test administrators/proctors), o computer equipment and peripherals on which to run the test, and other necessary equipment o administration monitoring capabilities

• Conducting quality checks and providing electronic test data to the FAA upon completion of the testing cycle.

• Tracking and maintaining metrics related to the test battery, such as number of tests taken per testing cycle, location, scores, incomplete tests, no-shows, non-responders, and other data that may help improve future administrations.

• Compiling Incident Reports that document any issues that may have transpired during the scheduling or test administration process. It may be necessary to coordinate with the

FAA to resolve some of these issues.

• Providing customer support to the applicants throughout the process

Specific contract requirements appear in Section 2, Task Requirements.

C.2. Task Requirements

At the direction of the FAA, the Contractor must administer the ATSA according to the requirements below. As stated previously, the FAA intends to replace the ATSA in 2026.

However, all requirements and restrictions below apply to the current and any future versions of the ATSA.

C.2.1 Program Management

C.2.1.1 Kickoff Meeting

Within 10 days of contract award, the FAA will host a kickoff meeting with the Contracting

Office, key FAA stakeholders and the Contractor’s designated representatives. It is possible for the meeting to be virtual.

C.2.1.2 Task Requirements

P a g e 3 | 13

Within 30 days of the contract kickoff meeting, the Contractor must work with the FAA to identify, validate, and document the requirements and steps needed to prepare for the first testing launch. Requirements may include but are not limited to:

• drafting the communications to candidates,

• determining data to be collected (and data that should not be collected),

• identifying the types of accommodations or comfort aids readily available,

• determining how results files are handled,

• creating the FAA landing page on the Contractor’s scheduling and informational website

• deciding which forms of candidate identification are required

• noting the break policy

• initial testing prior to launch

The Contractor must provide a timeline to address these steps.

C.2.1.3 Test Administration Manual

Using the information from the requirements document as well as the Contractor’s Standard

Operating Procedures, within 20 days of submitting the requirements document, the Contractor must prepare a Draft Test Administration Manual documenting how it intends to manage test preparation, administration, and security during the testing cycle. The test administration manual must be updated as needed and revisions agreed upon in advance with the FAA. At a minimum, the document should include:

• Step-by-step procedures and methods used to register applicants and administer the

ATSA, including the language to be used in emails to applicants. FAA will work with the

Contractor to develop specific language to applicants.

• Roles and responsibilities

• Applicant identity management

• Confidentiality and security processes (physical and electronic/data)

• Emergency protocol

• Incident management

• Proctor selection and training procedures

• Standards of Conduct

• Quality control procedures

FAA will have 20 days to review the Test Administration Manual and provide comments to the

Contractor. The Contractor will have 10 days to resolve FAA’s comments.

C.2.2 Facilities

At the direction of the FAA, the Contractor will be required to administer the ATSA in its testing facilities. The Contractor must have a minimum number of testing facilities in each of the 50

United States according to the physical size of the state. Specifically, states smaller than 50,000 square miles should have at least one testing facility. States ranging in size between 50,000 square miles and 100,000 square miles should have at least two testing facilities, and states larger

P a g e 4 | 13 than 100,000 square miles must have at least three testing facilities.3 In the state of Alaska, testing must occur in Anchorage and Fairbanks at a minimum. There must be at least one facility within a 200-mile radius of each of the 36 FAA-approved Air Traffic-Collegiate Training

Initiative (AT-CTI) schools. For Contractors whose network of testing centers exceeds this minimum, the FAA expects the Contractor to provide the ATSA at all its testing centers.

C.2.2.1 In cases in which established Contractor testing facilities are not close to desired testing locations, at the direction of the Contracting Officer’s Representative (COR), the Contractor must provide testing services in temporary testing facilities/sites. Temporary testing facilities/sites may include hotels, schools, and other public venues.

C.2.2.2 All testing facilities/sites and locations, including temporary sites, must ensure equivalent, high quality testing conditions for all applicants. Testing sites must meet all applicable Federal, state, and local standards for access by persons with disabilities and for safety.

C.2.2.3 The Contractor will be required to maintain a current list of all facilities, which shall be made available to the FAA. The testing facilities should be identified on a map that shows the city and zip code.

C.2.2.4 The Contractor must provide a mechanism (preferably lockers) for secure storage of applicants’ belongs while they are in the testing room. In the absence of lockers, the Contractor may offer another solution such as secure storage bags that may not be opened in the testing room.

C.2.3 Test Registration/Scheduling

As part of each ATSA testing cycle, the FAA will provide a list of referred applicants to the

Contractor. In turn, the Contractor will notify these applicants via email that they are authorized to test and will provide instructions on the registration and scheduling process. The FAA must have input into the correspondence to be sent to applicants. Once the applicants receive their authorization to test, they may schedule to test as soon as the next day.

The Contractor must provide a user-friendly, web-based scheduling system that allows applicants flexibility as to the date and location for taking the test while complying with testing deadlines and limitations imposed by the FAA. The system must provide the capability to reschedule or cancel a testing appointment within three days of the test. In addition, the Contractor must provide a means for applicants to seek assistance during the registration, scheduling, and testing process. An online chat capability for questions is preferred.

C.2.4 Test Administration

The test administration period usually lasts 30 days. Occasionally the FAA will extend the testing window by a week or two if needed.

3 For consistency, please refer to the following website for state size:

https://www.census.gov/geographies/reference- files/2010/geo/state-area.html https://www.census.gov/geographies/reference-%20files/2010/geo/state-area.html

P a g e 5 | 13

C.2.4.1 Preparation

• The FAA will alert the Contactor of upcoming testing windows as soon as the FAA

Vacancy Announcement opens.

• Testing shall be completed within the FAA’s timeline. As previously stated, testing usually lasts 30 days.

• Prior to 30 days of the first testing date, the Contractor must inform the FAA of any date(s) it is unable to provide testing services within the designated testing period.

• The Contractor shall attend meetings with the FAA during the testing cycle, either in– person or virtually, to discuss test preparations and logistical issues pertaining to testing.

The COR will work with the Contractor to determine the meeting schedule and agenda.

• The Contractor must maintain current and accurate information on its website regarding the FAA, ATSA, scheduled test dates, and announcement numbers that correspond to the testing cycles.

C.2.4.2 Special Accommodations

To comply with the Americans with Disabilities Act (Americans with Disabilities Act of 1990, P.L. 101-336, 42 U.S.C. § 12101 et seq.), the FAA reviews and processes all reasonable accommodations requests from applicants with disabilities. A reasonable accommodation is any modification to the workplace or to the standard operating procedures that affords equal employment opportunity to employees and applicants with disabilities. An applicant with a disability is a person who has a physical or mental impairment that substantially limits one or more major life activity. This includes people who have a record of such an impairment, even if they do not currently have a disability. It also includes individuals who do not have a disability but are regarded as having one. In this case, it applies to applicants who believe it to be difficult to take the ATSA using the standard testing equipment or within the standard assessment duration.

To receive an accommodation, the applicant must complete a request form provided by the FAA.

The request must be accompanied by relevant documentation such as a doctor’s note or

“Schedule A” letter.4 Applicants will submit their accommodation requests to FAA’s Applicant

Solutions mailbox at least seven days prior to registering for the ATSA. The FAA will review the request for accommodation and subsequently work with the Contractor to determine how to make accommodations, if applicable. The FAA will contact the applicant within three days of notification to provide the details of the requested accommodation(s) and/or, to request additional information or medical documentation.

In some cases, the accommodation may be minor and involve obtaining permission to have certain comfort aids in the testing room. The Contractor must coordinate with the FAA to

4 3 A Schedule A letter is a document that is used primarily in the hiring process. The letter demonstrates the applicant’s eligibility for Schedule A noncompetitive hiring due to a disability. It serves as proof of the applicant’s disability.

P a g e 6 | 13 identify which comfort aids do not require special approval in their testing facilities. A list of these comfort aids should be available to applicants on the scheduling site.

C.2.4.3 Applicant Check-in and Proctoring

• The Contractor’s test center staff shall not:

o Represent themselves as FAA employees o Provide advice on any portions of the test o Collect fees from applicants

• Test center receptionists must greet each applicant upon arrival, check and verify the identities of the applicants, and document that the applicants have arrived to take the exam. Two forms of identification are mandatory, one of which must be a Government-issued ID with a photograph (e.g. driver's license, passport, state ID).

• Test center receptionists must also make note of applicants who were scheduled to test but did not arrive for testing (no shows).

• Prior to entering the testing room, receptionists shall ensure that applicants secure all personal belongings (coats, purses, wallets, all electronic devices including cell phones and smart watches, and food/drinks) in a locker, or via another secure mechanism accessible only by the applicant.

• Proctors must provide a general briefing and orientation to applicants prior to test administration, including location of the rest rooms, break policy, emergency evacuation procedures, and other administrative information. The ATSA software provides all specific subtest instructions and regulates the timing of the individual tests within the battery.

• Proctors must establish a record of each applicant on the designated computer testing station for that person immediately prior to the start of each session.

• Proctors must assist applicants with general administrative matters pertaining to the test session and in the case of possible computer malfunctions. However, test administrators must not provide instructions included in the actual ATSA program or interpretations of the questions or problems.

• Proctors must monitor and record the start and stop times of breaks.

• Proctors must supervise the test room during the testing period and must maintain a ratio of at least one proctor for every fifteen testers in the testing room.

• Each session must be audio/video recorded from a minimum of two separate views that captures all applicants while testing as well as proctors and others who enter the testing room. The Contractor must maintain these video/audio recordings for a period for seven years and must provide copies of the recordings of any testing session to the FAA upon request.

• The Contractor must report cheating or any compromises of test security to the COR via email, within 4 hours of the incident. The contractor must retain all evidence of the incident (location, time, applicant names, applicant IDs, proctor names, computer number, a well-documented statement regarding the issues, and actions taken). The

Contractor must provide this information to the FAA in a full report within 24 hours of the incident.

P a g e 7 | 13

C.2.5 Safeguarding of Sensitive Information

C.2.5.1 The Contractor must apply following security measures to safeguard the ATSA, specifically:

• Test access, test scores, test software codes, applicant test files, and test copies, results, and reports must be strictly controlled such that no unauthorized person can obtain access.

• Access to FAA testing material is restricted to those Contractor employees who have a

"need to know." The COR must issue affirmative authorization to the Contractor to release test material or information to any person.

• For all persons who accessed the test or test material, the Contractor must record the names, dates, times, location, and purpose of access. The Contractor should maintain this information electronically and deliver it to the FAA upon request.

• Any method used to distribute the test software to testing facilities/sites and computers must be secure (see Section 2.5.2 for more information regarding security requirements).

C.2.5.2 The Contractor's systems that contain Personally Identifiable Information and sensitive information (including test content and applicants’ test responses and scores) must comply with the requirements of the e-Government Act of 2002, found in Title III of PL 107-347, the Privacy

Act of 1974, and the latest revision of FAA Order 1370.121 (FAA Information Security and

Privacy: Policy), and its supplemental directives.

C.2.6 Personnel

C.2.6.1 Each of the Contractor’s employees involved in the administration of the ATSA must sign a non-disclosure agreement provided by the FAA stating that he or she will maintain confidentiality of all ATSA information and will adhere to all test security policies and procedures, both during and after employment.

C.2.6.2 Neither the Contractor nor any of its employees may create or offer products and/or services relating to coaching on—or preparation for—the ATSA.

C.2.7 Equipment and Software

C.2.7.1 There must be at least two complete backup computer workstations (backup set of equipment) for every 15 testing workstations.

C.2.7.2 The Contractor will receive the software and/or Application Programming Interfaces

(APIs) used for ATSA from all ATSA content vendors. The Contractor is responsible for ensuring that the software is properly loaded and/or linked and distributed at each of the testing locations.

C.2.7.3 The Contractor’s computers must use the MS Windows operating system and possess sufficient processing capabilities to host and successfully run the ATSA software. The Contractor must ensure the ATSA software is compatible with its computer equipment before use in testing.

All testing facilities must offer the same experience to the applicants; therefore, the facilities must contain the same equipment. As a point of reference, the current minimum requirements are as follows:

P a g e 8 | 13

Equipment/Technology

Description

Minimum Specification

Operating System Windows 7 or greater

Browser Chrome 68 and above

Firefox 63 and above

Internet Explorer 11 and above

Network 1 Gb Network Card

Network Server SQL Server 2014 Express or greater

Microsoft Server 2012R2 Standard or greater

Memory 128MB or more system RAM

A minimum of 16MB video memory (apart from the main memory)

Storage 200 MB or more of free hard drive space

Monitor resolution 1368 X 768 or higher

Peripherals Mouse and Keyboard with number pad

Sound card and stereo headsets with adjustable volume

C.2.7.4 The Contractor shall maintain the existing GUI that allows applicants to interface seamlessly with the ATSA subtests owned by third party Contractors. Upon selection of a replacement test battery, the Contractor may be required to develop a GUI for the new testing material. Costs for this development will be negotiated via a contract modification.

C.2.8 Reports and Data Files

The Contractor must provide the FAA web-based access to the various reports and data files according to the schedule provided by the COR. These reports are listed as deliverables in

Section C.6.

C.2.8.1 During the testing cycle, the Contractor must provide online test activity data that is updated daily. Data must include:

Summary level data

• Total tests scheduled

• Total appointments canceled

• Total tests delivered

• Total reschedules

• Total number of no shows

Individual-level test detail data

• Applicant ID

• Country (if Contractor provides international testing sites)

• Applicant zip code

• Registration date

• Exam date

P a g e 9 | 13

• Status (delivered/no show)

• Site name, Site ID and location (Zip code, city, state)

C.2.8.2 If any issues or incidents occur during an applicant’s testing experience (e.g., fire alarm, temperature controls, computer issues, problems with applicant identification), the Contractor must notify the FAA using a standard incident report within four hours of the incident(s).

C.2.8.3 Within 10 days after the final test date, the Contractor shall provide a Final Test Data File that contains test scores for all applicants who took the test during the testing cycle. The file must contain the applicant IDs, test dates, and raw data from each subtest. If not all scores are provided, the report must indicate the reason.

C.2.8.4 The Contractor shall submit a report to the FAA detailing any planned equipment maintenance or outages.

C.2.9 Maintaining Records

The Contractor must adhere to the following with respect to maintaining records and documentation. Occasionally the FAA may request that the Contractor provide witnesses or pertinent records, videos or reports to support legal proceedings.

• The Contractor should retain incident reports for seven years.

• In accordance with the National Archives and Records Administration, General Records, Schedule 2, Employee Acquisition Records, Item 51, unless it is notified of a litigation hold, the Contractor shall destroy applicants’ test scores two years after the end of a testing cycle.

• In accordance with the National Archives and Records Administration, General Records, Schedule 2, Employee Acquisition Records, Item 51, photographs of candidates taken upon check-in shall be destroyed within 90 days of the end of the testing cycle.

• In accordance with the National Archives and Records Administration, General Records

Schedule 5.2, Transitory and Intermediary Records item 20, unless the Contractor is notified of a litigation hold, it shall destroy audio visual recordings of the testing room within four weeks of the test.

C.2.10 Configuration Management

C.2.10.1 The Contractor must manage and control any changes to its software or any

Contractor-initiated changes to the ATSA software within a formal configuration management system that conforms to a published and recognized standard such as ANSI/EIA-649 or configuration management as outlined in Section 4.1 of the FAA’s Acquisition Management

System (AMS) configuration management policy. See the following website:

https://fast.faa.gov/docs/acquisitionManagementPolicy/acquisitionManagementPolicy.pdf

C.2.11 Test Readiness

C.2.11.1 Prior to initial testing of applicants, and within 90 days of contract award, the

Contractor must conduct an end-to-end operational readiness test of the Contractor's equipment, staff, applicant contact and scheduling system, procedures, and all software required to execute

P a g e 10 | 13 the ATSA, to the satisfaction of the FAA. This must include a practical demonstration to the

FAA.

C.2.11.2 The Contractor will be required to repeat the end-to-end operational readiness test each time it alters any equipment or software.

C.2.12 Inspection

The FAA reserves the right to conduct periodic scheduled and unscheduled inspections of the testing facilities. During these inspections, the Contractor must accommodate the FAA at the testing facility where the ATSA is administered. The purpose of such inspections is to ensure the security and integrity of the exam and its associated procedures and ensure compliance with the standards and requirements in this Statement of Work.

C.3. Deliverables

The following deliverables apply to this contract.

P a g e 11 | 13

# SOW

Reference

Deliverable

Description

Due Date (in calendar days)

Acceptance Criteria

One-time deliverables to be accomplished upon contract award

1 2.1.1 Kickoff meeting Within 10 days of contract award

Key Contractor personnel are in attendance, in person (preferable) or virtual with meetings notes due

2 2.1.2 Requirements document

Within 30 days of contract kickoff meeting

Report is submitted on time in the form of a document or matrix that is easy to track to see the overview of all requirements and the timeline associated with the tasks.

3 2.1.3 Test Administration

Manual

(Draft and Final)

Draft: Within 20 days of submitting the project plan and the requirements document

Final: Within 10 days of FAA’s comments

Draft Test Administration manual is submitted on time, consistent with the requirements in Section 2.1.2.

Upon receiving FAA’s comments, the Contractor will make edits and provide the final within 10 days.

4 2.7.4 Graphic User

Interface (GUI) that integrates the test content from different vendors to create a seamless applicant experience.

The GUI becomes property of the FAA at the end of the contract.

As specified in contract modification

FAA will conduct usability testing to determine if the GUI is satisfactory.

The GUI should enable the ATSA to feel like one fluid test versus two separate segments. GUI should include visual and audio format.

5 2.11.1 Readiness demonstration to administer the

ATSA. (Operational

Readiness test)

Within 90 days of contract award

FAA is satisfied that the

Contractor’s equipment, staff, procedures, and software can consistently administer the ATSA test properly

The following deliverables occur only during testing cycles

6 2.4.1 Weekly teleconference to discuss logistics and test preparations

During the testing cycle, on a day of the week agreeable to both parties

Call happens on time and contains overall status and any issues

P a g e 12 | 13

7 2.8.1 Pre-testing analysis report that includes a zip code analysis showing the anticipated number of people testing at each testing site, balanced against existing demand.

Within 72 hours after completion of each applicant test registration period

Report contains a list of sites where volume could be extremely high during the testing cycle from ATSA candidates or any other major testing initiatives.

8 2.8.1 Test data file containing the exam data such as tests scheduled, tests administered, no shows, and applicant data by site.

At least weekly during testing cycles, by 5pm ET on Fridays

Data is provided on time in a standard format approved by the

FAA and is complete with the data indicated in Section 2.8.1

9 2.8.1 Final Test Data

File, containing test scores for all applicants who took the test during the testing cycle.

10 days after the final test date. Report is provided on time and is complete with the

• Announcement #

• Applicant name

• Applicant #

• Applicant Testing date

• Testing location

• Test score data

101 2.8.2 Email notification of any issues or incidents that arise during testing.

Note: a fully documented

Incident Report is provided separately

(per above) on a weekly basis.

Within 4 hours of occurrence

Emails are submitted within 4 hours of incident occurrence and contain details of the testing location, time, issue, and resolution. If the issue affects an applicant, the applicant’s

ID should be included.

11 2.8.4 Notice of planned maintenance, upgrades or outages

As needed Complete list of scheduled outages, by facility, with dates

C.4. Period Of Performance

The period of performance for this Firm-Fixed Price (FFP) IDIQ contract will be one base year with four possible option renewal periods (for a total of 60 months if all options are exercised).

P a g e 13 | 13

The FAA may order testing services during the final months of each performance year. In these situations, completion of testing may extend into the next option period or end of contract by a period of up to six (6) months.

C.5. Place Of Performance

Services occur Nationwide at the Contractor’s facilities.

End of Section C