6973GH-18-R-00014 - Amendment 0002.pdf

AMENDMENT OF SOLICITATION/MODIFICATION OF CONTRACT

1. CONTRACT ID CODE

5. PROJECT NO. (If applicable)2. AMENDMENT/MODIFICATION NO. 3. EFFECTIVE DATE 4. REQUISITION/PURCHASE REQ.NO.

PAGE OF PAGES

6. ISSUED BY CODE 7. ADMINISTERED BY (If other than Item 6)

8. NAME AND ADDRESS OF CONTRACTOR (No., street, county, State and ZIP Code)

CODE

9A. AMENDMENT OF SOLICITATION NO.

9B. DATED (SEE ITEM 11)

10A. MODIFICATION OF CONTRACT/ORDER NO.

10B. DATED (SEE ITEM 13)

CODE FACILITY CODE

11. THIS ITEM ONLY APPLIES TO AMENDMENTS OF SOLICITATIONS

The above numbered solicitation is amended as set forth in Item 14. The hour and date specified for receipt of Offers

Offers must acknowledge receipt of this amendment prior to the hour and date specified in the solicitation or as amended , by one of the following methods: (a) By completing

Items 8 and 15, and returning is extended, is not extended.

12. ACCOUNTING AND APPROPRIATION DATA (If required.)

13. THIS ITEM APPLIES ONLY TO MODIFICATIONS OF CONTRACTS/ORDERS. IT MODIFIES THE CONTRACT/ORDER NO. AS DESCRIBED IN ITEM 14.

(x) A. THIS CHANGE ORDER IS ISSUED PURSUANT TO: (Specify authority) THE CHANGES SET FORTH IN ITEM 14 ARE MADE IN THE CONTRACT

B. THE ABOVE NUMBERED CONTRACT/ORDER IS MODIFIED TO REFLECT THE ADMINISTRATIVE CHANGES (such as changes in paying office, C. THIS SUPPLEMENTAL AGREEMENT IS ENTERED INTO PURSUANT TO THE AUTHORITY OF:

D. OTHER (Specify type of modification and authority) appropriation date, etc.) SET FORTH IN ITEM 14.

E. IMPORTANT: Contractor is not, is required to sign this document and return

ORDER NO. IN ITEM 10A.

1 1

0002 10/25/2017

AAQ-811 FAC. & AVIATION SAFETY ACQ

FAA AERONAUTICAL CENTER

PO BOX 25082

MPB ROOM 380

AAQ0811-AFN

OKLAHOMA CITY OK 73125

AAQ-811 FAC. & AVIATION SAFETY ACQ

FAA AERONAUTICAL CENTER

PO BOX 25082

MPB ROOM 380

AAQ811-AFN

OKLAHOMA CITY OK 73125

x

14. DESCRIPTION OF AMENDMENT/MODIFICATION (Organized by UCF section headings, including solicitation/contract subject matter where feasible.)

separate letter or telegram which includes a reference to the solicitation and amendment number. FAILURE OF YOUR ACKNOWLEDGEMENT TO BE RECEIVED AT

THE PLACE DESIGNATED FOR THE RECEIPT OF OFFERS PRIOR TO THE HOUR AND DATE SPECIFIED MAY RESULT IN REJECTION OF YOUR OFFER. If by virtue of this amendment you desire to change an offer already submitted , such change may be made by telegram or letter, provided each telegram or letter makes reference to the solicitation and this amendment, and is received prior to the opening hour and date specified.

copies of the amendment; (b) By acknowledging receipt of this amendment on each copy of the offer submitted ; or (c) By ___________ x x

6973GH-18-R-00014

copies to the issuing office.

10/25/2017

(x)

Amendment 0002 is issued to provide responses to vendor's questions and changes / corrections to the SIR.

VENDOR'S WILL BE ALLOWED TO SUBMIT ADDITIONAL QUESTIONS / REQUEST FOR CLARIFICATIONS NO

LATER THAN 5 PM (CT) FRIDAY, DECEMBER 8, 2017. AFTER THIS DATE, NO ADDITIONAL QUESTIONS /

CLARIFICATIONS WILL BE ADDRESSED.

PROPOSAL DUE DATE REMAINS: 5:00 pm (CT), JANUARY 3, 2018.

All other terms and conditions remain the same.

16A. NAME AND TITLE OF CONTRACTING OFFICER (Type or print)15A. NAME AND TITLE OF SIGNER (Type or print)

15C. DATE SIGNED 16B. CONTRACT AUTHORITY 15B. CONTRACTOR/OFFEROR 16C. DATE SIGNED

(Signature of person authorized to sign) (Signature of Contracting Officer)

Connie M. Houpt

Except as provided herein, all terms and conditions of the document referenced in Item 9 A or 10A, as heretofore changed, remains unchanged and in full force and effect .

6973GH-18-R-00014 Amendment 0002, SIR changes:

1) Sec F, Clause AC1604 CONTRACT PERIOD (JUN 2015) - The effective period of this contract is 3 years from date of award whichever is later, or as maybe extended by options issued hereunder. (SIR pg 11)

ACTION: delete “whichever is later”

2) Sec G, Clause AC1401 IDENTIFICATION/DELIVERY OF GOVERNMENT PROPERTY (JAN 1997) - Within 30 calendar days after award of contract the Government-owned property listed in the Statement of Objectives (SOO) will be furnished to the contractor for use in the performance of this contract. (SIR pg 12)

ACTION: change “30 calendar days” to “6 months”; change “Statement of Objectives” to “Statement of Work”

3) Sec L, Clause 3.2.2.3-20 ELECTRONIC OFFERS (JUL 2004) - (a) The offeror (you) may submit responses to this SIR by the following electronic means email Your offer must arrive at the place and by the time specified in the SIR. (SIR pg 40)

ACTION: add after “email” the following “or surface mail (CD/DVD-ROM or USB Flash Drive with no .zip files included).”

4) Sec L, Clause 3.2.2.3-38 REQUIREMENTS FOR CERTIFIED COST OR PRICING DATA OR OTHER INFORMATION (JUL 2010) - ALTERNATE I (JUL 2010) (SIR pg 41)

ACTION: delete paragraph “(c)”.

5) Sec L, paragraph L.4 INSTRUCTIONS FOR PREPARATION AND SUBMISSION OF PROPOSALS– paragraph “(5) Proposals will be submitted electronically in accordance with Section L, Provision 3.2.2.3-20. All volumes should be submitted in electronic format via CD/DVD-ROM or USB Flash Drive with no .zip files included. Each volume should be submitted in separately marked electronic files. (SIR pg 45)

ACTION: delete second paragraph “All volumes should be submitted in electronic format via CD/DVD-ROM or USB Flash Drive with no .zip files included”.

6) Sec L, paragraph L.4 INSTRUCTIONS FOR PREPARATION AND SUBMISSION OF PROPOSALS– paragraph “(d) Technical Proposal – Volume II – (1) The technical proposal must be sufficiently detailed to enable technically oriented personnel to make a thorough evaluation and to arrive at a sound determination as to whether the proposed services meet the requirements of the Government and that the offered approach is valid and practical. The technical proposal must be specific, detailed and complete to clearly and fully demonstrate that the offeror has a thorough understanding of the requirements for, and the technical problems inherent in, providing services of the scope outlined in the Statement Of Objectives (SOO). Clear evidence of services previously demonstrated and currently in place relating to the factors and sub-factors should be included in each evaluation area.

ACTION: change “Statement of Objectives” to “Statement of Work”

7) Sec L, paragraph L.4 INSTRUCTIONS FOR PREPARATION AND SUBMISSION OF PROPOSALS– paragraph “(e) Cost/Price Information – Volume III – (2) Describe in detail for each cost element for each CLIN:

(A) Test delivery

(B) Psychometric services (everything except test delivery costs),

(C) Cost per test to delivery to the military

(D) applicant test fee amount.

When items (a) through (d) are added together the sum should include all costs for that year.

(SIR pg 46)

ACTION: change “When items (a) through (d) are added…” to “When items (a) and (b) are added…”

8) Sec L, paragraph L.5 Technical Proposals – Sub Factor 2.1 Worldwide Web-based Item and Test Development Process. (SIR pg 47)

ACTION: change to “Item and Test Development Workflow”.

9) Sec L, paragraph L.5 Technical Proposals – ‘Sub Factor 3.3 Test Administration - The Offeror shall describe the test administration process it intends to use to comply with the requirements of the contract including applicant identification, test environment, proctoring process and its test proctoring/monitoring process.” (SIR pg 49)

ACTION: change “test environment, proctoring process and its test proctoring/monitoring process.” to “test environment, test authorization validation and its test proctoring/monitoring process.”

10) Sec M, paragraph M.3 Technical Evaluation – (2) Factor 2 Item and Test Development Process, (C) Sub Factor 2.3 Random Test Generation - The FAA will evaluate the Offeror’s past experience using random test generation, evaluate the method of random test generation it intends to provide to meet the requirements of this contract and determine the Offeror’s capability of meeting the contract requirements.

For a “Satisfactory” rating the Offeror must provide evidence that it has several five years’ experience with random test generation. The proposed random test generation method meets the contract timeline and accommodates a variety of test blueprints.

ACTION: delete “several” in second paragraph.

11) Sec M, paragraph M.3 Technical Evaluation – (3) Factor 3 Test Delivery System, (A) Sub Factor 3.1 Web-based Test Delivery – “The FAA will evaluate the Offeror’s ability to provide webbased delivery of sample tests and deliver tests via the web to domestic and international sites provided and managed by the Vendor and sites that are not provided or managed by the Vendor.

For a “Satisfactory” rating the Offeror must provide evidence that it has five years’ experience with random test generation. The proposed random test generation method meets the contract timeline and accommodates a variety of test blueprints.”

ACTION: replace second paragraph with: “For a satisfactory rating, the Offeror must describe their capabilities for providing worldwide web-based test delivery to sites it is directly responsible for and sites it only provides oversight for. Test security is assured and supplemental material presentation capabilities are equal at all types of sites. Sample tests are delivered to all web-accessible computers."

Document Reference Question Question Response

SOW 8.1

How many individual candidate records will be transferred? How much data (expressed in GBs or TBs) does the transfer represent?

ACTS SOW, 21.1.3: 3 years of applicant test data will be furnished; there are on average 100,000 tests per year. (We don't have the GB or TB on this data.)

SOW 8.1 How much applicant data (expressed in GBs or TBs) will be transferred?

ACTS SOW, 21.1.3: 3 years of applicant test data will be furnished; there are on average 100,000 tests per year. (We don't have the GB or TB on this data.)

SOW 8.1 How much test content data (expressed in GBs or TBs will be transferred?

ACTS SOW, 21.1.1 and ACTS SOW, 21.1.4: about 14,000 active Domestic questions, about 11,000 International questions, and about 5,000 archived Domestic questions (Archived questions will be determined by the date they were archived; we will go back 2 years from the transfer month to accommodate the reconstruction of airman knowledge tests for the duration of a test's validity period. (We don't have the GB or TB on this data.)

SOW 8.2.3.1.c.1 Does the proctor qualification to be a U.S. citizen or legal resident alien of the U.S. extend to all personnel working on the contract?

Section 8.2.3.1.c. are only for proctor requirements

SOW 8.2.3.1.c.2 Please specify the "background check" that the vendor is required to conduct for personnel supporting the contract. A 5 year felony background check

SOW 8.2.5

We understand that the Vendor will provide an item development platform and the training on the tool; however, will the FAA continue to write, edit , and board/review items, and manage SMEs (including recruiting, contracting, compensation and performance management)?

The FAA will continue to write, edit, and board/review items, and manage SMEs

SOW 8.2.7

Could the Government please identify and define all the FAA roles and the pertinent data elements to be reflected in the dashboard?

Dashboard roles include specific roles and data elements for identified SMEs. For example, PVT Bank manager, AMT bank manager, UAS bank manager, statistician, editor, test delivery program manager, military program manager, etc etc.

SOW 8.2.2

Could the government please clarify the minimum pre-registration requirements to validate and accept a walk-in candidate for testing?

All test registration requirements must be met regardless if the applicant is a walk-in or schedules ahead of time. Registration requirements are outlined in section

8.2.2 of the SOW.

SOW 8.2.2 Is there a separate application for each exam? Each applicant must go through the registration process before each exam

SOW

SOW 8.4 SIR Section L.5, Sub Factor 2.3

Regarding "Random Test Generation," described in this section. Please confirm that the FAA is defining "Random Test Generation" as examinations dynamically built item-by-item from an item bank, at the moment of delivery to the candidate (i.e., Linear-on-the-Fly Testing), and not fixed test forms that are randomized.

Yes, that is correct

SOW SOW 1.0

Unlike traditional airman, 14 CFR 107.65 (b) requires these individuals to retest every two 2 years , what is the likelihood that the retest will be a standard proctored test administered by the vendor under this contract?

Yes, there will be a required retest every two years unless further information is provided.

SOW SOW 8.2.3.3.b please clarify i.e. The FAA understands the Vendor will incur costs “adopting” these third party sites and monitoring their compliance with the Vendor’s requirements and the requirements of this SOW. The Vendor will not use funds collected from other test sites in accordance with this SOW to pay fees incurred at U.S. Military test sites.

The vendor will provide the FAA's current military sites with the vendor's delivery system. Currently, military sites utilize the FAA’s test delivery platform, at no cost to the military or the airman applicant. The vendor will not build the cost of providing military testing into the fees charged to other (non-military) test applicants. Any fees charged to military applicants by the vendor should only be to recoop the costs of the vendor.

SOW SOW 8.2.6

Capability to increase sample tests, types of sample tests, and sample test items, are these practice tests that are to be published and delivered by the vendor? How many types of sample tests, sample test types, and sample items are there currently?

Yes, practice tests would be published and delivered by the vendor and available on the web (not for test site delivery). Sample tests reflect the same number of questions as the active tests; however, we do not currently have the lower volume tests available. It actually could be up to the full number of different tests we deliver.

SOW SOW 8.3

please confirm that the scope of work includes the 14 domestic test banks comprised of 14,000 items, but not the international test banks comprised of 11,000 items.

Yes, it includes domestic tests only. International test delivery would be contracted with AFS-50 and/or the Department of State.

SOW SOW 8.3

please clarify the vendor’s role, if any, in improving supplemental test materials such as Computer Testing Supplements, Learning Statement Reference Guide; and Airman Certification Standards.

Vendor will assess and support test types and associated supplemental material as stated in 8.3.1 and on page 4

SOW SOW 8.3.1

please clarify the vendor’s role in assessing each of the 89 test types and 14,000 items. Is the vendor responsible for leading the development of new content, or is the work scope limited to evaluating the test items and making recommendations for the FAA to develop new content?

The vendor is expected to support the FAA's development, assessment, maintenance, and enhancement of test items, tests, and supplementary test materials, with automated tools and acdemic expertise

SOW SOW 8.3.2

only 13 test types are listed. Is the RTG – Random Test excluded from the SOW to conduct task analysis and update test blueprints?

Task analysis and update test blueprints are not necessary on RTG/RTP/RTA exams.

SOW SOW 8.3.4 will the FAA provide meeting facilities and coordinate SME participation in the task analyses? Yes

SOW Introduction FAA has described, especially in paragraphs 2 and 3, the current airman certification test scheduling and delivery process.

Can FAA also provide bidders with a process flowchart that clearly depicts the overall process and the communication links between the various organizations?

No, the content discussed in paragraphs 2 and 3 of the SOW pertain to the current process which is going away as a result of this contract.

SOW section 1.2 page 3 it is stated that the current FAA airman test development system is very tenuous due to its age and vulnerabilities and that its capabilities are very limited and it cannot support the academic innovations AFS-630 must employ. Could FAA provide clarification of the academic innovations that are to be employed?

Please see section 2.0 Scope of Work 6. on page 5 "Item and exam assessment services for FAA airman knowledge form tests, test types, and test banks that employ psychometric techniques widely accepted by the academic community."

SOW 2.0 scope of work page 4 it is stated that "All computer systems are required to be compliant with the Federal Information Security Management Act of 2002 (FISMA) as described by the Vendor using the Memorandum of Understanding (MOU) for System Interface template in Appendix 7." Are offerors required to complete and submit this MOU with our proposal, or is this to be completed after award?

This will be completed upon contract award with winning vendor

SOW 2.0 page 5 next to the last paragraph, it is stated that this is a no-cost contract to the government, and that the “Vendor is expected to cover its costs and any profit by charging reasonable testing fees to applicants.” Can the FAA explain how the vendor selected will cover its costs during the transition phase, i.e., the first 18 months of the contract, when the vendor is not administering tests?

As it states in 2.0 page 5, "the vendor is expected to cover it's costs and any profity by charging reasonable testing fees to applicants." The vendor will start being able to cover it's costs from the transition phase once test delivery starts.

SOW

2.0 page 5 "reasonable testing fees" there are several elements that impact the determination of what would be considered a reasonable fee for an exam, including how long the exam session is, i.e., how long the applicant will need to consume test center workstation time and proctor time to complete the exam, as well as how many exams are given each year. While FAA has provided 2016 exam volumes in Appendix 3 – Test Types, we do not find an indication of exam session times allowed. Can FAA also provide the allowable test times for each exam in a table listing such as the one in Appendix 3?

Please see Test Time Table attachment

SOW page 5 it is stated that the FAA reserves the right to set a maximum testing fee of $160 per test for this contract and will re-evaluate this fee every 3 years. We have noticed, however, that higher FAA exam fees, such as $165, are listed on the sites of your current testing services providers. Can FAA please explain this discrepancy with the maximum fee quoted in the SOW, and if the $160 fee listed in the SOW will be changed to reflect the current fees posted on the testing services providers’ web sites?

There is no discrepancy. Current testing providers set a reasonable testing fee based on current FAA requirements which are different than what is indicated in the SOW. The FAA will not change the SOW maximum exam fee to reflect current testing provider fees based on current FAA requirements.

SOW testing fees it is not clear if amounts from the FAA exam fees the selected testing services provider will collect will be earmarked toward paying costs outside the testing provider’s services such as FAA costs, deductions allowed for certain applicant memberships, etc. Can FAA provide bidders with a clear picture of the proposed fee structure, including any amounts the testing services provider must distribute from the applicant fees it collects to pay for items that are not part of the testing provider’s services and what organizations receive compensation from the fees?

The FAA expects the vendor to cover it's costs using the exam fee collected from the applicant as indicated on page 5 of the SOW.

SOW

Section 8.0, Requirements, Phase 1 Data Transfer can the FAA provide more information about the size of the applicant dataset and historical data that will need to be transferred within 6 months of contract award? Is there a need to import applicant data? Will the selected vendor need to enforce retake rules for testing through the previous vendor?

ACTS SOW, 21.1.3: 3 years of applicant test data will be transferred; there are on average 100,000 tests per year. Yes, retake rules must follow the applicant.

SOW Phase 1, Data Transfer will there be open applicant testing appointments that will need to be managed between the current testing services provider(s) and the selected testing services provider during this transition phase?

No

SOW

Phase Two: Test Delivery, requirement 8.2.1.2 to provide reasonable accommodation for testing applicants with disabilities in compliance with applicable law, including the Rehabilitation Act of 1973 and the Americans with Disabilities Act (ADA), could FAA provide bidders with the number of applicants that have been granted accommodations in the last two years, and the type of accommodations they have been granted?

The FAA does not retain data on special accommodations testing. The FAA currently has provisions for applicants with learning disabilities in which approved applicants can choose one of three options: 1. Extented time for exam 2. Someone to read the exam 3. Use of a Franklin Speaking Master during test

SOW

8.2.1.5, and establishing and activating proctor access does the current test delivery provider only provide test administration updates to the IACRA system via file transmission, or can authorized proctors update and maintain applicant records directly in the IACRA system at the test site?

The data provided to IACRA orginates from the current FAA database. Current test delivery providers do not access the FAA's IACRA system. Test proctors will not maintain any applicant information in the IACRA system.

SOW 8.2.1.5, and the IACRA system can FAA please provide the technical specifications for the IACRA system, including required integration points as well as user roles and capabilities to manage access to the system?

This will be discussed and provided to the vendor at contract award once the technical specifications have been finalized.

SOW 8.2.1.6

do the current test center administrators or proctors review applicant documentation and hand it back to the applicants, or do they scan the documents and retain them in an electronic format to return to FAA in the data exchange? If so, how long does the provider retain them? If not, what applicant and testing data do the test sites exchange with the certification support system (IACRA)?

Applicant documentation requirements vary by the type of exam. Generally, applicant documentation is copied and retained by the testing provider for 5 years. Testing centers currently do not exchange data with the IACRA system.

SOW 8.2.3.2, bullet p can FAA describe what type of calculators would need to be used by applicants during the exams, and if they vary by exam?

The FAA allows all models of aviation-oriented caculators to be used during a test.

Applicants may use navigation computers and electronic or mechanical calculators that are related to the test. Electronic calculators incorporating permanent or continuious type memory circuits without erasure capability is prohibited.

SOW 8.2.3.2, bullet q could FAA provide samples of the pass and fail score reports that are printed at the conclusion of each test?

Please see current sample test report attachment. Please note that current test report requirements are different than what is stated in SOW.

SOW 8.2.3.2, bullet q (x), can FAA elaborate on the display of the LMS codes and the ACS codes on score reports, i.e., what the largest amount of information could be included on the report and how many pages that would require? Also, could FAA elaborate on the ACS code replacement of the LMS codes, and how soon this will be accomplished for the whole program?

Currently codes (even with all questions missed) do not require more than one page be printed. Codes are listed as PLT101, PLT325, PLT . . . Etc. We are currently not using the ACS codes, which will format as PA.III.A.K12b as a maximum size.

These will be comma separated and listed one time for possibly multiple missed questions using the same code. We don't expect this will require more than one printed page. We have several certifications (PAR - Private Pilot Airplane, CAX - Commercial Pilot Airplane, etc.) that can be initiated immediately. Others will be phased in as the ACS documents activate. This will most likely be spread out for several years, and will be activated by test type not by bank.

SOW 8.2.3.2, bullet t are the sample tests part of the no-cost contract, and if so, can FAA provide the number of sample tests that are taken per year and how many that are delivered at test sites and at home? Do current testing services providers charge an administrative fee to applicants if they choose to take a sample test at a test site?

Sample tests are part of the ACTS contract. The FAA and current test providers do not deliver sample tests. The FAA produces sample test questions that are available on the FAA website.

SOW 8.2.3.2, bullet u requires updates to the FAA certification system (currently IACRA) following the conclusion of the test. Can the FAA please provide any specifications regarding the file layouts for the data to be provided, including any required formats?

This will be discussed and provided to the vendor at contract award once the technical specifications have been finalized.

SOW 8.2.3.3, bullet a can the FAA provide bidders with an Excel spreadsheet listing applicant testing volume by the current ODA test site locations or by applicant zip code for the last 2 years?

No

SOW 8.2.3.3, bullet b it is unclear whether FAA requires the testing service provider to deliver AKT tests to military service members free of charge or at a discounted rate. Can FAA explain the current fees military members pay to FAA and/or to the testing service providers/ODA holders, if any, and if they do, what typical discount on the fee is offered to them? If they do not, are these fees covered by the DoD or the applicant’s service branch? Also, are family members of military personnel able to take AKT tests free of charge or at a discounted rate?

Currently, FAA tests are provided at no cost to eligible military applicants to test through the FAA's own test delivery platform. The FAA does not charge any of the military branches for this service; therefore, test applicants are not reimbursed by the DoD/military. FAA exams are also provided at no cost to: dependents (i.e., spouse, child(ren)) of active-duty, guard, and reserve component personnel of the United States Air Force), United States Army, United States Coast Guard, United States Marine Corps, and United Stated Navy (USN); United States military retirees; Department of Defense (DoD) civilians; and Department of Homeland Security (DHS) civilians. However, United States military retirees’ dependents, airman applicants in Individual Ready Reserve (IRR) status, and foreign military members are not eligible for no-cost FAA exams.

SOW

8.2.4 Item and Exam

Development please confirm that the selected vendor is not to directly develop any new test items with or without FAA-provided assistance, and that the “item and exam development” includes providing only the automated customizable workflow, tools, and associated training listed under 8.2.4.

The vendor is expected to support the FAA's development, assessment, maintenance, and enhancement of test items, tests, and supplementary test materials, with automated tools and acdemic expertise

SOW 8.2.4 does the FAA envision the vendor drafting test question content, or reviewing, editing, and managing the boarding process for content/draft items developed by SMEs?

Related in 8.3 of the SOW.

SOW 8.2.4 what percentage of the current test items does the FAA believe can be retained for use "as is" (i.e., no edits required) in the new ACTS system?

To begin with all items will be utilized in their transferred state.

SOW 8.2.4 what percentage of the current test items does the FAA believe can be retained after revision/editing/improvement? To begin with all items will be utilized in their transferred state.

SOW 8.2.4 please confirm that the vendor is not to directly conduct or facilitate the process for ensuring test accuracy and currency, and that with regard to determining test accuracy and currency, the vendor is limited to establishing a process for doing so.

It is expected that the vendor and the FAA will work in conjunction to make sure all testing content is accurate and current.

SOW Section 8.3 Phase Three Improve test bank items and exam blueprints, under 8.3.1, could FAA please state the number of test types that must be “assessed” each year?

Beginning in base year 2 of this contract the FAA is requiring two assessments be accomplished each year until all exams have been assessed. 8.3.2

SOW page 17 there is a request for “an ACS for each test type.” Does “test type” in this sentence refer to the 13 assessments listed in Section 8.3 under 8.3.2? If not, please state the total number of test types and the number of test types for which an ASC is expected each year.

There are currently 89 test types. For the first 3 years there will be 3-4 completed each year.

SOW Section 16 FAA asks that the vendor provide FAA with copies of exam authorization (listed by exam type in Appendix 4), and the Identification the applicant presented to take exam. Can FAA please clarify how the vendor captures this information and in what file format the vendor provides this information to FAA?

It is up to the vendor on how they want to capture this information

SIR L.2 Questions, page 43 will FAA please consider adding another opportunity for vendors to submit questions for clarification after FAA has responded to the initial round of questions from bidders?

Yes.

SIR L.4 (a) Is cost/price data allowed in Volume I: Contract Documentation? Yes.

SIR L.4 (b) 1 - 3

Section L requires each Volume include:

(1) A table of contents, a list of tables and figures, and a glossary of abbreviations and acronyms.

(2) ...Title Page

Would the Government consider exempting these items from the page limit in Volume II: Technical Proposal?

Those items are exempt from the page limit.

SIR

(1) Section L, Clause List, 3.2.2.3-20, Electronic Offers:

(2) Section L, L.4, Instructions for Preparation and Submission of Proposals (3) Section L, L.4, Instructions for Preparation and Submission of Proposals

There is a discrepancy among the following three sections regarding submission. Please clarify whether proposals are to be submitted via email to Bill White, or shipped to the FAA on CD/DVD-ROM or USB Flash Drive. (1) Section L, Clause List, 3.2.2.3-20, Electronic Offers: "(a) the Offeror (you) may submit responses…by the following electronic means email" and (e) "send your offer electronically to bill.white@faa.gov." (2) Section L, L.4, Instructions for Preparation and Submission of Proposals, (5) "Proposals will be submitted electronically in accordance with Section L, Provision 3.2.2.3-20."

(3) Section L, L.4, Instructions for Preparation and Submission of Proposals, (5) "All volumes should be submitted in electronic format via CD/DVD-ROM or USB Flash Drive…"

Offerors have the option of emailing submissions to bill.white@faa.gov or mailing / hand deliver a CD/DVD-ROM or USB Flash Drive.

SIR

Section M.3, Factor 2, Sub Factor 2.3

Please confirm that Vendors who do not have at least five years' experience with random test generation (i.e., LOFT) will not receive a "Satisfactory" rating for this Sub Factor.

That is correct. Vendors who do not have at least five years' experience with random test generation will not receive a "Satisfactory" rating.

SIR

Section M.3, Factor 3, Sub Factor 3.1

The current requirement reads "For a 'Satisfactory' rating the Offeror must provide evidence that it has five years' experience with random test generation. The proposed random test generation method meets the contract timeline and accommodates a variety of test blueprints." This is a repeat of Sub Factor (C) 2.3, Random Test Generation. Please confirm that this is a typo, and please provide the correct evaluation criteria for obtaining a "Satisfactory" rating for this sub factor (i.e., Sub Factor (A) 3.1, Web-based Test Delivery).

That is a typo. It should read "For a satisfactory rating, the Offeror must describe their capabilities for providing worldwide web-based test delivery to sites it is directly responsible for and sites it only provides oversight for. Test security is assured and supplemental material presentation capabilities are equal at all types of sites. Sample tests are delivered to all web-accessible computers."

SIR

Section L.5, Factor 1, Sub Factor 1.4

Are the completed "Past Performance Information Form"s to be included within our Technical Proposal under this Factor, or are the completed forms to be included as an Appendix?

The Past Performance Information Forms can be included as an Appendix.

SIR Section L.4, (e) The requirement reads "When items (a) through (d) are added together the sum should include all costs for that year." This is a discrepancy from Section B, Schedule, that reads "When items (a) and (b) are added together the sum should include all costs for that year." Please confirm which instructions are correct.

The instructions listed in section B should be "When items (a) and (d) are added together the sum should include all costs for that year."

SIR Section G, Clause AC1401

This clause reads that "within 30 days after award of contract the Government-owned property listed in the Statement of Objectives (SOO) will be furnished to the contractor for use in the performance of this contract." However, Section 21.0 of the SOW, Federal Government Furnished, states that "within 6 months of contract award the Government shall furnish..."

Please confirm the timeframe in which the Government will furnish data to the Contractor.

Clause AC1401 will be changed to read 6 months.

SIR Section L.5, Factor 2 Sub Factor 2.1 on page 47 is titled "Worldwide Web-based Item and Test Development Process." However, on page 48, Sub Factor 2.1 is titled "Item and Test Development Workflow." Please confirm the title for Sub Factor 2.1.

The title for subfactor 2.1 should be "Item and Test Development Workflow".

SIR

SIR document’s Section B – Supplies of Services/Prices, Section B – Schedule, 0001

BASE CONTRACT PERIOD –

(Years 1, 2, 3)

a. Test Delivery line, we note that there is no recognition of the time or effort required for the transition tasks. Can a line be added to recognize the transition Phase 1? Can the line a. Test Delivery be revised to reflect the fact that the selected vendor will not deliver tests until middle of Year 2 of the base contract period?

yes, agreed

SIR

Section F – Deliveries or Performance, AC1604 Contract period, page 11 can FAA clarify what the phrase "whichever is later" in the statement is referencing? Phrase "whichever is later" will be deleted.

SIR

Section F – Deliveries or Performance, AC1604 Contract period, page 11, can FAA clarify what the phrase "whichever is later" in the statement is referencing?

Under section 3.13-15

CONFIDENTIALITY OF DATA

AND INFORMATION (NOV

2016) it states that “(c) The contractor agrees to conduct formal training to make employees aware of the requirement to maintain confidentiality of data and information as required above. The contractor must obtain from each employee in connection with this contract a signed Non-Disclosure Agreement. This agreement must provide that the employee will not, during employment or anytime thereafter, disclose or use for current or future benefit of any party any of the data (to include any form of Sensitive Unclassified Information (SUI) described in FAA Order 1600.75) or information not publically available received in connection with the work under the contract.” Could the FAA please provide bidders with a copy of its typical NDA used to satisfy this requirement?

This will be discussed and provided to the vendor at contract award.

SIR

AC3212 LIABILITY INSURANCE

(JAN 1997) item a (3), page 31 if the bidder’s auto policy does not provide a per person limit, but carries a $2M combined single limit for bodily injury & property damage per accident, is that limit acceptable?

Yes.

SIR

AC3212 LIABILITY INSURANCE

(JAN 1997) item a (3) (b) if only the Commercial General Liability & Automobile Liability policies can provide additional insured status, can FAA confirm you are requesting additional insured status on the Commercial General Liability & Automobile Liability policies only?

Yes.

SIR

AC3212 LIABILITY INSURANCE

(JAN 1997), item a (3) c can this statement be amended to reflect the fact that any notice from the insurer would go to the bidder as the named insured, and the bidder would then provide the notices to the FAA?

No.

SIR

AC3212 LIABILITY INSURANCE

(JAN 1997), item a (3) (d) would it be possible to amend this requirement if a bidder does not have the ability to provide “certified” copies of policies to FAA?

No.

SIR

page 41, under the provision titled 3.2.2.3-38

REQUIREMENTS FOR

CERTIFIED COST OR PRICING

DATA OR OTHER

INFORMATION (JUL 2010) -

ALTERNATE I (JUL 2010),

paragraph (c) states “Submit the cost proposal electronically by electronic spreadsheet format. However, on page 46, based on the cost proposal instructions under paragraph (e) Cost/Price Information – Volume III, it seems Volume III should be submitted as a PDF. In what format does the FAA require the cost proposal (Volume III) to be submitted?

Paragraph (c) is deleted.

SIR

instructions on SIR page 45 and the FBO.gov notice, SIR

SIR page 45 states that “(6) Volumes II and III shall be marked 'Procurement Sensitive.' A cover sheet may be used for each volume for this designation along with the designation of the applicable page(s) the offeror deems competitive sensitive.”

However, the description on the FBO.gov page states that we should mark all documents submitted as “PROPRIETARY.”

What method should bidders use to mark information in their proposal confidential?

Vendors are urged to pull all material directly from the FAA contract opportunities website https://faaco.faa.gov. All instructions contained in the SIR supersede any other information.

SIR page 46 of the SIR instructions on page 46 of the SIR that state, “The proposal must include supporting documentation describing each subcontractor's technical qualifications and detailed pricing information to support all subcontractor costs.” Please confirm that the pricing information referenced in this statement needs to be included in Volume III and not Volume II.

That is correct. Subcontractor pricing information should be in Volume III.

SIR

SIR pages 47, 48, and 53, L.5 Technical Proposals, Sub Factor 2.1

L.5 Technical Proposals, Sub Factor 2.1 is referenced as “Sub Factor 2.1 Worldwide Web-based Item and Test Development Process” but elsewhere (pages 48 and 53) Sub Factor 2.1 is stated as “(A) Sub Factor 2.1 Item and Test Development Workflow.” Which is the correct title bidders should use for Sub Factor 2.1?

The title for subfactor 2.1 should be "Item and Test Development Workflow".

SIR

Sub Factor 3.3 Test Administration on pages 48-

“The Offeror shall describe the test administration process it intends to use to comply with the requirements of the contract including applicant identification, test environment, proctoring process and its test proctoring/monitoring process.” Is there a difference between the “proctoring process” and the “test proctoring/monitoring process” and if so, what is the difference between the two?

Sentence will be corrected to read “The Offeror shall describe the test administration process it intends to use to comply with the requirements of the contract including applicant identification, test environment, test authorization validation and its test proctoring/monitoring process.”

SIR

Regarding the submittal instructions on page 45 could the FAA please specify if the “Title Page” referenced in number (2), the “Volume cover” referenced in number (3), and the “cover sheet” referenced in number (6) are the same things?

1) The "title page" references any introductory letter or initial page of the offeror's proposal. 2) The "Volume cover" is the first page of each individual volume. 3) As an alternative for marking the entire volume "Procurement Sensitive", the "cover sheet" referenced in number (6) lists the pages of that volume that are marked "Procurement Sensitive".

SIR

regarding instructions on page 45, it is stated that the technical proposal should be no more than 100 pages. Are the following items exclusive of the 100-page limit for the technical proposal: volume cover, cover letter, table of contents, list of tables and figures, glossary of abbreviations and acronyms, and any appendices?

Yes.

SIR

L.4 Instructions for Preparation and Submission of Proposals, b.5, page 45 this requirement states vendors should submit proposals in compliance with Provision 3.2.2.3-20, which requires proposals to be submitted via email, but then goes on to require submission of proposals in electronic format on CD/DVD-ROM or USB Flash Drive with no .zip files included. Should vendors provide their proposals to FAA via email or ship them to FAA on CD/DVD-ROM or USB Flash Drive, or both?

Clause 3.2.2.3-20 and L.4(b)(5) will be amended to clarify that proposals can be submitted either via email or surface mail (CD/DVD-ROM or USB Flash Drive) with no .zip files included.

SOW 13.0

A draft QCP is required with the proposal. In which volume is the QCP required? If in Volume II, will the Government allow the QCP as an attachment not included in the page limitation?

QCP should be an attachment to Vol II and does not count against page limitation.