USCA18R0166.pdf

SOLICITATION, OFFER, AND AWARD 1. THIS CONTRACT IS A RATED ORDER UNDER DPAS

(15 CFR 700)

RATING

2. CONTRACT NUMBER 3. SOLICITATION NUMBER

USCA18R0166 SEALED BID (IFB)

X NEGOTIATED (RFP)

4. TYPE OF SOLICITATION 5. DATE ISSUED

09/07/2018

6. REQUISITION/PURCHASE NUMBER

PRO190003

CODE PMD

Procurement Management Division Administrative Office of the United States Courts One Columbus Circle, N.E.

Suite 3-250 Washington, DC 20544-0001

7. ISSUED BY 8. ADDRESS OFFER TO (If other than Item 7)

NOTE: In sealed bid solicitations "offer" and "offeror" mean "bid" and "bidder".

SOLICITATION

Linda_Williams@ao.uscourts.gov 15:00:00 10/29/2018

(Hour) (Date)

CAUTION - LATE Submissions, Modifications, and Withdrawals: See Section L, Provision No. 3-100. All offers are subject to all terms and conditions contained in this solicitation.

9. Sealed offers in original and ____________ copies for furnishing the supplies or services in the Schedule will be received at the place specified in item 8, or if handcarried, in the depository located in ______________________________________________________ until ________________ local time _________________

10. FOR

INFORMATION

CALL:

A. NAME

Linda M. Williams AREA CODE

NUMBER

502-2486

EXT.

B. TELEPHONE (NO COLLECT CALLS) C. E-MAIL ADDRESS

linda_williams@ao.uscourts.gov

11. TABLE OF CONTENTS

(X) SEC. DESCRIPTION PAGE(S) (X) SEC. DESCRIPTION PAGE(S)

PART I - SCHEDULE PART II - CONTRACT CLAUSES

X A SOLICITATION/CONTRACT FORM 1 X I CONTRACT CLAUSES 28-32

X B SUPPLIES OR SERVICES AND PRICES/COSTS 4-5 PART III - LIST OF DOCUMENTS, EXHIBITS AND OTHER ATTACH

X C DESCRIPTION/SPECS./WORK STATEMENT 6-24 X J LIST OF ATTACHMENTS 32

D PACKAGING AND MARKING PART IV - REPRESENTATIONS AND INSTRUCTIONS

X E INSPECTION AND ACCEPTANCE 24

X K REPRESENTATIONS, CERTIFICATIONS AND OTHER

STATEMENTS OF OFFERORS

32-35

X F DELIVERIES OR PERFORMANCE 24-25

X L INSTRS., CONDS., AND NOTICES TO OFFERORS 35-40X G CONTRACT ADMINISTRATION DATA 25-27

X M EVALUATION FACTORS FOR AWARD 40-42X H SPECIAL CONTRACT REQUIREMENTS 27-28

OFFER (Must be fully completed by offeror)

NOTE: Item 12 does not apply if the solicitation includes a minimum offer acceptance period.

In compliance with the above, the undersigned agrees, if this offer is accepted within _______________________ calendar days (60 calendar days unless a different period is inserted by the offeror) from the date for receipt of offers specified above, to furnish any or all items upon which prices are offered at the price set opposite each item, delivered at the designated point(s), within the time specified in the schedule.

12.

13. DISCOUNT FOR PROMPT PAYMENT 10 CALENDER DAYS(%) 20 CALENDER DAYS(%) 30 CALENDER DAYS(%) CALENDAR DAYS

NUMBER PERCENTAGE

14. ACKNOWLEDGEMENT OF AMENDMENTS (The offeror acknowledges receipt of amendments to the SOLICITATION for offerors and related documents numbered and dated):

AMENDMENT NO. DATE AMENDMENT NO. DATE

15A. NAME AND

ADDRESS OF

OFFEROR

CODE FACILITY 16. NAME AND TITLE OF PERSON AUTHORIZED TO SIGN

OFFER (Type or Print)

15B. TELEPHONE NUMBER

AREA CODE NUMBER EXT.

15C. CHECK IF REMITTANCE ADDRESS IS

DIFFERENT FROM ABOVE - ENTER SUCH

ADDRESS IN SCHEDULE.

17. SIGNATURE 18. OFFER DATE

AWARD (To be completed by Government)

19. ACCEPTED AS TO ITEMS NUMBERED 20. AMOUNT 21. ACCOUNTING AND APPROPRIATION

10 U.S.C. 2304(c) ( ) 41 U.S.C. 253(c) ( )

22. AUTHORITY FOR USING OTHER THAN FULL AND OPEN COMPETITION: 23. SUBMIT INVOICES TO ADDRESS SHOWN IN (4 copies unless otherwise specified)

ITEM

CODE24. ADMINISTERED BY (if other than Item 7) CODE AFSD Accounting Division, DAS-BAPO-ACC Administrative Office of the United States Courts One Columbus Circle, N.E., Suite 5-300 Washington, DC 20544-0001

25. PAYMENT WILL BE MADE BY

26. NAME OF CONTRACTING OFFICER (Type or print) 27. UNITED STATES OF AMERICA 28. DATE

IMPORTANT - Award will be made on this Form, or on Standard Form 26, or by other authorized official written notice.

AUTHORIZED FOR LOCAL REPRODUCTION

Previous edition is unusable

Judiciary Standard Form 33 (REV. 10-12)

USCA18R0166 - Page 1 of 45

Table of Contents

Section Description Page Number

B Supplies or Services and Prices/Costs B.2 Schedule of Items

C Descriptions/Specifications/Statement of Work C-1 FCICE Statement of Work

D Packaging and Marking E Inspection and Acceptance

2-5B Inspection of Services E-1 Inspection and Acceptance

F Deliveries or Performance 2-60 Stop-Work Order F-2 Place of Performance F-1 Schedule of Deliverable

G Contract Administration Data 3-305 Payment by Electronic Funds Transfer-System for Award Management (SAM) Registration ...24 7-1 Contract Administration 7-5 Contracting Officer's Representative 7-125 Alt I Invoices 7-125 Invoices G-1 Ordering Instructions G-2 Clause Guidance per Contract Type G-3 Invoicing Instructions 7-10 Contractor Representative

H Special Contract Requirements 1-1 Employment by the Government 7-45 Travel 2-65 Key Personnel

I Contract Clauses 1-10 Gratuities or Gifts 1-15 Disclosure of Contractor Information to the Public 1-5 Conflict of Interest 2-55 Privacy or Security Safeguards 2-80 Judiciary Property 3-105 Audit and Records 3-120 Order of Precedence 3-205 Protest after Award 3-25 Protecting the Judiciary's Interest When Subcontracting with Contractors Debarred, Suspended, or Proposed for Debarment 3-300 Registration in the System for Award Management (SAM) 3-35 Covenant Against Contingent Fees 3-40 Restrictions on Subcontractor Sales to the Judiciary 3-45 Anti-Kickback Procedures 3-50 Cancellation, Rescission, and Recovery of Funds for Illegal or Improper Activity 3-55 Price or Fee Adjustment for Illegal or Improper Activity 7-100B Limitation of Liability (Services) 7-110 Bankruptcy 7-130 Interest (Prompt Payment) 7-140 Discounts for Prompt Payment 7-15 Observance of Regulations/Standards of Conduct 7-150 Extras 7-175 Assignment of Claims 7-20 Security Requirements 7-210 Payment for Emergency Closures 7-215 Notification of Ownership Changes 7-235 Disputes 7-25 Indemnification 7-30 Public Use of the Name of the Federal Judiciary 7-35 Disclosure or Use of Information 7-40 Judiciary-Contractor Relationships 7-85 Examination of Records B-5 Clauses Incorporated by Reference 7-220 Termination for Convenience of the Judiciary (Fixed Price)

USCA18R0166 - Page 2 of 45

7-230 Termination for Default - Fixed-Price Products and Services 7-185 Changes 7-185 Alt II Changes 7-225 Alt I Termination (Cost-Reimbursement) 4-5 Ordering 4-25 Indefinite Quantity 5-30 Authorization and Consent 2-140 Judiciary IT Security Standards 6-110 Deferred Ordering of Technical Data or Computer Software 6-65 Rights in Data-Special Works 7-135 Payments 7-195 Excusable Delays 7-205 Payment for Judiciary Holidays 4-30 Payment (Time-and-Materials and Labor-Hour Contracts) 7-75 Subcontracts 4-10 Order Limitations

J List of Attachments K Representations, Certifications and Other Statements of Offerors or Respondents

3-130 Authorized Negotiators 3-20 Certification Regarding Debarment, Suspension, Proposed Debarment, and Other Responsibility Matters 3-30 Certificate of Independent Price Determination 3-5 Taxpayer Identification and Other Offeror Information

L Instructions, Conditions and Notices to Offerors or Respondents 3-210 Protests 3-85 Explanation to Prospective Offerors 3-95 Preparation of Offers 7-60 Judiciary Furnished Property or Services 3-100 Instructions to Offerors 4-1 Type of Contract B-1 Solicitation Provisions Incorporated by Reference L-1 Instructions to Offerors

M Evaluation Factors for Award 3-70 Determination of Responsibility M-1 Basis for Award

USCA18R0166 - Page 3 of 45

Section B - Supplies or Services and Prices/Costs

CLIN NO. Supplies or Services Quantity Unit of Issue Unit Price Extended Amount

0001 Test Administration 0 Each

Extended Description:Test administration includes all necessary pre and post activities, testing materials and personnel during the test cycle to validate scoring and report results to include test locations.

Period of Performance:12/03/2018 - 12/02/2023

CLIN NO. Supplies or Services Quantity Unit of Issue Unit Price Extended Amount

0002 Public and Non-Public Websites 0 Each

Period of Performance:12/03/2018 - 12/02/2023

CLIN NO. Supplies or Services Quantity Unit of Issue Unit Price Extended Amount

0003 Developmental Tasks 0 Each

Period of Performance:12/03/2018 - 12/02/2023

B.2 Schedule of Items

The Schedule of Items (Price Tables) is provided as an attachment and is referenced in the solicitation docu-ment under Sections J and L.

Section C - Descriptions/Specifications/Statement of Work

C-1 FCICE Statement of Work

C.1. Introduction

The Administrative Office of the United States Courts (AO) requires support for the Spanish/English Federal Court Interpreter Certification Examination (FCICE). The Court Interpreters Act (Act), 28 U.S.C. § 1827, re-quires the Director of the AO to prescribe, determine, and certify the qualifications of persons serving as certi-fied interpreters in federal courts when the Director considers such certification to be merited. The Act man-dates that such certifications shall be based on the results of a criterion-referenced performance examination.

The AO’s criterion-referenced FCICE is administered in two phases – a written examination and an oral exam-ination. The oral examination must test in the three modes of court interpreting: consecutive interpretation, sim-ultaneous interpretation, and sight translation.

C.1.1. Scope

The Contractor shall deliver complete administrations of both phases of the FCICE in 2019, in accordance with current industry standards as stated in section C.2.1 of this contract and according to other contractual require-

USCA18R0166 - Page 4 of 45 ments. The administrations shall produce valid candidate scores for the certification of interpreters of Spanish/ English. Delivery shall include, but not be limited to, registering candidates for the written and oral examina-tions, administering the AO-provided written and oral exam form 5.1 at multiple locations in 2019, training of expert raters, scoring examination performances, conducting psychometric analyses, reporting examination res-ults, providing examination information to candidates and other members of the public, and performing all other activities stated herein, and requested, or agreed to, by the AO.

C.1.2. AO-Furnished Property and Information

No later than two weeks after contract award, the AO will provide the Contractor with the following:

C.1.2.1. Registration Information

• An Examinee Handbook.

• The official FCICE logo

• A description of required written and oral registration procedures, the current registration fee, and required fee payment procedures.

• A list of expected dates and locations for examination administrations in 2018.

• A self-assessment questionnaire for candidates to complete which explains the knowledge, skills and abilit-ies needed to be an effective federal court interpreter.

• Full-length, sample written and oral examinations, with scoring keys, for candidates to use in preparation for the examinations.

C.1.2.2. Examination Information

• Three operational forms of the written examination.

• Electronic examination material and instructions for computer administration of the written examination.

• A Background Questionnaire that candidates shall be asked to submit at the end of the written examination, or at a time approved by the AO.

• Four operational forms of the oral examination, including electronic media with pre-recorded test stimuli.

• Examination administration instructions that vendor testing centers and proctors shall follow to support uni-form examination administrations and to help safeguard against breaches in examination content security.

• The list of candidates who are entitled to register for the oral examination to be administered in 2018.

C.1.2.3. Examination Scoring Information for the Written Examination

• The criterion statement/standard and rating scale and criteria for certification.

• Examination score reporting procedures and time lines.

• Psychometric data from past examination administrations for each written and oral operational form (Examination Statistical Analysis Report, also known as the Technical Report and the Examination Annual Reports)

• Rating keys for the written examination.

• Procedures and timelines for providing results to candidates.

C.1.2.4. Examination Scoring Information for the Oral Examination

• The criterion statement/standard and rating scale and criteria for certification.

• Examination score reporting procedures and time lines.

• Psychometric data from past examination administrations for each written and oral operational form (Examination Statistical Analysis Report, also known as the Technical Report)

USCA18R0166 - Page 5 of 45

• Rating scoring guides for the oral examination.

• A list of raters eligible to rate the oral examination in 2018.

• Rating materials for the oral examination, including rating keys, a list of qualified and experienced raters, rater training curriculum and materials, rating instructions, transcripts of examination items, and rating sheets.

• Procedures and timelines for providing results to the AO and candidates, upon approval by the AO.

C.2. Requirements for Deliverables

C.2.1. General Requirements

The Contractor shall ensure the delivery of complete written and oral examination administrations that shall in-clude, but not be limited to, examination registration, communication with candidates, test administration, psy-chometric evaluation, and scoring. In doing so, the Contractor shall ensure that all deliverables provided, and actions taken, under this contract conform to the standards set forth in the most current version, at the time of examination administration, of the publication, Standards for Educational and Psychological Testing. All ap-provals by the Contracting Officer’s Representative (COR) under this contract will be in writing.

C.2.2. Schedule for Delivery

The COR will provide to the Contractor a schedule for the completion of all deliverables under this contract.

The Contractor or the AO may propose alternative schedule(s), which must be approved by the COR.

C.2.3. Format and Manner of Delivery

The Contractor shall submit all deliverables to the COR, in a format and manner approved by the COR, and as specified in the contract (including complying with requirements herein to keep examination and candidate in-formation secure).

C.2.4. Implementation of Deliverables

The Contractor shall not use or implement any deliverables (such as technology, plans, schedules, or materials) without prior COR approval.

C.2.5. Rights in Data

All materials, work items, and recorded information, including examination forms, examination materials, data and all other information produced during or obtained for use under this contract, shall be subject to the Rights in Data clause under Section I and must be surrendered no later than 30 days of the conclusion of the perform-ance period of this contract, or at such times as the COR or this contract require.

C.2.6. Protection of Test Data

The Contractor, and all individuals under the Contractor’s control (employees, subcontractors, individuals at test centers where the exams are administered) who have access to the FCICE examinations and related materi-als, and/or are involved in the planning or administration of the FCICE, shall be required to adhere to strict con-fidentiality and security provisions contained in the Contractor’s Security Plan and in this contract.

C.3. Security

C.3.1. General Requirement

The Contractor shall ensure security of all FCICE data, including but not limited to operational examination

USCA18R0166 - Page 6 of 45 forms and related materials and information in normal operations and storage conditions and under such cir-cumstances as fire, earthquake, power interruptions, or other similar events. In accordance with Judiciary IT se-curity policies, no examination material may be accessed from outside the United States, unless expressly ap-proved by the COR for the purposes of written examination administration. If the Contractor anticipates housing any examination content at a facility or on a system other than the Contractor’s, either temporarily or for the duration of the contract, the Contractor shall first obtain COR approval.

C.3.2 Security Procedures

The Contractor shall develop security procedures to ensure the integrity and confidentiality of all FCICE Pro-gram materials. At a minimum the security procedures shall set forth steps to ensure examination content secur-ity before, during and after administration. The procedures shall also include steps to ensure the reliability and security of accounting and record keeping systems in the event of natural disasters or theft. The Security Plan shall not be distributed, or its contents disclosed to any party without explicit approval from the COR. The AO reserves the right to inspect the Contractor’s security and storage practices at the Contractor’s site or elsewhere at any time.

C.3.3. Reporting of Security Breaches

The Contractor shall report immediately to the COR, in writing, any instances of actual or suspected breach in the security of FCICE data in the manner specified in the sections below.

C.3.4. Security of Examination and Examination-related Information

C.3.4.1. Secure Communications: The Contractor shall submit to the COR, in writing for approval, its proposed method for securely delivering documents and electronic media to the AO, examination proctors, subject matter experts and others approved by the COR in a way that ensures examination content security. To the greatest ex-tent possible, U.S. Mail, Federal Express, United Parcel Service and similar methods of delivering confidential examination material should be avoided. The Contractor shall be responsible for reimbursing the judiciary the costs to redevelop any examination material that is lost, damaged or otherwise compromised during shipping via U.S. Mail, Federal Express, United Parcel Service and similar methods.

C.3.4.2. Examination Handling: The Contractor shall limit the risk of exposure of the examination or materials to unauthorized parties during storage, and before, during and after administration of the examination.

C.3.5. Examination Administration Security

C.3.5.1. Candidate Access to Examination Content: The Contractor shall ensure that candidates do not have ac-cess to equipment or material that could be used to assist in taking the examination or that could be used to re-cord or transmit any portion of the examination. Security measures that the Contractor shall take include, but are not limited to, ensuring that:

(1) only authorized candidates shall have access to examination facilities, equipment, and materials,

(2) only trained and authorized proctors shall have access to the examination content,

(3) candidates shall not be permitted, or allowed the opportunity, to view or record in any physical or electronic manner any examination materials, the computer monitor, and/or written notes of any other examination candid-ates, and

(4) candidates and proctors shall not be permitted to remove from the examination center, in physical or elec-tronic form, any examination materials, including notes, images, audio and/or visual recordings.

C.3.5.2. The Contractor shall publish these security requirements in all materials provided to potential candid-ates.

C.3.5.3. Any candidate who is found to be in breach of any part of the security requirements shall be perman-

USCA18R0166 - Page 7 of 45 ently disqualified from taking any examination in the FCICE program. In the event of breach, the Contractor shall promptly advise the AO in writing, of the breach, the circumstances, and the identity of the candidate. The Contractor shall provide a cumulative list of disqualified candidates on all subsequent Test Access lists.

C.3.6. Failure to Maintain Test Security

If the Contractor, its agents, employees, subcontractors, or others acting on its behalf cause a breach in examin-ation data security that compromises the examination form(s) to such an extent that all or part of the form(s) cannot be fairly administered to future test candidates, then the Contractor shall pay liquidated damages equal to the cost of replacing the compromised examination form(s) (or parts thereof) that were compromised, including the development, testing, and deployment of the new examination form(s) or parts thereof.

C.4. Preparation for Test Administration

C.4.1. Project Management Plan

C.4.1.1. Delivery: No later than thirty (30) days after contract award the Contractor shall submit a Project Man-agement Plan, for COR approval. The Project Management Plan shall set out how the Contractor shall manage the entire administration of the written examination, including the key personnel, their responsibilities for spe-cific tasks during each stage of the examination process, and the Contractor’s proposed procedures to ensure the quality and security of all exam material for both the written and oral test administrations, and that those admin-istrations produce examination valid results for the purpose of certifying interpreters of Spanish/English. The requirements for the components of this report are listed below. No other work shall proceed or be reimbursed or otherwise compensated until the Project Management Plan is received and approved by the COR. If the AO directs the modification of any portion of the Project Management Plan, the Contractor shall submit a revised Project Management Plan that will not become effective until approved by the COR. The Project Management Plan and its component parts shall not be distributed, or its contents disclosed to any party without explicit ap-proval from the COR.

C.4.1.2. Quality Assurance Procedures: The Contractor shall develop and follow Quality Assurance procedures that, at a minimum, ensure the ongoing professional review of the integrity, operational validity, and effective-ness of the written and oral examinations; and an operational design and procedures for accomplishing this on-going technical review.

C.4.1.3. Dispute Resolution Procedures: The Contractor shall develop a dispute resolution procedure no later than thirty (30) days after award that addresses candidate complaints and appeals regarding administration of the oral and written examinations. This plan shall identify those portions of the dispute resolution procedures that are for public dissemination and those that are confidential. Upon COR approval of the dispute resolution procedures, the Contractor shall post the public portion of the procedures on the FCICE public website. The Contractor shall address candidate complaints and appeals concerning the written and oral examinations in ac-cordance with the approved dispute resolution procedures. Confidential portions of the dispute resolution pro-cedures shall not be distributed to any party without explicit approval by the AO.

C.4.1.4. Accommodations Procedures: The accommodations procedures shall describe how the Contractor will address candidate requests for special accommodations for reasons of disability or otherwise. The Contractor shall consider and approve or deny accommodations requests on a case-by-case basis. The Contractor shall promptly inform the AO of any requests for accommodations that are not addressed in the approved accommod-ations procedures and propose a resolution for COR approval. The Contractor shall obtain COR approval before administering any examination with accommodations when those accommodations would incur additional costs to the AO. If the Contractor does not obtain prior COR approval, the Contractor shall be responsible for the costs of those accommodations. The accommodations procedures shall not be distributed to any party without explicit approval by the COR.

C.4.1.5. Registration Procedures: The Contractor shall propose registration procedures consistent with the

USCA18R0166 - Page 8 of 45 guidelines that were provided to it by the AO. The Contractor, at that time, shall also submit to the AO a plan for collecting and accounting for written and oral examination registration fees in accordance with the Court In-terpreters Act at 28 U.S.C. § 1827(g)(5). The fee is established by the AO Director pursuant to the Act. No ad-justments to fees for special circumstances or for candidates who participate in field testing can be made without AO approval. Such adjustments are rare, and if made, are allowed on a case-by-case basis only.

C.4.1.6. Communications Procedures: The Contractor shall propose communication procedures to directly re-spond to questions from prospective and actual candidates on topics relating to the FCICE examination process, the status of FCICE examination results, and other inquiries directly related to examination administration. All questions concerning other aspects of the FCICE shall be referred to the COR. The Contractor shall not make any promises, representations, or suggestions to candidates, members of the public, staff, or subcontractors that are not in strict conformance with this SOW and the FCICE’s Examinee Handbook.

C.4.2. FCICE Registration and Information Website

C.4.2.1. For the unit price, the Contractor shall develop a public website, or use an existing website owned and controlled by the Contractor, to develop a webpage on that site, that permits safe, user-friendly, free public ac-cess to FCICE Program information and registration for examinations. FCICE information shall be available on the webpage for online use and download at no charge to the public.

C.4.2.2. The Contractor shall submit for COR approval, no later than six weeks prior to registration for any ex-amination, a schedule for the development and deployment of the website. The Contractor’s schedule shall provide at least two weeks for the COR to access and test the website before it is launched or updated, and if updated, deliver a report of individual pages that are modified no later than ten (10) business days after updates are complete. The Contractor shall launch the site no later than 2 weeks before any registration period begins.

C.4.2.3. The Contractor shall ensure that the website allows for registration and secure payment for the appro-priate phase of the FCICE examination in a single transaction.

C.4.2.4. The Contractor shall ensure that the site is publicly accessible 24 hours a day, 7 days a week, with no more than 12 hours of interruption in service per week, including maintenance. The Contractor shall operate and maintain the website for the duration of this contract to ensure continuous online access to the public, including, but not limited to: implementing software patches, fixing problems, bugs, viruses, and/or reported issues, and troubleshooting and resolving all software problems.

C.4.2.5. The Contractor shall ensure that the website is secure and that personally identifiable information sub-mitted by the candidates through the test registration process is protected from theft or other breaches in pri-vacy.

C.4.2.6. The Contractor shall ensure that all material posted on the public website, as well as the website itself, use the official FCICE logo provided by the AO, and have a consistently branded, professional appearance, which shall be approved by the COR prior to launch.

C.4.2.7. The Contractor shall post the following FCICE information on the website. The information shall be updated at the request of the COR, and only with COR approval:

(1) The criterion statement/standard, rating scale and criteria for certification.

(2) The self-scoring inventory (self-assessment questionnaire) of knowledge, skill and ability factors related to the ability to be an effective interpreter in the federal court system.

(3) The full-length sample of both the written examination, with printable samples of the same difficulty level, item type and equivalent content as the operational examination forms.

(4) Scoring keys for the sample questions/examinations.

(5) The registration dates for the written and oral examinations.

(6) The written and oral examination administration dates and locations.

USCA18R0166 - Page 9 of 45

(7) A registration form that can be submitted to the Contractor online, and instructions for submittal.

(8) Information on registration fee and fee payment procedures.

(9) Examination score reporting procedures and time lines.

(10) References to additional resources related to court interpreting.

(11) A Background Questionnaire to be submitted to the Contractor by candidates before taking the written ex-amination.

(12) Test security information and policies relevant to the public.

(13) An Examinee Handbook, making all materials on the FCICE Program information website available in one document.

C.5. Examination Sites

C.5.1. Examination Locations

C.5.1.1. Four weeks before written or oral examination registration, the Contractor shall submit, for COR ap-proval, the Test Site List Report based on geographic locations recommended by the AO that best accommodate the needs of the candidates and the program.

C.5.1.2. The COR shall approve all test site locations and suggest additional or alternative locations, if needed.

For the written examination, there will be up to220 examination sites. (The Test site list is provided as a Sec-tion J attachment.) For the oral examination, there will be up to 20 examination sites.

C.5.2. Examination Facilities

C.5.2.1. The Contractor shall use appropriate facilities for the written and oral test administrations. All facilities are subject to approval by the COR.

C.5.2.2. As part of the unit price, the Contractor shall furnish all test locations with the staff, equipment, and other materials needed to properly administer each examination.

C.5.2.3. The COR and other AO staff may visit any testing facilities prior to, or during any test administration.

Visitors shall follow all Contractor protocols and requirements to avoid disruption to any testing process.

C.6 Equating of FCICE Examinations and Resulting Reports

C.6.1. Psychometric Services

The Contractor shall utilize the services of a psychometrician, identified in the project management plan, to equate and assist in the scoring of all FCICE written and oral examinations.

C.6.1.1. The Contractor shall substantiate the validity and reliability of all administered examinations for the certification of interpreters of Spanish/English. Emphasis shall be placed on determining and reporting the psy-chometric characteristics of the examinations used in the certification program.

C.6.1.2. The Contractor shall also perform detailed statistical analyses and other types of empirical research to substantiate the validity and reliability of the examination administrations and results.

C.6.1.3. Throughout the duration of the contract, the Contractor may also propose any new or alternative statist-ical analysis methods or other types of empirical research to accomplish these purposes. The Contractor may re-commend, for COR approval, that forms be equated prior to examination administration. In such case, the Con-tractor shall recommend to the COR the number of test forms to be administered.

C.6.2. Annual Report

USCA18R0166 - Page 10 of 45

No later than twelve weeks after the final test is administered under this contract (whether written or oral), the Contractor shall provide the COR with a report that contains, at a minimum, the following information:

• all program-related activities conducted by the Contractor since the execution of this contract;

• the reports delivered to the AO during the performance year (Project Management Plan, Test Site List, Score Report, etc.);

• statistics on the test administrations;

• candidate demographic data;

• all passing and failing examination scores (raw as well as converted); and

• the Test Access List Report. The Contractor shall maintain a list of the names of all individuals who have had access to the examinations, test items, or any other program-related materials not expressly available to the public, specifying for each individual the materials accessed, and shall provide this Test Access List Re-port to the AO prior to and after the administration of each test. The Test Access List Report shall also in-clude a separate cumulative list of any candidates disqualified at any time for security breaches.

C.6.3. Technical Report

No later than 20 weeks following the administration of the written or oral examination the Contractor, with the assistance of a psychometrician, shall determine and report the following characteristics of each of the tests ad-ministered:

• Range, standard deviation, and average of r-biserial correlations of all items in the tests, against a whole-test and test-part criterion;

• Internal (split-half test reliability coefficient);

• Correlation of each test part with the total test score and with each of the other test parts;

• If exams are equated after the test administration, procedures used for statistical results of the form-to-form equating process used (common-item or other technique); and

• Procedures used for and statistical results of the form-to-form equating process used (common-item or other technique).

C.7. Registration for Written and Oral Examinations

C.7.1. Registration and Examination Dates

The AO will provide the Contractor with the date(s) within which the Contractor shall conduct:

(1) registrations for the FCICE written and oral examinations; and

(2) administrations of the written and oral examinations to registered candidates.

If the Contractor or the COR seeks to modify the registration and/or examination window(s), it must obtain COR approval.

C.7.2. Online Registration

C.7.2.1. Examination Registration: The Contractor shall register individuals seeking to take both FCICE exam-inations, written and oral, online through its public FCICE website during the registration window. The Con-tractor shall ensure that it:

(1) Obtains a fully completed registration form from each candidate,

(2) Obtains each candidate’s preferred test dates and times within the testing window, and preferred test location of the test sites offered,

(3) Collects the $210 written examination fee and the $310 oral examination fee elec-tronically from each candidate in a secure manner approved by the COR before assign-ing a test date, time, and location to the candidate, and

(4) Not collect a registration fee from candidates for the oral examination re-test

USCA18R0166 - Page 11 of 45

(5) Does not issue refunds of fee collections, or refuse to give candidates refunds, without prior approval by the COR. The COR will describe any refund policies with which the Contractor shall comply.

(6) Ensure that each candidate provides proof of identity (copy of passport, driver’s li-cense, or a U.S. or state government-issued identification card) at the test location.

C.7.2.2. Test Equipment Tutorial – Written Examination Only: Prior to the written test examination date, the Contractor shall afford each candidate the opportunity to take hands-on written examination tutorials, admin-istered on the examination equipment, that explains how to use the equipment, including navigation, input devices, revising a selection, marking items for review, and returning to previous items for review or change.

For candidates who wish to take the tutorial prior to arriving at the test center, contractor shall provide access to the tutorial on the FCICE public website.

C.7.2.3. Oral Examination Registration: For 2019, in addition to the regular, cyclical administration of the oral examination, there shall be a re-test oral examination administration for candidates that experiences examina-tion irregularities during the 2017 administration. For the re-test, the Contractor shall register only those candid-ates who contact the Contractor and who are on the list that the AO provides the Contractor upon execution of this Contract.

C.8. Administration of Written Examination

C.8.1. Eligible Candidates

The Contractor shall administer the FCICE written examination to all candidates who properly registered for the examination and provided appropriate identification at the examination location on the date and at the time the candidate was scheduled for the examination. Appropriate identification shall consist of United States govern-ment-issued identification, or a foreign passport.

C.8.2. Electronic Written Examination Administration

The Contractor shall ensure that all written examinations are administered electronically in a reliable, standard-ized, and fair manner in accordance with the requirements of this contract and industry standards.

C.8.3. Written Examination Administration Features

The Contractor’s electronic administration of the written examination shall allow each candidate to skip one or more items and to mark one or more items for later review. The system shall allow a candidate to return easily to items that were skipped or marked for review. The electronic administration also shall provide each candidate with a clear indication of the individual’s progress on completing the examination and time remaining for com-pletion of the examination. The Contractor shall ensure that the candidate’s examination results are reliably as-sociated with that candidate.

C.8.4. Written Examination Comment Form for Candidates

The Contractor will develop a candidate comment form, for COR approval, no later than two weeks before the administration of the exam. Following administration of the exam and before leaving the test room, the Con-tractor shall give each candidate the option of completing a test comment form. The completed comment forms shall be scanned and provided to the COR no later than 30 days following the close of examination administra-tion. Results from the test comment forms will be reported in the Annual Report or as a separate report not later than seven weeks following administration of the examination.

C.8.5. Test Forms

The Contractor shall administer, in roughly equal proportions, three fixed exam forms, one per candidate, as

USCA18R0166 - Page 12 of 45 provided by the AO. Two of the exam forms are operational, and one is the base form to which the others will be psychometrically equated. Contractor may propose changes to this methodology, in writing, for COR ap-proval.

C.8.6. Proctors

The Contractor shall utilize proctors who have been properly trained by the Contractor in the administration of written examinations generally, and in the procedures provided by the AO for the administration of the FCICE examination. The Proctors must adhere to the relevant requirements of this contract and to industry standards in administering the FCICE.

C.8.7. Written Examination Timing

The Contractor shall ensure that each candidate is provided two and one half (2.5) hours to complete the exam-ination, unless the candidate received prior approval for an accommodation that allows for additional time.

C.9. Scoring and Reporting the Results of the Written Examination

The Contractor shall perform the following tasks related to scoring and reporting candidates’ written examina-tion results.

C.9.1. Scoring the Written Examination

The Contractor’s scoring of the FCICE written examination shall yield reliable scores that are valid to determ-ine which candidates have the Spanish/English proficiency to take the FCICE phase two oral examination. All raw scores shall be equated to the base form prior to reporting scores to the candidates and the AO. A passing score requires that the candidate achieve an equated score of no less than75 percent correct on each section of the written examination. For test analysis, research, and documentary purposes, the Contractor shall capture, store, and deliver to the AO in electronic form, item-level responses to each multiple-choice question for each examination candidate, across all administrations of the written examination.

C.9.2. Delivery of Written Examination Scores to the AO

The Contractor shall propose, for COR approval, a delivery date for test scores, immediately after the testing window. The Contractor shall report the results of written examinations to the AO in the Written Examination Score Report, with sample drafts of the report to be displayed and printed for the candidates indicating their ex-act score and stating whether they have passed or failed the examination.

C.9.3. Delivery of Written Examination Scores to the Candidates

Upon approval of the draft score report to candidates, the Contractor shall make visible, and make printable on demand, the score upon a candidate’s conclusion of the examination.

C.10. Administration of Oral Examination

The Contractor shall ensure that all oral examinations are administered in a reliable, standardized, and fair man-ner in accordance with the requirements of this contract and industry standards.

C.10.1. Eligible Oral Examination Candidates

The Contractor shall administer the FCICE oral examination to all eligible candidates who properly registered for the examination and provided appropriate identification at the examination location on the date and at the time the candidate was scheduled for the examination. The contractor shall not register or administer the oral examination to any candidate who has not previously passed the written examination, unless otherwise directed

USCA18R0166 - Page 13 of 45 to do so, in writing, by the COR.

C.10.2. Electronic Recording of Candidate Performances

The Contractor shall record digitally each candidate’s performance and everything said in the examination room during an examination administration. Recording shall be done in a manner that will allow for review of each candidate’s examination performance, for repeat listening due to need for clarification by the rating team, for second and/or third rating on an as-needed basis, and for quality control (as more fully described below). The Contractor shall use a redundant recording method in case of failure of the primary recorder. The Contractor shall ensure that it reliably associates the candidate with his or her performance recording through methods that include, but are not limited to, having the candidate and/or the proctor state his or her candidate identification number and the date of examination at the beginning and end of the examination recording.

C.10.3. Oral Examination Administration Features

The examination shall be administered by a specially trained proctor who shall administer exam stimuli and monitor performance recordings during the entire examination process. Test candidates must be allowed to take notes during administration of the examination on note paper and with writing supplies to be provided by the Contractor.

C.10.4. Test Comment Form for Candidates

The Contractor will develop a test comment form, for COR approval, no later than two weeks before the admin-istration of the exam. Following administration of the exam and before leaving the test room, the Contractor shall give each candidate the option of completing a test comment form. The completed comment forms shall be scanned and provided to the COR no later than 30 days following the close of examination administration. Res-ults from the test comment forms will be reported in the Annual Report or as a separate report not later than seven weeks following administration of the examination.

C.10.5. Oral Examination Forms

The Contractor shall administer examination form 5.1 to eligible test candidates and shall equate all raw scores to form 4.0 of the 2015 oral examination administration before exams are sent to re-score teams as provided un-der this contract.

C.10.6. Proctors

The Contractor shall utilize proctors who have been properly trained by the Contractor in the administration of the FCICE oral examination using the procedures provided by the AO. Proctor training, which may be accom-plished remotely online, or in person, shall take place no more than 90 days before the beginning to examination administration, The Proctors must adhere to the relevant requirements of this contract and with industry stand-ards in administering the FCICE. Proctors must submit reports regarding any exam irregularity during the ad-ministration process.

C.10.7. Oral Examination Timing

The oral examination takes approximately 45 minutes per candidate. The sight translation and consecutive inter-pretation portions of the examination are timed. The Contractor shall ensure that each candidate is provided five

(5) minutes for each sight translation section and eighteen (18) minutes for the consecutive interpretation sec-tion unless the candidate received previous approval for an accommodation that allows for additional time. Dur-ing the consecutive interpretation section of the examination, each candidate may request up to two (2) repeti-tions of stimuli.

C.11. Scoring and Reporting the Results of the Oral Examination

USCA18R0166 - Page 14 of 45

The Contractor shall perform the following tasks related to scoring and reporting candidates’ oral examination results.

C.11.1. Oral Examination Raters

The oral examination shall be scored by qualified raters who are federally certified court interpreters (FCCIs) of Spanish/English who have been trained by the Contractor in FCICE rating methodology. At least 60 days prior to rater training, the Contractor shall provide, for COR approval, a list of sufficient qualified raters to properly rate the performance of each oral examination candidate. These raters must be chosen from the list provided under this contract, section C.1.3.4.

C.11.1.1. Rater Qualifications: Raters shall have both excellent demonstrated theoretical knowledge of language interpretation and practical experience as FCCIs in federal courtroom interpreting; proficiency in both English and Spanish that is equivalent to a highly articulate educated native speaker in both languages and; have been certified by the AO as FCCIs no less than four years prior to rater training. After the AO has approved the Con-tractor’s final list of proposed raters, the Contractor shall contract for the raters’ services and provide raters with sufficient training to ensure proper performance during the scoring process.

C.11.1.2. Rater Training: No more than sixty (60) days prior to the examination rating event, the Contractor shall provide a formal, in-person training event using a curriculum and materials that are sufficiently thorough to develop reliable expert judgment skills for rating candidate performance on the oral examination. During the rater training event, the Contractor shall:

(1) Train raters to take reasonable precautions to ensure that rating procedures are fair to all candidates and are free of bias due to race, age, gender, ethnic background, religion, nationality, or disabling condition;

(2) Ensure that oral examination rater training materials address how different variants of Spanish are handled during the rating process;

C.11.2. Scoring the Oral Examination

The Contractor’s scoring of the FCICE oral examination shall yield reliable scores that are valid for the certific-ation of interpreters of Spanish/English. The rating scale shall be based on the criteria on 80 percent correct to achieve certification. For examination analysis, research, and documentary purposes, the Contractor shall deliv-er to the AO, no later than 60 calendar days, all audio recordings for each test candidate, across all administra-tions of the oral examination. Rating shall include both objective scoring (using a scoring key) and subjective scoring (relying on expert judgment) as appropriate to the test tasks. Rendering meaning correctly (substantive accuracy) shall be rated objectively by means of a key with acceptable renderings. Delivery, pronunciation, flu-ency, and similar factors shall be rated through expert judgment of the raters.

C.11.2.1 The Contractor shall convene rating teams as described below:

(1) Teams of two persons for the initial rating event;

(2) No more than 30 days after the initial rating event, a re-score team(s) composed of two persons to re-rate all examinations with initial equated scores between 77 and 81, but no less than ten percent of all examinations, for quality control purposes. If directed by the COR, additional rescores including all examinations with scores of 77 and above shall be rescored.

(3) No more than 45 days after the initial rating event, a final review team(s) that shall review and reconcile the scores assigned by the original and review teams which deviate by more than the standard error of measurement and/or which result in a difference in the pass/fail status for any candidate.

(4) Rescored examination shall be submitted to a final review team whenever rescores deviate by more than eight (8) points, or whenever the pass/fail status changes during the rescore process. All determinations by the final review team(s) are final.

USCA18R0166 - Page 15 of 45

C.11.2.2. The Contractor shall provide for the routine gathering and electronic storage of data needed to conduct item-level analysis and to determine inter-rater/inter-team reliability of the scoring process.

C.11.2.3. The Contractor shall review initial independent ratings to verify each rater’s ability to rate the oral ex-amination in accordance with the established standards. Raters whose item-level scores show a pattern of devi-ation from the norm or appear otherwise questionable shall be counseled, retrained, or dismissed as raters by the Contractor, as appropriate, to ensure that rating remains accurate and reliable. During the rescore process, the Contractor shall determine whether a rating team’s anomalous performance warrants rescoring examinations that fall outside of the regular rescore parameters established by this contract.

C.11.3. Delivery of Test Scores to the AO

The Contractor shall propose, for COR approval, a delivery date for final oral examination scores, not to exceed 12 weeks following the end of the testing window. The Contractor shall report the written examinations scores to the AO immediately after the written exam testing window concludes, but no more than two weeks after the test.

C.11.4. Delivery of Test Scores to the Candidates

C…