Draft Statement of Work.docx

Attachment 1 Statement of Work USCIS' Revised Naturalization Test

1. Background

In 2022 U.S. Citizenship and Immigration Services (USCIS) began the process of revising the speaking, understanding, and civics tests that are included on the naturalization test.[footnoteRef:1] The speaking and understanding tests will be consolidated into a single speaking and understanding test (speaking test), and the civics test will be converted from an orally administered test to a written, multiple-choice test. Both the speaking test and the civics test will be redesigned to be administered on a tablet during the naturalization interview in sequence with the existing reading and writing tests. As part of this redesign process, USCIS seeks to engage the services of a contractor to assist USCIS with facilitating a technical advisory group (TAG) consisting of external subject matter experts in fields relevant to the naturalization test content (e.g., adult ESL, civics, U.S. government, American history, citizenship education, psychometrics, assessment design, etc.). [1: The revised test consists of two parts: an English test (a test of a· naturalization applicant's ability to read, write, and speak English) as well as a "civics" test (a test of a naturalization applicant's knowledge of U.S. history and U.S. government.)]

USCIS will need input from the TAG to ascertain the relevance, accuracy, and clarity of test content for both the Speaking Test and Civics Test. The development of the revised test is based on Section 312 [8 USC 1423] of the Immigration and Nationality Act as well as on USCIS' regulation and policies regarding the naturalization test.

Additionally, USCIS’s Office of Citizenship (OoC) is mandated to develop educational materials and resources to prepare immigrants for the naturalization test and to help immigrants integrate into the United States. In order to adhere to this requirement, USCIS will need input from the TAG to ascertain that the educational materials and resources for the revised test are developed at the correct educational level and are developed in a way that meets the needs of immigrant learners and the organizations that serve them.

As USCIS moves forward with full implementation of the revised speaking test and civics test, it will require the assistance of the TAG not only to provide input on the revised tests, but also on the educational materials and resources related to them.

2. Scope

USCIS needs the TAG to review new test content, structure, and design methodology for the redesigned speaking test and civics test. TAG members will be divided into two groups based on their respective area of subject matter expertise: 1) Adult ESL Group; 2) Civics Group. USCIS will provide specific qualifications for TAG members to the contractors. The contractor will select TAG members with expertise in fields related to adult ESL, adult education, English language acquisition for the adult ESL group. The contractor will select TAG members with expertise in fields related to American government and American history for the Civics Group.

The scope of the contractor’s roles will include two duties:

1) assisting USCIS with administering the TAG,

2) capturing and documenting the redesigning of the Speaking Test and Civics Test.

The contractor’s administrative duties will include assisting with hiring TAG members (e.g., announcing available positions for TAG member, collecting and compiling resumes to provide to OoC/CETB, onboarding TAG members, etc.), distributing materials and information from OoC/CETB to the TAG, collecting written feedback from TAG members and compiling it into a report for OoC/CETB. The contractor’s documentation duties will include capturing notes from each TAG meeting, and compiling a comprehensive report documenting the work of the TAG and the process of redesigning the Speaking and Civics Tests.

There will be four TAG review cycles over the course of the 12 month base period. There will be one 12 month option period to include four optional TAG review cycles. Each TAG review cycle includes the following components:

· OoC/CETB will provide the contractor with pre-meeting materials and taskings to distribute to the TAG members prior to the scheduled TAG meeting

· The contractor will distribute the materials and taskings provided by USCIS to TAG members prior to the scheduled TAG meeting

· The TAG members will complete the assigned tasks prior to the scheduled TAG meeting.

· OoC will facilitate the TAG meeting with TAG members to provide TAG members with information, discuss revisions and comments with TAG members, provide TAG members with post-meeting tasks, answer TAG members questions. The contractor will document the meeting.

· Immediately following the TAG meeting, OoC will provide the contractor with updated drafts of the Speaking and Civics Tests to distribute to the TAG.

· The contractor will immediately distribute the updated drafts of the Speaking and Civics Test to the TAG.

· TAG members will complete their tasks and submit their input in writing to the contractor according to the deliverables timeline provided below.

· The contractor will collect the TAG’s input and compile it into a report and submit it to OoC/CETB according to the deliverables timeline provided below.

· OoC/CETB will review the TAG feedback and make revisions to the Speaking and Civics Tests.

USCIS will be conducting one external trial test over the course of the 12 month base period and one 12 month option period of the contract. There will be two TAG review cycles prior to the external trial test, and two TAG review cycles after the external trial test. Both the Speaking Test and Civics Test will be administered during the external trial test. The external trial test will be administered to non-native English-speaking students enrolled in adult citizenship education classes through Community Based Organizations (CBO’s). According to the project timeline that OoC/CETB provided to the USCIS leadership, the external trial testing period is anticipated to last for up to five months. After the trial testing, OoC/CETB will use data from the trial test to revise test items on both tests. Once OoC/CETB has completed its revisions to the tests, then the third TAG review cycle will begin.

3. Period of Performance

The period of performance consists of one 12 month base period and one 12 month option period.

4. Place of Performance

The work of OoC/CETB, the TAG, and the Contractor can be completed remotely.

5. Tasks

The contractor shall carry out the following activities:

Task 1: Identify and Select TAG Candidates

USCIS shall instruct the contractor to submit candidates that meet the government’s qualifications as identified in the list under Section 7. The contractor shall submit candidates that meet the government’s qualification to USCIS for review within one month after award of the contract. Once the contractor has selected ten candidates, then the contractor shall notify the ten candidates that they have been selected to serve on the TAG. Prior to the first meeting, the contractor shall also have the ten selected candidates sign a ''Non-Disclosure Form" and "Conflict of Interest Form" provided by USCIS.

The contractor must ensure that there are ten TAG members throughout the duration of the contract. The contractor must also ensure that there are five TAG members in each subject matter group (i.e. Adult ESL Group, Civics Group). Should a TAG member leave, the contractor shall identify and propose a replacement TAG Member according to the subject matter area in need, and based on the qualifications criteria under Contract Personnel, Key Personnel, TAG Members.

Task 2: Complete Pre-Meeting Work for Four TAG Revised Naturalization Test Review Meetings

The work the contractor needs to accomplish prior to the meeting includes:

1) Work with USCIS to determine and disseminate the pertinent information the TAG shall need to review or complete prior to the TAG meeting. The pertinent information will include, the TAG meeting agenda, project-related guidance, drafts of new educational resources and materials, memos or reports on revised test related subjects, etc.

Task 3: Facilitate Four Virtual TAG Meetings

TAG meeting will be conducted virtually on MS Teams. The first TAG meeting will be scheduled to last six to eight hours, and it may be broken up over two days over the course of one week. This will allow time for introductions, orientation, an overview of project, and questions from TAG members. The second, third, and fourth TAG meetings will last two – four hours per meeting and will allow time for TAG members to provide comments and ask questions.

The contractor's roles at each TAG meeting shall be to serve as facilitator, timekeeper, and note-taker.

The contractor shall submit the TAG meeting notes to USCIS and the TAG for review within one week after each TAG meeting.

USCIS will provide the contractor with post-meeting work to distribute to the TAG members after each TAG meeting.

A schedule for the TAG meetings will be provided after contract award.

Task 4: Complete Post-Meeting Work for Four TAG Meetings

After the completion of each TAG meeting, USCIS and contractor shall note follow up tasks and items.

USCIS will provide the contractor with post-meeting work to distribute to the TAG members after each TAG meeting.

TAG members will be expected to provide written feedback on post-meeting work.

TAG members will be required to submit written feedback to contractor one week (5 business days) after TAG meeting. The amount of post-meeting work expected from the TAG Civics Group shall consist of sixteen – twenty-four per TAG Civics Group member; and the work expected from the TAG Adult ESL Group shall consist of twenty-four to thirty hours per TAG Adult ESL Group member.

The Contractor will consolidate the TAG members’ comments into a single document and provides the document to USCIS within two weeks (10 business days) after TAG meeting.

The Adult ESL Group will review and provide feedback in writing on the Speaking Test content (e.g., question prompts, images, instructions, etc.), the Speaking Test structure (e.g., administration of the test, the delivery of individual test items), and the Speaking Test design methodology (e.g., the process for selecting test content and drafting test questions). The Adult ESL Group will review provide feedback in writing on the English language level appropriateness of the revised test items for both the Speaking Test and the Civics Test. They will use the National Reporting System for Adult Technical Assistance Guide for Performance Accountability (NRS Guide) to review the language level of individual test items and the overall test content. The Speaking Test and Civics Test will both be designed so that applicants who speak English at the high beginning level according to the NRS Guide can pass. Due to the cognitive difficulty of some content, some revised test items may register at a higher or lower language level according to the NRS Guide, but the scope of the recommended revisions should steer revised test items towards the high beginning level.

The Civics Group will review and provide feedback in writing on the Civics Test content (e.g., questions, answer choices, instructions, etc.), the Civics Test structure (e.g., administration of the test, the delivery of individual test items), and the Civics Test design methodology (e.g., the process for selecting test content and drafting test questions). The Civics Group will not review or provide input on the Speaking Test because the Speaking Test does not contain civics-related content.

Task 5: Submit a Summary Report of the Naturalization Redesign Process to USCIS

The Contractor will provide USCIS with a report that summarizes the process that USCIS followed in redesigning the Speaking Test and Civics Test. The report will be submitted to USCIS within one month of the last TAG meeting and will include:

· Summaries of the assessment design frameworks that USCIS used to design the Speaking and Civics Tests

· A summary of the structure and administration of the TAG (e.g., Descriptions and professional biographies of TAG members, processes for distributing information to and collecting information from TAG members, number of TAG meetings, etc.)

· A description of the role of the TAG and the work that it did during TAG meetings, and in reviewing drafts of the redesigned Speaking and Civics Tests.

· Summaries of the feedback that TAG members provide to USCIS and USCIS’s response to that feedback.

6. Schedule of Deliverables

The Contractor shall provide all submittals specified in the deliverable table for review and acceptance.

Unless otherwise specified, the USCIS will have ten business days to review and provide comments to the Contractor prior to acceptance of all deliverables. The COR will review the submittal and notify the contractor if it is rejected or accepted.

The USCIS has the right to reject or require corrections of any deficiencies. In the event of a rejected deliverable, the Contractor will be notified in writing by the COR of the specific reasons for rejection.

The Contractor shall have an opportunity to correct the rejected deliverable. The Contractor must respond with corrections within ten business days after receipt of USCIS comments.

If the deliverable is acceptable, the COR will send an e-mail to the Contractor notifying that the deliverable has been accepted.

Deliverable POC

Task 1: Identify and Select the TAG Members Deliverable 1: Submit names and qualifications of the ten TAG candidates (5 ESL SMEs and 5 Civics SMEs) to USCIS for review.

Task 2: Complete Pre-Meeting Work for Four TAG Revised Naturalization Test Review Meetings Deliverable 1: Disseminate pre-meeting work (to include the meeting agenda).

Task 3: Facilitate Four Virtual TAG Meetings Deliverable 1: Submit meeting notes to USCIS. (Each TAG Meeting)

Five business days after each TAG Meetings.

COR/PM

Task 4: Complete Post-Meeting Work for Four TAG Meetings Deliverable 1: Submit a report to USCIS that compiles the ESL TAG members’ recommendations and input on post-meeting work related to the Speaking Test. (Each TAG Meeting) Deliverable 2: Submit a report to USCIS that compiles the ESL TAG and Civics TAG members’ recommendation and input on post-meeting work related to the Civics Test. (Each TAG Meeting)

Task 5: Submit a Summary Report of the Naturalization Redesign Process to USCIS Deliverable 1: Submit a report to USCIS that summarizes the Naturalization Test Redesign Process, the role of the TAG, and the facilitation of TAG.

7. Contractor Personnel

The labor categories defined below are considered the minimum personnel necessary for the proposed contract. The qualifications listed below are the minimum acceptable level. The contractor shall determine all other labor categories and staffing levels appropriate for completing the objectives of this project. All contractor staff shall be United States citizens or lawful permanent residents.

Key Personnel: Key Personnel are required for successful performance of this contract. The contractor shall provide statements of qualifications for individuals identified as key personnel within ten calendar days after the date of the award.

TAG Members: The contractor shall identify ten total candidates to serve on the TAG. Of the ten candidates, five candidates will be selected to serve on the Adult ESL Group, and five candidates will be selected to serve on the Civics Group.

For the Adult ESL Group, the contractor will identify five candidates who all have recognized expertise, experience, and working knowledge in the areas:

· Expertise in Adult second language acquisition;

· Expertise and experience in psychometrics in test development and test administration;

· Experience or working knowledge of the U.S. Department of Education's National Reporting System for Adult Education (NRS)[footnoteRef:2]; [2: Information on the U.S. Department of Education's National Reporting System for Adult Education can obtained at www.nrsweb.org]

· Experience or working knowledge of the U.S. naturalization testing process and test;

For the Civics Group, the contractor will identify five candidates who collectively have recognized expertise, experience, and working knowledge in the following areas:

· U.S. history/government, civics, social studies and/or political science;

· Expertise and experience in psychometrics in test development and test administration;

· Experience or working knowledge of the U.S. naturalization testing process and test;

Candidates may not use any confidential information gained through participation on the TAG, or information not yet available to the general public for financial gain (i.e. authoring textbooks for profit)

Geographic and ethnic diversity of TAG members is strongly encouraged.

Consultant/Technical Advisor: This position requires expertise in a field related to research and evaluation of adult education initiatives and will provide expertise on an as-needed basis to all technical assignments.

· Education: Master's degree or higher from an accredited college or university in any field related to research, evaluation, statistics, social studies, political science, United States history/government, test development and administration, psychometrics in test development, second language acquisition.

· Experience: This position requires a minimum of five years’ experience in project management, including capturing and summarizing information from meetings, collecting and distributing resources, drafting reports that document the project, facilitating meetings, the ability to develop written documents and materials, good written and oral communication skills, excellent organizational skills, and a strong attention to detail The person shall coordinate with contractor management and government personnel to ensure that the problems have been properly defined and that the solution will satisfy the government's requirements.

Support Specialist: This position requires expertise in meeting facilitation and research support and will provide support on an as-needed basis to all technical assignment.

· Education: Bachelor's degree from an accredited college or university in research, evaluation, statistics, psychology, sociology, social studies, political science, United States history/government, test development and administration, psychometrics in test development, second language acquisition, or related field.

· Experience: This position requires a minimum of three years’ experience in their related field. This person shall have experience with facilitating meetings, the ability to develop written documents and materials, good written and oral communication skills, excellent organizational skills, and a strong attention to detail.