Statement of Work - Administration of 2025 MP Exam_DRAFT.docx
DOCX document 71 KB Posted
- Solicitation number
- M-24-A45-R-000023
About this file
This document is a draft Statement of Work (SOW) for the administration of the 2025 Merit Promotion Exam for the United States Marshals Service (USMS). The SOW describes the requirements for a contractor to administer a computer-based exam consisting of 185 multiple-choice questions to approximately 2,900 USMS criminal investigators. The exam will be offered at locations primarily via computer testing centers, with a limited number of paper-based exams delivered to remote sites. Key requirements include establishing an online practice site, transferring the exam to a computerized system, registering and scheduling test takers, securely administering the exam, providing preliminary and final exam scores, and handling appeals. The contractor must have significant experience administering exams nationally and meeting strict timelines. USMS will also issue a pre-solicitation sources sought notice for this requirement, with responses due by June 14, 2024.
View the file
Other files
| File | Type | Posted |
|---|---|---|
| MPE Administration SOW_Appendix I_USMS Sites.xlsx | XLSX spreadsheet |
On GovTribe
Work with this file
- Download the original file
- Contacts named in this file
- Similar government files
- Ask GovTribe AI about this file
Text version
Computer Administration of Merit Promotion Exam Statement of Work
A. Background:
The United States Marshals Service (USMS) consists of about 300 offices throughout the country and overseas. The primary mission of the agency is to support the Federal judiciary. Our employees assist the Department of Justice in virtually all Federal law enforcement activities. Further information about the USMS can be found on the Internet at www.usdoj.gov/marshals.
Criminal investigators (GS-1811) in the USMS compete within the agency for promotion to a grade 14 or 15 position. Grade 13 criminal investigators are full performance level criminal investigators or senior specialists and are competitively reassigned to areas such as judicial security, witness security, tactical operations, or asset forfeiture. Grade 14 criminal investigators are typically first-line supervisors in District offices. Operational positions at this grade level also serve as program managers in our Headquarters offices, and they manage the national programs of the USMS. Operational positions at this grade level assigned to a Division component also serve as Chief Inspectors and Deputy Commanders in our Regional offices. Grade 15 criminal investigators serve as Assistant Chief Deputy U.S. Marshals or Chief Deputy U.S. Marshals in District offices, or Division Chiefs or Deputy Assistant Directors in our Headquarters offices. Even though these positions have various duties and responsibilities, we have traditionally found that the job functions and required skills are consistent across positions, with a slight variation between grade 14 or 15 positions.
B. Description/Purpose of Requirement:
This USMS requirement is for a contractor to administer the writing skills exam (45 items) plus a job-knowledge exam (140 items in 3 parts), to USMS employees at various locations throughout the United States and its protectorates, primarily via a computer-based system. Based on USMS policy directive 3.1.F.5.a.1, “Employees applying for promotion must …have a current Merit Promotion Exam score on file in accordance with established timeframes”. The USMS will provide the vendor with the completed exam. It is expected that the vendor shall perform the following requirements:
1. Transfer the provided exam items to a computerized system.
2. Register the test takers and be responsive to the test takers emails and questions.
3. Securely administer the multiple-choice exam (140 items) at sites specified in Appendix I on mutually agreed upon date(s), primarily on Saturdays.
4. Securely administer make-up exams (typically 2 months after the last scheduled exam).
5. Provide the USMS test takers with the initial final exam scores and post-appeals modified scores by sections.
6. Provide the Contracting Officer’s Representative with the preliminary score report, initial exam scores, a classical item analysis report, post-appeals exam scores, and sample score report.
The contractor shall administer the examination via a computer system that is secure, proven reliable, and provides constant back-up. For example, if the system were to crash in the middle of an exam, only exam data on the current question would be lost. The contractor shall administer the exam via a secure computer network controlled by the vendor, rather than on-line or through a web-based system. This method will help ensure a more secure exam, and less variation in screen load time delays for each test taker. In addition, the contractor shall administer and score a very limited number of paper-and-pencil exams in remote locations where a computerized testing center is not available.
The USMS will update its job analysis and test plan for the 2025 exam, and subject-matter experts will have completed writing the exam by December 31, 2024. It is expected that the exam will be provided to the vendor two months prior to the primary exam dates.
Based upon the fact that the exam is designed to test USMS Criminal Investigators, the contractor and any sub-contractor(s) must be aware that test takers may show up at the testing facilities with concealed weapons.
C. Examination Procedures:
Exam Locations: The exam is to be offered in approximately 275 locations throughout the United States and its protectorates, along with a very limited number of foreign locations. The vendor shall provide a computerized testing site in at least 90% of the locations listed in Appendix I, or within 75 miles of these locations. The vendor will make testing arrangements for paper-based exams at those test sites where the vendor cannot provide a secure computer testing center.
See Appendix I for a listing of exam sites that must be matched at the rate of at least 90%. Please refer to Appendix I for more detailed instructions on how to respond to this part of Tasks G & H. It is mandatory that the contractor be able to match at least 90% of the locations listed under the attached Appendix I.
Exam Sites: Exam sites should be clean and provide one chair and computer workstation (with a computer mouse) for each test taker. Ample parking should be available for free or at a minimal fee. Exam takers may need to store a cell phone during the exam administration, so locked and secure storage must be available. In so much as possible, exam sites should provide a similar experience for all candidates, regardless of geographic location.
IT Systems: The exam must be stored and administered through a network (not Wi-Fi) secured and controlled by the vendor in order to ensure exam security and a consistent test-taker experience.
Exam entrance and Security: A) Test takers are required to show at least one government-issued picture ID upon entrance to the facility. B) Test takers must provide their signature on a registration form to verify that they arrived at the exam facility, or the vendor may provide some other type of proof to verify the names of all the test takers who signed in to take the exam on that date and time. All registration documents shall be provided to the USMS COR. Proctoring: C) The contractor shall provide no less than one proctor per exam site. At least one proctor must monitor the examination room and observe test takers throughout the exam session. The proctor will allow for up to a 30-minute late entry to the exam. Proctors must ensure test takers do not enter the exam room with material such as cell phones, calculators, written notes, etc.
D. Testing Preparation and Administration:
Exams shall be administered by the contractor, to participants, on a primary Saturday exam date, with some secondary Sunday exam date flexibility allowed for high-demand testing sites. Based upon the dates provided in this solicitation, it is anticipated the test will be administered in March/April 2025. If these dates are not available, please propose alternate dates, as well as any special provisions that are anticipated to be in place due to the ongoing COVID-19 pandemic. There are approximately 2,900 employees eligible to register for the exam, and historically, about 80% initially register for the exam and about 75% take the exam.
Test items should be randomized, so that the order of questions differs on each exam. This randomization should occur only within each of the job knowledge sections (e.g., typically 3 sections of 50 questions each).
Test takers should be allowed to appeal questions by entering a free-form response (e.g., “This question is not accurate because none of the possible responses are correct” or “The policy which is referred to in this question changed last month, and the responses are no longer correct.”). A comment box format may be used for this purpose if it is labeled with the term “appeal”. Ideally, this appeal item would be offered with each question or as a separate pull-down menu option but may also be offered as an option at the end of the exam. The vendor will provide an electronic or computer copy of the proposed appeals format to the USMS COR for review with the pilot exam.
The total exam time will require a 4-hour testing window. The writing exam section will be offered first, and each of the other 3 sections WILL NOT be timed separately. The test taker should be able to go back and change answers to questions that have already been completed and move from one section of the exam to another. They are not timed separately. This is important, since a few of the test takers may be taking a paper-based version of the test and will have that ability. In addition, if an appeals option is not provided with each question, the format will allow the test takers to skip forward to the appeals section while an issue is fresh in their minds, and then go back to the appropriate test question to finish the exam.
Exam Results: The contractor shall provide test takers with exam scores, preferably at the test site, but otherwise within one week of the testing date. The contractor shall provide a statement of test scores to the test taker in a format provided by the USMS. Example: a key shall be provided to explain each part of the exam, and how they are added together (e.g., Part B + Part C = job knowledge test score). The contractor shall provide the test taker with a unique registration number or other unique identifier other than his or her social security number. That number must be printed on the exam results or otherwise provided to the test taker. This number will be provided to the USMS as a unique identifier along with the computerized database of exam results.
E. Scope of Procurement:
Major activities and deliverables will include:
Task A Delivery Date: To be completed by vendor within 2 weeks after contract award.
Post-Award Conference. Within no more than two (2) weeks after contract award, a post award conference is to be held. At the conference (in-person or remotely), the contractor is to provide the following:
1. The USMS COR will schedule and coordinate the post-award conference, which will be held in the Arlington, VA area or remotely.
2. The USMS and the vendor will discuss possible exam dates and determine an exam date in Spring 2025 that provides the largest number of seats in the testing locations needed by the USMS.
3. The vendor will demonstrate 5-10 sample USMS test questions in the computerized testing system.
4. Within 2 weeks of the post-award conference, the vendor will provide
a. The web site and phone number that test takers may contact to register for the exam; and
b. A list of proposed exam sites available for online registration/scheduling.
Task B Target Delivery Date: To be completed by vendor no later than 3 months prior to the first primary exam administration date (Task A).
The vendor shall establish an online practice site so that USMS test takers can familiarize themselves with the contractor’s test taking methodology, prior to the exam. This site does not have to be USMS-specific but must be in a similar multiple-choice format. The proper web address shall be provided to the USMS COR.
Task C Target Delivery Date: To be provided by the USMS no later than December 31, 2024.
The multiple-choice exam will be transferred to the contractor no later than December 31, 2024.
Task D Target Delivery Date: To be completed by the vendor no later than 21 days prior to administration of the exam.
Transfer the multiple-choice exam from an excel spreadsheet to a computerized system. Each test item is in multiple-choice format with 4 or 5 possible responses. Most items and possible responses are only a few lines each, but some of the writing skills questions may be rather lengthy and include several 4 - 5 sentence paragraphs as prompts.
Task E Target Delivery Date: Registration for exam to be provided by vendor at least 60 days, but preferably 90 days, prior to first primary exam administration date (limited flexibility, depending on requirements of vendor).
The vendor will provide the USMS with a final list of computerized testing locations or a web link to this list, so that USMS employees can view options prior to registration.
The USMS will provide the vendor with a list of eligible employees to include first name, middle initial, last name, and last 5 digits of social security number. Upon registration, employees will provide contact information required by the vendor, such as addresses, phone numbers, etc.
At least 60 days prior to the exam (90 days is preferable) up to the actual exam administration, the vendor will provide a website link or scheduling tool to allow eligible U.S. Marshals Service employees to register/make appointments for the exam. Upon registration, the test taker will be provided with an exam date, location, and time, which will be confirmed in writing email. At this time, a unique confirmation number will be assigned by the vendor, which can then be used as an individual identifier throughout the testing and score reporting process. The vendor will contact the test taker if there are any changes in the exam date, location, or time. Be advised that due to the nature of the USMS mission, registration changes should be accommodated up to the day of the exam. Test takers may call to change an exam site or time, based on operational assignments or mission needs. As a result, there is no exam registration cut-off or deadline to change test sites. If a test-taker shows up on the exam date and space is available, the vendor will register the test-taker on-site and allow testing.
Note: Employees will be informed by the USMS that prime locations and times will be more available to those who register early in the registration window. At any given time, a very small percentage of the USMS workforce may be called upon at the last minute to perform a special operation or detail away from their normal duty site. Accommodations should be made when space is available at the testing site. This may include allowing a non-registered test taker to register on-site and take the exam if seats are still available. If a test seat is not available to accommodate a change or late registration, a make-up exam may be approved by the USMS, based on legitimate circumstances.
Task F Target Delivery Date: The vendor will provide an in person or virtual pilot exam administration and process at a limited number of sites 4 weeks prior to the primary exam date.
The vendor will allow 3 - 5 pilot test takers to take the exam at a limited number of locations (approximately 3-5 sites). The pilot will cover the entire test administration process, from the registration phase to the individual score report received after the session. The vendor will also report the scores to the USMS as a group to ensure computer system compatibility and the adequacy of the score reports and the item appeal process. The vendor will make critical adjustments as needed to ensure a successful exam delivery. Pilot test takers will not take the exam over again on the exam date, due to security concerns. The above is to be coordinated by the vendor with the COR.
Task G Target Delivery Date: The target delivery dates are Saturdays/Sundays in March and April 2025. (There is limited flexibility for USMS exam dates, depending on exam seat availability and CDC guidelines).
Exam Administration: Administer the computerized exam (185 multiple-choice questions (140 job knowledge and 45 written skills) with a 4-hour time limit) to USMS employees, preferably on five consecutive Saturdays, with seats at test sites primarily dedicated to the USMS.
The total exam time will be for up to 4 hours. Candidates will be able to determine time remaining via a constant timer on the computer screen, a timer pulled up on demand on the computer screen, or at minimum via a clock in plain sight in the testing facility. If a timer on the screen is available, the test takers will also receive a pop-up timer warning when 10 minutes remain in the testing window. Otherwise, a 10-minute verbal warning will be provided. Each of the four sections WILL NOT be timed separately. In addition, if an appeals option is not provided with each question, the test taker will be allowed to skip forward to the appeals section while an issue is fresh in their minds, and then go back to the appropriate test question to finish the exam. Candidates will also be allowed to navigate back and forth to review items or to skip items and then return later to answer and/or change answers.
Exam Date and Times: To be jointly determined by vendor and USMS to accommodate the 4-hour testing window. The goal is to administer all exams on five consecutive weekends, but there is limited flexibility to allow for sites with high demand and low supply or to meet potential COVID-19 CDC guidelines and restrictions.
Exam Results: The computer exam results, to include several different total scores, will be provided to the test taker upon exiting the facility in a format provided by the USMS. Otherwise, scores will be emailed to the test taker within one week of exam administration.
Exam Locations: The exam locations for the computerized version of the exam will vary based on the sites offered by the vendor, but must meet the 90% requirement described in Appendix I.
Exam Sites: Computer exam sites should be clean and provide one chair and desk/workstation for each test taker. The atmosphere should be conducive to test taking (e.g., no disruptively loud noises). Please provide known details regarding any special provisions that are anticipated to be in place due to COVID-19 pandemic restrictions. The work site configuration should be set up so that candidates cannot easily see each other’s screens. Ample parking should be available for free or a minimal fee. There should also be a secure locker available to store cell phones, etc.
Customer Satisfaction: At the end of the exam administration, an optional survey of customer satisfaction, to be provided by the USMS and consisting of up to 10 items, will be available to the test taker. Responses from this survey will be provided to the USMS, to include unique customer id, test site and test date.
Task H Target Delivery Date: The vendor will administer a paper-based version of the exam on at least one Saturday in March in USMS locations identified in Appendix I that are not within 75 miles of the vendor-provided automated testing sites, to correspond with the computerized exam delivery dates (limited flexibility for USMS exam date, depending on exam seat availability).
Exam Administration: Administer a proctored paper-based version of the computerized exam (185 multiple-choice questions with a 4-hour time limit) to USMS employees on at least one Saturday. The contractor shall provide USMS-approved instruction booklets for proctors, as well as answer sheets and a method for recording appeals. The vendor will deliver a paper-based version of the exam, within 75 miles of the USMS office location, to candidates in remote locations or in a very limited number of locations where automated testing site capacity does not meet USMS demand for seats.
For the paper-based exam, candidates will be able to determine time remaining via a clock in plain sight in the testing facility and will receive a verbal reminder with 10 minutes remaining in the test window. Each of the exam sections WILL NOT be timed separately. In addition, if an appeals option is not provided with each question, the test taker will be allowed to skip forward to the appeals section while an issue is fresh in their minds, and then go back to the appropriate test question to finish the exam. Candidates will also be allowed to navigate back and forth to review items or to skip items and then return later to answer and/or change answers.
Exam Times: To be jointly determined by vendor and USMS to accommodate the pre-determined testing window.
Exam Results: Proctors will mail exam booklets, answer sheets, and appeal sheets to the vendor using a traceable/accountable delivery method at the vendor expense. The vendor will grade the paper-based exams and provide final exam results to employees in the same manner as the computerized final exam results.
Exam Locations: The exam locations for the paper-based version are still to be determined and will vary based on the automated sites offered by the vendor. Historically, there are approximately 33 paper-based sites in remote locations, such as Fairbanks, Alaska; Eureka, California; Key West, Florida; Kingston, Jamaica; Marquette, Michigan; Aberdeen, Mississippi; Greenville, Mississippi; Missoula, Montana; Guadalajara, Mexico; Mexico City, Mexico; Monterrey, Mexico; Saipan, Northern Mariana Islands; Medford, Oregon; Hato Rey, Puerto Rico; Aberdeen, South Dakota; Pierre, South Dakota; Alpine, Texas; Del Rio, Texas; Laredo, Texas; Pecos, Texas; Victoria, Texas; Harrisonburg, Virginia; St Thomas, Virgin Islands; St Croix, Virgin Islands; Bluefield, West Virginia; and Lander, Wyoming. Most of these sites will have less than 10 test takers, while Texas sites may have up to 50 each. Fewer paper-based sites are preferred.
Exam Sites: The contractor shall provide the locations with sufficient paper-based exams for testing. Paper-based exam sites should be clean and provide one chair and desk for each test taker. (If desks are not available, sufficient space should be allowed around a table so that candidates cannot easily see each other’s answer sheet (e.g., one empty chair between each candidate.) Ample parking should be available for free or a minimal fee. There should also be a secure locker available to store cell phones, etc.
Customer Satisfaction: At the end of the exam administration, an optional survey of customer satisfaction, to be provided by the USMS and consisting of up to 10 items, will be available to the test taker. Responses from this survey will be provided to the USMS in electronic format, to include unique customer ID, test site and test date.
Task I Exam Security - To be provided by vendor at both computerized exam sites and paper-based exam sites on the primary exam dates and for the make-up exams.
Exam entrance and Security: A) Test takers are required to show at least one government-issued picture ID upon entrance to the facility. B) Test takers must provide their printed name and signature on a piece of paper to verify that they arrived at the exam facility, or the vendor may provide some other type of proof to verify the names of all the test takers who signed in to take the exam on that date and time.
C) Proctors must check to make sure test takers do not enter the exam room with material such as cell phones, calculators, written notes, etc.
Proctoring: It is expected that there will be no less than one proctor per exam site. At least one proctor must monitor the examination room and observe test takers throughout the entire exam session. The proctor will allow for up to a 30-minute late entry to the exam.
Task J Target Delivery Date – offered by vendor, make-up exam timeframe is typically from 2 weeks to 5 months after primary exam date.
The vendor will provide automated make-up exams for eligible employees designated by the USMS (typically 100-150 make-up exams).
Due to operational assignments, travel schedules, special assignments, illnesses, etc., some USMS employees will not be able to take the exam at the scheduled date or time. Within 2 weeks of the primary test dates, the USMS will provide the vendor with a list of names of those eligible for a make-up session and the window of testing opportunity (generally 5 months after the initial exam date.) A full list of names and make-up locations will not be finalized until 2 weeks after the last primary exam date. Make-up sessions will occur at multiple sites, depending on the locations of the employees who need to take the make-up exam. In general, make-up sessions can be offered on the vendor’s testing schedule, during regular testing hours and as seats are available at a particular testing site.
Note: Since the make-up window extends beyond the requirements for the provision of a final electronic file of scores (see Task L), the vendor will need to provide the USMS with a second final score report for the remainder of the make-up exams. This report needs to only include those not provided on the first report and should be received 2 weeks after the last scheduled vendor- administered make-up exam.
Task K Target Delivery Date – within 2 weeks of last primary exam date(s).
Provide, within two weeks of test completion, the preliminary exam results (to include survey responses) and a classical item analysis for the 185 items, using the exam results from the original test takers (make-up exam scores do not need to be included in this analysis). The preliminary results will show the number of test takers and the score distributions and averages for each section (e.g., Part A, writing skills, etc.). At a minimum, the item analysis should show the number and percent of valid responses, a score distribution for each question, point biserial, and also split the test takers into a high group (those who scored in the top 50%) and a low group (those who scored in the bottom 50%) and show the number and percent of responses for each of the four choices. For example, the percent responses could be shown as:
Part B, Question 1 A B C D % of high 75% 10% 5% 10% % of low 35% 50% 10% 5%
This data will be used during the appeals process, to help determine things such as whether all responses to a question should be granted credit or if more than one correct response was available. The item analysis data will also be used when writing the next exam, to determine if a question is worth reusing in a new exam.
In addition, the vendor will provide all appeals in an electronic spreadsheet format, in question order, to include test taker unique ID, exam section (e.g., Part A or writing), item number, correct answer, answer provided by test taker, and the text of the written appeal.
Task L Target Delivery Date(s) – Rescoring of exam based on appeals and item analysis – no later than 6 weeks after last primary exam date (2 weeks to provide USMS with score report, item analysis and appeals + 2 weeks for USMS to provide updated key for rescoring + 2 weeks for to provide final scores to USMS and candidates).
· Delivery of rescored exam (final) to USMS (electronic and hard copy)
· Emailing rescore (final score) to test takers
As stated in tasks F and K, test takers will be allowed to appeal individual exam items. An electronic or computer copy of this appeals information, with the question number identified, will be provided by the vendor to the USMS within 2 weeks of the last primary exam date, as stated in task K. In mid-April, the USMS will convene a group of subject matter experts to review each appeal. Over the past 25 years, appeals have always been received. Typically, the appeals for approximately 5 questions are upheld. In a very few cases, appeals have been upheld for up to 10 questions.
Once the appeals board has met, the USMS will update the scoring key and send it to the vendor to re-score all exams, including completed make-up exams. For example, Part B, item 31 may now have both option A and option C scored as correct, or for Part B, item 35, all responses may be scored as correct. Based on the updated scoring key, the vendor will rescore all tests (i.e., original exams and make-up exams) that have been received to that point and email the new exam results to the test takers at the email or mailing address designated by the test taker during registration. This format of results will be USMS specific and include information such as the unique ID of the exam taker, average exam scores and ranges.
In addition to providing the updated scores to the individuals, the vendor will provide the USMS with an electronic results file. Data downloads are typically not an option due to firewall issues. The results file will contain the following information: Test taker name, unique identifier number, exam date, exam site, and score for each exam section. This database should also contain the test taker responses for each item in the exam. This information can be split from the overall score information or can be combined into one database. The format for the computer file is flexible but must be able to be read into Microsoft Excel. Acceptable formats are Excel, tab-delimited, comma-delimited, Rich Text format, etc.
(The rescoring and provision of final scores will need to occur a second time, for the small number of test takers that complete make-up exams through the end of the make-up exam window.)
Task M Target Delivery Date(s) - Billing – The vendor will provide the USMS with an itemized bill within 4 weeks of the primary exam date(s), and for make-up exams, within 4 weeks of the closure of the window for make-up exams. (If the vendor prefers, make-up exams can also be billed monthly throughout the make-up exam window.)
The vendor will provide the USMS with an itemized bill that includes the name, unique registration id, registration date, exam site, and exam test date for each individual test taker.
Due to the variability of mission requirements, test takers will be able to cancel their registration and/or change test sites up to 48 hours prior to the exam start time without an additional charge or test cancellation fee. After the 48-hour window, the vendor will be able to charge the USMS for no-shows and/or late site changes.
N. Special Notes:
All test results are the property of the USMS. Once the make-up exams are completed, and the computer files are received and approved by the USMS, the vendor should destroy all exam materials and test taker records and provide written certification of such. Test taker records cannot be sold or shared with other vendors for statistical purposes, or to create or add to mailing lists.
All above listed dates are tentative. If the vendor cannot meet the proposed dates, it is requested that alternative dates be proposed. Please provide any alternative dates with your proposal submission.
F. Summary of Deliverables and Timeline:
| Task Item Number and Letter |
| Timeframe |
| Deliverable |
| 001 |
| Exam Project Management |
| Manage Exam project and preliminary steps to include registration, transfer of exam to automated system, security, delivery of test results to individuals. |
| Task A |
| 2 weeks post-award |
| Post-Award Conference – meet to discuss proposed schedule of activities and meetings for exam administration and to determine optimal exam date(s) |
| Task A (see also Task E below) |
| 3 weeks post-award |
| Block USMS seats on agreed upon exam date(s); provide proposed exam sites and times by location, as well as phone number and website for registration |
| Task B |
| Minimum of 3 months prior to first exam administration |
| Establish online practice site for USMS test takers |
| Task C |
| NLT Dec 31, 2024 |
| USMS provides Excel electronic copy of exam questions, answers, and key to vendor |
| Task D |
| Minimum of 21 days prior to first exam administration |
| Transfer exam from Excel to vendor computerized exam system |
| Minimum of 3 months prior to first exam administration |
| USMS provides list of eligible employees for registration |
| Task E |
| Minimum of 2 months (preferably 3) prior to first exam administration |
| Provide link or digital file of confirmed testing sites for USMS, with site address, for employees to view prior to registration |
| Minimum of 2 months (preferably 3) prior to first exam administration up until exam date |
| Open USMS exam registration, and upon registration, provide USMS employee with email confirmation that includes exam date, location, and time, along with unique identifier. |
| Task F |
| 2-4 weeks prior to first exam administration |
| Provide pilot test experience for 3-5 pilot test takers for entire process from registration through exam administration to results. |
| 002 |
| Exam Administration |
| Administer Automated, as well as Paper-based exam up to 2,300 test takers (approximately) |
| Tasks G & H |
| 185 questions/4 hours |
| Securely administer approximately 2,300 exams (i.e., automated and paper-based) using vendor sites and equipment to USMS employees and provide preliminary results to employees. (See tasks G & H for requirements) |
| Task I |
| Minimum of 3 months prior to first exam administration |
| Provide Exam Security Plan for both automated and paper-based sites (see task I for requirements) |
| Task J |
| List of USMS employees eligible for Exam |
| USMS to provide list of USMS employees approved to take the make-up exam, using a mutually agreed upon schedule |
| Task J |
| From post-exam to 3 months post-exam |
| Provide make-up exams to USMS employees |
| 003 |
| Psychometrician Services/Item Analysis |
| Provide test taker appeals and item analysis to USMS; rescore exam based on appeals result and provide final score to USMS test takers |
| Task K |
| Within 2 weeks of last primary exam administration |
| Preliminary Exam Results |
| Task K |
| Within 2 weeks of last primary exam administration |
| Classical Item Analysis |
| Task K |
| Within 2 weeks of last primary exam administration |
| Test taker appeals in electronic spreadsheet, in question order |
| Task L |
| Within 2 weeks of receiving appeals and item analysis |
| USMS provides any items that need to be rescored, to include updated key for each of these items |
| Task L |
| Within 2 weeks of USMS provision of rescoring |
| Rescore exams and provide final scores to USMS |
| Task L |
| Within 2 weeks of USMS provision of rescoring |
| Provide final scores to individual original test takers |
| Task L |
| Within 2 weeks of final USMS make-up exams |
| Provide final scores to make-up exam test takers |
| Task M |
| Within 2 months of final USMS make-up exam |
| With USMS permission, destroy all USMS exam materials and test taker records and provide written confirmation of action. |
G. USMS Responsibilities:
The U.S. Marshals Service agrees to perform the following activities and provide the following resources in support of these services. The vendor will treat these materials with the utmost awareness of their sensitivity.
· Materials and documentation. To execute various phases of the proposed project, the vendor will need to have access to exam items and answer key
· A list of USMS employees eligible to take the exam.
H. Contractor Qualifications:
It is of the utmost importance to the USMS that this project be conducted by an experienced project manager for exam management and testing. In addition, testing experts (psychometricians) of demonstrably superior credentials and expertise are required for the item analysis phase of this project.
The project manager needs to have 2+ years of experience administering 3 or more exams on a nationwide basis for 3,000+ test takers. Experience working with law enforcement or similar government organizations is preferable.
Accordingly, the psychometricians/statisticians who are selected to conduct the analysis for this project must have the following qualifications:
· A higher academic degree. The project leader must have a Ph. D.
· Demonstrated professional expertise and leadership in the analysis of test items.
I. Personnel Security Requirements:
Contractors will have no regularly scheduled interface with USMS or Government IT systems or facilities, however if required, they will be escorted by USMS employees. A signed statement acknowledging that each employee of the vendor will submit the required background paperwork in 10 business days or less of the contract award for security adjudication by the USMS.
Because vendor staff will have access to USMS employee information, all test registration activities must be performed by US-based employees rather than outsourced to a foreign call center.
J. Safeguard of Documents:
Any materials in possession of the contractor must be protected in accordance with the Privacy Act. Materials shall be secured in a locked file cabinet or password protected file when in the contractor’s possession and not in use.
K. Travel:
No travel will be required by the contractor other than to attend meetings with USMS personnel in the Washington, DC local commuting area.
L. Delivery Schedule:
The delivery schedules are provided above with each task. Delivery schedules are based on the date of award of this contract and the test date mutually decided upon by the vendor and the USMS.
M. Period of Performance:
This period of performance for this contract shall be from the date of award through the completion of the work product, estimated September 29, 2025.
Appendix I
The selected contractor must be able to match at least 90% of the USMS work sites on the attached spreadsheet, by offering an automated testing site within 75 miles of each location. The Excel spreadsheet contains a column for zip code, state/country, and city or location of the USMS work sites, along with the estimated maximum number of employees who work at or near that site.
Proposed Tasks: For every USMS work site, the vendor must compare the USMS sites and number of seats needed with a list of their exam site locations (including an address) available for Saturday and Sunday testing, and the number of seats available for Session I and Session II on the primary test date(s) for each location. It is not acceptable for a vendor to simply state that they can meet the site requirements. The vendor must provide the address and zip code of each computerized testing location in comparison to each USMS worksite listed on the Excel spreadsheets and approximate distance from the USMS work site. Without addresses, zip codes and approximate distances, the vendor will not be considered as meeting the requirements.
For example, the response should include:
mandatory worksite from USMS spreadsheet vendor exam site w/address and zip (If not available on Saturday, please note)
Approx. distance from USMS work site (miles)
# of seats needed based on excel
Session I # of seats available
Session II # of seats available
For each match to the USMS testing sites where a sufficient number of seats does not appear to be available for single date, two-session testing, or where Saturday testing is not available, the vendor must list alternatives, such as “employees located at zip code 55555 can take the exam on Sunday,” or “if registration at site XXX exceeds 50, a third testing session will be added on Sunday,” or “vendor will rent space and computer systems at site YYY to accommodate unusual number of test takers,” or “vendor will open site for Saturday testing at a fee of $$$ if less than xxx employees take exam at that site.”
In addition, please provide a separate, detailed plan for each of the following four USMS high workforce population/low testing center capacity regions. (For example, the requirements described above are not met if the vendor lists a single test site with 20 seats and maps this single testing site to five different locations, which has a USMS population of 20 each, for a total of 100 required seats). These three areas are:
· Washington, DC metropolitan area to include all MD, DC and Northern VA locations. The USMS population in this area is approximately 300 – how will this requirement be met?
· Georgia, specifically two regions:
· Atlanta metropolitan area to include Atlanta, Gainesville, Augusta, Macon, and Columbus. How/where will these 60+ employees be tested?
· Southeast, Georgia region to include Savannah, Brunswick, and Glynco, Georgia; and Jacksonville, FL: The normal population for these offices is 67, but the population may increase by at least another 50 if there is a large class taking place at the USMS Training Division in Glynco, GA. How will this requirement be met?
· Southwest Border districts in Arizona (Yuma, Phoenix, and Tucson), Southern California (Los Angeles, San Diego, El Centro, Riverside, and Santa Ana), Nevada (Las Vegas), New Mexico (Las Cruces) and Texas (Alpine, Brownsville, Corpus Christi, Del Rio, El Paso, Laredo, McAllen, Pecos and San Antonio). The Southwest Border offices are geographically dispersed and often employ a large population of USMS employees away from major metropolitan areas. How/where will these 450+ employees be tested, especially in areas such as Del Rio and Laredo, TX; and Pineville, LA area to include Alexandria, LA. The normal population for these offices is about 5, but the population may increase by another 25 if there is a training taking place at the USMS facility in Pineville, LA. How will this requirement be met?
File details come from the government source that posted it. Updated .