20200318_PWS_DLI_QCVal.pdf

W9124N20R0001 Attachment 01

Version date: 18 March 2020

PERFORMANCE WORK STATEMENT (PWS)

Defense Language Proficiency Test QC and Validations Contract Defense Language Institute Foreign Language Center (DLIFLC)

1. GENERAL:

1.1. Description of Services/Introduction: This is a non-personal services contract to provide Defense Language Proficiency Test (DLPT) Quality Control (QC) and Validations Support. The Contractor shall provide all personnel, equipment, supplies, facilities, transportation, tools, materials, supervision, and other items and non-personal services necessary to perform the services defined in this PWS except for those items specified as Government furnished property and services. The Contractor shall perform to the standards in this contract.

1.2. Background: DLIFLC is developing listening and reading proficiency tests -- DLPT to assess the foreign language proficiency of military and Government personnel. In order to certify the validity of these tests, DLIFLC is contracting with independent reviewers to ensure that these tests meet Government standards.

DLPT tests must adhere to standards established by the Interagency Language Roundtable (ILR). ILR reviews are required for the validation and certification of Government DLPT tests. All target-language test materials as well as their renderings in English and target language transcriptions of audio materials are additionally required to be certified as correct and complete by target-language experts before they are published.

1.3. Objectives: To ensure that all DLPT testing materials measure language proficiency in reading and listening to the same ILR standard.

1.4. Scope: The Contractor will provide foreign language service in the review process for:

a) ILR Review: ILR reviews will be guided by the ILR skill level descriptors and/or other US Government test development guidelines. These services shall ensure the integrity of the target language in both written and spoken forms as ILR level appropriate examples of natural language used for communicative purposes. For the purpose of supporting reviews and quality control, occasionally Target Language Experts (TLEs) will provide target language expertise in the review of listening and reading items/passages at the directorate level. These services will also ensure the accuracy and completeness of English renderings and target language transcriptions, and test item set adherence to the ILR proficiency guidance. To accomplish the foreign language services above, the Contractor shall provide language experts to assist with reviews of DLPT test items. These services will consist of reviewing the reading and listening texts for authenticity, naturalness, correctness of target language or dialect. These services will also ensure the accuracy and completeness of English renderings, the accuracy and completeness of target language transcriptions of audio materials and ensure that there is one correct answer and finally providing an ILR level for each item.

The Contractor shall also provide foreign language service for ILR review up to 9,000 items annually, in the possible languages listed below.

b) QC and Corrections review: QC and Corrections reviews will identify and correct any deficiencies found in the written and spoken target language test passages including the English renderings and target language transcripts. For the purpose of supporting reviews and QC, occasionally TLEs will provide target language expertise in the review of listening and reading items/passages at the directorate level. For the QC and Corrections Review, reviewers may be required to make corrections to the passages including re-recordings, re-typing, English renderings and target language transcriptions if errors are detected. For this type of review, they will not be required to assign ILR levels. The Contractor shall also provide foreign language service for QC and Corrections for up to 4,500 reading and listening comprehension passages annually, in the possible languages listed below.

Below is a list of possible languages involved in the ILR review, and QC and corrections reviews. The languages listed are subject to change because of geo-political shifts, the Government may add or subtract some languages:

Albanian, Algerian, Amharic, Armenian, Azeri, Bengali, Baluchi, Cantonese, Cebuano, Chavacano, French, Greek, Haitian-Creole, Igbo, Hausa, Hebrew, Hindi, Indonesian, Japanese, Javanese, Kazakh, Khmer, Korean, Kurmanji, Levantine, Malay, Mandarin- Chinese, MSA, Norwegian, Persian-Farsi, Portuguese (European and Brazilian), Punjabi, Russian, Serbian/Croatian, Telugu, Somali, Sorani, Swahili, Spanish, Sudanese, Swahili, Tagalog, Tamil, Tausug, Thai, Turkish, Turkmen, Uighur, Ukrainian, Urdu, Uzbek, Yemeni, Yoruba.

1.5. Period of Performance: This is a single award, Indefinite Delivery, Indefinite Quantity contract with an ordering period of 1 Sep 2020 to 31 Aug 2025.

1.6. General Information:

1.6.1. Quality Control: QC is the responsibility of the Contractor. The Contractor is responsible for the delivery of quality services/supplies to the Government (see FAR 52.246-4, Inspection of Services – Fixed-Price).

The Contractor shall develop, implement and maintain an effective Quality Control System which includes a written Quality Control Plan (QCP). The QCP shall implement standardized procedure/methodology for monitoring and documenting contract performance to ensure all contract requirements are met. The Contractors’ QCP must contain a systematic approach to monitor operations to ensure acceptable services/products are provided to the Government. The QCP, as a minimum, shall address continuous process improvement; procedures for scheduling, conducting and documentation of inspection; discrepancy identification and correction; corrective action procedures to include procedures for addressing Government discovered non-conformances; procedures for root cause analysis to identify the root cause and root cause corrective action to prevent re-occurrence of discrepancies; procedures for trend analysis; procedures for collecting and addressing customer feedback/complaints. The QCP shall be provided to the Contracting Officer (KO) and contract specialist via e-mail within 10 business days of the performance start date (PSD). The Government will accept, or return the QCP for revision within 10 business days. Any change to the QCP after initial acceptance requires the review and acceptance by the KO.

1.6.2. Quality Assurance: The Government will evaluate the Contractor’s performance under this contract in accordance with (IAW) the Quality Assurance Surveillance Plan. This plan is a Government only document primarily focused on what the Government must do to assure that the Contractor has performed IAW the requirements of the contract.

The acceptance of deliverables and satisfactory work performance required herein shall be based on the standards as specified in this task order. Only the COR, Client Representative, or an authorized alternate has the authority to inspect, accept, or reject work performed under this task order. Unless specified otherwise, the Government will provide acceptance, conditional acceptance, or rejection within seven calendar days. If the deliverable is conditionally accepted or rejected, the Contractor shall have seven days to re-submit unless a different date is mutually established. If the Government chooses to accept sub-standard work, personnel qualifications, or other Contractor shortfalls due to special circumstances, such acceptance does not obligate the Government to accept similar sub-standard performance in the future. In such cases the Government will inform the Contractor that standards are not being met per the Quality Assurance process, and the Contractor will correct the situation in a reasonable time.

1.6.3. Federal Government Holidays:

New Year’s Day 1st day of January Martin Luther King Jr.'s Birthday 3rd Monday of January Presidents Day 3rd Monday of February Memorial Day Last Monday of May Independence Day 4th day of July Labor Day 1st Monday of September Columbus Day 2nd Monday of October Veterans Day 11th day of November Thanksgiving Day 4th Thursday of November

Christmas Day 25th day of December

1.6.4. Hours of Operation: The Contractor is responsible for conducting business while the facility is open, between the hours of 7:45 ~ 16:45 Monday thru Friday except Federal holidays or when the Government facility is closed due to local or national emergencies, administrative closings, or similar Government directed facility closings.

1.6.5. Place of Performance: The work to be performed under this contract can be performed at Department of Defense Center, 400 Gigling Road Seaside, CA 93955 or off-site. The place of performance will be specified at the task order level.

1.6.6. Security Requirements: Contractor personnel (to include subcontractors) performing work under this contract must have a favorable background investigation to access the non-classified internet protocol router network (NIPRNET). In addition, access to the Fort Leavenworth NIPRNET is limited to United States citizens.

1.6.6.1. Physical Security: The Contractor shall be responsible for safeguarding all Government equipment, information and property provided for Contractor use. At the close of each work period, Government facilities, equipment, and materials shall be secured.

1.6.6.2. Key Control: NOT APPLICABLE.

1.6.6.3. Lock Combinations: NOT APPLICABLE

1.6.6.4. Key-pad Access Control: NOT APPLICABLE

1.6.7. Special Qualifications:

1.6.7.1 Reviewers Qualifications

a) Reviewers shall have an advanced professional proficiency equivalent to educated native speakers in reading and listening in the target language, and have general professional proficiency in reading, listening and writing in English. Government may allow exceptions to this requirement based on needs. Reviewers must have expertise in language assessment and testing experiences plus experience in one or more of the following fields: (1) linguistics, (2) language teaching, (3) literature, (4) translation, or (5) interpretation studies in the target language.

b) The Contractor shall document each reviewer’s qualifications. The Contractor shall provide resumes of all reviewers before they can begin work on the contract for Government’s review and validation based on paragraph a. above. The Contractor shall submit resumes no later than 14 calendar days from receipt of a task order. The Government will notify the Contractor of acceptance of reviewers within 2 days of receipt.

c) The Contractor shall ensure that all reviewers are trained in ILR listening and reading skills, as well as passage rating and text typology assessment under the ILR guidelines.

d) Each reviewer, including their replacements, shall be validated by the Government. It is recognized that, in some circumstances, it may not be possible to find reviewers meeting all of the required criteria. In such cases, the Government will determine reviewer eligibility on a case-by-case basis.

e) For each language, the Contractor shall normally provide two independent reviewers to conduct an ILR review and one independent reviewer to conduct a QC and Corrections. This will include reviewing a passage and its related test items. The Government may allow the use of one reviewer under certain circumstances such as fewer than 15 days notifications or when it is the best interest of the Government.

f) The Contractor shall ensure that all reviewers can conduct reviews independently of each other, i.e. without consulting or discussing with each other. Government test review experts will guide any discussion subsequent to the first independent analysis and rating for purposes of clarification.

1.6.7.2 Special Requirements

a) Test Security Confidentiality Agreement. The Contractor and all subcontractors employed by the Contractor shall sign Confidentiality to protect the security of all Government foreign language testing initiatives. The confidentiality agreement shall acknowledge that irreparable harm would be caused to Government if the confidential information obtained by Contractor in the performance of tasks included in this Performance Work Statement were to be revealed to third parties. This agreement shall include an agreement signed by Contractor and subcontractor personnel that the individual will not, directly or indirectly, use, make available, make copies of, sell, disclose, or otherwise communicate to any third party materials or services provided under this contract to the Government. The confidentiality agreement entered into by Contractor or subcontractor personnel in connection with this contract shall be in effect indefinitely.

b) Security Procedure. The Contractor or subcontractor personnel will not show any materials pertaining to this contract to any other person, or discuss its contents, beyond normal work requirements. Test materials will be kept safe and secured by the Government. (Army Regulation 611-5, Personnel and Classification Testing, paragraph 2-4b, 5 February 2008; AR 25-55, Dept. of the Army Freedom of Information Program, paragraph 4-401, 1 November 1997.)

c) Retention of Materials. The Contractor or subcontractor personnel will not retain paper or electronic copies of any contract deliverables or contract related materials following the delivery of contract deliverables to Government, or re-use any of these materials for future private, public, or commercial use. Inventory of Test Materials is restricted by the Government. The Contractor will not maintain an inventory of all test materials received.

1.6.7.3 Report of Destruction.

a) At the end of the on-site review session, the Contractor shall track and record the return or destruction process for all review materials used during the on-site review session. The return or destruction will be recorded by language and by passage and/or item number. The Contractor’s Designated Project Manager shall submit this record with the Final Report to the COR and KO.

b) At the end of the performance period, the Contractor shall submit the cumulative destruction record and a Certificate of Destruction to the COR. A template for the Certificate of Destruction will be provided by the Government. The report can be submitted in paper or electronic digitally-signed format.

c) The Certificate of Destruction accounts for any paper or CDs containing test materials generated by this contract for that set of materials. The Certificate of Destruction shall also include a certification that any electronic versions of test materials for that set of materials have been removed from the Contractor’s or Subcontractor’s computers or electronic storage media.

1.6.8. Post Award Conference/Periodic Progress Meetings: The Contractor agrees to attend any post award conference convened by the contracting activity IAW Federal Acquisition Regulation Subpart 42.5, Post Award Orientation. The KO, COR, and other Government personnel, as appropriate, may meet periodically with the Contractor to review the Contractor's performance. In progress reviews (IPR) will be at least quarterly. At a minimum, the KO, COR, and Contractor Lead shall attend these IPRs.

At these meetings the Government will apprise the Contractor of how the Government views the Contractor's performance and the Contractor will apprise the Government of problems, if any, being experienced. Appropriate action shall be taken to resolve outstanding issues. These meetings shall be at no additional cost to the Government.

1.6.9. Key Personnel: The follow personnel are considered key personnel by the Contractor shall provide a Project Manager/Task Order Manager (TOM) who shall be responsible for the performance of the work. The name of this person and an alternate who shall act for the Contractor when the manager is absent shall be designated in writing to the KO. The TOM or alternate shall have full authority to act for the Contractor on all contract matters relating to daily operation of this contract.

The TOM or alternate shall be available between 7:45am to 4:45PM, Monday thru Friday except Federal holidays or when the Government facility is closed for administrative reasons.

a) Standards of Conduct. The Contractor personnel’s conduct shall not reflect discredit upon the Government. Contractor shall ensure that personnel present a professional appearance. The Contractor’s employees shall observe and comply with all local policies and procedures concerning fire, safety, environmental protection, sanitation, security, and possession of firearms or other lethal or illegal weapons or substance. The Contractor is responsible for ensuring that any Contractor employees providing services under this contract conduct themselves and perform services in a professional, safe, and responsible manner. Contractor employees shall comply with all DoD and local requirements that establish uniform policies governing smoking in and around Government Owned buildings and vehicles. The Contractor shall remove from the job site any employee for reasons of misconduct or security. In accordance with DOD Directive 5500.7-R, “Joint Ethics Regulation,” Contractor employees must avoid being improperly influenced in the execution of their duties under the contract. Particular attention should be paid to acceptance of gifts/ gratuities, and on non-disclosure of sensitive or classified information. The Contractor shall ensure employee conduct complies with 41 U.S. C 423 relative to release of acquisition related information or actions or discussions which may prejudice future competitions. The Contractor shall ensure no Contractor employees conduct political related activities or events on the installation.

1.6.10. Identification of Contractor Employees: The Contractor shall provide each employee an Identification (ID) Badge, which includes at a minimum, the Company Name, Employee Name and a color photo of the employee. ID Badges for Key Personnel shall also indicate their job title. ID Badges shall be worn at all times during which the employee is performing work under this contract. Each Contractor (to include subcontractors) employees shall wear the ID Badge in a conspicuous place on the front of exterior clothing and above the waist except when safety or health reasons prohibit.

The Contractor (to include subcontractors) shall be responsible for collection of ID Badges upon completion of the contract or termination of employee. All contract personnel attending meetings, answering Government telephones, and working in other situations where their Contractor status is not obvious to third parties are required to identify themselves as such to avoid creating an impression in the minds of members of the public that they are Government officials.

1.6.11. Supervision of Contractor Employees: The Government will not exercise any supervision or control over Contractor employees while performing work under the contract. Such employees shall be accountable solely to the Contractor, not the Government. The Contractor, in turn, shall be accountable to the Government for Contractor and/or subcontractor employees.

1.6.12. Contractor Travel: The Contractor will be authorized travel expenses consistent with the substantive provisions of the Joint Travel Regulation (JTR) and the limitation of funds specified in this contract. All travel requires Government approval/authorization and notification to the COR, prior to making travel arrangements. Travel is required for all TLEs for conducting test reviews on-site at DLIFLC. The Contractor may also periodically be required to travel for the purpose of attending Government initiated meetings.

The Contractor shall submit proposed cost of contractor travel at the task order level.

Included in the task order proposal shall be a breakdown of costs related to the travel costs and shall include at a minimum, (1) transportation costs (airfare/car rental, etc.);

(2) lodging and; (3) per diem for each employee. The Contractor shall use Joint Travel Regulations rates as guideline for expenses (http://www.defensetravel.dod.mil/site/travelreg.cfm)

The Contractor shall submit an expense report to the COR when invoicing for travel costs.

The Contractor shall provide the following information with their WAWF invoice to substantiate any Cost Reimbursable Travel Expense payment claim:

a. A brief description of the travel to include traveler(s) name(s), dates of travel, destination and purpose of travel.

b. Copies of receipts for all airfare, lodging and any expense in excess of $75.00.

c. Claims for payment of per diem shall be accompanied by a corporate worksheet or copy of the employee’s per diem voucher listing the location, dates and per diem rate applied to the claim.

d. Acceptable submissions include paid receipts from airlines and hotels or copies of corporate credit / purchase card statements providing that the statement lists the date, vendor name and amount of purchase.

e. All backup documentation shall clearly indicate the actual date that the Contractor incurred the claimed expense.

f. Failure to provide the required documentation may result in rejection of the Contractor’s invoice as not being a proper invoice as described in FAR 32.905(b)(2).

g. Travel invoices shall be submitted separately from labor invoices.

h. Travel invoices shall be submitted within 60 calendar days of completion of the travel event. Failure to do so will be noted in the performance assessment.

The Contractor shall use only the minimum number of travelers and rental vehicles required to accomplish the requirement. Contractor personnel are responsible to make all arrangement for travel, lodging and local transportation without assistance of

Government personnel. The Contractor shall coordinate security related paperwork with COR (including obtaining access to military installations) and any other requirements pertinent to this PWS. Whenever travel is determined to be necessary by the Government; the Contractor personnel shall be available to travel within two weeks’ notice.

NOTE: Unapproved travel expenses are not a billable expense.

1.6.13. Other Direct Costs: NOT APPLICABLE

1.6.14. Organizational Conflict of Interest (OCI): Contractor and subcontractor personnel performing work under this contract will receive, have access to, or participate in the development of proprietary information which constitutes an OCI as defined in FAR Subpart 9.5. The Contractor shall notify the KO immediately whenever it becomes aware that such access or participation may result in any actual or potential OCI and shall promptly submit a plan to the KO to avoid or mitigate any such OCI. The Contractor’s mitigation plan will be determined to be acceptable solely at the discretion of the KO, and in the event the KO unilaterally determines that any such OCI cannot be satisfactorily avoided or mitigated, the KO may affect other remedies as he or she deems necessary, including prohibiting the Contractor from participation in subsequent contracted requirements which may be affected by the OCI.

1.6.14.1 Avoidance of Conflict of Interest. For reviewers conducting test reviews of DLPT materials, the Contractor will certify that no reviewers have a conflict of interest or the appearance of a conflict of interest. Potential sources of conflict of interest shall include but shall not be limited to previous exposure to the test items they will be reviewing, either as test writers or reviewers, personal, or business relationships with the development team for the items they will be reviewing, or being in a position in which they instruct people who will take the DLPT in the language they are reviewing, or being in a position in which they themselves will take the DLPT in the language they are reviewing.

1.6.14.2 The Contractor shall ensure that all reviewers shall sign the Confidentiality Agreement required by the US Government.

1.6.15. Anti-Terrorism (AT) Level I Training. All Contractor employees requiring access Army installations, facilities and controlled access areas shall complete AT Level I awareness training within 30 calendar days of the performance start date (PSD) or effective date of incorporation of this requirement into the contract, whichever is applicable. The Contractor shall submit certificates of completion for each affected Contractor employee to the COR or to the KO, if a COR is not assigned, within 15 calendar days after completion of training by all employees. AT level I awareness training is available at the following website: http://jko.jten.mil.

1.6.16. Access and General Protection/Security Policy and Procedures. Contractor and employees shall provide all information required for background checks to meet installation access requirements to be accomplished by installation Provost Marshal Office, Director of Emergency Services or Security Office. Contractor workforce must comply with all personal identity verification requirements (FAR clause 52.204-9, Personal Identity Verification of Contractor Personnel) as directed by DOD, HQDA and/or local policy. In addition to the changes otherwise authorized by the changes clause of this contract, should the Force Protection Condition (FPCON) at any individual facility or installation change, the Government may require changes in Contractor security matters or processes.

1.6.16.1. Contractors Requiring a Common Access Card (CAC). NOT APPLICABLE.

1.6.16.2. Contractors Not Requiring a CAC but Require Access to a DoD Facility or Installation. Contractor employees shall comply with adjudication standards and procedures using the National Crime Information Center Interstate Identification Index (NCIC-III) and Terrorist Screening Database (TSDB) (Army Directive 2014-05/AR 190- 13), applicable installation, facility and area commander installation/facility access and local security policies and procedures (provided by Government representative), or, at OCONUS locations, IAW status of forces agreements and other theater regulations.

1.6.17. AT Awareness Training for Contractor Personnel Traveling Overseas. NOT

APPLICABLE.

1.6.18. iWATCH Training. This standard language is for Contractor employees with an area of performance within an Army controlled installation, facility or area. The Contractor and all associated subcontractors shall brief all employees on the local iWATCH program (training standards provided by the requiring activity ATO). This locally developed training will be used to inform employees of the types of behavior to watch for and instruct employees to report suspicious activity to the COR. This training shall be completed within 30 calendar days of contract award and within five (5) calendar days of new employees commencing performance with the results reported to the COR NLT 30 calendar days after contract award.

1.6.19. Access to Government Information Systems. NOT APPLICABLE.

1.6.20. Operations Security (OPSEC) Standing Operating Procedure (SOP)/Plan. NOT

APPLICABLE.

1.6.21. OPSEC Training. NOT APPLICABLE

1.6.22. Information Assurance (IA)/Information Technology (IT) Certification. NOT

APPLICABLE

1.6.23. Threat Awareness Reporting Program (TARP). NOT APPLICABLE

2. DEFINITIONS AND ACRONYMS:

2.1. Definitions:

DLPT

Defense Language Proficiency Test. A high stakes (high stakes refers to the highly consequential outcomes for the examinees) proficiency test used by the Department of Defense to assess reading and listening comprehension in a language.

ILR

Passage Review

The analysis and examination of the properties of a passage in order to assign an ILR level to a passage.

Item Review

A process of assigning an ILR level to a test item by analyzing the passage, the stem, options, the credit scheme and protocol.

Additionally, the item review process also includes checking the accuracy of English renderings and Target Language transcriptions.

Test Item

The task on a test for which the examinee provides a response, along with the means for evaluating the response. For multiple choice tests, 1 question + 4 response choices. For a constructed- response test: 1 question + 1 scoring protocol.

Passage The listening or reading passage in the target language.

Item Set Each passage and its corresponding test items are collectively referred to as an “item set.”

CRT

Constructed-Response Test. Test takers are asked to write a short answer for each question.

MC

Multiple Choice Test. Test takers are asked to select the best possible answer from 4 options.

ILR Scale

Interagency Language Roundtable Skill Level Descriptions.

Descriptions of language proficiency levels. Levels range from “0” (no proficiency) to “5” (educated native speaker).

Blind Rating Reviewers assign an ILR rating to a passage and related items without knowing the test developer’s intended ILR rating.

Proprietary Information

All information and any idea in whatever form, tangible or intangible, pertaining in any manner to the business of the Government or the Contractor or Subcontractor, which was produced by any Contractor or subcontractor employee in the course of the employees’ employment or otherwise produced or acquired by or on behalf of the Government.

Confidential Information

Broadly defined, confidential information includes all information that has or could have commercial value or other utility in the business in which the Government is engaged in, and all information of which the unauthorized disclosure would be detrimental to the interests of the Government, whether or not such information is identified as confidential information by the Government.

The ILR level The level assigned for a test item or a passage per ILR guidelines. This level usually is referred to as “the level of difficulty” as specified by the ILR guidelines.

Independent Rating Rating in which two or more reviewers rate items and passages without knowing the ratings assigned by other raters.

2.1.1. Contractor: A supplier or vendor awarded a contract to provide specific supplies or services to the Government. The term used in this contract refers to the prime.

2.1.2. Contracting Officer: A person with authority to enter into, administer, and/or terminate contracts, and make related determinations and findings on behalf of the Government. Note: The only individual who can legally bind the Government.

2.1.3. Defective Service: A service output that does not meet the standard of performance associated with the PWS.

2.1.4. Deliverable: Anything that can be physically delivered, but may include non-manufactured things such as meeting minutes or reports.

2.1.5. Government Furnished Property (GFP) or Government Property (GP): Property in the possession of, or directly acquired by, the Government and subsequently made available to the Contractor.

2.1.6. Key Personnel: Contractor personnel that are evaluated in a source selection process and that may be required to be used in the performance of a contract by the Key Personnel listed in the PWS.

2.1.7. Physical Security: Actions that prevent the loss or damage of Government property.

2.1.8. Quality Assurance: The Government procedures to verify that services being performed by the Contractor are acceptable IAW established standards and requirements of this contract.

2.1.9. Quality Assurance Surveillance Plan: An organized written document specifying the surveillance methodology to be used for surveillance of Contractor performance.

2.1.10. Quality Control: All necessary measures taken by the Contractor to assure that the quality of an end product or service shall meet contract requirements.

2.2. Acronyms:

AR Army Regulation CFR Code of Federal Regulations CMR Contract Manpower Reporting CONUS Continental United States (excludes Alaska and Hawaii) COR Contracting Officer Representative DA Department of the Army DD254 Department of Defense Contract Security Requirement List DFARS Defense Federal Acquisition Regulation Supplement DLPT Defense Language Proficiency Test DMDC Defense Manpower Data Center DoD Department of Defense FAR Federal Acquisition Regulation ILR Interagency Language Roundtable KO Contracting Officer OCI Organizational Conflict of Interest OCONUS Outside Continental United States (includes Alaska and Hawaii) POC Point of Contact PRS Performance Requirements Summary PSD Performance Start Date PWS Performance Work Statement QA Quality Assurance QAP Quality Assurance Program QC Quality Control QCP Quality Control Program TLE Target Language Experts

3. GOVERNMENT FURNISHED ITEMS AND SERVICES:

3.1. Services: The Government will provide an escort to meet the Contractor employees at the lobby of the Government facility where the ILRs are being held.

3.2. Facilities: The Government will provide the necessary workspace for up to 10~20 Contractor employees to provide the support outlined in the PWS to include desk space and other items necessary to maintain an office environment while on-site for ILR reviews.

3.3. Utilities: The Government will provide all utilities associated with paragraph 3.2 for the Contractor’s use in performance of tasks outlined in this PWS. The Contractor (to include subcontractors) shall instruct employees in utilities conservation practices. The Contractor shall be responsible for operating under conditions that preclude the waste of utilities.

3.4. Equipment: IAW FAR Part 45.000(b)(5): This equipment is not considered true Government property because it is incidental to the place of performance, when the contract requires Contractor personnel to be located on a Government site or installation, and when the property used by the Contractor within the location remains accountable to the Government. Items considered to be incidental to the place of performance include, for example, office space, desks, chairs, telephones, computers, and fax machines. The Contractor shall use Government property only for performing this contract, unless otherwise provided for in this contract or approved by the KO.

Modifications or alterations of Government property are prohibited. Contractor personnel may be required to sign for electronics equipment provided herein. If travel is required, the Contractor assumes responsibility for the Government equipment when not located at the primary place of performance.

3.5. Materials: The Government will provide DLPT5 Familiarization Guide, Test Review and Education (TRE) Test Review Checklist, AR 611-5, dated 2/5/2008, DR 611-1, dated 7/1/1997, Test Security Agreement.

3.6. Reviewer Orientation: Each new reviewer shall be given a reviewer orientation.

A new reviewer’s orientation (onsite at DLIFLC) will be determined on a case-by-case basis depending on the reviewer’s previous related experience, such as serving as a certified OPI tester, or working in development or review of other tests of foreign language based on the ILR Scale, the ACTFL Scale, or other related proficiency scales such as the Common European Framework of Reference (CEFR) for Languages. Reviewers shall demonstrate familiarity with US Government ILR scale and have an advanced professional proficiency equivalent to that of educated natives in reading and listening comprehension of the target language, and have general professional proficiency in reading, listening, and writing in English. Government may allow exceptions to this requirement based on needs.

4. CONTRACTOR FURNISHED ITEMS AND RESPONSIBILITIES:

4.1. General: The Contractor shall provide the required equipment (laptops or desktops) to support the services outlined in the PWS.

4.2. Materials: See paragraph 4.1, above.

4.3. Equipment: The Contractor shall provide the required equipment (laptops or desktops) and will ensure that at no time the test items shall be copied in any shape or form during the review process. See paragraph 4.1, above.

5. SPECIFIC TASKS:

5.1. Task 1, Reviews:

a) ILR review (CLIN 0001): provide foreign language services for the review of items and by using the ILR descriptors and/or other US Government test development guidelines. The review will provide the linguistic and cultural input of one or two reviewers who have a high level of proficiency both in English and the target language.

Each reviewer shall assign independently an overall ILR level for each item reviewed.

Item review and the assigned ILR level (“the ILR level”) will be based on: analyzing the reading or listening passage that is used in developing the test item. This includes ensuring authenticity, naturalness, accuracy of the target language or dialect; analyzing linguistic, discourse and cultural features of each reading and listening passage;

ensuring accuracy and completeness of English renderings, and accuracy and completeness of target language transcriptions in listening passages. Ensuring that the input signal and the sound/voice quality are within required specifications in listening items. Review the task question(s) associated with the passage. Confirm the correctness of the key and scoring protocols (answer(s) required from the examinees in the case of Constructed-Response Test (CRT).

b. QC and Corrections Review (CLIN 0002): Provide foreign language services for the review of reading and listening comprehension passages for QC and Corrections review. These reviews will identify and correct any deficiencies found in the written and spoken target language test passages including correctness of target language in test passages, and quality and completeness of English renderings of target language texts as well as correctness and completeness of target language transcriptions of listening texts and their respective English renderings, any deficiency of sound quality in the listening passage. Identification and correction of any deficiency will be guided by Test Production (TP) staffs. The review sites will be at Defense Language Institute Foreign Language Center (DLIFLC), unless otherwise designated by the Government. For the purpose of supporting reviews and QC, occasionally TLEs will provide target language expertise in the review of listening and reading items/passages at the directorate level.

Provide review reports upon completion of each review session, including:

1. ILR levels assigned to each item by each reviewer using a set of rating criteria based on the ILR and related documents, supported by a corresponding analysis of the reading and listening passages,

2. Reviews of keys or scoring protocols provided by one or two independent reviewers,

3. Comments regarding correctness of target language in test passages, and quality and completeness of English renderings of target language texts as well as correctness and completeness of target language transcriptions of listening texts and their respective English renderings.

4. Comments regarding any deficiency of sound quality in the listening passage.

c. The Contractor shall provide ILR reviews and QC Corrections of DLPT test items in the languages listed under Section 2.0. Due to unforeseeable geopolitical events, the demand for languages is subject to change during the ordering periods. The KO or COR will notify the Contractor no less than thirty (30) days before review date if there is any change in languages other than the ones listed below.

d. The language tests, the number of items to be reviewed, and the review site(s) will be determined by the Government and provided in each task order. The number of materials to be reviewed will vary. Reviews will be conducted at a site determined by the US Government, who may exercise an option for on-line and/or on-site reviews. On-site working sessions may last approximately one to five days.

It is estimated that each session will produce between 10 to 100 item reviews per language, depending on the level(s) of the items under review. There will be occasions when the item count will be lower or higher than 10 to 100 per review session.

5.1.1. Deliverable:

5.1.1.1 ILR Item Review Report. The Contractor shall ensure that the reviewers submit their reviews at the end of each day to the Contractor’s Designated Project Manager. The reviews will be combined and compiled into a final report by the Contractor’s Designated Project Manager, who will submit the final report for each review session in a secure electronic format.

5.1.1.2 At the task order level, the final on-site ILR review report will be submitted to the COR. The on-site report shall include the following information:

a) The overall ILR rating assigned by each reviewer for each item,

b) Identification and explanations for the incorrect keys (MC) and key ideas (CRT);

c) Each reviewer’s comments, statements, justifications, or any other information that forms the basis for their overall ILR rating,

d) Information on the completeness and accuracy of the English rendering of the target language passage, this includes transcriptions of spoken passages in the target language.

e) Description of bad audio quality / background noise…etc.

f) Any other information useful for test development purposes as stipulated in Section 6.0. (Technical Exhibits and Definitions).

5.1.1.3 Test Item Review Report. The Contractor shall provide TRE each month with a monthly report, which shall include information on test items reviewed during that month, which shall include:

a) Total number of items reviewed during the month for all languages,

b) Any significant occurrence or event which took place during the on-sit review,

c) Significant Period Events (attendance at meetings/training/other),

d) Production and Funding Status as of the conclusion of the on-site review,

e) Production and Funding Projection as of the conclusion of the on-site review.

5.1.2. Standard: The Government will have 5 business days to review the deliverable and provide comments back to the Contractor. The Contractor shall submit the final deliverable within 5 business days of receiving the Government’s comments. A negative response from the Government within the time specified above constitutes acceptance.

No more than 2 reports submitted late. 100% accuracy of the content presented, (minor clerical mistakes that are merely a matter of form and not of substance will be noted for correction).

5.2 Task 2, In–process Reviews/Meetings (IPRs).

5.2.1 Deliverable: The Contractor shall be required to participate in meetings/conference calls conducted either in-person or over the phone on a monthly basis. The frequency of meetings may be subject to change upon advance notice (normally with five [5] working days). At the minimum, the COR or Government Representative will attend these IPRs. At these meetings the Government will apprise the Contractor of how the Government views the Contractor’s performance and the Contractor will apprise the Government of contract and task order progress and problems, if any, being experienced. Appropriate action shall be taken to resolve outstanding issues.

5.2.1 Standard: IPRs shall be held in the agreed upon format not later than close of business the 10th working day of the following month or COR agreed to schedule. The Contractor shall prepare and provide to the COR meeting minutes within 5 business days following the IPR.

5.3 Task 3, Issue and Track Payment to Field Test Participants. (CLINs 0003 & 0004)

a) As part of the test development process for DLPT test questions, field-test participants are needed to pilot test questions for various languages. In order to attract participants, the Government must provide a sufficient monetary incentive.

This contract shall fulfill the need to pay honoraria to non-military and non- Government field-test participants identified by the Government. The Government will plan, coordinate, and conduct field-testing sessions nationwide. On average, each field- testing session will involve approximately ten (10) field-test participants.

The Government shall provide the Contractor a list of field-test participants including names, addresses, telephone numbers, and payment amounts. Each list will generally have approximately ten (10) participants.

b) Check Processing. During the course of the contract period, approximately 80 checks shall be required to be issued by the Contractor. The cost of each Contractor-furnished check shall be in the range from $30 to $500 per check. The Contractor shall bear the cost of the checks issued and be later reimbursed by the Government through monthly invoices. Checks shall be issued within three (3) working days of receipt of the Government’s list of participants. The Contractor is required to include a Contractor-furnished stamped return envelope that shall be mailed back to the Contractor by each participant to acknowledge his/her receipt of payment/honoraria for his/her services. Included shall be a Contractor- furnished standard receipt form that shall be signed and dated by the participant. The Contractor shall provide a Point of Contact (POC) to provide assistance to Government personnel and field-test participants regarding payments and statuses of payments. The Contractor shall immediately correct Contractor-caused mistakes or errors.

c) Invoices. The Contractor shall submit monthly invoices that request payment for the issuance and tracking of payments to participants as well as the amount of each payment to each examinee. Submittal shall first be forwarded to the COR for review and approval. The Contractor shall clearly identify each Contract Line Item (CLIN) or Subcontract Line Item (SLIN) used for invoicing purposes. Included with the Contractor’s invoice shall be (1) a detailed listing of each participant paid; (2) acknowledgement of payment by each paid examinee (if no acknowledgement is furnished, an explanation why no acknowledgment must be attached); and (3) the amount paid to each participant.

5.4. Task 4, Contractor Manpower Reporting (CMR).

5.4.1. Deliverable: The Contractor shall report ALL Contractor labor hours (including subcontractor labor hours) required for performance of services provided under this contract for the Army via a secure data collection site. The Contractor is required to completely fill in all required data fields using the following web address:

www.ecmra.mil. Contractors may direct questions to the help desk by clicking on “Send an email” which is located under the Help Resources heading on the right side of the login page.

5.4.2. Standard: Reporting inputs will be for the labor executed during the period of performance during each Government fiscal year (FY), which runs October 1 through September 30. While inputs may be reported any time during the FY, all data shall be reported no later than October 31 of each calendar year. Submission extensions published on the eCMRA website will be honored.

6. APPLICABLE PUBLICATIONS:

DOCUMENT DESCRIPTION

Technical Exhibit A ILR Skill Level Descriptions (go to http//www.govtilr.org)

Language Proficiency Descriptions and Profiles. DLTP5 tests assess skill levels in accordance with ILR descriptions.

Technical Exhibit B DLPT5 Familiarization Guide

Guide intended for test users.

Technical Exhibit C ILR Review Checklist LPAD test review criteria

Technical Exhibit D ILR and Passage Review Form

ILR and Passage Review form to be filled out by reviewers.

Technical Exhibit E AR 611-5, dated 2/5/2008

Army regulations on test security.

Technical Exhibit F DR 611-1, dated 7/1/1997

DLIFLC regulations on test integrity.

Technical Exhibit G Test Security Agreement Document to be signed by anyone with access to test material.

Technical Exhibit H Travel Expense Tracking all the travel expenses for invoices.