ID06170027_Attach_1_PWS_QC_Validations_rev_1_08222017.pdf

Solicitation ID06170027 Attachment 1 revision 1

Performance Work Statement (PWS)

DLPT Quality Control and Validations Defense Language Institute Foreign Language Center (DLIFLC)

August 4, 2017 August 22, 2017

1.0 Introduction

The Defense Language Institute Foreign Language Center (DLIFLC) has a requirement to provide foreign language testing in both commonly and less-commonly taught languages.

1.1 Background

DLIFLC is developing listening and reading proficiency tests -- Defense Language Proficiency Test (DLPT) to assess the foreign language proficiency of military and government personnel. In order to certify the validity of these tests, DLIFLC is contracting with independent reviewers to ensure that these tests meet government standards. DLPT tests must adhere to standards established by the Interagency Language Roundtable (ILR).

ILR reviews are required for the validation and certification of government DLPT tests.

All target-language test materials as well as their renderings in English and target language transcriptions of audio materials are additionally required to be certified as correct and complete by target-language experts before they are published.

1.2 Objectives. To ensure that all DLPT-5 tests measure language proficiency in reading and listening to the same ILR standard.

1.3 Mission Statement. Support the DLIFLC testing mission by exercising target language and ILR quality control over the development of all Defense Language Proficiency Tests. Ensure compliance with US Government proficiency guidelines and the ILR proficiency standards in reading and listening test development through reviewing all DLPT-5 items developed by, or under the guidance of, DLIFLC in all languages.

2.0 Requirements.

2.1 General.

2.1.1 Scope.

The contractor will provide foreign language service in the review process for

a) ILR Review: ILR review approximately 9,650 items in the possible languages listed below. These ILR reviews will be guided by the ILR skill level descriptors and/or other US government test development guidelines. These services shall ensure the integrity of the target language in both written and spoken forms as ILR level appropriate examples of natural language used for communicative purposes. These services will also ensure the accuracy and completeness of English renderings and target language transcriptions, and test item set adherence to the ILR proficiency guidelines.

b) QC and Corrections review: The contractor shall also provide foreign language service for Quality Control (QC) and Corrections for 10,000 reading and listening comprehension passages in the possible languages listed below. These reviews will identify and correct any deficiencies found in the written and spoken target language test passages including the English renderings and target language transcripts.

Below is a list of possible languages involved in the ILR review, and QC and corrections reviews. The languages listed are subject to change because of geo-political shifts, the Government may add or subtract some languages:

Albanian, Algerian, Amharic, Armenian, Azeri, Bengali, Baluchi, Cantonese, Cebuano, Chavacano, French, Greek, Haitian-Creole, Igbo, Hausa, Hebrew, Hindi, Indonesian, Japanese, Javanese, Kazakh, Khmer, Korean, Kurmanji, Levantine, Malay, Mandarin- Chinese, MSA, Norwegian, Persian-Farsi, Portuguese (European and Brazilian), Punjabi, Russian, Serbian/Croatian, Telugu, Somali, Sorani, Spanish, Sudanese, Swahili, Tagalog, Tamil, Tausug, Thai, Turkish, Turkmen, Uighur, Ukrainian, Urdu, Uzbek, Yemeni, Yoruba.

2.1.2 Description of Services. To accomplish the foreign language services above, the contractor shall provide language experts to assist with reviews of DLPT test items.

These services will consist of reviewing the reading and listening texts for authenticity, naturalness, correctness of target language or dialect. These services will also ensure the accuracy and completeness of English renderings, the accuracy and completeness of target language transcriptions of audio materials and ensure that there is one correct answer and finally providing an ILR level for each item. For the QC and Corrections Review, reviewers may be required to make corrections to the passages including re-recordings, re-typing, English renderings and target language transcriptions if errors are detected. For this type of review, they will not be required to assign ILR levels.

The review sites will normally be at Defense Language Institute Foreign Language Center (DLIFLC), unless otherwise designated by the Government.

2.1.3 Period of Performance

The POP is as follows:

Base Year 09/17/2017 – 09/16/2018 Option Year I 09/17/2018 – 09/16/2019 Option Year II 09/17/2018 – 09/16/2020

2.1.4 Place of Performance: The review sites will normally be at Defense Language Institute Foreign Language Center (DLIFLC), unless otherwise designated by the Government. If determined more cost effective to the Government, some review sessions involving smaller quantities may also be conducted at the contractor facility via Defense Connect Online (DCO) if determined more cost effective to the Government

2.1.5 Recognized /Government Holidays.

New Year’s Day Labor Day Martin Luther King Jr.’s Birthday Columbus Day President’s Day Veteran’s Day Memorial Day Thanksgiving Day Independence Day Christmas Day

2.1.6 Travel / Temporary Duty.

2.1.6.1 Travel is required for all target language experts for conducting test reviews on-site at DLIFLC. The Contractor may also periodically be required to travel for the purpose of attending government initiated meetings.

2.1.6.2 The Contractor shall use only the minimum number of travelers and rental vehicles required to accomplish the requirement. Contractor personnel are responsible to make all arrangement for travel, lodging and local transportation without assistance of Government personnel. The Contractor shall coordinate security related paperwork with COR (including obtaining access to military installations) and any other requirements pertinent to this Performance Work Statement. Whenever travel is determined to be necessary by the Government; Contractor personnel shall be available to travel within two weeks’ notice.

2.1.6.3 The Contractor shall submit a proposed cost of Contractor travel. Included in their proposal shall be a breakdown of costs related to the travel costs and shall include as a minimum, (1) transportation costs (airfare/car rental, etc.); (2) lodging and; (3) per diem for each employee. The Contractor shall use Joint Travel Regulations rates as guideline for expenses (These can be found at http://www.defensetravel.dod.mil/site/travelreg.cfm as of May 10, 2011)

2.1.6.4 The Contractor shall submit an expense report to the COR when invoicing for travel costs.

2.1.7 Emergency Services. The Contractor shall have competent personnel trained and capable of dealing with minor personnel injuries. The Contractor or their employees shall immediately notify the COR or the Chief of any accident within 30 minutes of the incident.

2.1.8 Contract Administration and Management.

2.1.8.1 Post Award Conference / Periodic Progress Meetings. The Contractor agrees to attend any post award conference convened by the contracting activity or contract administration office in accordance with Federal Acquisition Regulation (FAR) Subpart

42.5. The CO and Contracting Officer’s Representative (COR) shall meet periodically with the Contractor to review the Contractor's performance. At these meetings the CO will apprise the Contractor of how the Government views the Contractor's performance and the Contractor will apprise the Government of problems, if any, being experienced.

Appropriate action shall be taken to resolve outstanding issues.

http://www.defensetravel.dod.mil/site/travelreg.cfm

2.1.9 Quality Control. The Contractor shall develop and maintain an effective quality control process to ensure services are performed in accordance with this PWS. The Contractor shall develop and implement procedures to identify, prevent, and ensure against recurrence of defective services. The contractor must submit its Quality Control Plan (QCP) to the CO through the COR for review and acceptance no later than 30 days after contract start contract start date.

2.1.10 Contracting Officer’s Representative (COR): The COR will be identified by separate letter. The COR monitors all technical aspects of the contract and assists in contract administration The COR is authorized to perform the following functions: Assure that the Contractor performs the technical requirements of the contract; perform inspections necessary in connection with contract performance; maintain written and oral communications with the Contractor concerning technical aspects of the contract; issue written interpretations of technical requirements, including Government drawings, designs, and specifications; monitor Contractor's performance and notifies both the CO and Contractor of any deficiencies; coordinate availability of Government furnished property, and provide site entry of Contractor personnel. A letter of designation will be issued to the COR (with the contractor cc’d), stating the responsibilities and limitations of the COR, especially with regard to changes in cost or price, estimates or changes in delivery dates.

The COR is not authorized to change any of the terms and conditions of the resulting contract.

2.1.11 Subcontract Management (if applicable): In accordance with Small Business Subcontracting Plan

2.1.12 Contract Employees

2.1.12.1 Key Personnel: Key Personnel: The following personnel are considered key personnel by the Government: Project Manager. The contractor shall provide a Project Manager who shall be responsible for the performance of the work. The name of this person and an alternate who shall act for the contractor when the manager is absent shall be designated in writing to the contracting officer. The project manager or alternate shall have full authority to act for the contractor on all contract matters relating to daily operation of this contract. The project manager or alternate shall be available during normal business hours, Monday thru Friday except Federal holidays or when the Government facility is closed for administrative reasons. The Contractor shall submit documentation of the qualifications of the Project Manager(s) for Government approval, which may include resumes and/or curriculum vitae.

Project Manager(s) Experience. Project Managers must have at least five years of experience in the field of language proficiency testing. Familiarity with the ILR scale is preferred. Project Managers must also have at least two years of experience in item development project management.

2.1.12 Identification of Contractor Employees: Contractor personnel required to wear issued badges by Government security in the performance of this service and return the issued badges at the end of the service.

2.1.13 Non-Personal Services. The FAR provides that each contract arrangement be judged in the light of its own facts and circumstances, with the key question always being whether the Government will exercise relatively continuous supervision and control over the Contractor personnel performing the contract. To ensure that services provided under this contract are non-personal in nature, employees performing services under this contract shall be controlled, directed and supervised at all times by management personnel of the Contractor. The Government will notify the Contractor’s management of all tasks that require completion and the Contractor’s management will assign those tasks to the appropriate Contractor personnel under this contract. The Contractor's management shall ensure that employees comply with the performance standards outlined in this PWS and as required by the contracting officer or the contracting officer's representative (COR).

Contractor employees shall perform independent of and without the supervision of any Government official. Actions of Contractor employees may not be interpreted or implemented in any manner that results in any Contractor employee creating or modifying Federal policy, obligating the appropriated funds of the U.S. Government, overseeing the work of Federal employees, providing direct personal services to any Federal employee or otherwise violating the prohibitions set forth in Parts 7.5 and 37.1 of the Federal Acquisition Regulation (FAR). The Government will control access to the facility and will perform the inspection and acceptance of completed work and tasks.

2.1.13.1 Standards of Conduct. Contractor personnel’s conduct shall not reflect discredit upon the Government. The Contractor shall ensure that personnel present a professional appearance. The Contractor’s employees shall observe and comply with all local policies and procedures concerning fire, safety, environmental protection, sanitation, security, and possession of firearms or other lethal or illegal weapons or substance. The Contractor is responsible for ensuring that any Contractor employees providing services under this contract conduct themselves and perform services in a professional, safe, and responsible manner. Contractor employees shall comply with all DoD and local requirements that establish uniform policies governing smoking in and around Government Owned buildings and vehicles. The Contractor shall remove from the job site any employee for reasons of misconduct or security. In accordance with DOD Directive 5500.7-R, “Joint Ethics Regulation”, Contractor employees must avoid being improperly influenced in the execution of their duties under the contract. Particular attention should be paid to acceptance of gifts/ gratuities, and on non-disclosure of sensitive or classified information.

The Contractor shall ensure employee conduct complies with 41 U.S. C 423 relative to release of acquisition related information or actions or discussions which may prejudice future competitions. The Contractor shall ensure no Contractor employees conduct political related activities or events on the installation.

2.1.14. Organizational Conflict of Interest (OCI). Contractor and subcontractor personnel performing work under this contract may receive, have access to, or participate in the development of proprietary or source selection information (e.g., cost or pricing information, budget information or analyses, specifications or work statements, etc.), or perform evaluation services which may create a current or subsequent Organizational Conflict of Interests (OCI) as defined in FAR Subpart 9.5. The Contractor shall notify the

CO immediately whenever it becomes aware that such access or participation may result in any actual or potential OCI and shall promptly submit a plan to the CO to avoid or mitigate any such OCI. The Contractor’s mitigation plan will be determined to be acceptable solely at the discretion of the CO, and in the event the CO unilaterally determines that any such OCI cannot be satisfactorily avoided or mitigated, the CO may affect other remedies as he or she deems necessary, including prohibiting the Contractor from participation in subsequent contracted requirements which may be affected by the OCI. The Government considers an Organizational Conflict of Interest to exist if the vendor performing under this contract also has a contract to develop Test Items for DLIFLC.

2.2. Special Requirements.

2.2.1 Security of Deliverables. The contractor and all associated sub-contractors’ employees shall comply with applicable garrison, facility, and area commander garrison/facility access and local security policies and procedures (provided by Government representative). The Contractor shall also provide all information required for background checks to meet garrison access requirements to be accomplished by garrison Provost Marshall Office, Director of Emergency Services, or Security Office. Contractor workforce must comply with all personal identity verification requirements as directed by DOD, HQDA, and/or local policy. In addition to the changes otherwise authorized by the changes clause of this contract, should the Force Protection Condition (FPCON) at any individual facility or garrison change, the Government may require changes in Contractor security matters or processes.

2.2.1.1 Confidentiality, Conflict of Interest, and Non-Disclosure Agreement. The Contractor and all subcontractors employed by the Contractor shall sign Confidentiality, Conflict of Interest, and Non-Disclosure Agreement to protect the security of all Government foreign language testing initiatives. The confidentiality agreement shall acknowledge that irreparable harm would be caused to Government if the confidential information obtained by Contractor in the performance of tasks included in this Performance Work Statement were to be revealed to third parties. This agreement shall include an agreement signed by Contractor and subcontractor personnel that the individual will not, directly or indirectly, use, make available, make copies of, sell, disclose, or otherwise communicate to any third party materials or services provided under this contract to the Government. The confidentiality agreement entered into by Contractor or subcontractor personnel in connection with this contract shall be in effect indefinitely. The contractor shall provide copies of the signed Confidentiality, Conflict of Interest, and Non- Disclosure Agreements at the Task order level.

2.2.1.2 Security Procedure. The Contractor or subcontractor personnel will not show any materials pertaining to this contract to any other person, or discuss its contents, beyond normal work requirements. Test materials will be kept safe and secured by the Government.

(Army Regulation 611-5, Personnel and Classification Testing, paragraph 2-4b, 5 February 2008; AR 25-55, Dept. of the Army Freedom of Information Act, paragraph 4-401, 1 November 1997.)

2.2.1.3 Retention of Materials. The Contractor or subcontractor personnel will not retain paper or electronic copies of any contract deliverables or contract related materials following the delivery of contract deliverables to Government, or re-use any of these materials for future private, public, or commercial use. Inventory of Test Materials is restricted by the Government. The Contractor will not maintain an inventory of all test materials received.

2.2.1.4 Report of Destruction.

a) At the end of the on-site review session, the Contractor shall track and record the return or destruction process for all review materials used during the on-site review session. The return or destruction will be recorded by language and by passage and/or item number. The Contractor’s Designated Project Manager shall submit this record with the Final Report to the COR and CO.

b) At the end of the performance period, the Contractor shall submit the cumulative destruction record and a Certificate of Destruction to the COR. A template for the Certificate of Destruction will be provided by the Government. The report can be submitted in paper or electronic digitally-signed format.

c) The Certificate of Destruction accounts for any paper or CDs containing test materials generated by this contract for that set of materials. The Certificate of Destruction shall also include a certification that any electronic versions of test materials for that set of materials have been removed from the Contractor’s or Subcontractor’s computers or electronic storage media.

2.2.1.5 Avoidance of Conflict of interest. For reviewers conducting test reviews of DLPT materials, the Contractor will certify that no reviewers have a conflict of interest or the appearance of a conflict of interest. Potential sources of conflict of interest shall include previous exposure to the test items they will be reviewing, either as test writers or reviewers, personal, or business relationships with the development team for the items they will be reviewing, or being in a position in which they instruct people who will take the DLPT in the language they are reviewing, or being in a position in which they themselves will take the DLPT in the language they are reviewing.

2.2.3 Force Protection/Anti-terrorism Requirements

2.2.3.1 AT Level I Training. This provision/contract text is for contractor employees with an area of performance within an Army controlled installation, facility or area. All contractor employees, to include subcontractor employees, requiring access Army installations, facilities and controlled access areas shall complete AT Level I awareness training within 30 calendar days after contract start date or effective date of incorporation of this requirement into the contract, whichever is applicable. The contractor shall submit certificates of completion for each affected contractor employee and subcontractor employee, to the COR or to the contracting officer, if a COR is not assigned, within 30 calendar days after completion of training by all employees and subcontractor personnel.

2.2.3.2 Access and General Protection/Security Policy and Procedures. This standard language is for contractor employees with an area of performance within Army controlled installation, facility, or area. Contractor and all associated sub-contractors employees shall provide all information required for background checks to meet installation access requirements to be accomplished by installation Provost Marshal Office, Director of Emergency Services or Security Office. Contractor workforce must comply with all personal identity verification requirements (FAR clause 52.204-9, Personal Identity Verification of Contractor Personnel) as directed by DOD, HQDA and/or local policy. In addition to the changes otherwise authorized by the changes clause of this contract, should the Force Protection Condition (FPCON) at any individual facility or installation change, the Government may require changes in contractor security matters or processes.

2.2.3.3 Contractor Employees Who Require Access to Government Information Systems.

All contractor employees with access to a government info system must be registered in the ATCTS (Army Training Certification Tracking System) at commencement of services, and must successfully complete the DOD Information Assurance Awareness prior to access to the information systems and then annually thereafter.

2.2.3.3 Contractors that do not require CAC, but require access to a DoD facility or installation. Contractor and all associated sub-contractors employees shall comply with adjudication standards and procedures using the National Crime Information Center Interstate Identification Index (NCIC-III) and Terrorist Screening Database (TSDB) (Army Directive 2014-05/AR 190-13), applicable installation, facility and area commander installation/facility access and local security policies and procedures (provided by government representative), or, at OCONUS locations, in accordance with status of forces agreements and other theater regulations.

2.2.3.4 iWATCH Training. This standard language is for contractor employees with an area of performance within an Army controlled installation, facility or area. The contractor and all associated sub-contractors shall brief all employees on the local iWATCH program (training standards provided by the requiring activity ATO). This locally developed training will be used to inform employees of the types of behavior to watch for and instruct employees to report suspicious activity to the COR. This training shall be completed within 30 calendar days of contract award and within five (5) calendar days of new employees commencing performance with the results reported to the COR NLT 30 calendar days after contract award.

2.2.3.5 Combating Trafficking In Persons (CTIP). In accordance with PGI 222.17 and FAR 52.222-50, the CO will be notified immediately of any information from any source that alleges a Contractor employee, subcontractor and/or subcontractor employee has engaged in conduct that violates the United States Government “zero” tolerance policy regarding trafficking in persons.

2.2.4 Environmental Requirements.

2.2.4.1 Environment and OSHA. The Contractor shall comply with all local, state, and Federal environmental and occupational safety laws, rules, and regulations. Any apparent conflict between compliance with such local, state and Federal laws, rules and regulations, and compliance with the requirements of the contract shall be immediately brought to the attention of the CO and/or the COR final resolution. The Contractor shall notify the CO and COR in writing in addition to any verbal notification of such conflicts. The Contractor shall be liable for all fines, penalties, and costs which result from violations of, or failure to comply with, all such local state, or Federal laws, rules, and regulations. All unsafe acts or conditions fostered by the Contractor or Contractor personnel may be grounds for the CO or COR, to halt all Contractor performance with a commensurate deduction of monies due the Contractor until such unsafe conditions are corrected. The Contractor shall take due caution not to endanger personnel during performance of this contract. Upon discovery of any hazard such as, but not limited to, fire, or large fuel spill, the Contractor shall immediately notify the CO or COR.

3.0 Tasks, Performance Requirements, and Standards.

3.1 Reviews. The Contractor shall:

a) ILR review (CLIN 0001): provide foreign language services for the review of 9,560 items and by using the ILR descriptors and/or other US government test development guidelines. The review will provide the linguistic and cultural input of one or two reviewers who have a high level of proficiency both in English and the target language.

Each reviewer shall assign independently an overall ILR level for each item reviewed.

Item review and the assigned ILR level (“the ILR level”) will be based on: analyzing the reading or listening passage that is used in developing the test item. This includes ensuring authenticity, naturalness, accuracy of the target language or dialect; analyzing linguistic, discourse and cultural features of each reading and listening passage; ensuring accuracy and completeness of English renderings, and accuracy and completeness of target language transcriptions in listening passages. Ensuring that the input signal and the sound/voice quality are within required specifications in listening items. Review the task question(s) associated with the passage. Confirm the correctness of the key and scoring protocols (answer(s) required from the examinees in the case of CRT tests).

b) QC and Corrections Review (CLIN 0002): Provide foreign language services for the review of approximately 10,000 reading and listening comprehension passages for QC and Corrections review. These reviews will identify and correct any deficiencies found in the written and spoken target language test passages including correctness of target language in test passages, and quality and completeness of English renderings of target language texts as well as correctness and completeness of target language transcriptions of listening texts and their respective English renderings, any deficiency of sound quality in the listening passage. Identification and correction of any deficiency will be guided by Test Production (TP) staffs. The review sites will be at Defense Language Institute Foreign Language Center (DLIFLC), unless otherwise designated by the Government.

c) Provide review reports upon completion of each review session, including:

1) ILR levels assigned to each item by each reviewer using a set of rating criteria based on the ILR and related documents, supported by a corresponding analysis of the reading and listening passages,

2) Reviews of keys or scoring protocols provided by one or two independent reviewers,

3) Comments regarding correctness of target language in test passages, and quality and completeness of English renderings of target language texts as well as correctness and completeness of target language transcriptions of listening texts and their respective English renderings.

4) Comments regarding any deficiency of sound quality in the listening passage

d) The Contractor will provide ILR reviews and QC Corrections of DLPT test items in the languages listed under Section 2.0. Due to unforeseeable geopolitical events, the demand for languages is subject to change during the twelve-month period of performance for the base period as well as any option period. The Contracting Officer (CO) or Contracting Office Representative will notify the contractor no less than thirty (30) days before review date if there is any change in languages other than the ones listed below.

e) The language tests, the number of items to be reviewed, and the review site(s) will be determined by the Government and provided in each task order. The number of materials to be reviewed will vary. Reviews will be conducted at a site determined by the US Government, who may exercise an option for on-line and/or on-site reviews. On-site working sessions may last approximately one to five days. It is estimated that each session will produce between 10 to 100 item reviews per language, depending on the level(s) of the items under review. There will be occasions when the item count will be lower or higher than 10 to 100 per review session.

3.2 Reports

3.2.1 The Contractor shall provide an ILR rating for each item and information on the justification and explanation for rating of each item, including the correct answer for each test item. Each passage and its corresponding test items are collectively referred to as an “item set”. Contractor shall provide target language reviewers no less than 15 working days after notification from the Government.

3.2.2 The Contractor shall ensure that the reviewers apply the ILR language proficiency descriptors and other USG testing documents provided in their ratings.

3.2.3 The Contractor, as well as the reviewers will document their rating with the support of an analysis of the linguistic, discourse and cultural features of the passages that are relevant to a given ILR level; this includes verification of authenticity and correctness of the language or dialect under consideration. In addition the Contractor, as well as the reviewers, shall review Target Language transcriptions and English renderings of each passage for accuracy and completeness.

3.2.4 For each multiple-choice item, the reviewer shall assess whether the key is the single correct answer to the question.

3.2.5 For each constructed response item, the reviewer shall assess whether the scoring protocol completely and accurately reflects the key ideas in the passage that address the question

3.3 Reviewers’ Qualifications

3.3.1 Reviewers shall have an advanced professional proficiency equivalent to educated native speakers in reading and listening in the target language, and have general professional proficiency in reading, listening and writing in English. Government may allow exceptions to this requirement based on needs. Preferred consideration will be given to the offer or who provides reviewers with expertise in language assessment and testing experiences plus one or more of the following fields: (1) linguistics, (2) language teaching, (3) literature, (4) translation, or (5) interpretation studies in the target language.

3.3.2 The Contractor shall document each reviewer’s qualifications. The Contractor shall provide resumes of all reviewers before they can begin work on the contract for Government’s review and approval. The Contractor shall submit resumes no later than 14 calendar days from receipt of a task order. The Government will notify the Contractor of acceptance of reviewers within 2 days of receipt.

3.3.3 The Contractor shall ensure that all reviewers are trained in ILR listening and reading skills, as well as passage rating and text typology assessment under the ILR guidelines.

3.3.4 Each reviewer, including their replacements, shall be approved by the Government.

It is recognized that, in some circumstances, it may not be possible to find reviewers meeting all of the required criteria. In such cases, the Government will determine reviewer eligibility on a case-by-case basis.

3.3.5 For each language, the Contractor shall normally provide two independent reviewers to conduct an ILR review and one independent reviewer to conduct a QC and Corrections.

This will include reviewing a passage and its related test items. The Government may allow the use of one reviewer under certain circumstances such as fewer than 15 days notifications or when it is the best interest of the Government.

3.3.6 The Contractor guarantees that all reviewers can conduct reviews independently of each other, i.e. without consulting or discussing with each other. Government test review experts will guide any discussion subsequent to the first independent analysis and rating for purposes of clarification.

3.4 Issue and Track Payment (Honoraria) to Field Test Participants (CLINs 0003 & 0004)

3.4.1 General. As part of the test development process for DLPT test questions, field-test participants are needed to pilot test questions for various languages. In order to attract participants, the Government must provide a sufficient monetary incentive. This contract shall fulfill the need to pay honoraria to non-military and non-government field-test participants identified by the Government. The Government will plan, coordinate, and conduct annually approximately 20 field-testing sessions nationwide. On average, each field-testing session will involve approximately twenty (25) field-test participants. The Government shall provide the Contractor a list of field-test participants including names, addresses, telephone numbers, and payment amounts. Each list will generally have approximately twenty (25) participants.

3.4.2 Check Processing. During the course of the contract period, approximately 500 checks shall be required to be issued by the Contractor. The cost of each Contractor-furnished check shall be in the range from $50 to $500 per check. The Contractor shall bear the cost of the checks issued and be later reimbursed by the Government through monthly invoices. Checks shall be issued within three (3) working days of receipt of the Government’s list of participants. The Contractor is required to include a Contractor-furnished stamped return envelope that shall be mailed back to the Contractor by each participant to acknowledge his/her receipt of payment/honoraria for his/her services.

Included shall be a Contractor-furnished standard receipt form that shall be signed and dated by the participant. The Contractor shall provide a Point of Contact (POC) to provide assistance to Government personnel and field-test participants regarding payments and statuses of payments. The Contractor shall immediately correct Contractor-caused mistakes or errors.

3.4.3 Invoices. The Contractor shall submit monthly invoices that request payment for the issuance and tracking of payments to participants as well as the amount of each payment to each examinee. Submittal shall first be forwarded to the COR for review and approval. The Contractor shall clearly identify each Contract Line Item (CLIN) or Subcontract Line Item (SLIN) used for invoicing purposes. Included with the Contractor’s invoice shall be (1) a detailed listing of each participant paid; (2) acknowledgement of payment by each paid examinee (if no acknowledgement is furnished, an explanation why no acknowledgment must be attached); and (3) the amount paid to each participant.

3.5 Contractor Manpower Reporting (CMR) (CLIN 0006):

3.5.1 The contractor shall report ALL contractor labor hours (including subcontractor labor hours) required for performance of services provided under this contract for the Army via a secure data collection site. The contractor is required to completely fill in all required data fields using the following web address: http://www.ecmra.mil/, and then click on “Department of the Army CMRA” or the icon of the DoD organization that is receiving or benefitting from the contracted services. Contractors may direct questions to the help desk by clicking on “Send an email” which is located under the Help Resources ribbon on the right side of the login page of the applicable Service/Component’s CMR website.

3.5.2 Reporting inputs will be for the labor executed during the period of performance during each Government fiscal year (FY), which runs October 1 through September 30.

While inputs may be reported any time during the FY, all data shall be reported no later than October 31 of each calendar year.

4.0 Performance Requirements Summary (PRS) Matrix.

Tasks/Deliverable Performance Standard

Performance Threshold / Minimum

Method of Surveillance http://www.ecmra.mil/

Acceptable Standard

(MAS)

2.1.14.

Organizational Conflict of Interest (OCI) The Contractor shall notify the CO immediately whenever it becomes aware that such access or participation may result in any actual or potential OCI and shall promptly submit a plan to the CO to avoid or mitigate any such

OCI.

The Contractor shall notify the CO immediately whenever it becomes aware that such access or participation may result in any actual or potential OCI and shall promptly submit a plan to the CO to avoid or mitigate any such OCI.

Notification within 24 hours

100% inspection

2.2.3 Force

Protection/Anti-terrorism Requirements

The various

ATOPSEC

requirements shall be done in accordance with PWS Paragraphs

2.2.3.1 through 2.2.3.5

No deviation allowed 100% Inspection

3.1(a) ILR Review Test Items

The language services shall be done in accordance with PWS Paragraph 3.0

Reports delivered on time with no acceptable deviation; ILR review in accordance with ILR Skill Level Descriptions (Tech Ex A) and Text Modes (Tech Ex H)

100% Inspection by Government Test Review and Education personnel

3.1 (b) DLPT TL Item Quality Control & Correction

The language services shall be done in accordance with PWS Paragraph 3.0

Reports delivered on time with no acceptable deviation

100% Inspection by Government Testing Production personnel

3.4 Field Test

Payment and Check Processing

The contractor shall pay 100% of the invoices as provided by DLIFLC.

No deviation allowed 100% check of invoices from Contractor;

Customer Feedback

3.5 Contractor

Manpower Reporting

The contractor shall report ALL contractor labor hours (including subcontractor labor hours) required for performance of services provided under this contract for the Army via a secure data collection site.

The contractor is required to completely fill in all required data fields using the following web address:

http://www.ecmra.mil/, and then click on “Department of the Army CMRA” or the icon of the DoD organization that is receiving or benefitting from the contracted services.

Contractors may direct questions to the help desk by clicking on “Send an email” which is located under the Help Resources ribbon on the right side of the login page of the applicable Service/Component’s CMR website..

Delivered 100% on-time, no deviation is allowed

100% Inspection Customer feedback

4.1 Quality control

Plan

Contractor shall deliver to the CO a written Quality Control Plan not later than thirty (30) days after the start of this contract and whenever a significant change is necessitated in the

QCP.

Timely submission and response to Government feedback.

100% inspection

4.0 Quality Control and Quality Assurance

4.1 Quality Control Plan: The contractor shall develop and maintain an effective quality control program to ensure services are performed in accordance with this PWS. The contractor shall develop and implement procedures to identify, prevent, and ensure non-recurrence of defective services. The contractor’s quality control program is the means by which he assures himself that his work complies with the requirement of the contract. In compliance with standards as specified in this document, the contractor shall provide and maintain a Quality Control Plan (QCP) that contains, as a minimum, the items listed below to the Contracting Officer (CO) for acceptance not later than thirty (30) days after the start of this contract and whenever a significant change is necessitated in the QCP. The CO will notify the Contractor of acceptability of the plan within seven days of receipt. The contractor shall make appropriate modifications and submit the revised QCP to the CO within seven days of notification of required changes. After acceptance of the quality control plan the contractor shall receive the contracting officer’s acceptance in writing of any proposed change to his QC system. The plan shall include the following requirements, as a minimum:

(1) A description of the inspection system to cover all services listed in the Performance Work Statement and Performance/Deliverables Matrix. Description shall include specifics as to the areas to be inspected on both a scheduled and unscheduled basis, frequency of inspections, and the title and organizational placement of the inspectors. Additionally, control procedures for any Government-provided keys or lock combination shall be included.

(2) A description of the methods to be used for identifying and preventing defects in the quality of service performed.

(3) A description of the records to be kept to document inspections and corrective or preventive actions taken.

(4) All records of inspections performed shall be retained and made available to the Government upon request.

4.2 Quality Assurance: The government shall evaluate the contractor’s performance under this contract in accordance with the Quality Assurance Surveillance Plan. This plan is primarily focused on what the Government must do to ensure that the contractor has performed in accordance with the performance standards. It defines how the performance standards will be applied, the frequency of surveillance, and the minimum acceptable defect rate(s

4.3 Acceptance: The acceptance of deliverables and satisfactory work performance required herein shall be based on the standards as specified in this task order. Only the COR, Client Representative, or an authorized alternate has the authority to inspect, accept, or reject work performed under this task order. Unless specified otherwise, the Government will provide acceptance, conditional acceptance, or rejection within seven calendar days. If the deliverable is conditionally accepted or rejected, the Contractor shall have seven days to re-submit unless a different date is mutually established. If the Government chooses to accept sub-standard work, personnel qualifications, or other contractor shortfalls due to special circumstances, such acceptance does not obligate the government to accept similar sub-standard performance in the future. In such cases the Government will inform the contractor that standards are not being met per the Quality Assurance process, and the contractor will correct the situation in a reasonable time.

5. Deliverables.

5.1 Pre-Performance Meeting. A meeting between the Government and the Contractor will take place within seven (7) business days of the award of the contract. At the meeting the two parties will review contract deliverables, proposed milestones submitted with the proposal, and implementation issues.

5.2 Reviews. There will be one or more TRE representatives at the review site to oversee the process and ask/answer any questions related to the review of the items or any other process or content related issues. TRE personnel will be responsible for providing the test items. The Contractor shall provide the required equipment (laptops or desktops) and will ensure that at no time the test items shall be copied in any shape or form during the review process. The Contractor shall ensure that all reviewers shall sign the Confidentiality, Conflict of Interest, and Non-Disclosure Agreement required by the US government.

5.2.1 On-site ILR Item Review Report. The Contractor shall ensure that the reviewers submit their reviews at the end of each day to the Contractor’s Designated Project Manager.

The reviews will be combined and compiled into a final report by the Contractor’s Designated Project Manager, who will submit the final report for each review session in a secure electronic format.

5.2.1.1 The final on-site ILR review report will be submitted to the COR and CO. The on-site report shall include the following information:

a) the overall ILR rating assigned by each reviewer for each item,

b) identification and explanations for the incorrect keys (MC) and key ideas (CRT);

c) each reviewer’s comments, statements, justifications, or any other information that forms the basis for their overall ILR rating,

d) information on the completeness and accuracy of the English rendering of the target language passage, this includes transcriptions of spoken passages in the target language.

e) description of bad audio quality / background noise…etc

f) any other information useful for test development purposes as stipulated in Section 6.0. (Technical Exhibits and Definitions).

5.2.2 Monthly Test Item Review Report. The Contractor shall provide TRE each month with a monthly report, which shall include information on test items reviewed during that month, including but not limited to the following list. The report is due by the 5th working day of the following month.

a) total number of items reviewed during the month for all languages,

b) any significant occurrence or event which took place during the on-sit review,

c) significant Period Events (attendance at meetings/training/other),

d) production and Funding Status as of the conclusion of the on-site review,

e) production and Funding Projection as of the conclusion of the on-site review.

5.2.3 In–process Reviews/Meetings. The Contractor shall be required to participate in meetings/conference calls conducted either in-person or over the phone on a quarterly basis. The frequency of meetings may be subject to change upon advance notice (normally with five (5) working days).

5.2.4 Reviewer Orientation. The requirement for a new reviewers orientation (onsite at DLIFLC) will be determined on a case by case basis depending on the reviewer’s previous related experiences like OPI. Reviewers shall demonstrate familiarity with US Government ILR scale and have an advanced professional proficiency equivalent to educated native speakers in reading and listening in the target language, and have general professional proficiency in reading, listening and writing in English. Government may allow exceptions to this requirement based on needs.

6.0 Related Documents.

6.1 Definitions

DLPT Defense Language Proficiency Test. A high stakes (high stakes refers to the highly consequential outcomes for the examinees) proficiency test used by the Department of Defense to assess reading and listening comprehension in a language.

ILR

Passage Review

The analysis and examination of the properties of a passage in order to assign an ILR level to a passage.

Item Review

A process of assigning an ILR level to a test item by analyzing the passage, the stem, options, the credit scheme and protocol.

Additionally, the item review process also includes checking the accuracy of English renderings and Target Language transcriptions.

Test Item

The task on a test for which the examinee provides a response, along with the means for evaluating the response. For multiple choice tests, 1 question + 4 response choices. For a constructed-response test: 1 question + 1 scoring protocol.

Passage The listening or reading passage in the target language.

Item Set Each passage and its corresponding test items are collectively referred to as an “item set.”

CRT Constructed-Response Test. Test takers are asked to select best possible answer out of 4 choice.

MC Multiple Choice Test. Test takers answer multiple questions.

ILR Scale

Interagency Language Roundtable language proficiency scale. Descriptions of language proficiency levels. Levels range from “0” (no proficiency) to “5” (educated native speaker).

Blind Rating Reviewers assign an ILR rating to a passage and related items without knowing the test developer’s intended ILR rating.

Proprietary Information

All information and any idea in whatever form, tangible or intangible, pertaining in any manner to the business of the Government or the Contractor or Subcontractor, which was produced by any Contractor or subcontractor employee in the course of the employees’ employment or otherwise produced or acquired by or on behalf of the Government.

Confidential Information

Broadly defined, confidential information includes all information that has or could have commercial value or other utility in the business in which the Government is engaged in, and all information of which the unauthorized disclosure would be detrimental to the interests of the Government, whether or not such information is identified as confidential information by the Government.

The ILR level The level assigned for a test item or a passage per ILR guidelines.

This level usually is referred to as “the level of difficulty” as specified by the ILR guidelines.

Independent Rating

Rating where two or more reviewers rate without knowing the ratings assigned by other raters.

6.2 Technical Exhibits. The Test Review and Education Division (RE) at the Defense Language Institute Foreign Language Center (DLIFLC) will provide the following Technical Exhibits:

Technical Exhibits DOCUMENT DESCRIPTION Technical Exhibit A ILR Skill Level

Descriptions (go to http//www.govtilr.org)

Language Proficiency Descriptions and Profiles.

DLTP5 tests assess skill levels in accordance with ILR descriptions.

Technical Exhibit B DLPT5 Familiarization Guide Guide intended for test users.

Technical Exhibit C ILR Review Checklist LPAD test review criteria

Technical Exhibit D ILR and Passage Review Form ILR and Passage Review form to be filled out by reviewers.

Technical Exhibit E AR 611-5, dated 2/5/2008 Army regulations on test security.

Technical Exhibit F DR 611-1, dated…