ID06180011_SF1449_Exhibit_07_DLIFLC_Reg_611-1_08312015.pdf

* DLIFLC Regulation 611 -1

DEPARTMENT OF THE ARMY

Defense Language Institute Foreign Language Center and Presidio of Monterey 1759 Lewis Road, Monterey, California 93944-5006 31 August 2015

Personnel Selection and Classification

TEST INTEGRITY

Summary. This regulation sets policy for the Defense Language Institute Foreign Language Center and Presidio of Monterey (DLIFLC & POM), DLIFLC Washington (DLI-W) and Language Training Detachments (LTD) in order to secure a fair and equitable language testing program.

Applicability. This regulation applies to all faculty, staff, and students assigned or attached to the DLIFLC & POM, DLI-W and LTD.

Suggested Improvements. The proponent of this regulation is the Language Proficiency Assessment Directorate. Users are invited to send comments or suggested improvements on DA Form 2028 (Recommended Changes to Publications and Blank Forms) to the Commander, DLIFLC & POM, ATTN: ATFL-LPAD, Monterey, CA 93944- 5006.

Distribution. This publication is available electronically.

Table of Contents Page

SECTION I. General Applicability References Purpose Enforceability Scope Definitions General Standards of Conduct SECTION II. School Administered Evaluations Policy Test Security SECTION Ill. Evaluations Administered by the LPAD Directorate Scope Policy

Appendices A. Code of Conduct for OPI Testers and OPI Third Raters B. Checklist of Suggested Policies

* This regulation supersedes DLIFLC Reg. 611 -1, Test Integrity, dated 1 April 2001

DLIFLC Regulation 611-1

SECTION I. General Applicability

1. References.

a. AR 611-5, Personnel and Classification Testing, 5 February 2008.

b. Army Personnel Testing, Test Control Officer Handbook, September 2014.

c. DLIFLC Regulation 350-10, Student Management, Education, Training, and Administration of Resident Programs, 12 December 2014.

d. Command Policy Letter 37, ATZP-CMDT, Command Policy on Academic Integrity, 18 February 2015.

2. Purpose.

a. This regulation defines the responsibilities for all DLIFLC & POM, DLI-W and LTD personnel to include all military and civilian faculty, staff, and students to ensure the integrity and validity of language testing.

b. Establishes policy and responsibilities to review and assess the security and integrity of all language training programs, testing materials, and conduct that determine the overall performance abilities of the students on completion of the program.

3. Enforceability. This regulation is punitive. Individuals who violate this regulation may be subject to criminal, disciplinary or other adverse administrative action. All alleged violations of this regulation shall be referred to the appropriate military commander, supervisor, Dean, or Associate Provost for disposition after consultation with the Office of the Staff Judge Advocate and/or the Civilian Personnel Advisory Center as appropriate.

4. Scope. This regulation covers language tests developed by and used in the schools, such as semester tests or school-level end-of-program tests. This regulation also applies to the formal end-of-course (EOG) Defense Language Proficiency Test (DLPT), Oral Proficiency Interview (OPI), In Course Proficiency Test (ICPT), Final Learning Objective (FLO) tests, Defense Language Aptitude Battery (DLAB) or any other test that may be developed and administered by the school of Language Proficiency Assessment Directorate (LPAD) personnel. For the purpose of this regulation, a language test is defined as:

a. Any test booklet, recorded audio for listening comprehension, recorded audio for speaking proficiency and any ancillary materials such as answer sheets, OPI scoring sheets, scoring keys, conversion tables, material used for test development, and test administration instructions.

b. Manuals or automated systems which are used for administration, scoring, and recording in order to determine an individual's level of proficiency in a particular modality in a particular language.

5. Definitions.

a. Tester: A person who administers and scores any test as described in paragraph 4 above.

b. Test Developer: An individual who has at any time participated in test development or scoring. This includes DLPT passage selector and item writer, recording voice, and FLO sub-skill speaking test or other constructed-response test scorer. In addition , this includes anyone who has served as a test content reviewer or any other capacity involving operational contact with end-of-program or DLPT materials or procedures beyond that normally encountered as a test examinee.

c. Test Material: Includes, but not limited to,

(1) All media on which end-of-program, DLPT, DLAB, OPI , ICPT, FLO tests or their individual components are delivered, including printed booklets, audio tapes, CD ROM disks, computer files, etc.

(2) Any printed, audio, or electronic media containing draft or final-form test passages, questions, spoken stimuli or scripts thereof, answer keys, scoring protocols, or other similar material used in the development, validation, or production of end-of program and DLPTs.

(3) Examinee response data files, test answer keys, or completed answer sheets, including any notes from examinee scratch sheets or pads, when their use is authorized for a particular test.

d. Test Control Officer (TCO): A military member or federal civil service employee appointed by the Commandant and approved by the Army Personnel Testing (APT) Program Manager who is fully responsible for test account management and the safeguarding of examinations. One TCO will be appointed for each test account.

e. Test Examiner: A person who administers tests to Army personnel. Test examiner must be physically present during all phases of testing . Linguists, language instructors, basic skills instructors, retention non-commissioned officers (NCOs), and recruiters (or spouses of linguists, language instructors, basic skills instructors, retention NCOs, and recruiters) or any person who may have a conflict of interest will not be designated a test examiner.

6. General Standards of Conduct.

a. Faculty, staff, and students must not attempt to obtain information regarding test scores for unofficial or non-mission related reasons prior to official public release of the results.

b. Faculty, staff, or students will not ask a certified OPI tester for OPI test results at any time nor will certified OPI testers volunteer OPI test results at any time. No person

DLI FLC Regulation 611-1 will use position, seniority, or other means to exert pressure on a certified oral proficiency tester to deviate from official OPI procedures and lnteragency Language Roundtable scoring standards. All oral proficiency testers must sign a Tester Code of Conduct statement certifying they will adhere to the oral testing procedures and principles prior to certification as an oral tester. (Appendix A)

c. A faculty or staff member certified as a test examiner must remain in the room at all times when a test is being administered. The tester will be in the room from the beginning until the last student has completed the test and the last of the testing materials have been collected from the room and the last computer testing session has been completed.

d. Any person charged with the responsibility to score end-of-program, DLPTs, ICPTs, FLOs or DLABs will accurately record and report test score data. Falsification of scores or the intentional entry of incorrect test scores on official records is strictly prohibited.

(1) In any case where any falsification is suspected at the school level, the responsible school TCO will immediately notify the school Dean. The school Dean will notify the Director of LPAD who will immediately notify both the LPAD TCO and the Assistant Commandant of the suspected or confirmed falsification case. The LPAD TCO will submit a Memorandum For Record (MFR), based on initial facts collected from the school, describing the incident with recommended actions to the Director of LPAD who will notify the Assistant Commandant and Commandant. The Commandant will determine if an investigation is warranted .

(2) If falsification is suspected at the Test Management Division (LPAD-TM) level, the LPAD TCO will immediately notify the Director of LPAD and will follow-up with a MFR describing the incident and recommended actions. The Director of LPAD will immediately notify the Assistant Commandant and Commandant. The Commandant will determine if an investigation is warranted.

e. DLIFLC Regulation 350-10, Chapter 4, specifies the current grading criteria for DLIFLC. Faculty members must not deviate from these criteria or attempt to manipulate test scores. Teachers must not use test scores as a form of incentive or motivation for students. All grading must adhere to current guidance.

SECTION II. School Administered Evaluations

1. Policy.

a. The school will inform examinees that any discussion or disclosure of the content of the school-level tests is strictly prohibited . Any discussion or disclosure of test material that can potentially compromise the integrity of the testing program constitutes a violation of this regulation.

b. Except for providing general feedback on test results, to the extent and in a manner authorized by the school Dean or designee, faculty and staff members will not seumas.rogan Highlight discuss or permit discussion of the content of any test materials with or within the vicinity of students.

c. Beyond the feedback authorized by the school Dean, students will not solicit any information regarding test materials from faculty or other personnel or from students who have previously taken the test, nor will they maintain or attempt to develop lists of topics, passages, or questions which purportedly appear on the tests. Such effort to solicit, maintain or attempt to develop lists of topics, passages, or questions that may compromise the integrity of the testing program is prohibited.

d. The school Dean, through the school TCO, will ensure that the security of the school's testing program is maintained at all times. This security includes protection against loss and against compromise because the unauthorized disclosure of test materials will adversely affect the educational mission of DLIFLC. Examples of test compromise situations that require investigation and appropriate action include, but not limited to:

(1) Review of, or access to, specific test material by students in any circumstance other than during a normal scheduled test administration. Formative feedback given to students by faculty after test sessions does not necessarily constitute test compromise.

(2) Oral or written discussion about the contents of test material with students, except in a manner and to the extent explicitly authorized by the school Dean.

(3) Except as required for official purposes and under proper security conditions, the reproduction or copying of any test material to include sample items used for test development, in whole or in part, for any purpose.

(4) Removal of test material by an examinee from the testing room.

(5) Leaving an examinee, or group of examinees, unsupervised at any _time during a test administration session.

2. Test Security.

a. Overall accountability for the security and control of tests rests with the school Dean. Responsibility for individual functions may be delegated by the school Dean, in writing, to a primary and alternate staff member within the school. The appointed individual(s) act as the school TCO.

b. All schools that have geographically separated facilities (schools with more than one distinct location) are authorized at the discretion of the school Dean to appoint a primary and alternate TCO for each location.

c. All physical media containing school-level major tests or module, semester, unit, and combined tests will be stored in a centrally located and secure location or cabinet.

seumas.rogan Highlight

The keys or combinations of the central storage area or cabinet will be maintained by the school TCO and kept secure within the school premises.

d . A security log for the test storage area will be maintained. The log must record storage facility opening and closing activities each time test material is accessed from the secure storage area.

e . A complete inventory of test material will be conducted at least annually or upon change of school Dean or TCO. Any discrepancies will be immediately brought to the attention of the school Dean. Test materials inventory files will be retained for three years.

f . All school-level test material will be consecutively numbered, with the copy number indelibly marked on each item. The words FOR OFFICIAL USE ONLY will be stamped or preprinted on each test booklet, audiotape, CD, or other test component.

g. Destruction of obsolete or unusable test material will be accomplished by the TCO, with the details of destruction annotated in a memorandum and stored with the previous inventory list to which they relate.

h. Test components will be logged out and in on a test control log maintained for this purpose. This log will be retained for three years.

i. Loss, compromise, or suspected compromise of tests will be reported to the School Dean not later than the first working day following known or suspected loss or compromise. The school Dean will immediately notify the Director of LPAD and the LPAD TCO. The LPAD TCO will provide a MFR, based on initial facts collected from the school, describing the incident and recommended actions. The Director of LPAD will immediately notify the Assistant Commandant and Commandant.

j. The Director of LPAD is responsible for oversight of all DLIFLC test security and administration procedures and practices. To ensure compliance with this regulation and other Department of Defense (DoD) and Army testing regulations, LPAD is authorized to review school test security practices and procedures on a quarterly basis at a minimum, or at the request of the school when necessary.

SECTION Ill. Evaluations Administered by the Language Proficiency Assessment Directorate

1. Scope. This section covers end-of-program tests, including all examinations used to evaluate the lnstitute's Final Learning Objectives (FLOs), including the reading comprehension and listening comprehension DLPTs; OPI ; the performance sub-skill tests; ICPT, EOC tests; any other subsequently developed FLO tests, and the Defense Language Aptitude Battery (DLAB).

2. Policy.

a. Test examiners will inform examinees that discussion of the content of any LPAD administered test is prohibited and can-warrant disciplinary action. Any discussion or disclosure of test material that can potentially compromise the integrity of the testing program constitutes a violation of this regulation.

b. Faculty and staff members will not discuss or otherwise permit discussion or disclosure of the content of EOG tests with any person. Only those LPAD personnel who are currently assigned as test developers or LPAD personnel whose duties require the discussion of test content, may do so only within the confines of their assigned work area and only to the extent required for their official duties.

c. Faculty, staff, and students will not solicit any information regarding the DLAB or end-of-program tests from testers, test developers, test examiners or from examinees who have taken the test, nor will they maintain or attempt to develop lists of topics, passages, or questions which purportedly appear on the test.

d. Unless currently assigned to an officially recognized end-of-program test development, review, revision, or validation project, or otherwise officially authorized to handle secure test material as a necessary component of their regular employment, DLIFLC faculty and staff members will not have access to end-of-program test material.

Specifically, DLIFLC faculty and staff are prohibited from taking EOC tests as examinees unless they are requested to do so as participants in an official LPAD piloting or validation program.

e. Loss, compromise, or suspected compromise of DLAB or end-of-program test material has potential DLIFLC, military service-wide, and DoD implications. Upon discovery of loss, compromise, or suspected compromise the individual who discovers or becomes aware of the incident will immediately report the incident up their chain of command to reach the Director of LPAD within the same duty day. The Director of LPAD will immediately notify both the LPAD TCO and Assistant Commandant. The LPAD TCO will submit a MFR, based on initial facts collected, describing the incident and with recommended actions to the Director of LPAD. The Director of LPAD will notify the Assistant Commandant and Commandant. The Director of LPAD will determine further actions IAW AR 611-5 and the specific situation. Examples of test compromise situations that require investigation and appropriate action include, but not limited to:

(1) Review of, or access to, end-of-program, EOG test material, or the DLAB by other than authorized individuals, as described in paragraph 2d above.

(2) Oral or written discussion about the content of end-of-program, EOG test material or the DLAB by an authorized person with an unauthorized person.

(3) Oral or written communication about the content of end-of-program, EOC test material or the DLAB by an examinee with any other person, under any circumstances.

(4) Oral or written communication about the content of end-of-program, EOC test material or the DLAB by a Test Developer who participated in test development, review, or revision with an instructor, examinee, or other person not officially assigned to the project.

(5) The development, reproduction, distribution, or possession of a practice test not authorized by LPAD, or of any other material or media containing identical or very similar content to that of an official end-of-program or EOC test.

(6) Except as required for official test development or administration purposes and under proper security conditions, the reproduction or copying of any end-of program or EOC test material to include sample items used for test development, in whole or in part, for any purpose.

(7) Removal of test material by an examinee from the testing room.

(8) During a test administration session, leaving an examinee or group of examinees unsupervised while test material is in the possession of or otherwise accessible to the examinee(s).

(9) Any official or unofficial, formal or informal training that concentrates on teaching end-of-program or EOC tests, or that emphasizes information known or purported to be on specific tests. A specific exception allows for conversational practice of language tasks in many diverse content areas throughout the course, without specifically rehearsing students for any particular line of questions. In order to discourage the practice of rehearsing students, all testers are instructed to identify and discount rehearsed language as part of a speaking sample during an OPI.

(10) Review of test content by members of inspection teams or similar parties, without specific approval from the DLIFLC Commandant or authorized representative.

This does not prohibit the visual inspection, in the presence of the LPAD TCO or other authorized representative, of the front covers of test booklets or test-material labels for test inventory verification .

f. All test material will be consecutively numbered, with the copy number indelibly marked on each item. The words FOR OFFICIAL USE ONLY will be stamped or preprinted on each test booklet, audiotape, CD, or other test component.

DISTRIBUTION:

G

Ov\~~Oni\ p~u}p J. D~lE;T COL, Ml Commanding

Appendix A:

Defense Language Institute Foreign Language Center Language Proficiency Assessment Directorate Proficiency Standards

CODE OF CONDUCT FOR OPI TESTERS

AND

OPI THIRD/FOURTH RATERS

General Code of Conduct:

• I will strictly adhere to the official OPI testing procedures and DLIFLC Regulation No. 611-1 .

• I will use only the lnteragency Language Roundtable (/LR) Language Skill Level Descriptions for Speaking and the Rating Factor Grid to rate the performance of an examinee in any test.

• I will notify Proficiency Standards (LPAD-PS) of any attempt to solicit information about the test, or to negotiate or influence my ratings.

• I will not discuss the content/rating of any Oral Proficiency Interview that I conduct or third/fourth rate before, during, or after the test, with anyone, including:

examinees, other students/job applicants, other testers, non-testers, and civilian or military supervisors.

• I will consult with personnel of Proficiency Standards, or Test Management, if I have any questions or concerns.

• I will adhere to Command EEO/AA policies in the conduct of tests and reporting of test scores.

OPI Tester Code of Conduct:

• I will rate independently, and not discuss the test or my rating with my co-tester.

• I will not seek any information pertaining to other OPls from anyone.

• I will notify the Dean or Associate/Assistant Dean of my School or Proficiency

Standards (LPAD-PS) of any attempt to solicit information about the test, or to negotiate or influence my ratings.

• I will not bring a cell phone or any electronic devices into the testing room.

• I agree to attend Test Review sessions as required .

• I understand that all OPI tests are the property of the US Government. Any recording may be used for statistical, research and training purposes by the Defense Language Institute. No personal information will be disclosed. I give my consent for the use indicated above.

OPI Third/Fourth Rater Code of Conduct:

• I understand that the provided recordings constitute controlled test material For Official Use Only (FOUO).

• I will listen to the recordings using a headset so that no one else can hear the test material.

• I will ensure that no unauthorized person reads the 3rd or 4th rating forms.

I have read, I understand, and I agree to abide by this Code of Conduct. I understand that failure to abide by it, is grounds for decertification by LPAD-PS and may result in adverse administration action.

Full Name:

Signature:

Date Signed:

Appendix B: Checklist of Suggested Policies

D (Who has) Access to Extremely Sensitive Materials o Develop policies and procedures regarding access to extremely sensitive information (such as test items, test forms, and test-takers data) , non-disclosure agreement, confidentiality and conflict of interest agreements, security audits, proprietary rights, and equipment and materials management, and removal of access rights at termination of relationship with organization.

• Confidentiality and Conflict of Interest

• Equipment and Materials Management

• Removal of Access rights at Termination of Relationship

D Information Security Policy o Establish an Information Security Policy to safeguard the security of information and to monitor its compliance.

D Employment Agreement(s) o Employees, contractors, and vendors must agree to terms of confidentiality and conflicts of interest agreements by signing a document stating such terms.

o Employees, contractors, and vendors must agree to the code of ethical and lawful business practices by signing a document stating such terms.

o Employees, contractors, and vendors must participate in training regarding security measures and the appropriate course of action in probable scenarios to maintain confidentiality, security of information and safety of equipment at test centers.

o Contractors and vendors must agree to allow announced and/or unannounced audits at their sites.

o Suggested policies and documents to be instituted for monitoring compliance of this standard

D Pre-employment Agreement o See above (Employee Agreement(s))

D Employee Handbook o See above (Employee Agreement(s))

D Non-Disclosure/Confidentiality Agreement - Vendor o See above

D Conflict of Interest Policy for Employees, Contractors and Vendors o See above (Employee Agreement(s))

D Policy on Handling of Exam Scores o Test Control Officers and assigned staff must agree to terms of test security operations by signing a document stating such terms.

o Test Control Officers and assigned staff agree to perform the following functions:

• Safeguarding test materials and information

• Managing security compromise incidents as directed

• Allowing announced or unannounced reviews of test security operations

• Providing regular reports on all test security operations o Suggested policies and documents to be instituted for monitoring compliance of this standard include:

• Policy on New Test Center Consideration

• Policy on Closing a Test Center

• Policy on Handling of Exam Scores

• Examination Administration Manual

• Crisis Management Plan

• Examination Irregularity Report

• Warning Notice To Test-Takers Failing To Follow Test Examiner's Instructions

• Conflict of Interest Policy for Test Administrators o See above (Employee Agreement(s))

• Remote Access and Wireless Access o The organization has a policy regarding remote access and wireless access to secure and sensitive data.

• Frequency of Security Procedure Training o Policies on how often individuals in the organization are trained/retrained on security procedures will be produced.

• Security Plan o The organization's program has a comprehensive Security Plan. Actions are defined for security breaches and a written, complete, Security Plan exists.

• Security Incident Response Plan o The organization has a documented Security Incident Response Plan.

• Security Manual o Providing all individuals with a current security manual (or with the portion that applies to them) which describes the security procedures for which they are responsible.

• Tracking Mechanism for each Policy and Procedure o Develop a tracking mechanism for each policy and procedure and specify how compliance is defined and achieved.

• Information Security Policy o See Appendix B. (Information Security Policy)

• Non-Disclosure/Conflict of Interest Policy for Employees, Contractors and Vendors o See Appendix B.

• Test-takers Attestation (Non-Disclosure/Non-Conflict of Interest Declaration)

• Test-takers Code of Conduct

• Test Administration Manual

• Crisis Management Plan

• Examination Irregularity Report

• Warning Notice to Test-Takers Failing to Follow Proctor's Instructions

• Non-disclosure/Confidentiality Agreement for Item Writers

• Prohibited Candidate Registry o A Prohibited Candidate Registry is kept and maintained of those individuals who are permanently or temporarily ineligible for testing.