20170518_Attach3_PWS-Tech-Exhibits_12-22.pdf

Technical Exhibit 12

Defense Language Proficiency Test

DLPT5

Upper Range Constructed-Response Test

Specifications

The full specification document for the DLPT5 consists of a number of parts:

1. A General Description of the test, which is for public dissemination (pp. 2-6)

2. Test Development Specifications, which are kept private and internal to EST. This includes:

a. Tables of Specifications for the Operational Linear Test (p. 8)

b. A table of content areas to be covered in the test (p. 9)

c. The Composition of the Sub-Pools for Each ILR Level (pp. 10-19)

3. General Guidelines on Text Selection (pp. 20-21)

4. Specifications for Test Administration and Scoring

a. Administration of the Operational Forms of the DLPT5 (p. 22)

b. Standard instructions for test booklets/computer sessions (pp. 23-26)

c. Scoring of the DLPT5 (p. 27)

DLPT5

General Test Specifications Revision: February 15, 2006

DLPT5-CRT

General Description (For Public Dissemination)

Purpose of the DLPT The purpose of the DLPT5 is to assess the language proficiency of military and civilian government employees, by testing comprehension of a wide variety of realistic written and spoken materials over a wide range of topics. Each DLPT5 is designed to measure proficiency in one target language, regardless of how it has been acquired. Scores are used to determine eligibility for certain jobs and for pay bonuses. The test is designed for native speakers of American English, although it may be used for native speakers of other languages who have near-native English proficiency. Typically, examinees taking the upper-range DLPT5 are those who have taken the lower-range test and achieved a rating of level 3 within the previous three-month period.

Administration of the DLPT5 The DLPT5 has two separate sections, Reading Comprehension and Listening Comprehension, which are administered as two separate tests. Each section is allotted 3 hours for administration.

Each section is constructed from a separate pool of available questions, and different forms of the test are constructed from the pools. Tests will be administered in linear form by computer, and will assess proficiency from Interagency Language Roundtable (ILR) levels 3 to 4. Some passages and questions may also appear on the lower-range test.

Test Format

Reading Comprehension Section The test consists of a series of approximately 14 written passages with questions based on each passage. Passages vary in length and have up to 3 associated questions. The total number of questions in this section is approximately 35.

The tasks given to the examinee are all constructed-response questions, each of which consists of a question or prompt given in English. For each question, there is a box on the screen in which examinees type their answer in English. A brief orientation in English precedes each passage and sets the stage, informing the test-taker from what kind of source a reading text was selected.

Examinees are able to navigate freely between passages, both forward and backward.

Listening Comprehension Section The test consists of a series of approximately 14 spoken passages with questions based on each passage. Passages vary in length and have up to 3 associated questions each. The total number of questions in this section is approximately 35.

The tasks given to the examinee are all constructed-response questions, each of which consists of a question or prompt given in English. For each question, there is a box on the screen in which examinees type their answer in English. A brief orientation in English precedes each passage and sets the stage, informing the test-taker under what circumstances a listening text would have been heard.

DLPT5

General Test Specifications

In the Listening Comprehension section, examinees are able to control when the audio for the next passage begins, but have to listen to the entire audio for each passage and to all of the passages in order.

Additional Screens In addition to the test content, the examinee sees a number of introductory screens including personal-data-gathering screens, disclaimer screens, instructional screens (both navigational and content-related), and a sample passage and questions. Also, following the final question there is a review screen that allows the examinee to answer any unanswered questions or change any previous answers.

Scores Reported A separate score is provided for Reading Comprehension and Listening Comprehension. These are reported on the ILR scale, from levels 3 through 4. Scores are based on the number of passages credited to the examinee at each ILR level. The examinee is given a rating roughly corresponding to the highest level at which he or she is given credit for at least 67% of the passages at that level, as long as he or she achieved that percentage on the level(s) below that one.

Each test is rated by at least 2 trained scorers using a detailed scoring protocol that includes instructions on the credit-awarding requirements for each passage. Disagreements are resolved by asking more scorers to rate the test.

Communicative Skills Assessed Although DLPT5 is a test of comprehension, not production, it is designed to test a wide range of communicative language ability, including:

• Grammatical knowledge: Test-takers are tested on their knowledge of vocabulary, syntax, morphology, phonology, the writing system, and so forth, by means of tasks requiring understanding of explicitly stated information and of expressions in context.

• Discourse knowledge: Test-takers are tested on their ability to understand discourse features, such as cohesive devices, pronominal reference, and rhetorical schemata, as well as the ability to deal with the structure of unplanned spoken discourse.

• Sociolinguistic knowledge: Test-takers are tested on their knowledge of the appropriateness of language use; at the higher proficiency levels, they are also tested on their ability to understand cultural references and non-standard features of the language.

The balance of these abilities will vary depending on the targeted ILR level of the text.

Text Selection Texts are selected based on a variety of criteria, principally, ILR proficiency descriptors, topic, and degree of authenticity. Texts are selected from sources intended for the general reader or listener.

ILR Descriptors The ILR descriptors provide general guidelines for the texts appropriate at each level, and these are used as the main basis for text selection. However, some languages have characteristics that are not covered by the ILR descriptors (e.g., honorifics, use of classical language, tones, etc.), and text selection will take these language-specific issues into account where appropriate.

Topical Content Topics for both listening and reading are based on the following areas:

• military and security issues

• economic and political matters

DLPT5

General Test Specifications

• scientific and technical issues

• cultural and social issues

• physical, political and economic geography.

The balance of topics commonly found in any target-language situation, or culture, varies considerably from one culture to another—especially at the higher levels. (For example, very little public discussion of military issues is allowed in some countries.) Hence, emphasis is placed on those topics that are relevant to the particular target-language culture.

Degree of Authenticity The DLPT5 will emphasize authentic language. This means that to the extent that it is practical, texts have the characteristics of texts found in the target-language situation, and texts will require the language-processing skills typical of target-language use. Some texts may be modified as necessary.

Reading Comprehension Section

Construct Definition:

Reading Comprehension is defined as the ability to:

• process samples of realistic written discourse automatically and reasonably quickly;

• understand the linguistic information that is unequivocally included in the text;

• make any author-intended inferences that are unambiguously implied by the content of the text.

DLPT5 emphasizes the ability to understand transactional language (i.e., language intended to convey ideas), but may also include sufficient interactional language (i.e., language intended to maintain social relationships) in order to preserve the authenticity of the text.

Reading Texts These will vary based on characteristics stated in the ILR Skill Level Descriptions. Some examples and characteristics are given below. Note that the examples given do not exhaust the possible text types that might be used.

Level 3 Editorials, commentaries, biographies with critical interpretations, criticisms, reports intended for the general reader on complex issues or specialized topics, philosophical argumentation, opinion pieces, political analysis. These texts demonstrate a wide variety of discourse structures and a wide range of vocabulary. They contain complex argumentation, including hypothesis, supported opinion, analysis, and subtle implications. Main ideas are often not stated explicitly and require “reading between the lines.” From two to three questions may accompany a Level 3 text. A Level 3 text can be up to 300 words longs, measured by its English rendering.

Level 4 Think-pieces, essays, philosophical expositions, satiric writings, other sophisticated humor. These texts are linguistically and conceptually extremely dense. They reflect the author’s unique thinking about an abstract topic or complex issue and usually display unexpected means of argumentation or unusual turns of thought. They use sophisticated and complex language with virtuosity, often in figurative and subtle ways. From one to five questions may accompany a Level 4 text.

DLPT5

General Test Specifications

Proficiency Skills Tested These vary according to the targeted ILR level of individual items, but overall the DLPT5 targets the following reading skills. The ability to:

• understand main ideas, or the gist of a text.

• understand supporting detail.

• understand the cohesive relationships between parts of the text.

• understand grammatical relationships within the text.

• understand the meaning of vocabulary, or expressions, in context.

• make inferences that the author intended the reader to make.

• understand the author’s tone or attitude.

• compare and contrast points of view.

• synthesize information or ideas.

Listening Comprehension Section

Construct Definition:

Listening Comprehension is defined as the ability to:

• process samples of realistic spoken language automatically and in real time;

• understand the linguistic information that is unequivocally included in the text;

• make any speaker-intended inferences that are unambiguously implied by the content of the text or by intonation, accent, or other oral characteristics.

Listening Texts Spoken texts are either taken from prerecorded sources or created in-house. Texts are genuine or realistic samples of the language as it is spoken in the target-language environment.

The spoken texts used vary with regard to two factors that affect difficulty:

• Content, which ranges from less challenging to more challenging, based on the characteristics of the ILR Scale and the characteristics of the target language.

• Oral characteristics, (e.g., phonological modification, hesitation phenomena, speech rate, level of formality, colloquialisms, number of speakers, etc.), with informal texts having more oral characteristics, and more formal texts having fewer of them.

Texts with more oral characteristics tend to be more difficult to process than those with fewer.

The degree to which a text has oral characteristics will be based on the characteristics of the target-language-use situation.

Texts are good-quality audio, although some texts may have background sounds added to provide authenticity, and other texts that have been recorded from authentic media sources will have the sound quality of the original source. Texts can be up to about 2 minutes long, although most are considerably shorter.

Examinees will hear each text twice.

Some examples of possible text types and their general characteristics are given below. Note that the examples given do not exhaust the possible text types that might be used.

Level 3 Debates, talk shows, movies, discussion forums, lectures, interviews, speeches, etc. Texts deal with complex concrete or abstract topics and generally contain analysis, opinions, and/or subtle

DLPT5

General Test Specifications implications. They may be spoken at a normal rate of speech or sometimes at a somewhat faster rate than normal. Texts may have a wide range of phonological modifications and demonstrate a variety of discourse structures. They may contain colloquial or common slang expressions. Two questions accompany each Level 3 text. A Level 3 text may be up to 2 minutes long.

Level 4 Debates, movies, discussion forums, lectures, speeches, intellectual discussion, formal oratory.

These texts may be formal or informal in nature and typically deal with abstract topics that are conceptually dense. They are spoken at normal or fast rates of speech and have an extensive range of “oral characteristics.” They may also contain non-intrusive background noise. The general language use and word choice in Level 4 texts is creative, figurative, and often idiosyncratic in nature, demonstrating the speaker’s virtuosity in expressing him-/herself. From one to three questions may accompany a Level 4 text.

Proficiency Skills Tested These will vary according to the targeted ILR level of individual items, but overall the DLPT5 targets the following listening skills. The ability to:

• understand explicit main ideas.

• understand major details.

• understand the sequence of events.

• understand common indirect speech acts.

• understand speaker-intended implications.

• understand implications conveyed by suprasegmentals (e.g., intonation, stress, etc.).

• separate fact from opinions.

• understand register.

• understand colloquial and idiomatic expressions.

• understand the speaker’s tone or attitude.

DLPT5

General Test Specifications

Defense Language Proficiency Test

DLPT5

Constructed-Response Test Development Specifications

Private and Confidential For the Internal Use of the Test Development and Standards Division, DLIFLC

Not for External Distribution

DLPT5

General Test Specifications

Table of Specifications for Operational Linear Forms

For all languages, there will also be a DLPT5 available to assess examinees whose proficiency is at or below ILR level 3. That test may also be a constructed-response test, or it may be a multiple-choice test. For some languages, the range of the test as a whole will be limited to the lower range (0 through 3).

The length of passages for Reading is calculated in terms of the number of words in an English translation of the passage.

Reading Comprehension Section

Level Approx. % of Total Test

Range of Items

Approx. Length of Passages (in words)

Range of Items per Passage

Number of Passages

3 10% 4-6 Up to 300 2-3 2 3+ 45% 14-16 Up to 400 2-3 6 4 45% 14-16 Up to 400 2-3 6

Total 100% 34-36 14

Listening Comprehension Section

Level Approx % of Total Test

Number of Items

Max. length of a Passage* (in seconds):

(monologs/dialogs)

Number of Items per Passage

Number of Passages

# of Voicings per Passage

3 10% 4-6 100 120 2-3 2 2 3+ 45% 14-16 120 140 2-3 6 2 4 45% 14-16 140 160 2-3 6 2

Total 100% 34-36 14

* In rare cases, a passage exceeding the maximum length may be divided into 2 segments, each administered independently. Such dividing of a listening passage into segments is to be considered an exceptional procedure, not standard practice.

DLPT5

General Test Specifications

Content Areas The DLPT generally covers all the Final Learning Objectives (FLO) content areas. The types of topics within the content areas will, of course, be different for different levels, as will the proportion of each content area represented at each level. Passages should represent both a wide range of topics within each content area and particular situations that are not likely to be known to the examinee. The distribution of content areas at each level should be as balanced as possible, although language- or culture-specific constraints may make an even distribution difficult or impossible to achieve.

The FLO Content Areas

• Military-Security

• Economic-Political

• Scientific and Technological

• Cultural and Social

• Geography: Physical, Political, Economic

DLPT5

General Test Specifications

The Composition of the Sub-pools for Each ILR Level

General Comments A separate sub-pool of passages with their associated items is created for each of the base and plus ILR levels represented in the test. Each sub-pool is balanced according to a number of criteria—text characteristics, sources, topics, skills, and questions to be asked. The content of each sub-pool is intended to provide a representative sample of the universe of all possible items at that level. Explanations and discussion of the specifications for the sub-pools are provided for each level on the following pages. The balance of items testing global tasks and those testing non-global tasks is shown in the table below. Global tasks are those for which the question focuses on the passage as a whole, for example, asking about the author’s general attitude, the main idea of the passage, or the principle underlying the author’s position. Non-global tasks are those for which the question focuses on part of the passage (even if understanding the whole passage is necessary for answering correctly). Examples might be an item asking about a detail of a plane crash, the function of a particular metaphor in the author’s argumentation, or the author’s tone in a particular paragraph or when addressing a particular sub-issue.

Reading Item type Percent of items in pool for each level

3 3+ 4 Global 40-50% 20-30% 20-30% Non-Global 50-60% 70-80% 70-80%

Listening Item type Percent of items in pool for each level

3 3+ 4 Global 50-60% 40-50% 40-50% Non-Global 40-50% 50-60% 50-60%

DLPT5

General Test Specifications

Reading: Level 3

Part I: The texts Length Texts should not typically be longer than the equivalent of 300-word English passages. It is desirable to have a substantial proportion of the texts considerably shorter than this.

Linguistic Characteristics

• Texts should show a wide range of grammatical structures and vocabulary.

• Texts should demonstrate a variety of discourse structures.

• Texts should deal with abstract topics. They should generally contain hypothesis, argumentation, analysis, supported opinions and/or subtle implications.

• Texts should usually reflect the target-language culture, and cultural references/cues play an increasingly large role as the difficulty of these texts rises.

• The organization of the texts should:

a. be complex and personalized (‘shaped’ by the author)

b. show clear cohesive discourse with adequate cohesive devices characteristic of the target language.

Text Types Texts may be of the types shown below, but the ultimate selection of a text depends on its meeting the ILR Skill Level criteria.

1. Evaluative texts (whose function is to analyze or assess events).

Examples: Editorials, commentaries, literature or art reviews.

2. Expository texts (whose function is to explain or discuss issues).

Examples: Letters to the editor explaining a problem, articles from specialized magazines, essays, textbooks. There are two subcategories of texts:

a. Specialized texts that discuss concrete but technical issues (scientific, technological or legal texts)

b. Abstract texts that discuss ideas, concepts, or notions.

3. Argumentative texts (whose function is to defend or refute a policy, a strategy, or a program).

Examples: Platforms of political parties, proposals put forth by politicians or officials, letters to the editor to argue in favor of or against an issue, editorials, etc.

Sources Leading newspapers, serious magazines, position papers, reports, the Internet, books, essays, textbooks.

Examples Editorials, commentaries, biographies with critical interpretations, criticisms, reports, philosophical argumentation, opinion pieces, political analysis.

DLPT5

General Test Specifications

Examples of possible topics within the FLO Content Areas

FLO Content Area Example Sub-categories Military-Security Analysis of military events and intelligence activities Assessment of military cooperation or alliance among countries Rationales for arms sales or military aid Description of advanced weapons systems Analysis of historical battles Analysis of security issues: justifications for security measures, etc.

Economic-Political Economic analysis

Political analysis Economic policies Assessment of a country’s economic situation Development and growth issues Political strategies Political reforms and democracy Analysis of international political and trade relations Scientific and Technological Explanation of scientific, medical, and technological issues Analysis of scientific achievements, discoveries, or findings (e.g., the ethics of cloning) Research policies of governments or institutions Impact of scientific progress on families, behavior, society, etc.

Scientific theories

Cultural and Social Evaluation or analysis of social events and relationships

Discussion of societal issues (discrimination, violence, drugs, privacy, etc.)

Private vs. public education, role of government in education, standards, etc.

Religious issues: church/state relationship, fundamentalism, persecution, value systems

Freedom of the press Commentary on books, movies, artworks

Geography: Physical, Political, Economic

Analysis of environmental policies Opinions on water issues (dam-building, etc.)

Analysis of border disputes Use of natural resources Plans for maintaining habitat (e.g., rainforest preservation , anti-desertification plans)

DLPT5

General Test Specifications

Part II: Skills assessed On the above texts, level 3 readers can:

Global:

o understand major ideas which may be explicitly stated or be implicit in the text.

o draw appropriate conclusions from an author’s remarks.

Non-global:

o understand major points supporting a line of argumentation.

o understand author-intended implications.

o understand vocabulary in context, including idiomatic expressions and figurative speech.

o understand grammatical and other references that connect parts of the text and that control meaning.

Can be either global or non-global:

o separate fact from opinion.

o identify the attitude of the author.

The predominance of items should test comprehension of major ideas, conclusions, and author-intended implications; it is desirable to have at least a few items in the test that address supporting a line of argumentation. A text must have either an item testing comprehension of a major idea, an item testing a conclusion, or an item testing an author-intended implication in order to be included in an operational test.

DLPT5

General Test Specifications

Reading: Level 3+ Texts should not typically be longer than the equivalent of 400-word English passages. It is desirable to have a substantial proportion of the texts considerably shorter than this.

Level 3+ text and item characteristics are similar to those for level 3, but items assessing ability to recognize basic rhetorical structure are permissible at this level.

DLPT5

General Test Specifications

Reading: Level 4

Part I: The texts Length Texts should not typically be longer than the equivalent of 500-word English passages. It is desirable to have a substantial proportion of the texts considerably shorter than this.

Linguistic Characteristics

• Texts are linguistically and conceptually extremely dense.

• Texts may display unusual/unexpected means of argumentation or turns of thought.

• Texts are highly individualized, i.e., reflect the author’s unique thinking about or synthesis of a topic.

• Texts are extremely sophisticated and use complex, subtle language with virtuosity, often in figurative and subtle ways.

Sources Intellectual periodicals, books, the Internet, collections of essays, etc.

Examples Think-pieces, essays, philosophical expositions, satiric writings, other sophisticated humor.

Possible topics within the FLO Content Areas Essentially any topic is appropriate, but many topics are not usually discussed at level 4. Many of the example topics at level 3 would also be appropriate for level 4; see the level 3 table for details.

Part II: Skills assessed On the above texts, level 4 readers can:

Global or non-global:

o understand lines of argumentation by synthesizing information and ideas.

o understand major points supporting the author’s argumentation.

o understand the rhetorical structure (i.e., the relationship between the parts of the text).

o understand subtle author-intended implications.

o follow unpredictable turns of thought/understand subtle argumentation.

o read beyond the lines (i.e., understand the full ramifications of a text as it is situated in a wider context).

o understand the author's attitude or tone.

o understand vocabulary and expressions in context.

o understand grammatical and other references.

The items should assess comprehension of the features indicated above. A text must have either an item testing comprehension of a line of argumentation, an item testing comprehension of a conclusion, an item testing the synthesis of information central to the author’s argument, or an item testing an author-intended implication in order to be included in an operational test.

DLPT5

General Test Specifications

Listening: Level 3

Part I: The texts Length Texts should not typically be longer than 100 seconds for monologues, 120 seconds for dialogues.

It is desirable to have a substantial proportion of the texts considerably shorter than this.

Linguistic Characteristics

• Texts are either dialogues or monologues; most will be delivered at a normal speech rate, and some may be at a faster rate than normal. Texts may have authentic background noise.

• Texts deal with abstract topics or concrete topics that are complex, and they generally contain hypotheses, analyses, opinions, and/or subtle implications.

• Texts may have a wide range of phonological modifications and demonstrate a variety of grammatical and discourse structures. Texts may have colloquial and common slang expressions.

Sources

• Authentic audio media sources (e.g. TV/radio programs, movies, public announcements, the Internet).

• Recorded in-house: debates, lectures, interviews, speeches.

Examples of possible topics within the FLO Content Areas

FLO Content Area Example Sub-categories Military-Security Analysis of military events and intelligence activities Assessment of military cooperation or alliance among countries Rationales for arms sales or military aid Description of advanced weapons systems Analysis of historical battles Analysis of security issues: justifications for security measures, etc.

Economic-Political Economic analysis

Political analysis Economic policies Assessment of a country’s economic situation Development and growth issues Political strategies Political reforms and democracy Analysis of international political and trade relations

DLPT5

General Test Specifications

Scientific and Technological Explanation of scientific, medical, and technological issues

Analysis of scientific achievements, discoveries, or findings (e.g., the ethics of cloning)

Research policies of governments or institutions Impact of scientific progress on families, behavior, society, etc.

Scientific theories

Cultural and Social Evaluation or analysis of social events and relationships

Discussion of societal issues (discrimination, violence, drugs, privacy, etc.)

Private vs. public education, role of government in education, standards, etc.

Religious issues: church/state relationship, fundamentalism, persecution, value systems

Freedom of the press Commentary on books, movies, artworks

Geography: Physical, Political, Economic

Analysis of environmental policies Opinions on water issues (dam-building, etc.)

Analysis of border disputes Use of natural resources Plans for maintaining habitat (e.g., rainforest preservation , anti-desertification plans)

Part II: Skills assessed The listening skills necessary to comprehend Level 3 texts are as follows:

Global:

o Understand major ideas that may be explicitly stated or be implicit in the text.

o Draw appropriate conclusions from a speaker’s remarks.

Non-global:

o Understand major points supporting a line of argumentation.

o Understand implications conveyed by supra-segmental features (e.g., intonation, stress, etc.).

Can be global or non-global:

o Understand speaker-intended implications.

o Separate fact from opinion.

o Identify the intent of the speaker.

o Understand register.

Most items should test the understanding of major ideas, conclusions, and speaker-intended implications.

DLPT5

General Test Specifications

Listening: Level 3+ Texts should not typically be longer than 120 seconds for monologues, 140 seconds for dialogues.

It is desirable to have a substantial proportion of the texts considerably shorter than this.

Level 3+ text and item characteristics are similar to those for level 3.

DLPT5

General Test Specifications

Listening: Level 4

Part I: The texts Length Texts should generally not be longer than 140 seconds for monologues, 160 seconds for dialogues.

Linguistic characteristics

• Texts are either dialogues or monologues delivered at normal or fast speech rate. They may have non-intrusive background noise.

• Texts have an extensive range of phonological modification.

• Texts may deal with abstract topics. They typically are linguistically and conceptually dense.

• Texts may consist of formal or informal speech.

• Texts have much low-frequency vocabulary, with creative, figurative, or idiosyncratic use of language. They may include features of non-standard dialects.

Sources

• Authentic audio media sources (e.g. TV and radio programs, movies, public addresses, the Internet).

• Recorded in-house: informal discussions, formal speeches, debates, intellectual discussions.

Possible topics within the FLO Content Areas Essentially any topic is appropriate, but many topics are not usually discussed at level 4. Many of the example topics at level 3 would also be appropriate for level 4; see the level 3 table for details.

Part II: Skills assessed On the above texts, level 4 listeners can:

Global or non-global:

o Understand lines of argumentation by synthesizing information and ideas.

o Understand major points supporting the speaker’s argumentation.

o Draw appropriate conclusions.

o Understand subtle speaker-intended implications.

o Follow unpredictable turns of thought and understand subtle argumentation.

o Go “beyond the lines” of the text, i.e., place the speaker’s message in a wider contextual framework.

o Understand register.

o Understand the speaker’s attitude or tone.

o Understand colloquial and idiomatic use of the language.

o Understand implications represented by supra-segmental features.

Most items should test comprehension of lines of argumentation, conclusions, and speaker-intended implications.

DLPT5

General Test Specifications

General Guidelines on Text Selection Authenticity of Reading Texts Texts are genuine reading texts, taken from the target-language use situation, and intended for the general reader. If necessary, they may be edited, but ideally editing should be kept to a minimum, or avoided altogether. If the range of authentic texts available is too restricted to provide an appropriate range of test items, texts may be purpose-written, but in such cases they are expected to have all the characteristics of genuine reading texts, and require reading skills typical of authentic texts.

Authenticity of Listening Passages Passages are genuine samples of spoken language, intended for the general listener. They may be recorded directly from the target-language use situation, with or without some editing, or they may be created by the test-development team. As far as possible, created passages should be spontaneous, although semi-scripting can be used if necessary. The reading of scripted passages should be avoided if at all possible. The important thing is that the passages contain the characteristics of realistic spoken language, and require listening skills and language knowledge typically required to listen to such passages in the target-language use situation.

The Characteristics of Spoken Language The selection of listening passages should cover the range of spoken-language characteristics found in the target-language use situation, from very oral texts to very literate texts. All the passages should attempt to replicate the characteristics of real-world speech, and at least half of the passages used should be on the oral side of the continuum.

The most common characteristics of oral language to be included in texts are:

• Considerable phonological modification (assimilation, elision, vowel reduction, etc.).

• Relatively rapid speed, even at lower levels, requiring listeners to process speech in real time, automatically and effectively.

• Use of idea units (roughly the amount a listener can store in working memory at one time) rather than sentences. (Idea units typically are short, about 7 words or 2 seconds long, and are connected more often by conjunction than by embedding.)

• Hesitations: pauses, fillers, repairs, repetitions.

• Slang and linguistic forms not found in writing.

• More colloquial, and less conservative, vocabulary and grammar.

• More personal involvement.

• Approximation and exaggeration.

Level of Tasks In order to target a particular proficiency level, tasks should focus on aspects of a text at that level that are part of the core of the text (rather than its periphery). The English used in the question statements and/or prompts of items should be as simple and straightforward as possible (rather than necessarily matching the level of the target-language text).

DLPT5

General Test Specifications

Sources The DLPT should provide a representative sample of types of sources from the domain of interest, the target-language use situation. Web pages, for example, are just one text genre out of the many that should be included on the test. Generally, no more than 30% of the test content should come from sources that were written specifically for the Web. Other text genres, such as reports, book reviews, news reports, etc., that were not written specifically for the Web, are accessible on-line and are not included in the 30% figure cited above (provided they are in their original form and have not been modified for the Web). In the case of low-volume languages, where sources are not otherwise available, this figure of 30% may be increased. The same considerations apply to other text genres, such as radio broadcasts, television broadcasts, newspapers, textbooks, etc.

DLPT5

General Test Specifications

Administration of the Operational Forms of the DLPT5

Each DLPT5 will be administered operationally as an upper-range or lower-range linear test. The operational linear forms of the DLPT5 will be delivered to examinees by computer. The upper-range reading section and listening section will each consist of approximately 35 test items. (See tables on p. 8.)

Examinees have 3 hours to complete the Reading test and 3 hours to complete the Listening test.

Approximately 90 minutes into each test, the computer will initiate a 15-minute break. For each test, examinees control when to move to the next passage by clicking a [Next] button on the computer screen. Countdown timers are visible on the screen to help the examinee manage the time; the screen also displays how many passages are on the test and where the examinee is in the test. Following the final question in both the listening and the reading sections, a review screen will appear that allows the examinee to use whatever time remains to answer any questions left unanswered or change answers.

Presentation of the Listening Texts

For all listening passages, examinees see on their computer screen the orientation to the passage and all questions associated with that passage. Each listening passage in the test will be preceded by an orientation statement in English. The purpose of the orientation is to let the examinee know what source the listening passage came from or under what circumstances the examinee would hear such a text. After hearing the orientation, examinees are given 5 seconds to read the questions, after which the playing of the passage automatically begins. Passages will be played twice, with a short tone in between the two playings. Then, the examinee hears instructions to answer the questions. Examinees cannot replay the audio or stop it before it has finished playing.

After the final playing of the passage, examinees will have as much time as they want to answer the questions for that passage before moving on to the next passage. The test as a whole is timed, so examinees have to manage their time spent per passage, as they do on the reading section.

DLPT5

General Test Specifications

Standard Instructions for Examinees (operational administration)

Reading Comprehension This test measures your reading comprehension. It contains a number of reading passages, and there are two or three questions based on each passage. In taking the test you should do the following:

• Read the short statement before each passage that tells you where the passage came from.

• Read the passage and questions carefully.

• Type your answers in English in the text boxes provided. Your answer should be complete, concise, and relevant. You must provide all the information requested to receive credit. The answer to each question may be stated overtly in the passage or only implied.

• Click on the [Next] button at the bottom right of the screen to go to the next passage.

A sample passage follows on the next screen.

If you need help during the test, click on the [Help] button.

DLPT5

General Test Specifications

Listening Comprehension

• This measures your listening comprehension. It contains a number of listening passages, and there are two or three questions based on each passage.

• Before each passage you will hear a statement in English that explains the context from which the passage was taken.

• Then you will be given a short amount of time to read the questions pertaining to the passage.

• You will then hear the passage itself. Each passage will be played twice. There will be a short tone signal between the two playings.

• You do not have to wait until the passage has finished playing to begin answering the questions. After hearing the passage you will be reminded to answer the questions. Read each question carefully.

• Type your answers in English in the text boxes provided. Your answers should be complete, concise, and relevant. You must provide all the information requested to receive credit. The answer to each question may be stated overtly in the passage or only implied.

• Click on the [Next] button at the bottom right of the screen to go to the next passage.

A sample passage follows on the next screen.

If you need help during the test, click on the [Help] button.

DLPT5

General Test Specifications

Standard Instructions for Examinees (paper-and-pencil pilot administration)

Reading Comprehension

This test measures your reading comprehension in XXXXXX. It contains a number of reading passages, and there are two or three questions based on each passage. In taking the test you should do the following:

• Read the short statement before each passage that tells you where the passage came from.

• Read the XXXXXXX passage and the questions carefully.

• Write down your answer in English in the separate answer form. Your answers should be complete, concise, and relevant. You must provide all the information requested to receive credit. The answer to each question may be stated overtly in the passage or only implied.

Before starting the test, please read the sample passage and questions provided on the following page.

DLPT5

General Test Specifications

Listening Comprehension This test measures your listening comprehension in XXXXX. It contains a number of listening passages, and there are two or three questions based on each passage.

• Before each passage you will hear a statement in English that explains the context from which the passage was taken. This statement is also printed in your test booklet.

• Then you will be given a short amount of time to read the questions pertaining to the passage.

• You will then hear the XXXXX passage itself. Each passage will be played twice. There will be a short tone between the two playings.

• After hearing the passage, you will be reminded which questions are to be answered at this point. Read each question carefully. The amount of time you have to answer depends on the difficulty of the passage.

• Write down your answer in English in the separate answer form. Your answer should be complete, concise, and relevant. You must provide all the information requested to receive credit. The answer to each question may be stated overtly in the passage or only implied.

You will now hear a sample passage and see sample questions based on that passage.

DLPT5

General Test Specifications

Scoring of DLPT5

Scoring will be performed by trained scorers at the Defense Language Institute using a detailed Scoring Protocol. Scorers work independently, and disagreements in ratings are resolved by asking more scorers to rate the test. Scorers refer to the examinee’s answer output file and to the Scoring Protocol booklet to determine whether to award credit for each passage. If the examinee includes all the essential information correctly, and also writes incorrect information, the examinee is given credit and not penalized for the incorrect extra information.

Raw scores are then calculated for each ILR level. In order to be rated level 4, an examinee must be awarded credit for at least 9 of the 14 total passages, at least 4 of which must be level-4 passages. If an examinee fails to meet this requirement, in order to be rated level 3+, an examinee must either be awarded credit for at least 7 of the 14 total passages or else be awarded credit for at least 5 of the 8 passages that are below level 4, at least 4 of which must be level-3+ passages. If an examinee fails to meet both of these requirements, then the examinee’s rating remains 3.

Scores will be closely examined in the early stages of use of a test to check for the incidence of scoring anomalies and inaccuracies; scoring protocols may be subsequently modified if necessary.

Scoring Protocol A Scoring Protocol booklet will be developed for each section of each section of the test. The booklet will contain instructions for scorers and, for each passage, the following elements:

• target-language text or transcript

• an English rendering of the passage

• the questions pertaining to the passage

• the Key Answer Information, which includes the information in the passage that the questions are intended to elicit.

• the Credit Scheme, which makes clear how much of the information pertaining to each passage is required in the examinee’s response in order for that response to be awarded credit.

Scorers The scorers for a given test must either have been involved in the development of the test or else trained by one of the developers. Until a test has been scored enough times that the test developers are confident in the protocol, scoring will be restricted to a small group of people.

Scorers may also suggest changes to the Scoring Protocol in cases in which an examinee’s response appears to answer a question correctly but fails to correspond to the Key Answer Information in the Protocol. Such answers may be added to the Protocol. As the Protocol stabilizes over time, the pool of scorers can be expanded.

At first, scoring is performed independently and simultaneously by three trained scorers, after which their ratings are immediately resolved with each other, yielding a consensus awarding of credit for each question and a consensus overall rating for the test. At a later stage, scoring may be performed by only two independent scorers, with a third scorer involved only in case of a discrepancy.

DLPT5

General Test Specifications

Technical Exhibit 13

DLPT 5 Style Guide 9/11/2015

Style Guide for DLPT 5 Materials Update: 9/11/2015

This style guide is offered as a support to the test writers, test developers, project managers and contracting office representatives of the Test Production division of DLIFLC, Language Proficiency Directorate. It will serve also to provide information to help editors maintain consistency across products and languages. It is a work in progress, that is, additional materials as well as examples to illustrate the guidelines given are still in development. Please consult primary sources and then contact the editor for discussion of any new concerns.

The primary source for spelling of English words is Merriam-Webster’s Collegiate Dictionary (currently the Eleventh Edition), also available online at:

http://www.merriam-webster.com/.

The primary source for style decisions is The Chicago Manual of Style 16th Edition. A handy list of changes from the 15th Edition is available online at:

http://www.chicagomanualofstyle.org/about16_rules.html. And additional information is available online at:

http://www.chicagomanualofstyle.org/CMS_FAQ/new/new_questions01.html.

A secondary, but often consulted, source is The Gregg Reference Manual by William Sabin. This source balances the publishing style guidelines of CMS, focusing more on usage for academic and business education.

Spelling of place names will be determined according to the NGA (National Geospatial- Intelligence Agency) GEOnet Names Server (GNS) at http://geonames.nga.mil/gns/html/. Frequency and familiarity of English spellings as determined by usage in The New York Times will also be consulted. Exceptions will be considered on a case by case basis.

Spelling/transliteration of personal names will follow established, recognized English language conventions, as determined by usage in The New York Times. For names not previously published by TNYT, a transliteration scheme for each language should be chosen and followed consistently.

Important Note: This style guide uses examples designated as Correct and Incorrect as a matter of convenience only. Many examples labeled Incorrect are merely not preferred DLPT5 style rather than being strictly incorrect according to English language rules.

abbreviations common abbreviations used in Test Production materials will follow CMS style, including whether periods will be included.

Some of the most frequently used are as follows:

http://www.merriam-webster.com/ http://www.chicagomanualofstyle.org/about16_rules.html http://www.chicagomanualofstyle.org/CMS_FAQ/new/new_questions01.html http://www.chicagomanualofstyle.org/CMS_FAQ/new/new_questions01.html http://geonames.nga.mil/gns/html/ a.m ./p.m. (lowercase, with periods)

EU

exam ID (not I.D. or identification, referring to official card or papers) tech (technician, technical) OK (rather than the word okay)

TV

UK

UN (not U.N.)

US (not U.S.)

Some short forms that were previously considered abbreviations are now the preferred forms for DLPT5.

Examples: ad exam (medical and academic) phone photo quote

TV

above-level Some words may have been identified as above grade 8 vocabulary vocabulary level, yet have been accepted as essential for DLPT5 testing or as specific military-related terms. This is a partial list of these accepted English words:

allocate [funds/revenue] (12) anecdote (9) bilingual (10) cardiac combatant (12) consulate corruption (12) deploy(ment) (13) extradite (12) guerrilla (10) harass(ment) (10) implement (v) (12) infer (11) infiltrtate (10) infrastructure (12) insurgent (13) logistic(al) (12) negotiate (10) ordnance (16) perpetrator (12) personnel (10) retaliate/retaliation (11) skirmish (10) sniper (10) subordinate surveillance (12) technology (10)

In addition to the above, exceptions will be made for passage-specific vocabulary or for cognates or recognizable [REC] words within a passage. A higher-level passage that refers to a concept and discusses it extensively will often require the English equivalent in the items. If a passage uses a recognizable variation of an English word, then the English word can be used in the items. For example, if the word ideology (which is grade 12 level) is used in a passage, then the items could use the English word and its different forms.

Examples of words that often appear but have NOT been approved are shown below.

Do NOT use: entrepreneur ironic/irony marginalize proactive sibling absolutes Avoid absolutes (such as always or never or even only) in options of a multiple choice test; they usually clue the key or create an outlier.

according to Since this phrase literally means “by the authority of,” its use is NOT appropriate to test an examinee’s inference or deliberated conclusion. Use only when information in a passage is stated outright.

This phrase should NOT be used unless it is necessary to limit responses to the information provided in the passage versus other logical explanations or to limit responses to one speaker rather than another.

For a stem that begins with the compound preposition “according to,” do not use the demonstrative pronoun this, use the definite article the.

Incorrect: According to this article . . .

According to this news report . . .

Correct: According to the article . . .

According to the news report . . .

acronyms In general, acronyms should be avoided in stems. However, very well-known organizations or entities (e.g., US, UN, EU) or organizations whose acronyms may be in general use or have been established by spelling out in the passage may be repeated in a stem.

Examples: EU

NATO

NGO

UN

US

action The idiomatic phrase “take action” cannot be broken apart into a verb and a count noun. The passive form of this phrase, i.e., “action taken,” is permissible but beware the temptation to add a number, i.e., the word action cannot become plural actions.

Substitute, as appropriate, “steps taken” or “efforts made,” but NOT “measures taken” (because the word measures is above grade level).

Incorrect: What TWO actions were taken to resolve the issue?