Attachment_2_APA_Test_Users_DJF-18-4210-PR-0008262.pdf

APA's Guidelines for Test User Qualifications An Executive Summary

Samuel M. Turner Stephen T. DeMers

Heather Roberts Fox

Geoffrey M. Reed

University of Maryland University of Kentucky American Psychological Association and Towson University American Psychological Association

At the direction of the Council of Representa-tives of the American Psychological Associa-tion (APA), the Task Force on Test User Qual-ifications (TFTUQ) was established in October 1996 to develop guidelines that inform test users and the general public of the qualifications that the APA considers im-portant for the competent and responsible use of psy-chological tests. The TFTUQ reviewed the relevant literature related to test user qualifications (see, e.g., Eyde, Moreland, Robertson, Primoff, & Most, 1988, and Tyler, 1986), as well as policy statements developed by the APA (1950, 1992) and other groups both national (e.g., American Educational Research Association [AERA], APA, & National Council on Measurement in Education [NCME], 1999; American Association for Counseling and Development, 1988) and international (British Psychological Society, 1995, 1996; Interna-tional Test Commission, 2000). The task force then developed a set of comprehensive guidelines and so-licited comments from numerous individuals and groups involved with test use both within and outside the APA.

The final report of the TFTUQ was approved by the APA Council of Representatives in August 2000. This article provides a brief summary of the Guidelines on Test User Qualification (APA, 2000) that are now APA policy.

The TFTUQ was established in part because of evi-dence that some current users of psychological tests may not possess the knowledge and skill that the APA considers desirable for optimal test use (see, e.g., Aiken, West, Sechrest, & Reno, 1990). The phrase test user qualifica-tions refers to the combination of knowledge, skills, abili-ties, training, experience, and, where appropriate, practice credentials that the APA considers desirable for the respon-sible use of psychological tests. The guidelines in the TFTUQ's report are intended to apply to persons who use psychological tests in a variety of settings and for diverse purposes. The APA's purpose in developing these guide-lines is to inform test users as well as individuals involved with training programs, regulatory and credentialing bod-ies, and the public about the qualifications that promote high professional standards in the use of tests with the public.

Historical Background The reason that the APA has sought to develop and pro-mulgate guidelines for the use of psychological tests evolves from the historical role the APA has played in the science and practice of testing and assessment. The disci-pline of psychology is the historical root for psychological testing and provides the research evidence and professional training to advance competent psychological assessment.

Since 1950, the APA has addressed the issue of test user qualifications broadly in its ethical principles (APA, 1950, Editor's Note. This article is an executive summary of a larger report that was adopted by the American Psychological Association's Council of Representatives on August 6, 2000. To obtain a copy of the full Report of the Task Force on Test User Qualifications, contact the Science Direc-torate, American Psychological Association, 750 First Street, NE, Wash-ington, DC 20002-4242; phone number: (202) 336-6000.

Author's Note. Samuel M. Turner, Maryland Center for Anxiety Disorders, Department of Psychology, University of Maryland; Stephen T. DeMers, Department of Educational and Counseling Psychology, Uni-versity of Kentucky; Heather Roberts Fox, Science Directorate, American Psychological Association, Washington, DC, and Department of Reading, Special Education, and Instructional Technology, Towson University;

Geoffrey M. Reed, Practice Directorate, American Psychological Associ-ation, Washington, DC.

The full report was developed by the American Psychological As-sociation (APA) Task Force on Test User Qualifications (TFTUQ). The TFTUQ cochairs were Stephen T. DeMers, EdD, and Samuel M. Turner, PhD. The TFTUQ members included Marcia Andberg, PhD; William Foote, PhD; Leaetta Hough, PhD; Robert Ivnik, PhD; Scott Meier, PhD;

Kevin Moreland, PhD (deceased); and Celiane M. Rey-Casserly, PhD.

The TFTUQ wishes to acknowledge Stephen DeMers, EdD; Nadine Lambert, PhD; and Leona Aiken, PhD, for their role in the creation of the task force. It was their foresight regarding the need for an official policy on qualifications necessary for the competent use of tests that brought this motion to the APA Council of Representatives. In addition, the TFTUQ extends thanks to Wayne Camara, PhD; Rodney Lowman, PhD; Karen O'Brien, PhD; and many other APA colleagues for the consultation and assistance they gave to this project; to the Board of Professional Affairs, the Board of Scientific Affairs, the Committee on Legal Issues, and especially the Committee on Psychological Tests and Assessment for their kind support; to Dianne Maranto and Dianne Schneider, PhD, for staff support from the Science Directorate; and to Robert Walsh and Georgia Sargeant for staff support from the Practice Directorate.

Correspondence concerning this article should be addressed to Sam-uel M. Turner, Maryland Center for Anxiety Disorders, Department of Psychology, University of Maryland, College Park, MD 20742. Electronic mail may be sent to turner@psyc.umd.edu.

December 2001 • American Psychologist Copyright 2001 by the American Psychological Association, Inc. OO03-O66X/O1/S5.0O Vol. 56, No. 12, 1099-1113 DOI: 10.1037//OOO3-O66X.56.12.1O99

Th is d oc um en t i s c op yr ig ht ed b y th e A m er ic an

P sy ch ol og ic al A ss oc ia tio n or o ne o f i ts a lli ed p ub lis he rs

Th is a rti cl e is in te nd ed so le ly fo r t he p er so na l u se o f t he in di vi du al u se r a nd is n ot to b e di ss em in at ed b ro ad ly

1981, 1992). The APA also has participated in formulating standards on the development and use of psychological and educational tests (APA, AERA, & NCME, 1954, 1966, 1974; AERA, APA, & NCME, 1985, 1999). Other profes-sional groups that use psychological tests also have pro-mulgated ethical guidelines that address qualifications for test use. For example, the American Counseling Associa-tion (formerly the American Association for Counseling and Development) has a specific set of Responsibilities of Users of Standardized Tests (American Association for Counseling and Development, 1988).

The task force found that concern over the misuse of tests has been growing in the international psychology community over the past few years. Several countries and international groups, including the British Psychological Society (1995, 1996), the Canadian Psychological Associ-ation (Simner, 1994), and the International Test Commis-sion (2000), have launched initiatives to address concerns about test user qualifications.

Review of extant literature suggests that most of the problems associated with test use are related to the com-petence of individual test users, although the uneven qual-ity of test construction and the ease with which test instru-ments can be obtained from some test publishers also contribute to these problems (Tyler, 1986). In devising the present set of guidelines, the TFTUQ kept in mind the types of problems identified by the empirical research and the conclusion that much of the difficulty lies with test users. The APA formed the TFTUQ in the belief that previous efforts to specify test user qualifications, although useful, did not provide the kind of specific guidance that many APA members and others were seeking.

Scope of the Guidelines The use of psychological tests should typically be viewed within the broader concept of assessment. Psychological assessment is a complex activity requiring the interplay of knowledge of psychometric concepts with expertise in an area of professional practice or application. Assessment is a conceptual, problem-solving process of gathering de-pendable, relevant information about an individual, group, or institution to make informed decisions.

These guidelines describe two types of test user qual-ifications: (a) generic psychometric knowledge and skills that serve as a basis for most of the typical uses of tests and

(b) specific qualifications for the responsible use of tests in particular settings or for specific purposes (e.g., health care settings or forensic or educational decision making). The guidelines apply most directly to standardized tests, such as tests of ability, aptitude, achievement, attitudes, interests, personality, cognitive functioning, and mental health. The guidelines define a psychological test as any measurement procedure for assessing psychological characteristics in which a sample of an examinee's behavior is obtained and subsequently evaluated and scored using a standardized process. The guidelines do not apply to unstandardized questionnaires and unstructured behavior samples or to teacher- or trainer-made tests used to evaluate performance in education or training.

Various activities included in the testing process may be appropriately conducted by different people working collaboratively. Each participant should possess the knowl-edge, skills, and abilities relevant to his or her role. For example, different individuals may be responsible for de-ciding what constructs, conditions, or characteristics need to be assessed, selecting the appropriate tests, administer-ing and scoring tests, and interpreting and communicating the results. Moreover, some testing activities may involve tasks that require limited professional knowledge (e.g., administering or scoring some paper-and-pencil tests). In such circumstances of multiple participants in the testing process or participants with limited expertise, test use should be directed or supervised by a qualified test user. It is this qualified responsible test user to whom these guide-lines apply.

Persons whose psychological test use is confined to research will find that the degree to which these guidelines apply to their work depends on the focus and setting of their research. The sections of the guidelines that address competencies related to psychometrics, statistics, test ad-ministration, and scoring are applicable to research that uses psychological tests. When research is conducted with clinical populations or in settings where there are likely to be real or perceived implications for the test taker, addi-tional guidelines may be applicable.

Generic Knowledge and Skills The TFTUQ began by conceptually dividing those skills and knowledge considered important for good test use into two main categories: first, core knowledge and skills and second, context-related qualifications. The core knowledge and skills discussed in this section are deemed essential for all test users who make decisions or formulate policies that directly affect the lives of test takers. This core set of knowledge and skills is considered to be relevant for all test users; however, the level of skill and depth of knowledge in these domains may vary depending on the testing purpose and context.

Psychometric and Measurement Knowledge

In general, it is important for test users to understand classical test theory and, when appropriate or necessary, item response theory (IRT). When test users are making assessments on the basis of IRT, such as adaptive testing, they should be familiar with the concepts of item parame-ters (e.g., item difficulty, item discrimination, and guess-ing), item and test information functions, and ability pa-rameters (e.g., theta).

Descriptive statistics. Basic to any test use is the ability to define, apply, and interpret concepts of de-scriptive statistics. For example, means and standard devi-ations are often used when comparing different groups on test scales, whereas correlations are frequently used for examining the degree of convergence and divergence be-tween two or more scales. Similarly, understanding how frequency distributions describe the varying levels of a behavior across a group of persons is essential. Persons

1100 December 2001 • American Psychologist

Th is d oc um en t i s c op yr ig ht ed b y th e A m er ic an

P sy ch ol og ic al A ss oc ia tio n or o ne o f i ts a lli ed p ub lis he rs

Th is a rti cl e is in te nd ed so le ly fo r t he p er so na l u se o f t he in di vi du al u se r a nd is n ot to b e di ss em in at ed b ro ad using tests should have sufficient knowledge and under-standing of descriptive statistics to select and use appro-priate test instruments, as well as to score and interpret results. The most common descriptive statistics relevant to test use include frequency distributions, descriptive statis-tics characterizing the normal curve (e.g., kurtosis, skew-ness), measures of central tendency (e.g., mean, median, and mode), measures of variation (e.g., variance and stan-dard deviation), indices of relationship (e.g., correlation coefficient), and scales, scores, and transformations.

Test results frequently represent information about individuals' characteristics, skills, abilities, and attitudes in numeric form. Test users should understand issues related to scaling, types of scores, and methods of score transfor-mation. For example, test users should understand and know when to apply the various methods for representing test information (e.g., raw scores, standard scores, and percentiles). Relevant concepts include types of scales, types of scores (e.g., raw, transformed, percentile, standard, normalized), scale score equating, and cut scores.

Reliability and measurement error. Test users should understand issues of test score reliability and measurement error as they apply to the specific test being used, as well as other factors that may influence test results, and the appropriate interpretation and application of differ-ent measures of reliability (e.g., internal consistency, test-retest reliability, interrater reliability, and parallel forms reliability). Similarly, test users should understand the stan-dard error of measurement, which presents a numerical estimate of the range of scores consistent with the individ-ual's level of performance. Additional constructs related to reliability and measurement that should be understood by test users are delineated in Figure 1.

Validity and meaning of test scores. The interpretation and uses of test scores, not the test itself, are evaluated for validity. Responsibility for validation belongs both to the test developer, who provides evidence in sup-port of test use for a particular purpose, and to the test user, who ultimately evaluates that evidence, other available data, and information gathered during the testing process to support interpretation of test scores. Test users have a particularly important role in evaluating validity evidence when the test is used for purposes different from those investigated by the test developer.

Contemporary discussions of validity have focused on evidence that supports the test as a measure of a construct (sometimes called construct validity). For example, evi-dence for the uses and interpretation of test scores may come through evaluation of the test content (content rep-resentativeness), through evidence of predictions of rele-vant outcomes (criterion-related validity), or from a num-ber of other sources of evidence. Test users should understand the implications associated with the different sources of evidence that contribute to construct validity, as well as the limits of any one source of validity evidence (i.e., criterion, convergent, and discriminant validity).

Normative interpretation of test scores.

Norms describe the distribution of test scores in a sample from a particular population. Test users should understand

Figure 1 Sources of Variability or Measurement Error

• Characteristics of test takers (e.g., motivation)

• Characteristics of tests (e.g., domain sampling, test length, and test heterogeneity)

• Characteristics of construct and intended use of test scores (e.g., stability of characteristic)

• Characteristics and behavior of the test administrator (e.g., importance of standardized verbal instructions)

• Characteristics of the testing environment

• Test administration procedures

• Scoring accuracy

• Types of reliability and their appropriateness for different types of tests and test use

• Test-retest reliability, parallel or alternative forms reliability, internal consistency, scorer and interrater reliability

• Change scores (or difference scores)

• Standard error of measurement (i.e., standard error of a score) and validity and meaning of test scores.

how differences between the test taker and the particular normative group affect the interpretation of test scores.

Issues to be considered include the types of norms and their relevance for interpreting test taker scores, characteristics of the normative group, type of score referent (e.g., domain referenced, self-referenced), and expectancy tables.

Selection of appropriate test(s). Test users should select the best test or test version for a specific purpose and should have knowledge of testing practice in the context area and of the most appropriate norms when more than one normative set is available. Knowledge of test characteristics such as psychometric properties (pre-sented above), basis in theory and research, and normative data (where appropriate) should influence test selection.

For example, normative data or decision rules may not be accurate when (a) important characteristics of the examinee are not represented in the norm group, (b) administration or scoring procedures do not follow those used in standardiz-ing the test, (c) characteristics of the test may affect its utility for the situation (e.g., ceiling and floor effects), (d) the test contains tasks that are not culturally relevant to the test taker, or (e) the validity evidence does not support decisions made on the basis of the test scores.

Those using tests should have an understanding of how the construction, administration, scoring, and interpre-tation of tests under consideration match the current needs.

Mismatches in these dimensions between the selected test

December 2001 • American Psychologist 1101

Th is d oc um en t i s c op yr ig ht ed b y th e A m er ic an

P sy ch ol og ic al A ss oc ia tio n or o ne o f i ts a lli ed p ub lis he rs

Th is a rti cl e is in te nd ed so le ly fo r t he p er so na l u se o f t he in di vi du al u se r a nd is n ot to b e di ss em in at ed b ro ad and the current testing situation represent important factors that should be considered and that may invalidate usual test interpretation. More specifically, for test users to select an appropriate test for a particular use, it is important that they understand and consider such issues as the intended use of the test score, the method and procedures used to develop or revise the test being considered, the definition of the construct that the test purports to measure, and the defini-tion of the test purpose and its intended context of use.

Additional knowledge needed in this area is listed in Figure 2.

Test administration procedures. Knowl-edge about procedural requirements, confidentiality of test information, communication of results, and test security is important for many testing applications, as is familiarity with standardized administration and scoring procedures and understanding a test user's ethical and legal responsibilities and the legal rights of test takers.

Similarly, it is important that test users understand the legal and ethical issues related to the release of test materials, including issues of confidentiality, de-pending on the context of the testing and the character-istics of the test taker. Test users should be able to explain test results and test limitations to diverse audiences. Written communication should include the purpose of the test and the setting in which the testing occurred. In preparing written reports on test results, test users should be aware that test scores might be-come separated from the interpretive report over time and should be familiar with the areas in Figure 3.

Ethnic, Racial, Cultural, Gender, Age, and Linguistic Variables Consideration of these variables may be important to the proper selection and use of psychological tests. For certain purposes, legal requirements influence or restrict the test-ing, scoring, interpretation, analysis, and use of test data of individuals in different subgroups. In some cases (e.g., employment testing), the use of gender, race, and ethnicity in test interpretation is illegal. Test users should consider and, where appropriate, obtain legal advice on legal and regulatory requirements to use test information in a manner consistent with legal and regulatory standards. Issues asso-ciated with testing individuals from particular subgroups, such as race or ethnicity, culture, language, gender, age, or other classifications, are addressed in greater detail in the

Figure 2 Knowledge Needed for the Appropriate Selection of Tests

• Type of keying or scaling used—rational or theoretical, • empirical, internal consistency or construct homogeneity (e.g., factor analysis) •

• Scoring procedures (e.g., clinical, mechanical, and • correction for guessing)

• Type of score interpretation (criterion or domain referenced, norm referenced, ipsative)

• Item and scale score characteristics

• Item format

• Difficulty level

• Reliability (e.g., internal consistency and test-retest) •

• Validity evidence of test scores

• Construct validity evidence

• Content representativeness •

• Criterion-related •

• Validity generalization (e.g., effects of sample size, test and criterion reliability and range restriction, and dichotomization of variables)

• Convergent

• Discriminant •

Cross-validation

Criterion characteristics (e.g., sufficiency, relevance)

Test bias

Description of validation, normative, and/or standardization group(s) (characteristics of groups such as age, gender, race, culture, language, disabilities, geographic region, socioeconomic status [SES], educational or grade level, motivational set, mental status, and item format familiarity), sample size(s), and recency of data

Test administration procedures (standardization procedures, time limits—power vs. speed)

Knowledge of test taker variables that may moderate validity and interpretation of scores (such as age, gender, race, culture, language, disabilities, geographic region, era or time period tests, SES, educational or grade level, motivational set, mental status, and item format familiarity)

Other or special requirements and limitations of test

Adequacy of the match between test characteristics and present need in terms of construct measured, difficulty level, validity, reliability, test bias, normative data, similarity of normative group with present group, test administration procedures (accommodations for disabilities when appropriate, characteristics of test administrator, adaptation for those with different primary language when appropriate)

Special requirements and limitations of test

1102 December 2001 • American Psychologist

Th is d oc um en t i s c op yr ig ht ed b y th e A m er ic an

P sy ch ol og ic al A ss oc ia tio n or o ne o f i ts a lli ed p ub lis he rs

Th is a rti cl e is in te nd ed so le ly fo r t he p er so na l u se o f t he in di vi du al u se r a nd is n ot to b e di ss em in at ed b ro ad

Figure 3 Additional Knowledge Required for Test Administration

• Legal rights of test takers

• Standardized administration procedures

• Scoring procedures

• Confidentiality of test materials and test information

• Safeguards for protecting test material (protection against copyright infringement, protection against unauthorized dissemination of test items, keys, and scoring procedures)

• Safeguards for protecting protocols and test results (legal issues, ethical issues)

• Reporting results to the test taker, caregiver, or others as appropriate (characteristics of meaningful reports, amount of information to report, legal and ethical issues)

1999 version of the Standards for Educational and Psy-chological Testing (AERA, APA, & NCME, 1999).

The APA's promulgated Guidelines and Principles for Accreditation of Programs in Professional Psychology (APA, 1996) discussed the need for psychology training programs to address issues of cultural diversity. The APA demonstrated its interest in and sensitivity to these issues by establishing the Commission on Ethnic Minority Re-cruitment, Retention, and Training in Psychology. In addi-tion, the Task Force on Delivery of Services to Ethnic Minority Groups, under the auspices of the Board of Ethnic Minority Affairs, published Guidelines for Providers of Psychological Services to Ethnic, Linguistic, and Cultur-ally Diverse Populations (APA, 1990). These guidelines were approved by the APA's Council of Representatives.

In addition, the International Test Commission has issued "Guidelines for Adapting Educational and Psychological Tests: A Progress Report" (Hambleton, 1994), which pro-vides recommendations about adapting tests for cross-cul-tural testing. For test users using tests with different ethnic, racial, cultural, gender, and language groups, knowledge of the constructs listed in Figure 4 is essential.

Testing Individuals With Disabilities Tests are administered to increasing numbers of persons with disabilities in a variety of settings and for a multitude of purposes. The requirement to accommodate an individ-ual with a disability in the testing situation raises many complex issues for test users. Test users must frequently make decisions regarding the use of tests that were not developed and normed for individuals with disabilities. In such circumstances, confidence in the inferences drawn from test results may be diminished. There may be legal requirements concerning the accommodation of individuals with disabilities in test administration and the use of mod-ified tests. Test users should consider and, where appropri-ate, obtain legal advice on legal and regulatory require-ments regarding appropriate administration of tests and use of test data when assessing individuals with disabilities.

Test users should be familiar with several efforts initiated during the 1990s to provide guidance to test users for assessing individuals with disabilities. The APA Task Force on Test Interpretation and Diversity published a book identifying the scientific and policy issues related to the interpretation of tests used with individuals for whom the tests were not developed, standardized, and validated (San-doval, Frisby, Geisinger, Scheuneman, & Grenier, 1998).

Additionally, the Joint Committee on Testing Practices is publishing a sourcebook for practitioners that describes some of the pertinent legal and regulatory information, as well as types of accommodations, required documentation, and the use of tests with disabled individuals in various contexts (Ekstrom & Smith, in press). Finally, the 1999 Standards for Educational and Psychological Testing (AERA, APA, & NCME, 1999) includes a chapter on technical considerations for testing individuals with dis-abilities. Those who administer tests to individuals with disabilities should be familiar with the legal, technical, and professional issues governing the use of tests with individ-uals with disabilities, including those listed in Figure 5.

Supervised Experience In addition to test users having knowledge and skills needed for appropriate test use, it is important that they have the opportunity to develop and practice their skills under the supervision of appropriately experienced profes-sionals. This supervision typically begins in graduate school and continues throughout training until any creden-

Figure 4 Factors Associated With Test Use in Diverse Groups

• Construct equivalence (information concerning the influence of psychological characteristics such as motivation, attitudes, and stereotype threat on test performance

• Orientations and values that may alter the definition of the constructs(s) being assessed and how those factors may affect the interpretation of test results

• Requirements of the testing environment and how that may affect the performance of different groups

• Test bias

• Laws and public policies concerning use of tests that may have implications for test selection, as well as administration and interpretation

• Procedures for examining between-groups differences in test performance

• Empirical literature concerning differential validity for racial or cultural groups

December 2001 • American Psychologist 1103

Th is d oc um en t i s c op yr ig ht ed b y th e A m er ic an

P sy ch ol og ic al A ss oc ia tio n or o ne o f i ts a lli ed p ub lis he rs

Th is a rti cl e is in te nd ed so le ly fo r t he p er so na l u se o f t he in di vi du al u se r a nd is n ot to b e di ss em in at ed b ro ad

Figure 5 Parameters Associated With Testing of the Disabled

• Legal issues

• Test selection

• Test accommodation

• Effects of the testing environment and the tests being used on the performance of individuals with disabilities

• Inferences based on the test scores accurately reflect the construct, rather than construct-irrelevant, characteristics associated with the disability

• Knowledge of whether regular norms or special norms are appropriate for the characteristic in question tials that are necessary to practice independently have been attained. The structure and focus of supervision vary de-pending on the domain(s) in which supervision is being administered. Because testing is conducted by psycholo-gists with different specialties, as well as by nonpsycholo-gists, a specific prescribed format or mechanism for super-vision cannot be described for each test user. However, focused and setting-specific supervision of sufficient inten-sity and duration is important for those who use tests.

Summary of Core Knowledge and Skills for Test Users The intent of this section has been to delineate the generic domains and competencies important for users of psycho-logical tests. Although the extent of knowledge of these generic domains may vary by practice area, some knowl-edge of these core assessment domains in combination with context specific expertise and a high level of professional judgment is important for appropriate test use. The test user's key function is to make valid interpretations of test scores and data, often collected from multiple sources, using proper test selection, administration, and scoring procedures. For test users to provide valid interpretation, it is important that they be able to integrate knowledge of applicable psychometric and methodological principles, the theory behind the measured construct and related empirical literature, the characteristics of the particular tests used, and the relationship between the selected test and the particular testing purpose, the testing process, and, in some contexts, the individual test taker.

Test User Qualifications in Specific Contexts The context in which psychological tests are used includes both the setting and the purpose of testing. Test user qualifications vary across settings, as well as within set-tings, depending on the purpose of testing. This section addresses the context-relevant qualifications that build on the generic qualifications described above. Regardless of the setting, psychological tests are typically used for the following purposes:

Classification—to analyze or describe test results or conclusions in relation to a specific taxonomic system and other relevant variables to arrive at a classification or diagnosis.

Description—to analyze or interpret test results to understand the strengths and weaknesses of an individual or group. This information is integrated with theoretical models and empirical data to improve inferences.

Prediction—to relate or interpret test results with re-gard to outcome data to predict future behavior of the individual or group of individuals.

Intervention planning—to use test results to determine the appropriateness of different interventions and their rel-ative efficacy within the target population.

Tracking—to use test results to monitor psychological characteristics over time.

The sections that follow describe five major contexts in which tests are commonly used: employment, education (both individual and large-scale testing), vocational and career counseling, health care, and forensic assessment.

Although there may be other contexts that require specific competencies, the test user qualifications (including appro-priate training and supervision) important in the major contexts where tests are used are discussed below.

Employment Context Many employers use tests as part of the assessment process to develop work-related information and recommendations or decisions about people who work for them or are seeking employment with them. Test users in this context should have not only the qualifications identified as core knowl-edge and skills but also an understanding of the work setting, the work itself, and the worker characteristics re-quired of the work situation. They should strive to know what skills, abilities, or other individual difference charac-teristics enable people to perform effectively (as defined in a variety of ways) in a particular work setting. Test users should consider the strengths and weaknesses of different methods for determining the human requirements of the work situation and how to conduct such job, work, or practice analyses. They also should consider and, where appropriate, obtain legal advice about employment law and relevant court decisions (see Dunnette & Hough, 1990, 1991, 1992, 1994; Guion, 1998).

Classification. Organizations seek to classify or place people in jobs to maximize overall utility to both the individuals and the institution. To perform these activities well, test users should strive to be knowledgeable about job clustering (e.g., creation of job families), validity, cost-benefit analysis, utility analysis, and measurement of work outcomes.

Psychological tests are sometimes used to certify peo-ple as qualified to perform certain job or work activities.

Test users should have knowledge of the task or work and knowledge of the level of performance required for com-petent practice. This means that test users should define the

1104 December 2001 • American Psychologist

Th is d oc um en t i s c op yr ig ht ed b y th e A m er ic an

P sy ch ol og ic al A ss oc ia tio n or o ne o f i ts a lli ed p ub lis he rs

Th is a rti cl e is in te nd ed so le ly fo r t he p er so na l u se o f t he in di vi du al u se r a nd is n ot to b e di ss em in at ed b ro ad task or criterion, measure the required knowledge and skills, and identify the required performance level. They should strive to have a thorough knowledge of job, work, or practice analysis and of content validation principles and strategies.

Description. Description of an individual's cur-rent abilities, skills, interests, personality, knowledge, or other personal characteristics can be a significant part of the assessment process. This information is the starting point for determining the fit between an individual and work in a given setting; identifying areas of needed individual, team, or organizational development; providing feedback about likely success in different work activities and settings;

planning career choices and paths; and auditing organiza-tional or unit readiness. Those who use psychological tests to describe individual, team, or organizational characteris-tics in the employment setting should have knowledge about job, work, or career analysis (see, e.g., Campion, 1994; Goldstein, Zedeck, & Schneider, 1993).

Prediction. Psychological tests may be used as part of a larger assessment process to help make predictions about an individual's future training performance, job per-formance, trustworthiness, attrition, or a variety of other work-related criteria. These predictions are often made to facilitate recommendations or decisions about selection, promotion, or succession planning.

Test users involved in testing to predict future em-ployment criteria should make every effort to be knowl-edgeable about the work setting, the work itself, and, hence, job or work analysis methods. They also should understand performance measurement, criterion constructs and their measurement, relationships between various pre-dictor constructs and criterion constructs, research methods and design, validity concepts and evidence, test bias, ad-verse impact analysis, utility analysis, validity generaliza-tion, and group differences, and they should, where appro-priate, obtain legal advice.

Intervention planning. Employment testing may be part of an analysis of the test taker's training and development needs. Test results may provide information for developing plans to improve skill and performance of current work responsibilities and anticipated work respon-sibilities. Test results also may be used as part of career planning activities. When tests are used for these purposes, test users should make every effort to be knowledgeable about such matters as the work itself, the work setting, performance appraisal and performance measurement, cri-terion constructs and their measurement, training and de-velopment, career development, coaching and mentoring, and training needs analysis.

Employment testing may be part of an outplacement process. If testing is done as part of an involuntary process that determines who is to be retained and who is to be laid off, test users should be knowledgeable about the work itself, the work setting (hence, job, work, or practice anal-ysis methods), performance measurement, criterion con-structs and their measurement, validity concepts and evi-dence, test bias, adverse impact analysis, and group differences, and they should obtain appropriate legal ad-vice. If testing is done as part of a voluntary job search process, test users should be knowledgeable about voca-tional and career guidance, job loss, and labor markets.

Employment testing also may be a part of a monitor-ing system designed to identify individuals who are at risk for performing below an acceptable level. The individuals may be employed in sensitive-duty (high cost for mistakes) jobs. Those who use tests to identify at-risk individuals should have the qualifications listed under the Classifica-tion and Prediction subsections above.

Tracking. Psychological tests may be used in pre-dictive, criterion-related validation studies in which indi-viduals and their performance are tracked over time. In addition to the knowledge recommended for the use of psychological tests for prediction purposes, test users who track individuals or their performance also need to under-stand how task or work performance and criterion perfor-mance requirements may change over time. In addition, test users who conduct reassessments should be familiar with the effects of repeated use of assessment procedures on both the individual and the findings obtained.

Training and supervision. Training for test use in the employment context is best obtained by success-ful completion of an integrated program of study that includes industrial psychology; psychology of individual differences; measurement theory; job, work, and practice analysis; performance measurement; and employment law relevant to the testing situation. Experience and supervision using tests in settings similar to those in which employment tests are used are important. For test users who provide assessment of health outcomes or understanding of health problems of individuals and groups (e.g., those working in employee assistance programs [EAPs]), the qualifications described in the Health Care Context section below also apply.

Educational Context

The results of psychological tests often serve as relevant information to guide educational decisions about both stu-dents and programs. Psychological tests are used in a variety of educational settings, including preschools, ele-mentary and secondary schools, higher education, technical schools, business training programs, counseling centers, health and mental health settings that offer educational services, and educational consulting practices. Psycholog-ical tests are typically used to acquire information about students to make informed decisions about such issues as student admissions and placement, educational pro-gramming, student performance, and teacher or school effectiveness.

On an individual level, psychological tests are often used to describe a student's learning or behavioral strengths and weaknesses. The results may then be used to develop educational interventions, to determine appropriate educa-tional placements (e.g., special education, gifted education, magnet school program, or alternative educational setting), or as part of clinical diagnostic assessment to guide thera-peutic services.

December 2001 • American Psychologist 1105

Th is d oc um en t i s c op yr ig ht ed b y th e A m er ic an

P sy ch ol og ic al A ss oc ia tio n or o ne o f i ts a lli ed p ub lis he rs

Th is a rti cl e is in te nd ed so le ly fo r t he p er so na l u se o f t he in di vi du al u se r a nd is n ot to b e di ss em in at ed b ro ad

Assessment of groups of individuals, often called large-scale testing, typically addresses questions about ed-ucational programs or policies. Decision makers may ag-gregate results from psychological tests and use this infor-mation to evaluate program effectiveness and to develop recommendations for changes to educational programs or systems. Test users in these cases may use standardized tests or nonstandardized procedures (e.g., performance events or portfolios of student work) to obtain information about cognitive ability or academic achievement levels of a group of students (Fuchs & Fuchs, 1990).

Test user qualifications that have particular relevance in educational settings include the representativeness of the test sample, attention to language and cultural diversity, and the use of cut scores in selection for special programs.

Test users also should understand the cognitive and emo-tional factors that affect student learning, as well as the social and political factors that affect schools as learning environments. Those who use psychological tests in social institutions like schools should be particularly skilled at communicating the results of testing to many different audiences, including educational decision makers, teachers, students, parents, and the public.

Classification. Tests are often used to identify or classify individual students or groups of students for ad-mission to special programs. In public elementary and secondary schools, the most frequently used formal classi-fication system is probably the one used to determine eligibility for special education services as required by federal and state law (e.g., the Individuals With Disabilities Education Act, 1990/1997). Therefore, test users in educa-tional contexts should consider and, where appropriate, obtain legal advice regarding state and federal laws related to the provision of educational and related services to disabled students. Many schools also use curriculum-track-ing schemes (e.g., general vs. college-preparatory classes) that categorize students and then place them in separate instructional tracks or ability groupings, often on the basis of test data. Individuals using psychological tests for clas-sification purposes, in both individual and large-scale as-sessments, should be familiar with the taxonomic systems used by schools and other educational settings as well as the psychometric limitations of the tests used.

Test users also should possess the knowledge to select instruments that are appropriate for the characteristics of the student being evaluated. For example, tests that have adequate reliability and validity for assessing school-age students may be inappropriate for use with preschool chil-dren. Similarly, tests normed and validated for use with individuals from one culture or ethnic group may not be appropriate for assessing individuals from other cultural or ethnic populations. Also, when making high-stakes deci-sions about individuals, test users should integrate infor-mation when appropriate from multiple sources, such as psychological and educational test data, behavioral obser-vations and ratings, school records, and interviews with parents and teachers (Salvia & Ysseldyke, 1995).

Large-scale tests are used for a variety of purposes, including program accountability and decisions related to admissions and educational placement. When schools, dis-tricts, or states develop or select a test to determine student achievement relative to state standards, test users should have the skills and knowledge to determine the degree of correspondence among the standards, curricula, and test content. When critical decisions, such as graduation or retention, are based on test results, test users should strive to consider students' opportunity to learn the stated content and to identify other sources of relevant data that reflect student proficiency. When tests are used for college place-ment, test users should determine the degree of alignment between the test's content and the college curriculum and should understand the relationship between predicted and actual performance in college before determining a cut score or other classification criteria. Legal requirements may influence or restrict the use of rank ordering or cut scores, particularly if these practices have a disproportion-ate effect on one or more subgroups.

Description. Psychological tests also are used in educational settings to describe aspects of learners' skills and abilities, such as learning styles, motivation, reading readiness, and emotional maturity. Group measures of in-terests, attitudes, cognitive abilities, or emotional adjust-ment also may provide a basis for interventions designed to remediate current problems or to prevent future difficulties.

Large-scale assessments are often used by schools, districts, and states to measure the general level of student performance or to evaluate the effects of curricular deci-sions. In some instances, schools or teachers may be held accountable for their students' test results, with penalties imposed for scores below expectations. Therefore, it is important that test users attend to the multiple factors that contribute to test score differences between schools, class-rooms, or districts (e.g., student motivation, quality of prior educational experiences, and parental support of educa-tional goals).

Prediction. In the educational context, tests are often used to predict the future behavior or academic suc-cess of a student or group of students. In individual assess-ment, tests are often used to screen students for placement in special programs or to place them in an instructional group or track on the basis of a prediction of expected future performance.

In large-scale testing, admissions tests are required for entry into most undergraduate, graduate, and professional programs. Admissions tests also are useful in college coun-seling, providing students with useful information on their potential for academic success at different colleges and universities. In addition, most colleges use specially devel-oped placement tests to determine a student's eligibility for particular courses.

Test users in educational settings should have the skills and knowledge to evaluate the relative contribution of teacher competence and motivation, school and class-room climate, peer group influence, class size, and other factors .that play a critical role in determining a student's future performance. Test users should understand how group differences (e.g., ethnicity, gender, race, and socio-economic status [SES]) may affect performance on stan-

1106 December 2001 • American Psychologist

Th is d oc um en t i s c op yr ig ht ed b y th e A m er ic an

P sy ch ol og ic al A ss oc ia tio n or o ne o f i ts a lli ed p ub lis he rs

Th is a rti cl e is in te nd ed so le ly fo r t he p er so na l u se o f t he in di vi du al u se r a nd is n ot to b e di ss em in at ed b ro ad dardized tests, grades, school completion, and other out-comes that may be used in predicting academic success.

Intervention planning. Psychological tests are frequently used to plan interventions for one student or a group of students. Psychological tests are commonly used as part of the individual diagnostic assessment of students with learning or behavioral problems. The results from these tests help to describe or diagnose the educational strengths and weaknesses of students or their behavioral difficulties and contribute to the development of educa-tional, behavioral, or mental health interventions. Those who use tests to prescribe interventions based on assessed student characteristics should be familiar with the empirical evidence for using test data to make such decisions.

Test results sometimes provide a rationale for educa-tional interventions that affect a large number of students, such as a modification in instructional approach (see, e.g., Gettinger & Stoiber, 1999). Test users should strive to clearly communicate to decision makers the appropriate-ness of inferences based on test data and the likely effects of program changes on various groups of students. Test results may also be used as a basis for individual interven-tions, such as removing a student from school. Test users should consider and, where appropriate, obtain legal advice about relevant state and federal laws dealing with changes in school placement as well as the legal protections af-forded to parents and students, including a student or parent's due process rights and requirements of informed consent (Jacob-Timm & Hartshorne, 1994).

Tracking. Test users in school settings often ad-minister tests multiple…