Attachment L - AMS Overview.pdf

Assessment Management System Overview

Activities

NAEP’s data collection is organized into four main activities, which are supported by the NAEP Assessment Management System (AMS). These major tasks in each NAEP cycle’s data collection activities include:

• Recruitment and Sampling – releasing the school sample for a NAEP cycle to the states, recruiting public and private schools to participate in NAEP, collecting information about school characteristics, collecting student lists in support of student sampling, and scheduling assessments

• Preassessment – collecting logistical and technical information needed to conduct assessments in schools, collecting information about students needed for reporting and accommodations, and collecting school and teacher questionnaires

• Assessment – conducting assessments, collecting information about which students were assessed

• Special Studies – conducting special studies in selected NAEP cycles (such as the High School Transcript Study)

Characteristics

NAEP State and TUDA Coordinators, district staff, school staff, and data collection staff use the AMS to conduct these data collection tasks and activities. The AMS includes the following characteristics:

• Clean, modern, responsive, and user-friendly design

• Reflects feedback from panels of actual users who review proposed AMS designs and features before each NAEP cycle

• Incorporates feedback received from actual users on the AMS after each NAEP cycle

• Conforms with web accessibility standards

• Conforms with FISMA-Moderate Federal security standards

• Supports internationalization and localization for US English and Puerto Rico Spanish

• Supports scenario-based online training

• Includes integrated monitoring and reminders for data collection tasks

• Built with modern web application tools and a scalable cloud-based architecture

Finally, the AMS integrates with other NAEP components including the NAEP assessment platform, the NAEP questionnaire platform, the NAEP digital form selection tool platform, and workflows for sampling, weighting, and reporting.

Features

Key AMS features include:

Attachment L

• Update school participation status – Each assessment program includes a sample of schools, and tracking each school’s participation in the program (such as participating or refused) is key.

• Monitor school participation – Allows NAEP State/TUDA Coordinators and data collection staff to monitor school participation.

• Provide school characteristics – Collect and confirm basic school information such as school name, address, enrollment, and schedule.

• Manage school team – Provides a way to manage registrations for school staff and indicating which roles on the assessment (such as principal) each school staff member fills.

• Monitor provide school characteristics – Allows NAEP State/TUDA Coordinators and data collection staff to ensure each sampled school is ready for the student list import process.

• Import Students – Collects a list of all students in the selected grades from in-scope sampled schools. Schools, districts, or NAEP State/TUDA Coordinators can import this list.

• Monitor Import Students – Allows NAEP State/TUDA Coordinators and data collection staff to ensure each sampled school has an imported student list.

• Perform Student Sampling – Student sampling occurs after the student list submission phase ends.

• Respond to Internet Connectivity Survey (ICS) – This survey determines whether NAEP can use the school’s Internet/Wi-Fi.

• Manage data collection staff team – Each school requires a team of data collection staff to prepare for and conduct the assessment, and this feature supports tracking assignments.

• Add Students – Allows schools to add students who do not appear on the original student list frame previously submitted by states, districts, and schools.

• Update Economically Disadvantaged information for students – Allows NAEP State/TUDA Coordinators to update Economically Disadvantaged information for students.

• Monitor data collection staff teams – Allows data collection staff to manage their teams.

• Monitor Internet Connectivity Survey responses – Allows NAEP State/TUDA Coordinators and data collection staff to ensure the survey is completed.

• Schedule Assessment Planning Meeting – Allows data collection staff to conduct meetings with school coordinators to ensure the school is prepared for the assessment day.

• Provide assessment logistics – Allows the school coordinator and data collection staff collaborate to provide assessment logistics in advance of the assessment.

• Provide technical logistics – Allows the school coordinator and data collection staff collaborate to provide technical logistics in advance of the assessment.

• Monitor assessment activities – Allows NAEP State/TUDA coordinators and data collection staff to ensure required tasks are complete.

• Notify parents/guardians – Each assessment study requires implicit or explicit parent/guardian consent for their students to participate in the assessment.

• Monitor parent/guardian notification – Allows NAEP State/TUDA coordinators and data collection staff to ensure required parent/guardian notification tasks are performed.

• Provide student information – Allows collection of detailed information from school coordinators on the sampled students needed for the assessment.

• Provide inclusion policies – State coordinators share with schools the NAEP inclusion policies that describe Universal Design Elements and accommodations offered to English learners and students with disabilities.

• Manage questionnaires – School coordinators need to identify school questionnaire and teacher questionnaire respondents to answer the NAEP questionnaire.

• Monitor questionnaires – Allows NAEP State/TUDA Coordinators and data collection staff to ensure required questionnaires are completed.

• Conduct assessment – Allows ACs to send student registration information to launch a NAEP assessment.

• Monitor assessment – Provides a centralized place for data collection staff to monitor progress through the assessment including devices and students.

• Download assessment day documents – School coordinators can download key assessment documents such as student appointment cards.

• Monitor assessment tasks – Allows NAEP State/TUDA Coordinators and data collection staff to monitor required assessment tasks.

• Monitor transcript study tasks – Allows NAEP State/TUDA Coordinators and data collection staff to monitor required transcript study tasks.

• Submit course catalogs – Allows transcript study coordinators to submit course catalogs.

• Complete transcript information survey – Allows transcript study coordinators to submit a survey to provide information on their transcripts.

• Submit student transcripts – Allows schools to upload student transcripts.

Assessment Management System Component Diagram

Assessment Management System Data Flow Diagram