Intelligent simulation-based learning about natural disasters

The National Science Foundation's Division of Research on Learning in Formal and Informal Settings awarded a $3.81 million Project Grant under the STEM Education program (CFDA 47.076) to Concord Consortium Inc. on September 15, 2025, for the five-year ISLAND (Intelligent Simulation-Based Learning About Natural Disasters) initiative. The project will develop and research Hazbot, an artificial intelligence-enhanced pedagogical system leveraging machine learning and large language models to provide automated, personalized feedback to middle school students engaged in simulation-based scientific inquiry into wildfires, floods, and hurricanes. The system will guide students through data collection, analysis, and interpretation while generating targeted instructional recommendations for teachers. All educational materials developed through the project will be made freely available to educators, students, and researchers beyond project participants.

The project incorporates expertise from three sub-awardees: WestEd will serve as external evaluator, conducting psychometric analyses, pilot study design and validation, and impact analysis using hierarchical linear modeling through August 2030; the American Meteorological Society will contribute subject matter expertise by recruiting scientists and educators, reviewing instructional materials, developing professional development content, and securing formal approval for the finalized modules; and Physics Front will provide specialized knowledge in learning analytics, large language model-based automated scoring development, and simulation-based inquiry pedagogical theory. The deliverables will advance understanding of how automated feedback combined with teacher facilitation supports middle school students' evidence-based reasoning about natural disaster hazards.

Abstract

While simulations are powerful tools for scientific inquiry, most students need scaffolding to engage productively in simulation-based inquiry. This project will develop and study an automated feedback system designed to support middle school students' simulation-based inquiry into wildfires, floods, and hurricanes. The system, called hazbot, will leverage advanced artificial intelligence (AI) technologies – including machine learning and large language models (llms)--to provide timely, personalized feedback as students investigate the three different natural hazards. Hazbot will guide students to collect, analyze, and interpret data from simulations and develop scientific arguments based on that data. Hazbot will also synthesize the automated performance diagnosis and feedback information provided to students and offer teachers targeted instructional suggestions to support individual students and the whole class. The project will research the automated scoring methods, the automated feedback system, the combinations of teacher facilitation and automated feedback needed to support students' simulation-based inquiry, and the impact of hazbot-integrated wildfire, flood, and hurricane modules on student learning outcomes. The materials generated through design and development will be made available for free to all future students, teachers, and researchers beyond the participants outlined in the project. Island (intelligent simulation-based learning about natural disasters) is a five-year level III design and development project aimed at advancing middle school students' understanding of wildfires, floods, and hurricanes – and their ability to construct evidence-based arguments about these hazards – through simulation-based inquiry supported by automated feedback. The project will design a fully integrated AI-enhanced two-tier pedagogical agent to (1) diagnose student performance in simulation-based scientific inquiry and respond in real time to their evolving needs and (2) support teachers by synthesizing student learning in an actionable teacher dashboard. In the first three years, the project will employ design-based research to develop and integrate the hazbot system into three modules in collaboration with 9 teachers and their 900 students across geographically and demographically diverse schools. This phase will investigate how hazbot's automated scoring models capture students' simulation-based inquiry behaviors and performance; how its feedback supports students in collecting, analyzing, and interpreting data and constructing evidence-based arguments; and what combinations of teacher facilitation and automated feedback are most effective. In the final two years, the project will conduct three randomized controlled trials (rcts)--one for each hazard module (wildfires, floods, and hurricanes)--to measure the impact of the hazbot system on students' understanding of natural hazards and risk, as well as their ability to construct scientific arguments. These rcts will involve a nationally recruited sample of 72 teachers and 3,600 students with half of the teachers randomly assigned to implement a hazbot-integrated version of the module and the other half implementing the same module without hazbot integration. This project will generate critical insights for designing LLM-based feedback systems that can (1) be trained to uphold disciplinary standards, (2) systematically scaffold simulation-based inquiry, and (3) integrate meaningfully with teachers who bring valuable contextual insights to classroom implementation. This project is supported by two NSF programs: the discovery research prek-12 (DRK-12) program, which aims to significantly improve the learning and teaching of science, technology, engineering, and mathematics (stem) for prek-12 students and teachers through research and development of innovative resources, models, and tools; and the innovative technology experiences for students and teachers (itest) program, which supports projects that advance understanding of the practices, program elements, contexts, and processes that foster students' knowledge of and interest in stem disciplines and information and communication technology (ICT) careers. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the foundation's intellectual merit and broader impacts review criteria.- subawards are planned for this award.