Safety from weak feedback: inverse constraint reinforcement learning for safe control and safe online adaptation

The National Science Foundation Division of Electrical, Communications and Cyber Systems awarded New Jersey Institute of Technology $324,116 on October 1, 2026, for research into inverse constraint reinforcement learning for safe autonomous systems. The project, "Safety from Weak Feedback: Inverse Constraint Reinforcement Learning for Safe Control and Safe Online Adaptation," runs through September 30, 2029, and is performed in Newark, New Jersey.

The research establishes mathematical foundations enabling learning-based autonomous systems to discover safety limits from coarse, after-the-fact feedback rather than precise moment-by-moment safety specifications. The project develops theory determining when safety constraints can be learned from limited feedback, methods measuring uncertainty relevant to safety maintenance, and strategies for cautiously exploring safety boundaries without taking unacceptable risks. Deliverables include theoretical advances in learning safety from weak trajectory-level feedback, open educational resources, a free online course, public software and benchmarks, and research training for students from high school through doctoral levels. The work targets improvements in reliability of autonomous systems deployed in healthcare, transportation, and other safety-critical settings. This award is funded under NSF's Engineering program (CFDA 47.041).

Abstract

This NSF project aims to make learning-based autonomous systems provably safe in the realistic cases where the rules defining safe behavior are never written down and cannot easily be specified at all. Today's methods for safe machine learning usually assume someone can hand the machine a precise, moment-by-moment definition of danger. In the real world, machines instead receive only coarse, after-the-fact feedback?whether an entire trip ended safely or came dangerously close to failing?and must work out the hidden safety limits for themselves. The project will establish the mathematical foundations for a machine to discover its own safety limits from weak feedback, to recognize honestly what IT does not yet know, and to act, explore, and improve without ever taking unacceptable risks. This will be achieved by developing theory that determines when safety can even be learned from such limited feedback, methods that measure uncertainty specifically where IT matters for staying safe, and a novel strategy that turns a shortage of data into a controllable process of cautiously probing to reveal the safety boundary. The intellectual merit of the project includes advancing the fundamental understanding of when safety constraints can be learned from limited feedback and how to guarantee safe behavior despite that uncertainty. The broader impacts of the project include strengthening the reliability of autonomous systems in healthcare, transportation, and other safety-critical settings, along with open educational resources, a free online course, public software and benchmarks, and research training for students from high school through the doctoral level. The central technical challenge in safe machine learning is that safety specifications are often unknown and observed only through weak, trajectory-level feedback?a single safe/unsafe label per attempt, rather than a detailed cost at every step. This makes the problem an inverse one: the system must reconstruct hidden safety limits without knowing where along a trip a violation occurred, and available data rarely approaches the true danger boundary. The project addresses this through three connected directions: first, establishing when hidden safety limits can be identified at all and building calibrated confidence sets that capture what remains uncertain; second, using those sets to compute policies that stay safe under worst-case uncertainty while taking small, deliberately low-risk actions that most efficiently reduce IT; and third, enabling systems to keep improving after deployment with near-zero safety violations, even as conditions drift. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the foundation's intellectual merit and broader impacts review criteria.- subawards are not planned for this award.