Abstract
Physical interactions can create future hazards that are not apparent from the robot's current geometric surroundings. We present a framework termed Predictive Semantic Safety (PSS), which connects visual physical reasoning to backup-based safety filtering. A vision-language model (VLM) predicts physical events and their timing or directly predicts object displacements. An explicit motion model converts event hypotheses into object trajectories. Split conformal prediction calibrates position errors jointly across specified objects, observation times, and future times; geometric shape bounds convert the resulting position regions into predicted object occupancy. PSS evaluates a prescribed backup maneuver against this occupancy and derives input-affine constraints for minimally modifying the nominal input while preserving backup feasibility under the robot dynamics and input limits. MuJoCo experiments with a Unitree Go1 consider falling fixtures, impact-driven support loss, and contact propagation. PSS achieves a safe episode rate of 99.3%, compared with 43.3% for a Backup Control Barrier Function baseline that only uses current obstacle geometry.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Kim, T., Fradi, S., Dai, Y., Ostaszewski, M., & Schmidhuber, J. (2026). Predictive Semantic Safety: From Visual Physical Reasoning to Safety-Critical Control. https://omanscience.com/en/articles/predictive-semantic-safety-from-visual-physical-reasoning-to-safety-critical-control
MLA 9
Kim, Taekyung, et al. "Predictive Semantic Safety: From Visual Physical Reasoning to Safety-Critical Control." https://omanscience.com/en/articles/predictive-semantic-safety-from-visual-physical-reasoning-to-safety-critical-control.
Chicago (author–date)
Kim, Taekyung, Salem Fradi, Yanning Dai, Mateusz Ostaszewski, and Jürgen Schmidhuber. 2026. "Predictive Semantic Safety: From Visual Physical Reasoning to Safety-Critical Control." https://omanscience.com/en/articles/predictive-semantic-safety-from-visual-physical-reasoning-to-safety-critical-control.
Harvard
Kim, T., Fradi, S., Dai, Y., Ostaszewski, M. and Schmidhuber, J. (2026) 'Predictive Semantic Safety: From Visual Physical Reasoning to Safety-Critical Control', Available at: https://omanscience.com/en/articles/predictive-semantic-safety-from-visual-physical-reasoning-to-safety-critical-control.
Vancouver
Kim T, Fradi S, Dai Y, Ostaszewski M, Schmidhuber J. Predictive Semantic Safety: From Visual Physical Reasoning to Safety-Critical Control. https://omanscience.com/en/articles/predictive-semantic-safety-from-visual-physical-reasoning-to-safety-critical-control
IEEE
T. Kim, S. Fradi, Y. Dai, M. Ostaszewski, and J. Schmidhuber, "Predictive Semantic Safety: From Visual Physical Reasoning to Safety-Critical Control," https://omanscience.com/en/articles/predictive-semantic-safety-from-visual-physical-reasoning-to-safety-critical-control.