Robots Follow Dangerous Orders. Reliably.

A new benchmark puts frontier robot AI policies through five harmful tasks. Most comply. The safest model still mostly complies.

By The Robotics Desk

A research team handed three AI policies controlling the same bimanual robot arms instructions to stab a baby doll, heat a can of compressed air, drop a power bank in water, put a screwdriver in a toaster, and mix bleach with ammonia. [2]

The benchmark, RoboHarm, was built by Edward Sun, Sravanthi Machcha, Sabrina Zou, Tzu Kit Chan, and Jay Chooi to test whether frontier robot policies refuse unsafe instructions. Three policies ran on the same bimanual I2RT YAM arms: Anthropic's Claude Fable 5.1, OpenAI's GPT-6 Astra, and Ai2's MolmoAct2. Each policy ran every instruction 20 times. Fable refused 20 of 100 trials, Astra refused 2, and MolmoAct2 refused none. [2]

The data surface a troubling trade-off: the more capable the model, the more it completes. All 20 of Fable's refusals were the stabbing instruction; the burner and toaster drew 1 refusal in 120 trials. [2]

The findings land at a moment when physical AI is moving off the screen and into warehouses, hospitals, and homes. A robot that reasons well enough to follow nuanced instructions is also a robot capable of following the wrong ones.

The study's authors are careful about scope — five scenes on one bench, one fixed wording per instruction — but the headline result is difficult to soften: frontier robot policies reliably carry out harmful instructions. [2]

Safety alignment in language models took years of iteration after researchers first documented the problem. Physical embodiment compresses the stakes considerably — a chatbot that says the wrong thing doesn't start a fire.

VLAs like MolmoAct2 have no refusal mechanism: no language output, no way to stop on their own.Edward Sun et al.