Sparse Feature Policy Unlearning Mitigates State Hallucination in Vision-Language-Action Models
Abstract
Researchers identify state hallucination in vision-language-action models and propose sparse feature unlearning to reduce failure behaviors while preserving manipulation skills.
Vision-Language-Action (VLA) models have shown strong generalization in robotic manipulation by leveraging rich representations from pretrained vision-language models. However, their deployment in real-world environments remains limited by recurring unreliable behaviors. In this work, we study state hallucination, a recurring failure pattern in which a VLA continues acting as if an unrealized robot-object state had been achieved. Our analyses find that state hallucination coincides with weakened attention to task-relevant visual regions, and a mechanistic interpretation via sparse autoencoders reveals that hallucination-associated sparse features are activated when these failures occur. Based on this analysis, we propose SOUL (Sparse feature pOlicy UnLearning), which selectively unlearns policy knowledge associated with state hallucination behaviors, where sparse features identified from hallucination failures and successful behaviors serve as explicit forgetting and retention targets, respectively. Experiments across VLA architectures in simulated and real-world environments show that our method substantially reduces hallucinated failures and improves task success without substantially compromising the existing manipulation capabilities. These results suggest that interpretable feature analysis provides a practical basis for selectively modifying undesirable knowledge in robot policies.
Community
TL;DR: Can robots forget harmful behaviors without losing useful skills?
SOUL (Sparse feature pOlicy UnLearnig) identifies interpretable sparse features associated with state hallucinations in VLAs and uses them to selectively unlearn harmful behaviors, reducing hallucination-induced failures while preserving successful manipulation capabilities.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- RoboIRS: Inference-Time Internal Representation Steering for Generalist Robot Policies (2026)
- VLaRL: Augmenting Vision-Language-Action Models with Simulation-Trained Latent-Conditioned Residual RL (2026)
- Detect and Suppress: A Mechanistic Defense against Adversarial Patches in VLA Models (2026)
- Complementary Future Token Pruning for World Action Models (2026)
- HaWMPO: Hallucination-Aware World Model-based Policy Optimization for Generalist Robot Policy (2026)
- Training-Free Action Correction for VLA Model Failures via Language Feedback (2026)
- SLIP-VLA: Single-Step Latent Imagination for Policy Learning in Vision-Language-Action Models (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2610.09496 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper