Do OpenVLA Transformers Encode Motion Primitives? A Layer-wise Attention Analysis

Abstract

Vision-Language-Action (VLA) models such as OpenVLA have demonstrated strong performance in robotic manipulation by jointly modeling visual observations and action generation. In this work, we investigate whether specific hidden layers specialize in distinct motion primitives during manipulation. We use a layer-wise attention visualization method over OpenVLA and analyze action sequences from egocentric manipulation datasets. Through the analysis of different motion stages at different depths, our findings suggest that the motion primitive is not explicitly constructed, but rather emerges naturally through hierarchical transformer representations, providing new interpretability insights for embodied foundation models.