Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior

Gabriel B. Margolis     Pulkit Agrawal

Improbable AI Lab
Computer Science and Artificial Intelligence Laboratory (CSAIL)
Massachusetts Institute of Technology

Conference on Robot Learning 2022 (Oral)

Learned locomotion policies can rapidly adapt to diverse environments similar to those experienced during training but lack a mechanism for fast tuning when they fail in an out-of-distribution test environment. This necessitates a slow and iterative cycle of reward and environment redesign to achieve good performance on a new task. As an alternative, we propose learning a single policy that encodes a structured family of locomotion strategies that solve training tasks in different ways, resulting in Multiplicity of Behavior (MoB). Different strategies generalize differently and can be chosen in real-time for new tasks or environments, bypassing the need for time-consuming retraining. We release a fast, robust open-source MoB locomotion controller, Walk These Ways, that can execute diverse gaits with variable footswing, posture, and speed, unlocking diverse downstream tasks: crouching, hopping, high-speed running, stair traversal, bracing against shoves, rhythmic dance, and more.

All behaviors are achieved by a single neural network policy with its gait parameters modulated by a human operator or procedurally generated by a script.

        @article{margolisyang2022rapid,
            title={Rapid Locomotion via Reinforcement Learning},
            author={Margolis, Gabriel B and Yang, Ge and Paigwar, Kartik and Chen, Tao and Agrawal, Pulkit},
            journal={Robotics: Science and Systems},
            year={2022}
        }
        

        @article{margolis2022learning,
            title={Learning to Jump from Pixels},
            author={Margolis, Gabriel B and Chen, Tao and Paigwar, Kartik and Fu, Xiang and Kim, Donghyun and Kim, Sangbae and Agrawal, Pulkit},
            journal={Conference on Robot Learning},
            year={2021}
        }