Learning to see physical properties with active sensing motor policies.
Consider a drone image of a terrain and a few candidate paths for a legged robot.
Depending on the task, the relative frictions of terrains will have different bearing on the optimal path.
We develop a pipeline to inform such plans by perceiving terrain properties.
We propose Active Sensing Motor Policies, which adapt the robot’s behavior to accurately estimate physical information,
for example, intentionally swiping the feet against the ground to feel the friction parameter.
Active Sensing Motor Policies allow us to label the terrain properties from a few minutes of traversal.
The robot then learns a self-supervised vision module that densely predicts the terrain physics from RGB images.
The vision module can generalize to images from a teamed drone which facilitates semantics-aware path planning.
We can select efficient locomotion paths under new conditions using the same vision module.
The robot decides to cross the grass when walking alone but keeps the payload on the sidewalk to walk faster.
Thank you