Auto-Tuned Sim-to-Real Transfer
Auto-Tuned Sim-to-Real Transfer

Yuqing Du*
UC Berkeley
Olivia Watkins*
UC Berkeley
Trevor Darrell
UC Berkeley
Pieter Abbeel
UC Berkeley
Deepak Pathak
CMU
* equal contribution
International Conference on Robotics and Automation (ICRA), 2021
[Paper]
[GitHub Code]

Policies trained in simulation often fail when transferred to the real world due to the `reality gap' where the simulator is unable to sufficiently accurately capture the dynamics and visual properties of the real world. Current approaches to tackle this problem, such as domain randomization, require prior knowledge and engineering to determine how much to randomize system parameters in order to learn a policy that is robust to sim-to-real transfer while also not being too conservative.


Auto-Tuned Sim2Real

We propose a method for automatically tuning system parameters of simulator to match the real world using only raw observation images without the need to define rewards or estimate state in the real world itself. Our key insight is to reframe the auto-tuning of parameters as a search problem where we iteratively shift the simulation system parameters to approach the real world system parameters. We propose a Search Param Model (SPM) that, given a sequence of observations and actions and a set of system parameters, predicts whether the parameters are higher or lower than the true parameters used to generate the observations. We evaluate our method on multiple robotic control tasks in both sim-to-sim and sim-to-real transfer, demonstrating significant improvement over the conventional approach of domain randomization.


Overview of the Algorithm