Learning Interaction-Aware Robot Hand Retargeting from monocular RGB videos
Human hand demonstrations provide useful training data for dexterous robots. However, directly mapping human hand motion to a robot hand often produces unstable contact, object drops, or incorrect object motion.
In this Master’s thesis, you will train a temporal model to improve robot hand controls using corrections generated by an interaction-based teacher. The teacher uses the object mesh during training, but the learned model will only use observed hand and object motion during inference. The project will test whether this approach can improve manipulation success while avoiding costly physics optimization for every new demonstration.
In this Master’s thesis, you will train a temporal model to improve robot hand controls using corrections generated by an interaction-based teacher. The teacher uses the object mesh during training, but the learned model will only use observed hand and object motion during inference. The project will test whether this approach can improve manipulation success while avoiding costly physics optimization for every new demonstration.