🤗HF Dataset | arXiv | XHand Coupling CAD
LeRobot Extension for Franka FER Robot & XHand Robot. An instantiation of the LeVR framework.
There are three main parts to this LeRobot Franka robot extension, franka_server, franka_xhand_teleoperator, and added class implementation under src/lerobot. Check out system flowchart below:
Project was tested on LeRobot commit ce3b9f627e55223d6d1c449d348c6b351b35d082, with Ubuntu 24.04 and Python 3.12. To use this extension, copy and paste all content inside the repo over to your LeRobot directory and do the following:
This needs to be built and deployed on your real-time machine that controls the robot.
cd franka_server
bash build.shBuild franka_server and run on your robot RTPC (or run in a second terminal if it's the same PC)
franka_server Dependencies
Note
You will need to run the following commands on RTPC to start up the server before using any LeRobot utilities to move the arm.
./franka_server [YOUR_FRANKA_ROBOT_IP]This needs to be built and added to your environment
cd franka_xhand_teleoperator
[uv] pip install -e .Additionally, you will also need to set up Meta Quest VR App from this repo
XHand Dependencies
For XHand, we will use a repository modified based on Yuzhe Qin's amazing work on dex-retargeting to map human hand motion to the robot hand.
To enable XHand Motion Retargeting:
# First, update all git submodule
git submodule update --init --recursive
# Build dependencies
cd vr-dex-retargeting
[uv] pip install -e .You will also need to install RobotEra's python API to control the XHand robot. Download the python wheel from RobotEra's document center, and install to your environment.
# Use your corresponding python version
[uv] pip install xhand_controller-1.1.7-cp312-cp312-linux_x86_64.whl Copy to merge with files under LeRobot's src directory. This includes new Robot and Teleoperator class implementations needed to work with the rest of the framework.
Note
Due to time constraints and the nature of the project, I didn't develop an interface for the Franka Hand gripper since I didn't use it, but contributions are welcome!
Call any LeRobot utility as you would with the new Robots! Examples can be found under scripts.
Caution
If you're comboing robots together like I did with Franka + XHand, it might be a good idea to call utility methods directly in your own python script as opposed to using the existing python implementations with arguments (for training and rollout, etc.). Combo robot setup will cause circular import calls with Draccus; I'm currently bypassing this issue by constructing train and deployment scripts directly. Please let me know if you would have a better method to deal with this.
orange_cube.mp4
toaster.mp4
box_pnp.mp4
Open-source datasets for the demo tasks could be found on HuggingFace here.
If you find this work helpful, please consider citing as:
@misc{weng2025levr,
title={LeVR: A Modular VR Teleoperation Framework for Imitation Learning in Dexterous Manipulation},
author={Zhengyang Kris Weng and Matthew L. Elwin and Han Liu},
year={2025},
eprint={2509.14349},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2509.14349},
}
Apache-2.0