This project ports HimLoco from IsaacGym to Isaac Lab.

HimLoco is a reinforcement learning-based quadruped robot gait controller with a dual-network architecture. Through this project, you can train, export, and deploy HimLoco policies in the Isaac Lab environment.

himloco_lab/
├── scripts/                          # Script files
│   ├── himloco_rsl_rl/
│   │   ├── train.py                  # Training script
│   │   ├── play.py                   # Batch inference and policy export
│   │   ├── play_interactive.py       # Interactive control script
│   │   └── cli_args.py               
│   ├── list_envs.py                  # List available environments
│   ├── zero_agent.py                 
│   └── random_agent.py               
│
├── source/himloco_lab/               # Main source code
│   └── himloco_lab/
│       ├── tasks/                    # Task definitions
│       │   └── locomotion/
│       │       ├── mdp/              # Observations, actions, rewards, terminations
│       │       └── robots/go2/       # Training configuration
│       │
│       ├── rsl_rl/                   # HimLoco algorithm implementation
│       │   ├── config/               # Algorithm configuration
│       │   ├── modules/              # HIMActorCritic, HIMEstimator networks
│       │   ├── algorithms/           # HIMOnPolicyRunner training logic
│       │   ├── wrappers/             # HimlocoVecEnvWrapper environment adapter for IsaacLab
│       │   └── env/                  # VecEnv interface
│       │
│       ├── utils/                    # Utility functions
│       │   └── export_policy.py      # JIT and ONNX export tools
│       │
│       └── assets/                   # Asset files
│           └── unitree/              # Unitree Go2 URDF and configuration
│
├── deploy/                           # Robot deployment code
│   └── robots/go2/                   # Go2 controller implementation
│
└── README.md                         # This file

HimLoco

Follow the official installation guide to install Isaac Lab. We recommend using the pip installation method.

Clone this repository into a separate directory outside of Isaac Lab:

git clone https://github.com/IsaacZH/himloco_lab.git
cd himloco_lab

python -m pip install -e source/himloco_lab

Method 1: Using USD Files

• Download unitree usd files from unitree_model, keeping folder structure

  git clone https://huggingface.co/datasets/unitreerobotics/unitree_model

• Config UNITREE_MODEL_DIR in source/himloco_lab/himloco_lab/assets/robots/unitree.py.

  UNITREE_MODEL_DIR = "</home/user/projects/unitree_usd>"

Method 2: Using URDF Files [Recommended] Only for Isaacsim >= 5.0

• Download unitree robot urdf files from unitree_ros

  git clone https://github.com/unitreerobotics/unitree_ros.git
  

• Config UNITREE_ROS_DIR in source/himloco_lab/himloco_lab/assets/robots/unitree.py.

  UNITREE_ROS_DIR = "</home/user/projects/unitree_ros/unitree_ros>"

• [Optional]: change robot_cfg.spawn if you want to use urdf/usd files

Train a HimLoco policy on Unitree Go2:

python scripts/himloco_rsl_rl/train.py --task Unitree-Go2-Velocity --headless

Run a trained policy for batch inference:

python scripts/himloco_rsl_rl/play.py --task Unitree-Go2-Velocity-Play

Use keyboard to control the robot in real-time in IsaacLab:

# Launch interactive control
python scripts/himloco_rsl_rl/play_interactive.py --task Unitree-Go2-Velocity-Play

# Keyboard controls:
#   Numpad 8 / ↑   : Move forward
#   Numpad 2 / ↓   : Move backward
#   Numpad 4 / ←   : Strafe right
#   Numpad 6 / →   : Strafe left
#   Numpad 7 / Z   : Rotate counter-clockwise
#   Numpad 9 / X   : Rotate clockwise

Export trained models for deployment:

python scripts/himloco_rsl_rl/play.py --task Unitree-Go2-Velocity-Play 

The script generates files in the log directory, including:

• policy.pt - TorchScript JIT model (single file deployment)
• encoder.onnx & policy.onnx - ONNX format (separate models)

After model training, you need to verify the trained policy in Mujoco (Sim2Sim) to test model performance. Only then can you proceed to real robot deployment (Sim2Real).

# Install dependencies
sudo apt install -y libyaml-cpp-dev libboost-all-dev libeigen3-dev libspdlog-dev libfmt-dev

# Install unitree_sdk2
git clone [email protected]:unitreerobotics/unitree_sdk2.git
cd unitree_sdk2
mkdir build && cd build
cmake .. -DBUILD_EXAMPLES=OFF # Install to /usr/local directory
sudo make install
# Compile the robot_controller
cd himloco_lab/deploy/robots/go2 
mkdir build && cd build
cmake .. && make

Install unitree_mujoco.

• Set the robot at /simulate/config.yaml to go2
• Set domain_id to 0
• Set enable_elastic_hand to 1
• Set use_joystck to 1.

Launch Mujoco simulation environment:

cd unitree_mujoco/simulate/build
./unitree_mujoco

Launch the controller:

cd himloco_lab/deploy/robots/go2/build
./go2_ctrl

You can use this program to directly control the real robot, but make sure the robot's motion controller is disabled.

./go2_ctrl --network eth0 # eth0 is the network interface name.

• deploy on real robot and mujoco
• deploy on jetson
• migrate to latest rsl_rl version

• HimLoco
• Isaac Lab
• unitree_rl_lab