Learning Goal-Directed Object Pushing in Cluttered Scenes with Location-Based Attention
| Paper | Video |

Authors: Nils Dengler*, Juan Del Aguila Ferrandis*, João Moura, Sethu Vijayakumar, and Maren Bennewitz (*equal contribution)

This repository contains the implementation of our method from our IROS 2025 paper Learning Goal-Directed Object Pushing in Cluttered Scenes With Location-Based Attention.

Work in Progress: This repository will be regularly updated with new features and improvements.

Overview

Our framework learns goal-directed object pushing using a location-based attention module:

1. We represent the clutter in the environment with an occupancy grid map.
2. This map is fed to the RL agent along with object and target poses, and the pusher's position.
3. We extract features from the cluttered scene via a location-based attention module.

Environment

The PushingClutterTask in pushing_gym/tasks/pushing_clutter.py is a reinforcement learning simulation environment designed for goal-directed object pushing in cluttered scenes using Isaac Sim. It features domain randomization and synthetic observation noise to robustly train agents that can be zero-shot deployed in hardware. The environment consists of a table surface, a pusher that is controlled by the RL policy, a movable box, static obstacles, and a visual target. Observations combine spatial grid representations with precise pose information, while the reward structure encourages accurate placement of the box at the target location.

Installation

This project uses Docker for environment consistency. To run:

1. Install Docker.
2. Clone this repository.
3. Run the appropriate script below.
4. If you encounter issues, please open an issue.

Running with Docker

We provide two modes:

1. Interactive (Bash):

# Outside Docker
./docker_run_bash.sh
# Inside Docker
./docker_train.sh

2. Non-Interactive:

# Outside Docker
./docker_run.sh

Note: The initial setup may take up to 5 minutes as Isaac Sim is reconfigured each time. To avoid this delay, you may install Isaac Sim locally (no detailed support provided).

Configuration

By default, training runs in headless mode (no visualization). Our reinforcement learning implementation leverages the SKRL library. The training can be customized with:

• Enable visualization:
  Set headless=False in pushing_gym/docker_train.sh.

• Weights & Biases (Wandb):
  Set your API key WANDB_API_KEY=YOUR_KEY in docker_run.sh:

• SKRL & Isaac Sim parameters:
  Edit pushing_gym/cfg/config.yaml. Important parameters:

  • num_envs: Number of parallel Isaac Sim environments (1440 in the paper).
  • seed: Seed used for training.
  • max_iterations: Max training iterations (set to empty string for infinite).
  • checkpoint: Path for saving model checkpoints.

• Training parameters:
  Edit pushing_gym/cfg/general_training_config.yaml:

  • train_model: true to train, false to evaluate.
  • save_experiment: Save models and enable Wandb logging.
  • use_wandb, wandb_entity, wandb_project, wandb_run: Wandb settings.
  • experiment_name: Custom name for this training run.
  • load_model: Load a pretrained model for further training.
  • load_model_path: path to the model to evaluate or to fine-tune.
    • Default path: /workspace/pushing-gym/pushing_gym/trained_model/demo_agent.pt
  • rollout_size, mini_batches: num_envs must be divisible by mini_batches.

Running Locally

1. Install Omniverse Isaac Sim.
2. Install OmniIsaacGymEnvs.
3. (Optional) Install Weights & Biases for tracking.
4. In the skrl directory, install the library:

~/.local/share/ov/pkg/isaac_sim-*/python.sh -m pip install .["torch"]

5. In the pushing_gym directory, install the library:

~/.local/share/ov/pkg/isaac_sim-*/python.sh -m pip install -e .

6. Start training:

./local_train.sh

7. To enable visualization, set headless=False in local_train.sh. (Not recommended for training due to slower performance.)

Reference

If you find this project useful, please cite our paper as:

@inproceedings{dengler2025learning,
  title={Learning Goal-Directed Object Pushing in Cluttered Scenes with Location-Based Attention},
  author={Dengler, Nils and {Del Aguila Ferrandis}, Juan and Moura, Jo{\~a}o and Vijayakumar, Sethu and Bennewitz, Maren},
  booktitle={Proc. of the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS)},
  year={2025}
}