This repository hosts the official PyTorch implementation of "Learning Implicit Credit Assignment for Cooperative Multi-Agent Reinforcement Learning". The implementation is based on the PyMARL framework and SMAC.

[PDF]

Set up the working environment:

pip3 install -r requirements.txt

Set up the StarCraftII game core

bash install_sc2.sh

To train LICA on the LBF scenario,

python3 main.py --config=lica --env-config=gymma with env_args.time_limit=100 env_args.key='LBF'

Change the map_name accordingly for other scenarios (e.g. mmm2). Remember to update hyperparameters accordingly (e.g. regularization coefficient). All results will be saved in the results folder.

The training tensorboards are saved in the results/tb_logs directory, containing useful info such as policy entropy trajectory and test battle win rate during training.

Same as PyMARL, set save_model to True in src/config/default.yaml will save the learnt model during training (saved in result/ directory be default). The frequency for saving models can be adjusted using the parameter save_model_interval.

Saved models can be loaded by adjusting the checkpoint_path parameter in src/config/default.yaml. For instance, to load model under path result/model/[timesteps]/agent.th, set checkpoint_path to result/model/[timesteps].

The learned model loaded from checkpoint_path can be evaluated by setting evaluate to True in src/config/default.yaml. To save the Starcraft II replays, please make sure configure save_replay to True, and use the episode_runner.

Check out PyMARL documentation for more information.

The demo videos of two of the best battles performed by LICA agents (red) are available at visualizations/. The GIF previews are shown below:

5m_vs_6m	MMM2

Please cite this work if you find it useful:

@article{zhou2020learning,
  title={Learning Implicit Credit Assignment for Cooperative Multi-Agent Reinforcement Learning}, 
  author={Meng Zhou and Ziyu Liu and Pengwei Sui and Yixuan Li and Yuk Ying Chung},
  journal={arXiv preprint arXiv:2007.02529},
  year={2020}
}

Please email mzho7212 AT gmail.com and kenziyuliu AT outlook.com for further questions.

See SMAC and PyMARL for additional instructions.