The entire codes for the paper 'Dynamic multi-tour order picking in an automotive-part warehouse based on attention-aware deep reinforcement learning' published in 'Robotics and Computer-Integrated Manufacturing'.

In the context of an automotive-part warehouse, this paper addresses a dynamic multi-tour order-picking problem based on a novel attention-aware deep reinforcement learning-based (ADRL) method. The multi-tour represents that one order-picking task must be split into multiple tours due to the cart capacity and the operator’s workload constraints. First, the multi-tour order-picking problem is formulated as a mathematical model, and then reformulated as a Markov decision process. Second, a novel DRL-based method is proposed to solve it effectively. Compared to the existing DRL-based methods, this approach employs multi-head attention to perceive warehouse situations. Additionally, three improvements are proposed to further strengthen the solution quality and generalization, including (1) the extra location representation to align the batch length during training, (2) the dynamic decoding to integrate real-time information of the warehouse environment during inference, and (3) the proximal policy optimization with entropy bonus to facilitate action exploration during training. Finally, comparison experiments based on thousands of order-picking instances from the Swedish warehouse validated that the proposed ADRL could outperform the other twelve DRL-based methods at most by 40.6%, considering the optimization objective. Furthermore, the performance gap between ADRL and seven evolutionary algorithms is controlled within 3%, while ADRL can be hundreds or thousands of times faster than these EAs regarding the solving speed.

The dataset was generated in simulation to avoid the confidential issue, the methods include the propsed attention-based methods, commonly-used deep reinforcement learning-based methods, and evolutionary algorithms. Thanks for the opensource codes used in the paper:

https://github.com/ai4co/rl4co

https://github.com/p-christ/Deep-Reinforcement-Learning-Algorithms-with-PyTorch

https://github.com/MISTCARRYYOU/solvehtspwithfcrnrde

To run the project, just 'python main_att.py/main_ea.py/main_drl.py'

If you could find something valuable for your work (perhaps), please cite our work as:

@article{wang2025dynamic, title={Dynamic multi-tour order picking in an automotive-part warehouse based on attention-aware deep reinforcement learning}, author={Wang, Xiaohan and Zhang, Lin and Wang, Lihui and Zu{~n}iga, Enrique Ruiz and Wang, Xi Vincent and Flores-Garc{'\i}a, Erik}, journal={Robotics and Computer-Integrated Manufacturing}, volume={94}, pages={102959}, year={2025}, publisher={Elsevier} }