Can Deep Reinforcement Learning Improve Inventory Management? Performance on Dual Sourcing, Lost Sales and Multi-Echelon Problems
We study the effectiveness of Deep Reinforcement Learning (DRL) algorithms on classic inventory problems in operations. Despite the excitement about Deep Reinforcement Learning in many industries, such as gaming, robotics and self-driving cars, DRL applications in operations and supply chain management remain rather scarce. We identify this gap and provide a rigorous performance evaluation of DRL on three classic and intractable inventory problems: dual-sourcing, lost sales and multi-echelon inventory management. We model each inventory problem as a Markov Decision Process and apply a state-of-the-art DRL algorithm: the A3C algorithm. We show how to apply and tune the A3C algorithm to achieve good performance across three sets of inventory problems, for a variety of parameter settings. We demonstrate that the A3C algorithm can match performance of many state-of-the-art heuristics and other approximate dynamic programming methods, with limited changes to the tuning parameters across all studied problems. Yet, tuning DRL algorithms remains computationally burdensome while the resulting policies often lack interpretability. Generating structural policy insight or designing specialized policies that are (ideally provably) near optimal thus remains indispensable. DRL provides a promising research avenue, especially when problem-dependent heuristics are lacking. In this case DRL may be used to set new benchmarks or provide insight in the development of new heuristics.