Comparative Analysis of Novel Deep Reinforcement Learning Methods for Food Distribution Optimization

Jeperson Hutahaean, Yessica Siagian, Endra Saputra
Pages 2726-2740
Abstract Views 460
PDF Downloads 697
Crossref Citations -
DOI https://doi.org/10.47738/jads.v6i4.956

Abstract


Uneven food distribution across various regions in Indonesia often results in supply-demand imbalances, leading to price surges, stock shortages, and overall market instability. This challenge is compounded by the limitations of conventional distribution systems, which are ill-equipped to respond to rapidly changing market dynamics. In response, this study introduces a novel, AI-driven approach by implementing Deep Reinforcement Learning (DRL) to optimize food distribution policies using real-world data. Specifically, we perform a comparative evaluation of four emerging DRL models—Double Deep Q-Network (Double DQN), Dueling DQN, Proximal Policy Optimization (PPO), and Advantage Actor-Critic (A2C)—to determine their effectiveness in learning adaptive distribution strategies from national food logistics data provided by Indonesia’s Central Bureau of Statistics (BPS). Each model was trained within a custom simulation environment based on the Markov Decision Process (MDP) framework and evaluated using five core performance metrics: cumulative reward, average reward, success rate, sample efficiency, and best reward. The results reveal that A2C consistently outperformed the other models, delivering the highest average reward and most stable training performance, while PPO demonstrated strong efficiency and success rate. These findings underscore the potential of policy-gradient methods—particularly A2C—as robust and intelligent solutions for dynamic food logistics management. This research offers one of the first comparative benchmarks of DRL methods in the food distribution domain and highlights their applicability for future integration into national AI-powered logistics systems.

Full Text:

PDF

References


J. Liu, “Can digital inclusive finance ensure food security while achieving low-carbon transformation in agricultural development? Evidence from China,” J. Clean. Prod., vol. 418, 2023, doi: 10.1016/j.jclepro.2023.138016.

A. H. Abdi, “Exploring climate change resilience of major crops in Somalia: implications for ensuring food security,” Int. J. Agric. Sustain., vol. 22, no. 1, 2024, doi: 10.1080/14735903.2024.2338030.

M. F. B. Alam, “Analysis of the enablers to deal with the ripple effect in food grain supply chains under disruption: Implications for food security and sustainability,” Int. J. Prod. Econ., vol. 270, 2024, doi: 10.1016/j.ijpe.2024.109179.

A. Morchid, “Applications of internet of things (IoT) and sensors technology to increase food security and agricultural Sustainability: Benefits and challenges,” Ain Shams Eng. J., vol. 15, no. 3, 2024, doi: 10.1016/j.asej.2023.102509.

S. Oh, “Vertical farming - smart urban agriculture for enhancing resilience and sustainability in food security,” J. Hortic. Sci. Biotechnol., vol. 98, no. 2, pp. 133–140, 2023, doi: 10.1080/14620316.2022.2141666.

A. Saleem, “Securing a sustainable future: the climate change threat to agriculture, food security, and sustainable development goals,” J. Umm Al-Qura Univ. Appl. Sci., 2024, doi: 10.1007/s43994-024-00177-3.

B. Subedi, “The impact of climate change on insect pest biology and ecology: Implications for pest management strategies, crop production, and food security,” J. Agric. Food Res., vol. 14, 2023, doi: 10.1016/j.jafr.2023.100733.

C. Ume, “The role of improved market access for small-scale organic farming transition: Implications for food security,” J. Clean. Prod., vol. 387, 2023, doi: 10.1016/j.jclepro.2023.135889.

S. Bhardwaj, “Determining Point of Economic Cattle Milk Production through Machine Learning and Evolutionary Algorithm for Enhancing Food Security,” J. Food Qual., vol. 2023, 2023, doi: 10.1155/2023/7568139.

K. Sar and P. Ghadimi, “A systematic literature review of the vehicle routing problem in reverse logistics operations,” Comput. Ind. Eng., vol. 177, no. January 2022, p. 109011, 2023, doi: 10.1016/j.cie.2023.109011.

J. Zhang, Y. Liu, G. Yu, and Z.-J. (Max) Shen, “Robustifying humanitarian relief systems against travel time uncertainty,” Nav. Res. Logist., vol. 68, no. 7, pp. 871–885, Oct. 2021, doi: 10.1002/nav.21981.

S. Jain, M. L. Meena, V. Kumar, and P. K. Detwal, “Route Optimization as an Aspect of Humanitarian Logistics: Delineating Existing Literature from 2011 to 2022,” in Intelligent Manufacturing Systems in Industry 4.0, B. B. V. L. Deepak, M. V. A. R. Bahubalendruni, D. R. K. Parhi, and B. B. Biswal, Eds., Singapore: Springer Nature Singapore, 2023, pp. 647–661.

C. Liu, G. Kou, X. Zhou, Y. Peng, H. Sheng, and F. E. Alsaadi, “Time-dependent vehicle routing problem with time windows of city logistics with a congestion avoidance approach,” Knowledge-Based Syst., vol. 188, p. 104813, 2020, doi: https://doi.org/10.1016/j.knosys.2019.06.021.

A. Wahid, “Smart Blended Learning Framework based on Artificial Intelligence using MobileNet Single Shot Detector and Centroid Tracking Algorithm,” Int. J. Adv. Comput. Sci. Appl., vol. 13, no. 5, pp. 364–369, 2022, doi: 10.14569/IJACSA.2022.0130543.

X. Huo, J. Xu, M. Xu, and H. Chen, “Artificial Intelligence in the Life Sciences An improved 3D quantitative structure-activity relationships ( QSAR ) of molecules with CNN-based partial least squares model,” Artif. Intell. Life Sci., vol. 3, no. November 2022, p. 100065, 2023, doi: 10.1016/j.ailsci.2023.100065.

B. J. De Moor, “Reward shaping to improve the performance of deep reinforcement learning in perishable inventory management,” Eur. J. Oper. Res., vol. 301, no. 2, pp. 535–545, 2022, doi: 10.1016/j.ejor.2021.10.045.

Q. Wu, “Mobility-Aware Cooperative Caching in Vehicular Edge Computing Based on Asynchronous Federated and Deep Reinforcement Learning,” IEEE J. Sel. Top. Signal Process., vol. 17, no. 1, pp. 66–81, 2023, doi: 10.1109/JSTSP.2022.3221271.

J. Kang, “UAV-Assisted Dynamic Avatar Task Migration for Vehicular Metaverse Services: A Multi-Agent Deep Reinforcement Learning Approach,” IEEE Caa J. Autom. Sin., vol. 11, no. 2, pp. 430–445, 2024, doi: 10.1109/JAS.2023.123993.

L. Zhang, “Deep reinforcement learning for dynamic flexible job shop scheduling problem considering variable processing times,” J. Manuf. Syst., vol. 71, pp. 257–273, 2023, doi: 10.1016/j.jmsy.2023.09.009.

C. Huang, “Mixed Deep Reinforcement Learning Considering Discrete-continuous Hybrid Action Space for Smart Home Energy Management,” J. Mod. Power Syst. Clean Energy, vol. 10, no. 3, pp. 743–754, 2022, doi: 10.35833/MPCE.2021.000394.

M. A. Dhuheir, “Deep Reinforcement Learning for Trajectory Path Planning and Distributed Inference in Resource-Constrained UAV Swarms,” IEEE Internet Things J., vol. 10, no. 9, pp. 8185–8201, 2023, doi: 10.1109/JIOT.2022.3231341.

V. Singh, “How are reinforcement learning and deep learning algorithms used for big data based decision making in financial industries–A review and research agenda,” Int. J. Inf. Manag. Data Insights, vol. 2, no. 2, 2022, doi: 10.1016/j.jjimei.2022.100094.

W. Terapaptommakol, “Design of Obstacle Avoidance for Autonomous Vehicle Using Deep Q-Network and CARLA Simulator,” World Electr. Veh. J., vol. 13, no. 12, 2022, doi: 10.3390/wevj13120239.

X. Geng, “Deep Q-Network-Based Intelligent Routing Protocol for Underwater Acoustic Sensor Network,” IEEE Sens. J., vol. 23, no. 4, pp. 3936–3943, 2023, doi: 10.1109/JSEN.2023.3234112.

S. Shen, “MFGD3QN: Enhancing Edge Intelligence Defense Against DDoS With Mean-Field Games and Dueling Double Deep Q-Network,” IEEE Internet Things J., vol. 11, no. 13, pp. 23931–23945, 2024, doi: 10.1109/JIOT.2024.3387090.

X. Liu, “Proximal Policy Optimization-Based Transmit Beamforming and Phase-Shift Design in an IRS-Aided ISAC System for the THz Band,” IEEE J. Sel. Areas Commun., vol. 40, no. 7, pp. 2056–2069, 2022, doi: 10.1109/JSAC.2022.3158696.

M. Yin, “Collision avoidance control for limited perception unmanned surface vehicle swarm based on proximal policy optimization,” J. Franklin Inst., vol. 361, no. 6, 2024, doi: 10.1016/j.jfranklin.2024.106709.

L. Liu, “Dynamic hedging of 50ETF options using Proximal Policy Optimization,” J. Autom. Intell., 2025, doi: 10.1016/j.jai.2025.04.001.

K. Belattar, “Parallel multiple DNA sequence alignment using genetic algorithm and asynchronous advantage actor critic model,” Int. J. Bioinform. Res. Appl., vol. 18, no. 5, pp. 460–478, 2022, doi: 10.1504/IJBRA.2022.10051235.

B. Ye, “Traffic signal control method based on asynchronous advantage actor-critic,” Zhejiang Daxue Xuebao Gongxue Ban J. Zhejiang Univ. Eng. Sci., vol. 58, no. 8, pp. 1671–1703, 2024, doi: 10.3785/j.issn.1008-973X.2024.08.014.

J. Xu, “Optimization of Robot Environment Interaction Based on Asynchronous Advantage Actor-Critic Algorithm,” Int. J. Adv. Comput. Sci. Appl., vol. 15, no. 6, pp. 1350–1359, 2024, doi: 10.14569/IJACSA.2024.01506136.

M. H. Alabdullah, “Microgrid energy management using deep Q-network reinforcement learning,” Alexandria Eng. J., vol. 61, no. 11, pp. 9069–9078, 2022, doi: 10.1016/j.aej.2022.02.042.

L. Zhang, C. Yang, Y. Yan, Z. Cai, and Y. Hu, “Automated guided vehicle dispatching and routing integration via digital twin with deep reinforcement learning,” J. Manuf. Syst., vol. 72, no. February, pp. 492–503, 2024, doi: 10.1016/j.jmsy.2023.12.008.

H. Guo, “Deep-q-Networks-Based Adaptive Dual-Mode Energy-Efficient Routing in Rechargeable Wireless Sensor Networks,” IEEE Sens. J., vol. 22, no. 10, pp. 9956–9966, 2022, doi: 10.1109/JSEN.2022.3163368.

S. Song, “Proximal policy optimization through a deep reinforcement learning framework for remedial action schemes of VSC-HVDC,” Int. J. Electr. Power Energy Syst., vol. 150, 2023, doi: 10.1016/j.ijepes.2023.109117.

K. Kavya, “Dynamic Service Migration in Multi-Cloud Architectures Using Proximal Policy Optimization and Reinforcement Learning,” Int. J. Environ. Sci., vol. 11, no. 3, pp. 425–433, 2025.

M. Kaloev, “Segmented Actor-Critic-Advantage Architecture for Reinforcement Learning Tasks,” TEM J., vol. 11, no. 1, pp. 219–224, 2022, doi: 10.18421/TEM111-27.

Y. I. Cho, “Locating algorithm of steel stock area with asynchronous advantage actor-critic reinforcement learning,” J. Comput. Des. Eng., vol. 11, no. 1, pp. 230–246, 2024, doi: 10.1093/jcde/qwae002.

L. Xu, “Deep Reinforcement Learning-Based Routing and Spectrum Assignment of EONs by Exploiting GCN and RNN for Feature Extraction,” J. Light. Technol., vol. 40, no. 15, pp. 4945–4955, 2022, doi: 10.1109/JLT.2022.3175865.

G. Huang, “Real-Time Battery Thermal Management for Electric Vehicles Based on Deep Reinforcement Learning,” IEEE Internet Things J., vol. 9, no. 15, pp. 14060–14072, 2022, doi: 10.1109/JIOT.2022.3145849.

R. Hashemi, “Deep Reinforcement Learning for Practical Phase-Shift Optimization in RIS-Aided MISO URLLC Systems,” IEEE Internet Things J., vol. 10, no. 10, pp. 8931–8943, 2023, doi: 10.1109/JIOT.2022.3232962.

J. Zheng, “Exploring Deep-Reinforcement-Learning-Assisted Federated Learning for Online Resource Allocation in Privacy-Preserving EdgeIoT,” IEEE Internet Things J., vol. 9, no. 21, pp. 21099–21110, 2022, doi: 10.1109/JIOT.2022.3176739.

A. Nouriani, R. Mcgovern, and R. Rajamani, “Intelligent Systems with Applications Activity recognition using a combination of high gain observer and deep learning computer vision algorithms,” Intell. Syst. with Appl., vol. 18, no. March, p. 200213, 2023, doi: 10.1016/j.iswa.2023.200213.

L. I. Kesuma, “ELREI: Ensemble Learning of ResNet, EfficientNet, and Inception-v3 for Lung Disease Classification based on Chest X-Ray Image,” Int. J. Intell. Eng. Syst., vol. 16, no. 5, pp. 149–161, 2023, doi: 10.22266/ijies2023.1031.14.

C. Li, “Deep reinforcement learning in smart manufacturing: A review and prospects,” CIRP J. Manuf. Sci. Technol., vol. 40, pp. 75–101, 2023, doi: 10.1016/j.cirpj.2022.11.003.

G. Mi, “Multi-agent cooperative confrontation with proximal policy optimization in urban environments,” Tongxin Xuebao J. Commun., vol. 46, no. 3, pp. 94–108, 2025, doi: 10.11959/j.issn.1000-436x.2025049.

W. Funika, “Automated cloud resources provisioning with the use of the proximal policy optimization,” J. Supercomput., vol. 79, no. 6, pp. 6674–6704, 2023, doi: 10.1007/s11227-022-04924-3.

C. Zhang, “Proximal Policy Optimization Based Intelligent Energy Management for Plug-In Hybrid Electric Bus Considering Battery Thermal Characteristic,” World Electr. Veh. J., vol. 14, no. 2, 2023, doi: 10.3390/wevj14020047.




DOI: https://doi.org/10.47738/jads.v6i4.956

Refbacks

  • There are currently no refbacks.


Barcode

Journal of Applied Data Sciences

ISSN:2723-6471 (Online)
Publisher:Bright Publisher
Website:http://bright-journal.org/JADS
Email:taqwa@amikompurwokerto.ac.id (principal contact)
  support@bright-journal.org (technical issues)

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0