Rescheduling Under Disruption: A Product-Driven Framework with Heuristic and Reinforcement Learning Strategies

Authors

  • Patricio Sáez Department of Statistics, Universidad de Concepción, Chile
  • Jaime Rivera Department of Industrial Engineering, Universidad de Concepción, Chile
  • Patricio Salas Department of Statistics, Universidad de Concepción, Chile
  • Victor Parada Department of Informatics Engineering, Universidad de Santiago de Chile, Chile

DOI:

https://doi.org/10.24425/mper.2025.156145

Keywords

product-driven system, agent-based model, manufacturing planning, shifting bottleneck heuristic, intelligent products, reinforcement learning

Abstract

In modern manufacturing, addressing disruptions across multi-stage production requires adaptive and intelligent scheduling. This study evaluates two rescheduling strategies within a product-driven system for the Job Shop Scheduling Problem under disturbances: one based on the Shifting Bottleneck Heuristic (PDS-SBH), and another using a Monte Carlo Reinforcement Learning agent (PDS-RL). Products act as intelligent agents capable of autonomous decisions. A total of 151 simulations were conducted across 14 benchmark instances, with machine-level disruptions modeled as 100%, 200%, and 300% increases in processing times. PDS-SBH achieved average makespan reductions up to 5.2%, serving as a reactive and interpretable baseline. In contrast, PDS-RL consistently outperformed it, achieving reductions of 22.12%, 37.13%, and 53.87%, respectively. These results highlight the superior adaptability of reinforcement learning in uncertain production contexts. The study contributes to the understanding of how combining product-driven architectures with heuristic and learning-based strategies enables the development of intelligent, autonomous, and resilient scheduling systems.

References

Adams, J., Balas, E., & Zawack, D. (1988). The shifting bottleneck procedure for job shop scheduling. The Intitute of Management Sciences, 34 (2), 391–400. DOI: 0025-1909/88/3403/0391

Bhongade, A.S., Khodke, P.M., Rehman, A.U., Nikam, M.D., Patil, P.D., & Suryavanshi, P. (2023). Managing Disruptions in a Flow-Shop Manufacturing System. Mathematics, 11 (7). DOI: 10.3390/ math11071731

Bi, M., Kovalenko, I., Tilbury, D.M., & Barton, K. (2023). Dynamic distributed decision-making for resilient resource reallocation in disrupted manufacturing systems. International Journal of Production Research. DOI: 10.1080/00207543.2023.2200567

Bouazza, W., Sallez, Y., & Trentesaux, D. (2021). Dynamic scheduling of manufacturing systems: a product-driven approach using hyper-heuristics. International Journal of Computer Integrated Manufacturing, 34 (6), 641–665. DOI: 10.1080/0951192X. 2021.1925969

Bożek, A., & Werner, F. (2018). Flexible job shop scheduling with lot streaming and sublot size optimisation. International Journal of Production Research, 56 (19), 6391–6411. DOI: 10.1080/00207543.2017.1346322

Campos, J.T. de G.A.A., Blumelova, J., Lepikson, H.A., & Freires, F.G.M. (2020). Agent-based dynamic scheduling model for product-driven production. Brazilian Journal of Operations & Production Management, 17 (4), 1–10. DOI: 10.14488/bjopm.2020.044

Chang, X., Jia, X., & Ren, J. (2024). A reinforcement learning enhanced memetic algorithm for multiobjective flexible job shop scheduling toward Industry 5.0. International Journal of Production Research, 63 (1), 119–147. DOI: 10.1080/00207543.2024.2357740

Cui, W.W., & Lu, Z. (2017). Minimizing the makespan on a single machine with flexible maintenances and jobs’ release dates. Computers and Operations Research, 80, 11–22. DOI: 10.1016/j.cor.2016.11.008

Gao, K., Yang, F., Li, J., Sang, H., & Luo, J. (2020). Improved Jaya Algorithm for Flexible Job Shop Rescheduling Problem. IEEE Access, 8, 86915–86922. DOI: 10.1109/ACCESS.2020.2992478

Gao, K., Yang, F., Zhou, M., Pan, Q., & Suganthan, P.N. (2019). Flexible job-shop rescheduling for new job insertion by using discrete Jaya algorithm. IEEE Transactions on Cybernetics, 49 (5), 1944–1955. DOI: 10.1109/TCYB.2018.2817240

Hwangbo, S., Liu, J.J., Ryu, J.H., Lee, H.J., & Na, J. (2024). Production rescheduling via explorative reinforcement learning while considering nervousness. Computers and Chemical Engineering, 186 (March), 108700. DOI: 10.1016/j.compchemeng.2024.108700

Kardos, C., Laflamme, C., Gallina, V., & Sihn, W. (2020). Dynamic scheduling in a job-shop production system with reinforcement learning. Procedia CIRP, 97 (March), 104–109. DOI: 10.1016/j.procir. 2020.05.210

Kim, Y.I., & Kim, H.J. (2021). Rescheduling of unrelated parallel machines with job-dependent setup times under forecasted machine breakdown. International Journal of Production Research, 59 (17), 5236–5258. DOI: 10.1080/00207543.2020.1775910

Ku, W.Y., & Beck, J.C. (2016). Mixed Integer Programming models for job shop scheduling: A computational analysis. Computers and Operations Research, 73, 165-173. DOI: 10.1016/j.cor.2016.04.006

Li, H., Gajpal, Y., & R. Bector, C. (2017). A survey of duedate related single-machine with two-agent scheduling problem. Journal of Industrial & Management Optimization, 13 (5), 1–19. DOI: 10.3934/jimo.2019005

Liang, Z., Zhong, P., Zhang, C., Yang, W., Xiong, W., Yang, S., & Meng, J. (2023). A genetic algorithmbased approach for flexible job shop rescheduling problem with machine failure interference. Eksploatacja i Niezawodnosc, 25 (4), 0–3. DOI: 10.17531/ein/171784

Mahmoodjanloo, M., Tavakkoli-Moghaddama, R., Baboli, A., & Bozorgi-Amiri, A. (2022). Distributed jobshop rescheduling problem considering reconfigurability of machines: a self-adaptive hybrid equilibrium optimiser. International Journal of Production Research, 60(16), 4973–4994. DOI: 10.1080/00207543.2021.1946193

Meyer, G.G., Hans Wortmann, J.C., & Szirbik, N.B. (2011). Production monitoring and control with intelligent products. International Journal of Production Research, 49 (5), 1303–1317. DOI: 10.1080/ 00207543.2010.518742

Muhuri, P.K., & Biswas, S.K. (2020). Bayesian optimization algorithm for multi-objective scheduling of time and precedence constrained tasks in heterogeneous multiprocessor systems. Applied Soft Computing Journal, 92, 106274. DOI: 10.1016/j.asoc.2020.106274

Pannequin, R., & Thomas, A. (2012). Another interpretation of stigmergy for product-driven systems architecture. Journal of Intelligent Manufacturing, 23 (6), 2587–2599. DOI: 10.1007/s10845-011-0588-3

Pérez-Salazar, M. del R., Aguilar-Lasserre, A.A., CedilloCampo, M.G., Posada-Gómez, R., del MoralArgumedo, M.J., & Hernández-González, J.C. (2019). An agent-based model driven decision support system for reactive aggregate production scheduling in the Green Coffee Supply Chain. Applied Sciences (Switzerland), 9 (22). DOI: 10.3390/app9224903

Rasheed, M.B., Javaid, N., Arshad Malik, M.S., Asif, M., Hanif, M.K., & Chaudary, M.H. (2019). Intelligent Multi-Agent Based Multilayered Control System for Opportunistic Load Scheduling in Smart Buildings. IEEE Access, 7, 23990–24006. DOI: 10.1109/ACCESS.2019.2900049

Roesch, M., Linder, C., Bruckdorfer, C., Hohmann, A., & Reinhart, G. (2019). Industrial load management using multi-agent reinforcement learning for rescheduling. Proceedings - 2019 2nd International Conference on Artificial Intelligence for Industries, AI4I 2019, 99–102. DOI: 10.1109/AI4I46381.2019.00033

Sáez, P., Herrera, C., Booth, C., Belmokhtar-Berraf, S., & Parada, V. (2023). A product-driven system with an evolutionary algorithm to increase flexibility in planning a job shop. PLoS ONE, 18 (2 February), 1–12. DOI: 10.1371/journal.pone.0281807

Salido, M.A., Escamilla, J., Barber, F., & Giret, A. (2017). Rescheduling in job-shop problems for sustainable manufacturing systems. Journal of Cleaner Production, 162, S121–S132. DOI: 10.1016/j.jclepro.2016.11.002

Sutton, R., & Barto, A. (1998). Reinforcement learning: An Introduction. In Cambridge: MIT press.

Upasani, A., & Uzsoy, R. (2008). Integrating a decomposition procedure with problem reduction for factory scheduling with disruptions: A simulation study. International Journal of Production Research, 46 (21), 5883–5905. DOI: 10.1080/00207540601156215

Wu, C.-C., Chen, J.-Y., Lin, W.-C., Lai, K., Bai, D., & Lai, S.-Y. (2019). A two-stage three-machine assembly scheduling flowshop problem with both two-agent and learning phenomenon. Computers and Industrial Engineering, 130, 485–499. DOI: 10.1016/j.cie.2019.02.047

Wu, X., & Yan, X. (2023). A spatial pyramid poolingbased deep reinforcement learning model for dynamic job-shop scheduling problem. Computers and Operations Research, 160 (September 2022), 106401. DOI: 10.1016/j.cor.2023.106401

Yamamoto, M., & Nof, S.Y. (1985). Scheduling rescheduling in the manufacturing operating system environment. International Journal of Production Research, 23 (4), 705–722. DOI: 10.1080/00207548508904739

Yang, S., & Xu, Z. (2022). Intelligent scheduling and reconfiguration via deep reinforcement learning in smart manufacturing. International Journal of Production Research, 60 (16), 4936–4953. DOI: 10.1080/00207543.2021.1943037

Zhang, S., Xiang, L., Bowen, Z., & Shouyang, W. (2020). Multi-objective optimisation in flexible assembly job shop scheduling using a distributed ant colony system. European Journal of Operational Research, 283 (2), 441–460. DOI: 10.1016/j.ejor.2019.11.016

Zhou, T., Tang, D., Zhu, H., & Zhang, Z. (2021). Multiagent reinforcement learning for online scheduling in smart factories. Robotics and Computer-Integrated Manufacturing, 72 (June), 102202. DOI: 10.1016/j.rcim.2021.102202

Downloads

Published

2025-09-30

How to Cite

Sáez, Patricio, et al. “Rescheduling Under Disruption: A Product-Driven Framework With Heuristic and Reinforcement Learning Strategies”. Management and Production Engineering Review, vol. 16, no. 3, Sept. 2025, pp. [nr art. 11], s. 1-15, doi:10.24425/mper.2025.156145.

Issue

Vol. 16 No. 3 (2025)

Section

Artykuły