Decision-making in a complex, dynamic, interconnected, and data-intensive industrial environment can be improved with the assistance of machine-learning techniques. In this work, a complex instance of industrial assembly line control is formalized and a parallel deep reinforcement learning approach is presented. We consider an assembly line control problem in which a set of tasks (e.g., vehicle assembly tasks) needs to be planned and controlled during their execution, with the aim of optimizing given key performance criteria. Specifically, the aim will be that of planning the task in order to minimize the total time taken to execute all the tasks (also called cycle time). Tasks run on workstations in the assembly line. To run, tasks need specific resources. Therefore, the tackled problem is that of optimally mapping tasks and resources to workstations, and deciding the optimal execution times of the tasks. In doing so, several constraints need to be respected (e.g., precedence constraints among the tasks, constraints on needed resources to run tasks, deadlines, etc.). The proposed approach uses deep reinforcement learning to learn a tasks/resources mapping policy that is effective in minimizing the resulting cycle time. The proposed method allows us to explicitly take into account all the constraints, and, once training is complete, can be used in real time to dynamically control the execution of tasks. Another motivation for the proposed work is in the ability of the used method to also work in complex scenarios, and in the presence of uncertainties. As a matter of fact, the use of deep neural networks allows for learning the model of the assembly line problem, in contrast with, e.g., optimization-based techniques, which require explicitly writing all the equations of the model of the problem. In order to speed up the training phase, we adopt a learning scheme in which more agents are trained in parallel. Simulations show that the proposed method can provide effective real-time decision support to industrial operators for scheduling and rescheduling activities, achieving the goal of minimizing the total tasks’ execution time.

A Parallel Deep Reinforcement Learning Framework for Controlling Industrial Assembly Lines / Tortorelli, Andrea; Imran, Muhammad; Delli Priscoli, Francesco; Liberati, Francesco. - In: ELECTRONICS. - ISSN 2079-9292. - 11:4(2022). [10.3390/electronics11040539]

A Parallel Deep Reinforcement Learning Framework for Controlling Industrial Assembly Lines

Tortorelli, Andrea;Imran, Muhammad;Delli Priscoli, Francesco;Liberati, Francesco
2022

Abstract

Decision-making in a complex, dynamic, interconnected, and data-intensive industrial environment can be improved with the assistance of machine-learning techniques. In this work, a complex instance of industrial assembly line control is formalized and a parallel deep reinforcement learning approach is presented. We consider an assembly line control problem in which a set of tasks (e.g., vehicle assembly tasks) needs to be planned and controlled during their execution, with the aim of optimizing given key performance criteria. Specifically, the aim will be that of planning the task in order to minimize the total time taken to execute all the tasks (also called cycle time). Tasks run on workstations in the assembly line. To run, tasks need specific resources. Therefore, the tackled problem is that of optimally mapping tasks and resources to workstations, and deciding the optimal execution times of the tasks. In doing so, several constraints need to be respected (e.g., precedence constraints among the tasks, constraints on needed resources to run tasks, deadlines, etc.). The proposed approach uses deep reinforcement learning to learn a tasks/resources mapping policy that is effective in minimizing the resulting cycle time. The proposed method allows us to explicitly take into account all the constraints, and, once training is complete, can be used in real time to dynamically control the execution of tasks. Another motivation for the proposed work is in the ability of the used method to also work in complex scenarios, and in the presence of uncertainties. As a matter of fact, the use of deep neural networks allows for learning the model of the assembly line problem, in contrast with, e.g., optimization-based techniques, which require explicitly writing all the equations of the model of the problem. In order to speed up the training phase, we adopt a learning scheme in which more agents are trained in parallel. Simulations show that the proposed method can provide effective real-time decision support to industrial operators for scheduling and rescheduling activities, achieving the goal of minimizing the total tasks’ execution time.
2022
deep reinforcement learning; parallel training; Industry 4.0
01 Pubblicazione su rivista::01a Articolo in rivista
A Parallel Deep Reinforcement Learning Framework for Controlling Industrial Assembly Lines / Tortorelli, Andrea; Imran, Muhammad; Delli Priscoli, Francesco; Liberati, Francesco. - In: ELECTRONICS. - ISSN 2079-9292. - 11:4(2022). [10.3390/electronics11040539]
File allegati a questo prodotto
File Dimensione Formato  
Tortorelli_A-Parallel-Deep_2022.pdf

accesso aperto

Note: https://doi.org/10.3390/electronics11040539
Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Creative commons
Dimensione 1.68 MB
Formato Adobe PDF
1.68 MB Adobe PDF

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1610610
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 6
  • ???jsp.display-item.citation.isi??? 3
social impact