Self-learning Processes in Smart Factories: Deep Reinforcement Learning for Process Control of Robot Brine Injection

Rasmus Eckholdt Andersen, Steffen Madsen, Alexander Bendix Krukow Barlo, Sebastian Blegebrønd Johansen, Morten Nør, Rasmus Skovgaard Andersen, Simon Bøgh

Publikation: Bidrag til bog/antologi/rapport/konference proceedingKonferenceartikel i proceedingForskningpeer review

Resumé

The goal of this paper is to investigate the application of adaptive learning algorithms, which enables industrial robots to cope with natural variations exhibited in a brine injection process related to the production of bacon. Due to the variations in bacon meat, the traditional needle-based brine injection process is not capable of injecting the correct amount of brine, leading to either ruined or unflavored bacon. In the presented work a Deep Deterministic Policy Gradient (DDPG) reinforcement learning algorithm is introduced in the injection process to improve process control. To accelerate training of the reinforcement learning algorithm, a simulation environment of the brine absorption is generated based on 64 conducted experiments. The simulation environment estimates the amount of absorbed brine given injection pressure and injection time. Tests are run in the simulation where the starting mass is generated from a normal distribution with mean 80.5g, and a standard deviation of 4.8 g and 20.0 g respectively. With a target of 15 % mass increase, the agent can produce an average mass increase of 14.9 % for the first test and 14.6 % for the second test. This indicates that the model can successfully adapt to a high variety input, thereby showing potential for process control in brine injection, coping with natural variation in meat structure.
OriginalsprogEngelsk
Titel29th International Conference on Flexible Automation and Intelligent Manufacturing : FAIM 2019
Antal sider7
Publikationsdato2019
StatusAccepteret/In press - 2019

Emneord

  • Deep Reinforcement Learning
  • artificial intelligence (AI)
  • robotics
  • Self-learning Smart Factories
  • process control

Citer dette

Andersen, R. E., Madsen, S., Barlo, A. B. K., Blegebrønd Johansen, S., Nør, M., Andersen, R. S., & Bøgh, S. (Accepteret/In press). Self-learning Processes in Smart Factories: Deep Reinforcement Learning for Process Control of Robot Brine Injection. I 29th International Conference on Flexible Automation and Intelligent Manufacturing: FAIM 2019
Andersen, Rasmus Eckholdt ; Madsen, Steffen ; Barlo, Alexander Bendix Krukow ; Blegebrønd Johansen, Sebastian ; Nør, Morten ; Andersen, Rasmus Skovgaard ; Bøgh, Simon. / Self-learning Processes in Smart Factories: Deep Reinforcement Learning for Process Control of Robot Brine Injection. 29th International Conference on Flexible Automation and Intelligent Manufacturing: FAIM 2019. 2019.
@inproceedings{445dfbccdd9f47ffb9e4e9ef7ef14112,
title = "Self-learning Processes in Smart Factories: Deep Reinforcement Learning for Process Control of Robot Brine Injection",
abstract = "The goal of this paper is to investigate the application of adaptive learning algorithms, which enables industrial robots to cope with natural variations exhibited in a brine injection process related to the production of bacon. Due to the variations in bacon meat, the traditional needle-based brine injection process is not capable of injecting the correct amount of brine, leading to either ruined or unflavored bacon. In the presented work a Deep Deterministic Policy Gradient (DDPG) reinforcement learning algorithm is introduced in the injection process to improve process control. To accelerate training of the reinforcement learning algorithm, a simulation environment of the brine absorption is generated based on 64 conducted experiments. The simulation environment estimates the amount of absorbed brine given injection pressure and injection time. Tests are run in the simulation where the starting mass is generated from a normal distribution with mean 80.5g, and a standard deviation of 4.8 g and 20.0 g respectively. With a target of 15 {\%} mass increase, the agent can produce an average mass increase of 14.9 {\%} for the first test and 14.6 {\%} for the second test. This indicates that the model can successfully adapt to a high variety input, thereby showing potential for process control in brine injection, coping with natural variation in meat structure.",
keywords = "Deep Reinforcement Learning, artificial intelligence (AI), robotics, Self-learning Smart Factories, process control, Deep Reinforcement Learning, Process Control, Artificial Intelligence (AI), Machine Learning, Robotics, Reinforcement Learning",
author = "Andersen, {Rasmus Eckholdt} and Steffen Madsen and Barlo, {Alexander Bendix Krukow} and {Blegebr{\o}nd Johansen}, Sebastian and Morten N{\o}r and Andersen, {Rasmus Skovgaard} and Simon B{\o}gh",
year = "2019",
language = "English",
booktitle = "29th International Conference on Flexible Automation and Intelligent Manufacturing",

}

Andersen, RE, Madsen, S, Barlo, ABK, Blegebrønd Johansen, S, Nør, M, Andersen, RS & Bøgh, S 2019, Self-learning Processes in Smart Factories: Deep Reinforcement Learning for Process Control of Robot Brine Injection. i 29th International Conference on Flexible Automation and Intelligent Manufacturing: FAIM 2019.

Self-learning Processes in Smart Factories: Deep Reinforcement Learning for Process Control of Robot Brine Injection. / Andersen, Rasmus Eckholdt ; Madsen, Steffen; Barlo, Alexander Bendix Krukow; Blegebrønd Johansen, Sebastian; Nør, Morten; Andersen, Rasmus Skovgaard; Bøgh, Simon.

29th International Conference on Flexible Automation and Intelligent Manufacturing: FAIM 2019. 2019.

Publikation: Bidrag til bog/antologi/rapport/konference proceedingKonferenceartikel i proceedingForskningpeer review

TY - GEN

T1 - Self-learning Processes in Smart Factories: Deep Reinforcement Learning for Process Control of Robot Brine Injection

AU - Andersen, Rasmus Eckholdt

AU - Madsen, Steffen

AU - Barlo, Alexander Bendix Krukow

AU - Blegebrønd Johansen, Sebastian

AU - Nør, Morten

AU - Andersen, Rasmus Skovgaard

AU - Bøgh, Simon

PY - 2019

Y1 - 2019

N2 - The goal of this paper is to investigate the application of adaptive learning algorithms, which enables industrial robots to cope with natural variations exhibited in a brine injection process related to the production of bacon. Due to the variations in bacon meat, the traditional needle-based brine injection process is not capable of injecting the correct amount of brine, leading to either ruined or unflavored bacon. In the presented work a Deep Deterministic Policy Gradient (DDPG) reinforcement learning algorithm is introduced in the injection process to improve process control. To accelerate training of the reinforcement learning algorithm, a simulation environment of the brine absorption is generated based on 64 conducted experiments. The simulation environment estimates the amount of absorbed brine given injection pressure and injection time. Tests are run in the simulation where the starting mass is generated from a normal distribution with mean 80.5g, and a standard deviation of 4.8 g and 20.0 g respectively. With a target of 15 % mass increase, the agent can produce an average mass increase of 14.9 % for the first test and 14.6 % for the second test. This indicates that the model can successfully adapt to a high variety input, thereby showing potential for process control in brine injection, coping with natural variation in meat structure.

AB - The goal of this paper is to investigate the application of adaptive learning algorithms, which enables industrial robots to cope with natural variations exhibited in a brine injection process related to the production of bacon. Due to the variations in bacon meat, the traditional needle-based brine injection process is not capable of injecting the correct amount of brine, leading to either ruined or unflavored bacon. In the presented work a Deep Deterministic Policy Gradient (DDPG) reinforcement learning algorithm is introduced in the injection process to improve process control. To accelerate training of the reinforcement learning algorithm, a simulation environment of the brine absorption is generated based on 64 conducted experiments. The simulation environment estimates the amount of absorbed brine given injection pressure and injection time. Tests are run in the simulation where the starting mass is generated from a normal distribution with mean 80.5g, and a standard deviation of 4.8 g and 20.0 g respectively. With a target of 15 % mass increase, the agent can produce an average mass increase of 14.9 % for the first test and 14.6 % for the second test. This indicates that the model can successfully adapt to a high variety input, thereby showing potential for process control in brine injection, coping with natural variation in meat structure.

KW - Deep Reinforcement Learning

KW - artificial intelligence (AI)

KW - robotics

KW - Self-learning Smart Factories

KW - process control

KW - Deep Reinforcement Learning

KW - Process Control

KW - Artificial Intelligence (AI)

KW - Machine Learning

KW - Robotics

KW - Reinforcement Learning

M3 - Article in proceeding

BT - 29th International Conference on Flexible Automation and Intelligent Manufacturing

ER -

Andersen RE, Madsen S, Barlo ABK, Blegebrønd Johansen S, Nør M, Andersen RS et al. Self-learning Processes in Smart Factories: Deep Reinforcement Learning for Process Control of Robot Brine Injection. I 29th International Conference on Flexible Automation and Intelligent Manufacturing: FAIM 2019. 2019