Self-learning Processes in Smart Factories: Deep Reinforcement Learning for Process Control of Robot Brine Injection

Rasmus Eckholdt Andersen, Steffen Madsen, Alexander Bendix Krukow Barlo, Sebastian Blegebrønd Johansen, Morten Nør, Rasmus Skovgaard Andersen, Simon Bøgh

Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

13 Downloads (Pure)

Abstract

The goal of this paper is to investigate the application of adaptive learning algorithms, which enables industrial robots to cope with natural variations exhibited in a brine injection process related to the production of bacon. Due to the variations in bacon meat, the traditional needle-based brine injection process is not capable of injecting the correct amount of brine, leading to either ruined or unflavored bacon. In the presented work a Deep Deterministic Policy Gradient (DDPG) reinforcement learning algorithm is introduced in the injection process to improve process control. To accelerate training of the reinforcement learning algorithm, a simulation environment of the brine absorption is generated based on 64 conducted experiments. The simulation environment estimates the amount of absorbed brine given injection pressure and injection time. Tests are run in the simulation where the starting mass is generated from a normal distribution with mean 80.5g, and a standard deviation of 4.8 g and 20.0 g respectively. With a target of 15 % mass increase, the agent can produce an average mass increase of 14.9 % for the first test and 14.6 % for the second test. This indicates that the model can successfully adapt to a high variety input, thereby showing potential for process control in brine injection, coping with natural variation in meat structure.
Original languageEnglish
Title of host publication29th International Conference on Flexible Automation and Intelligent Manufacturing: FAIM 2019
Number of pages7
PublisherElsevier
Publication date2019
Publication statusPublished - 2019

Keywords

  • Deep Reinforcement Learning
  • Process Control
  • Artificial Intelligence (AI)
  • Machine Learning
  • Robotics
  • Reinforcement Learning

Cite this

Andersen, R. E., Madsen, S., Barlo, A. B. K., Blegebrønd Johansen, S., Nør, M., Andersen, R. S., & Bøgh, S. (2019). Self-learning Processes in Smart Factories: Deep Reinforcement Learning for Process Control of Robot Brine Injection. In 29th International Conference on Flexible Automation and Intelligent Manufacturing: FAIM 2019 Elsevier.
Andersen, Rasmus Eckholdt ; Madsen, Steffen ; Barlo, Alexander Bendix Krukow ; Blegebrønd Johansen, Sebastian ; Nør, Morten ; Andersen, Rasmus Skovgaard ; Bøgh, Simon. / Self-learning Processes in Smart Factories: Deep Reinforcement Learning for Process Control of Robot Brine Injection. 29th International Conference on Flexible Automation and Intelligent Manufacturing: FAIM 2019. Elsevier, 2019.
@inproceedings{445dfbccdd9f47ffb9e4e9ef7ef14112,
title = "Self-learning Processes in Smart Factories: Deep Reinforcement Learning for Process Control of Robot Brine Injection",
abstract = "The goal of this paper is to investigate the application of adaptive learning algorithms, which enables industrial robots to cope with natural variations exhibited in a brine injection process related to the production of bacon. Due to the variations in bacon meat, the traditional needle-based brine injection process is not capable of injecting the correct amount of brine, leading to either ruined or unflavored bacon. In the presented work a Deep Deterministic Policy Gradient (DDPG) reinforcement learning algorithm is introduced in the injection process to improve process control. To accelerate training of the reinforcement learning algorithm, a simulation environment of the brine absorption is generated based on 64 conducted experiments. The simulation environment estimates the amount of absorbed brine given injection pressure and injection time. Tests are run in the simulation where the starting mass is generated from a normal distribution with mean 80.5g, and a standard deviation of 4.8 g and 20.0 g respectively. With a target of 15 {\%} mass increase, the agent can produce an average mass increase of 14.9 {\%} for the first test and 14.6 {\%} for the second test. This indicates that the model can successfully adapt to a high variety input, thereby showing potential for process control in brine injection, coping with natural variation in meat structure.",
keywords = "Deep Reinforcement Learning, artificial intelligence (AI), robotics, Self-learning Smart Factories, process control, Deep Reinforcement Learning, Process Control, Artificial Intelligence (AI), Machine Learning, Robotics, Reinforcement Learning",
author = "Andersen, {Rasmus Eckholdt} and Steffen Madsen and Barlo, {Alexander Bendix Krukow} and {Blegebr{\o}nd Johansen}, Sebastian and Morten N{\o}r and Andersen, {Rasmus Skovgaard} and Simon B{\o}gh",
year = "2019",
language = "English",
booktitle = "29th International Conference on Flexible Automation and Intelligent Manufacturing: FAIM 2019",
publisher = "Elsevier",
address = "United Kingdom",

}

Andersen, RE, Madsen, S, Barlo, ABK, Blegebrønd Johansen, S, Nør, M, Andersen, RS & Bøgh, S 2019, Self-learning Processes in Smart Factories: Deep Reinforcement Learning for Process Control of Robot Brine Injection. in 29th International Conference on Flexible Automation and Intelligent Manufacturing: FAIM 2019. Elsevier.

Self-learning Processes in Smart Factories: Deep Reinforcement Learning for Process Control of Robot Brine Injection. / Andersen, Rasmus Eckholdt ; Madsen, Steffen; Barlo, Alexander Bendix Krukow; Blegebrønd Johansen, Sebastian; Nør, Morten; Andersen, Rasmus Skovgaard; Bøgh, Simon.

29th International Conference on Flexible Automation and Intelligent Manufacturing: FAIM 2019. Elsevier, 2019.

Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

TY - GEN

T1 - Self-learning Processes in Smart Factories: Deep Reinforcement Learning for Process Control of Robot Brine Injection

AU - Andersen, Rasmus Eckholdt

AU - Madsen, Steffen

AU - Barlo, Alexander Bendix Krukow

AU - Blegebrønd Johansen, Sebastian

AU - Nør, Morten

AU - Andersen, Rasmus Skovgaard

AU - Bøgh, Simon

PY - 2019

Y1 - 2019

N2 - The goal of this paper is to investigate the application of adaptive learning algorithms, which enables industrial robots to cope with natural variations exhibited in a brine injection process related to the production of bacon. Due to the variations in bacon meat, the traditional needle-based brine injection process is not capable of injecting the correct amount of brine, leading to either ruined or unflavored bacon. In the presented work a Deep Deterministic Policy Gradient (DDPG) reinforcement learning algorithm is introduced in the injection process to improve process control. To accelerate training of the reinforcement learning algorithm, a simulation environment of the brine absorption is generated based on 64 conducted experiments. The simulation environment estimates the amount of absorbed brine given injection pressure and injection time. Tests are run in the simulation where the starting mass is generated from a normal distribution with mean 80.5g, and a standard deviation of 4.8 g and 20.0 g respectively. With a target of 15 % mass increase, the agent can produce an average mass increase of 14.9 % for the first test and 14.6 % for the second test. This indicates that the model can successfully adapt to a high variety input, thereby showing potential for process control in brine injection, coping with natural variation in meat structure.

AB - The goal of this paper is to investigate the application of adaptive learning algorithms, which enables industrial robots to cope with natural variations exhibited in a brine injection process related to the production of bacon. Due to the variations in bacon meat, the traditional needle-based brine injection process is not capable of injecting the correct amount of brine, leading to either ruined or unflavored bacon. In the presented work a Deep Deterministic Policy Gradient (DDPG) reinforcement learning algorithm is introduced in the injection process to improve process control. To accelerate training of the reinforcement learning algorithm, a simulation environment of the brine absorption is generated based on 64 conducted experiments. The simulation environment estimates the amount of absorbed brine given injection pressure and injection time. Tests are run in the simulation where the starting mass is generated from a normal distribution with mean 80.5g, and a standard deviation of 4.8 g and 20.0 g respectively. With a target of 15 % mass increase, the agent can produce an average mass increase of 14.9 % for the first test and 14.6 % for the second test. This indicates that the model can successfully adapt to a high variety input, thereby showing potential for process control in brine injection, coping with natural variation in meat structure.

KW - Deep Reinforcement Learning

KW - artificial intelligence (AI)

KW - robotics

KW - Self-learning Smart Factories

KW - process control

KW - Deep Reinforcement Learning

KW - Process Control

KW - Artificial Intelligence (AI)

KW - Machine Learning

KW - Robotics

KW - Reinforcement Learning

M3 - Article in proceeding

BT - 29th International Conference on Flexible Automation and Intelligent Manufacturing: FAIM 2019

PB - Elsevier

ER -

Andersen RE, Madsen S, Barlo ABK, Blegebrønd Johansen S, Nør M, Andersen RS et al. Self-learning Processes in Smart Factories: Deep Reinforcement Learning for Process Control of Robot Brine Injection. In 29th International Conference on Flexible Automation and Intelligent Manufacturing: FAIM 2019. Elsevier. 2019