Development of an intelligent metal forming robot and application to multi-stage Cold Forging

Development of an intelligent metal forming robot and application to multi-stage Cold Forging | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Development of an intelligent metal forming robot and application to multi-stage Cold Forging Papdo Tchasse, Tahsin Deliktas, Mathias Liewald This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5109889/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 9 You are reading this latest preprint version Abstract Metal forming processes often undergo different instability phases due to different factors like tool and part temperature variations, tool vibrations or frictional interactions between workpiece and tool. In the absence of an experienced process operator, these instabilities can induce a very considerable production loss. This study addresses this issue and proposes a method to develop a data-based virtual process operator equipped with the appropriate hardware and physical components that allow it to constantly monitor and if necessary regulate the process. The resulting system is introduced as the intelligent metal forming robot. The objective of this self-learning system is first to stabilize the process and ensure a certain part quality despite the noises, dynamical disturbances and user-defined changes of the part quality requirements, then, to control the process even in states that have not yet been experienced and at last to improve the control precision based on the updated process experience. This intelligent metal forming robot has been implemented and applied on a two-stage cold forging process, where the target quality feature was the part head height of a screw-like part. The results showed that, based on a qualitative process experience and on effective actuators, an intelligent self-learning system can significantly increase the robustness of a metal forming process. Metal forming multi-stage cold Forging self-learning system intelligent process control machine learning Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 Figure 9 Figure 10 Figure 11 Figure 12 Figure 13 Figure 14 Figure 15 Figure 16 1 Introduction During metal forming processes, the shape of solid metal bodies is plastically changed, while their mass and material cohesion are retained [ 1 ]. Every metal forming part is the result of a prior thorough and detailed process design, which involves many numerical simulations with varying input parameters. Then, based on the simulations results, try-out experiments and expert knowledge, a process range and the corresponding press settings are deduced, which should enable the production the intended part. During the manufacturing however, metal forming processes experience different phases like ramp-up phases and are subject to many influences related to the part material properties, the tool dynamics and the complex frictional interactions between the forming tool and the workpiece. These aspects of the process often represent sources of failures that can lead to downtimes and the production of defective parts. To solve these issues, many press shops rely on the operator knowledge, which is rather an unsafe and unsystematic approach. Also, many researches addressed this problem by developing different control strategies to ensure the process robustness. However, many of these control strategies have been designed only for sheet metal forming applications [ 2 ] and the control goal was to compensate the difference between the current and a reference value of the control variable, which was determined via either experiments or simulations. That means, the process control was based on a static assumption on the process evolution. Such assumptions have been proven to be limited as different unknown and known factors like temperature changes, tool wear or vibrations can strongly affect the relationship between process parameters and the part quality [ 3 ]. For this reason, this work aims to tackle the problem by proposing an inclusive method to automatically improve the resilience of metal forming processes despite their different instability phases and states. The objective here is to develop intelligent systems out of classical metal forming tools, that are able to make use of the past and current process experience in order to overcome dynamical challenges and stabilize the process, reducing the downtimes and the production of defective parts. For this purpose, the forming tool is upgraded into an intelligent robot, which learns over the time from the process environment and is able to use that knowledge to make the optimal decisions, given the process goals. In this work, the concept and implementation steps of such an intelligent metal forming robot were developed and then applied on a multi-stage cold forging process. 2 Concept definition According to the international federation of robotics and ISO 8373–20213, a robot is defined as a programmed actuated mechanism with a degree of autonomy to perform locomotion, manipulation or positioning [ 4 ]. Following this definition, a metal forming robot was defined in this study as the result of integrating controllable components in a forming tool or in a press, which gives the possibility to purposefully and automatically affect the process online. Furthermore, such a robot would become intelligent if it is extended with sensors, able to collect process data, and a processing hardware as well as a software agent, that is able to learn from the past and the current process data in order to optimally operate controllable actuators, stabilize the process and fit it to user-defined quality requirements despite process disturbances. This intelligence of the metal forming robot is the main ability, which enhances its robustness and allows it to control the process even through yet unknown process states. Hence, an intelligent metal forming robot is a system comprising: A metal forming tool or press, equipped with the appropriate sensors for the process observation as well as actuators able to effectively influence the process evolution and the part quality, A control hardware as central information processing unit and A control software agent, which, based on the past and current process experience, is able to automatically determine the actual process working range, given the user-defined quality criteria and targets. 3 State of the Art There have been many different approaches and studies of the application of Machine Learning (ML) to automatically control and optimize different aspects of metal forming processes. In [ 5 ], the authors explored the simulative optimization of processing parameters in hot forging of AISI 4340 steel using instability map and reinforcement learning. In this study, a self-learning algorithm based on Q-Learning was developed in order to find the optimal part temperature and stroke speed, which lead to the most stable material range. Another study, implemented using finite element methods (FEM) is presented in [ 6 ]. Here, the authors proposed a simulative approach to develop an optimization strategy for the heating process of a forging line based on a digital twin, the objective being the automation of the development of an optimal control policy that adjusts the power for the heating coils in an induction oven based on temperature data recorded via pyrometers. In this use case, a reinforcement learning agent was trained to optimally adjust the heating power in order the keep the temperature of steel bars in the appropriate range. In Hot rolling Technology, Gamal et al. focused on the bar and wire hot rolling process and proposed a control strategy based on reinforcement learning in order to adaptively control the roll force, the roll torque and the roll gap [ 7 ]. For this study, the self-learning agent was trained in a simulative environment and the input parameters were chosen depending on the control parameter. Also, in [ 8 ] Scheiderer et al. considered the heavy plate rolling process and designed a simulation-based process control in order to adaptively control the process parameters for each pass. Using input parameters like the height, the grain size and the temperature of the work piece, a reinforcement learning agent was trained to find the optimal roll gap and the inter-pass time, so that the process result fits to the target settings. In the sheet metal forming, some studies also focused on applying ML-based control. In [ 9 ] for example, Molitor et al. focused on the springback compensation in the air bending process and proposed a control approach, for which the input was the part image and the punch force. Based on the force signals and the part image, the bending angle was predicted using a multilayer perceptron, a multioutput linear regression model and a convolutional neural work. Then, the predicted result was processed using a transfer function that provided the required vertical position of the press ram, and thus the punch, in order to compensate the springback. In [ 10 ], Liu et al. explored the use of reinforcement learning in free sheet metal free form stamping. In this study, the authors trained a reinforcement learning algorithm in order to predict the optimal forming route of the hammer in order to achieve the desired part shape. This study was carried out in a simulative environment without providing any kind of prior expertise to the learning agent. Furthermore, [ 11 ] presents a self-learning optimal control algorithm for episodic fixed-horizon manufacturing processes with time discrete control actions. This optimal control is based on model free reinforcement learning and is applied in deep drawing in order to find the optimal blank holder force for each stroke based on the process experience. In this use case, the process state parameters were the punch position, the blank draw-in, and the position of the blank holder. The control algorithm in this study was developed and validated in a simulative environment. According the state of the art, the practical application of ML Methods in metal forming has mainly been in order to monitor or predict the value of defined quality parameters. Then, the subsequent process control was done either incrementally, with a transfer function or a pre-control based on static or limited assumptions on the process behaviour and without considering short and long-term process instabilities. Furthermore, the few works that investigated a model-driven process control have been mostly implemented in simulative environments, based on input and output parameters that are hardly accessible in the practice like the sizes of finite elements [ 12 ]. Hence, it becomes a considerable challenge to evaluate the industrial applicability of the developed approaches. Also, because the process control has been principally developed in FEM Environments, different essential and practical aspects like the tool design, the sensor positioning, the actuator definition and the process speed have been strongly neglected. The ML applications so far also mainly cover the sheet metal forming and cutting technologies, so that practical insights of ML Methods applied to bulk metal forming seem not to be frequent. The focus of artificial Intelligence (AI) or ML in bulk metal forming so far have principally been on wear and maintenance prediction due to the high tool loads [ 13 ]. This paper addresses the practical and industry-oriented model-driven process control of metal forming processes. Model-driven means that the target values to control actuators are generated by an AI-model, depending on the current process state. In other words, the process knowledge is replaced by an AI-model, that can learn long and short-time patterns and so, adapt to the different dynamical changes occurring during the process evolution. Furthermore, because the control software directly depends on different physical process components, it becomes obvious that in order to successfully deploy the control software, the process environment has to be upgraded. Therefore, this paper introduces the concept of intelligent metal forming robot, which includes the software, the hardware and the physical components required in order to build a practical and effective intelligent model-driven process control. Of course, the integration of Actuators into a metal forming tool or press and the use of sensors in order to monitor different aspects of the process are not a new concept in the field of metal forming. Also, The concept of a robot as a supporting asset for the forming operation is already present in some metal forming technologies like incremental sheet metal forming [ 14 ] or roll forming [ 15 ]. However, this study introduces the robot as the principal active forming component, which explores and interacts with each process stroke while learning from them. The purpose and the novelty of introducing the metal forming tool or machine as an intelligent metal forming robot is to present an integral system, which end-use, the intelligent adaptive process control, should be considered from the early stage of process design. The objective in this approach is to define a systematic methodology which considers the specific manufacturing environment and allows the development of a self-learning system that is able to exploit the extent and the versatility of the modern ML solutions in order to interact with the process and make it more robust in the short and long term. This approach is particularly relevant for one or multi-stage processes like deep drawing, forging or stamping. 4 System architecture of an intelligent metal forming robot In this section, the different system components as well as the implementation steps required to build an intelligent metal forming robot are presented. 4.1 Actuators and sensors in Metal forming The concept of intelligent metal forming robot is compatible not only with modern sensors and actuator technologies but also with sensors and actuators that are already familiar to the metal forming industry. For a process, different sensors can be used before, during and after a forming operation. Before the forming operation, different sensors like eddy current sensors, balances or optical sensors can be used to measure different aspects of the workpieces like the surface quality, the volume and the material properties [ 2 ]. Then, during the process, force, pressure, energy or material flow sensors can be used to record the evolution of the formed part, which can later, after the forming operation, be measured using product sensors like cameras. The actuators are usually integrated to purposefully influence the forming operation. This can be done via a position, force, speed, temperature or lubrication adjustment [ 2 ]. Depending on the application scenario, the positioning of the required sensors and actuators for an intelligent system may happen to be a challenge. In this case, FEM Simulations combined with mathematical correlation analysis would allow their appropriate disposition in or around the forming tool. 4.2 Control Hardware The further development of digitization solutions for the industry has put the conventional programmable logic controllers (PLC) under a considerable pressure [ 16 ]. To solve these new challenges and allow the deployment of novel software solutions like AI-models, the traditional PLCs have been upgraded in performance and modulated, so that they can be extended with additional hardware through different communication technologies like the open platform communications unified architecture (OPC UA) or the message queuing telemetry transport (MQTT). An example of additional hardware are edge devices, which act as an entry point in the internal process control and allows the extension of the standard control loop with additional software and also enables an access to cloud-based solutions [ 17 ]. Furthermore, due to their low-cost, compared to conventional PLC, micro-controller-based PLC are also being further developed and extended so that they can support the requirements of the latest data processing technologies [ 18 ]. 4.3 Control Software Two particular software approaches can be considered for the process control of the intelligent forming robot. The first approach presented in this work is based on reinforcement learning (RL) and relies on the hypothesis that an online adaptive control of metal forming processes using RL would allow not only to take advantage of the process experience in order to guarantee a certain process stability but also to overcome dynamic instabilities like fluctuations of part material properties, tool temperatures changes and tool wear. In RL, a software agent makes observations and takes actions within an environment and in return it receives rewards , the objective of the agent being to learn to act in a way that will maximize its expected rewards over the time [ 19 ]. The solution proposed with this approach consists in two steps. The first step consists in using process simulations in order to find and equip the metal forming tools with the appropriate sensors and actuators. Furthermore, the suitable hardware set would be incorporated in the system in order to process the sensor information and control the actuators. Then, the second and most important step is to design and develop the data processing software, which will make a resilient and self-learning system out of the equipped process tool. Thus, in this case: The environment is the metal forming process including the tool and the integrated hardware and the process disturbances and influencing factors, The agent is the control software to be developed, The observations and the rewards are derived from the process or sensors data, And the actions are carried out via actuator integrated in or around the tool. A learning step is defined as an action leading to an observation and a reward. The principle of RL applied on metal forming processes is shown in Fig. 1 . The training algorithm is the key point in this approach, especially because RL Tasks are known to be complex. Two families of training algorithms are popular for solving RL Tasks, Policy gradients (PG) algorithms and Markov decision processes (MDP) algorithms. In practice, popular and well performing algorithms are based on a combination of PG and MDP. But of course, The righteous choice of the algorithm depends on the considered environment and the specific task. The second approach proposed in this work is based on the hypothesis that if a metal forming tool is equipped with sensors able to record the process settings, noise and the part quality, then supervised and transfer learning methods can be used to generate an extensible virtual model of the process environment that allows to precisely control the process. The fundamental idea in this second approach is to use supervised learning methods in order to build a data-based model of the process and its environment and use it to build a virtual process operator. For this purpose, raw data are to be acquired and the suitable features for the specific use case are to be extracted. Then, after the data pre-processing, hyperparameter optimization is to be used in order to generate a ML-model architecture, that minimizes the validation loss between predicted and real targets and allows to avoid overfitting and underfitting [ 22 ]. An important aspect in this procedure is the choice of the input and output features. Given a system with control parameters, noise parameters and system output, the features of the ML-model are to be chosen so that the input features of the ML-model are the noise parameters and the system output and the targets features are the control parameters. One key aspect of such a ML-model is its ability to be used as a prediction model, which can interpolate between the known process states and even extrapolate, controlling the process even in unknow states, depending on the quality of the training data. Furthermore, as it is shown in the following section of this paper, the resulting virtual process operator doesn’t even need to directly monitor the part quality in order to control the actuators like in conventional control loops. Instead, it acts as a generative ML-model that provides new control targets for the actuators based only on the process state and the operator indication of the desired target quality. One essential advantage of using supervised learning is the possibility to upgrade the trained model by using domain adaption or transfer learning and hence create a dynamical and extensible ML-model for the process control. Transfer learning is used to fit pretrained ML-models so that they can be extended to new tasks or new data distributions of a given task [ 23 ]. Applying transfer learning means that the data of every stroke is not processed directly after it is done and the software agent is not constantly training like in RL. Instead, each ML-model is trained and redeployed periodically, after a certain amount of process experience has been collected in form of labelled data. This way, the ML-models are trained at regular or irregular time intervals and then deployed online. Updating the ML-model means to apply the following data processing steps after a defined time interval: data acquisition and cleaning, data shuffle and splitting, data standardization, model training and model application. If the validation loss after each periodical model training seems unsatisfactory, then hyperparameter optimization can be used once again in order to build up another ML-model, that will better capture the correlations between the features in the whole dataset. The advantage of this approach is the simplicity of the model training and the achievable process speed when deploying the ML-models, since they are trained offline and deployed only after their training. In Fact, Transfer learning could be combined with the first approach in order to limit the training cost of the RL agents. The Fig. 2 illustrates the idea of the control loop for the second approach. 4.4 Implementation steps of an intelligent metal forming robot Developing an intelligent tool for a metal forming process requires implementing the following steps: Process definition and specification of the quality criteria Determination of the quality influencing factors and quality parameters Development of an appropriate sensor and actuator concept and tool design Design of a Signal processing model and choice of the applicable processing hardware Development of the signal processing and control software These implementation steps were applied to a multi-stage cold forging process, which is described in the following sections. 5 Application of the intelligent metal forming robot to cold forging The first use case for the aforementioned method is a cold forging application, in which the objective is to reduce scrap and hence the loss during the ramp-up phase of a two-stage cold forging process. A ramp-up phase is inevitable every time presses are started after a standstill, downtimes or because of a different process tool mounted in the press. In this phase, defective parts are produced at a higher rate than in the steady phase, which often represents a non-negligible financial loss. Consequently, being able to better understand and intelligently influence the ramp-up phase of the process would lead to the reduction of the amount of scrap and therefore the financial loss. 5.1 Process Design For this use case, a two-stage cold forging process was defined, consisting of a forward extrusion stage and an upsetting stage. The objective was to form a cylindrical billet of diameter 15 mm and height 20 mm into a screw-like part without thread, drive and tip and with head diameter 20 mm ± 0.15 and head height 6.25 mm ± 0.1 and body diameter 10 mm and body height 20 mm. The shape of the part during the process is shown in Fig. 3 . During the first stage, the punch pressure forces the billet to flow in the punch movement direction, which results in the part body being created with a smaller cross section than the part head. Then, this part head is reduced in the second stage in a way that two parallel surfaces, the limits of the part head, are obtained, as show in Fig. 3 and Fig. 4 . For this process, the material considered was 28B2 (1.5510) KGK. the billet was cut from a rod, that was drawn, annealed and trailed. Also, the rod surface was coated with phosphate and was soaped for temperature resistance. The formability study was mainly done using FEM simulations with the software DEFORM. For the simulation to be started, compression tests were carried out in order to acquire the material properties. After the numerical design of the multi-stage forming process, the forming tool shown in Fig. 5 was designed with the three tool slots arranged in line for the implementation of three forming dies, although in this case the last tool stage was used as a measuring stage. The tool was installed in a servo mechanical knuckle joint press from the Schuler company with a ram upsetting capacity of 5000 kN and a stroke rate range of 3 to 45 min − 1 . Furthermore, in order to move the workpiece from one stage to the next, the press was equipped with a gripper transfer system. Figure 5 shows the experimental process tool and the press used for the experimental investigations. 5.2 Process parameters Both the forward extrusion and the upsetting operation cause a significant increase of the die temperature. This feature is one key factor in cold forging, as its evolution often characterizes the different process phases including the ramp-up and the steady phases of a production run. For every stroke, the die temperature increases are caused mainly by the frictional interactions between the flowing workpiece and the die shoulder, for the forward extrusion, as well as the deformation work. Furthermore, the temperatures of every stroke affect the next ones, and therefore act as a thermal noise during the process evolution. This complex interactions between the die temperature of different strokes strongly affect the production runs, so that they often experience a ramp-up phase and a steady phase. Because of this importance of the die temperature, it was the main process parameter that was considered for this application. 5.3 Sensor and Actuator concept The choice and positioning of the sensors and the actuators were deduced from the process design and simulation studies. For the temperature measurements, temperature sensors were integrated in the tool, according to the simulation results, so that they would be as near as possible of the hottest point in the dies during the process. Two temperature sensors were used for each stage. Also, other sensors for the part quality measurement and the process control via actuators were considered. The target quality feature in this use-case was the head height of the final part. Because this study addressed the forming operations involved for the production of screw-like parts, this quality feature was chosen to be industry-relevant. In order to measure the head height, a laser profiler sensor was installed in the tool and was used to capture the difference between the two surfaces, that bound the Part head as shown in Fig. 6 . For this application, actuators were used to control the vertical position of the punches. Since the main factor during the ramp-up phase is the die temperature evolution, which is caused by frictional and compression forces, vertically adjusting the punch position appeared to be a direct way of controlling the extent of the frictional interactions and the deformation work deployed for the workpiece. For this reason, a wedge system was developed with the objective of converting the rotational motion of a motor into a linear vertical motion of the punches. One wedge system was tied to each punch and distance sensors were integrated in order to measure the actual position of the punches. The wedge systems were conceived so that the punch adjustment range was 0.4 mm for both process stages. The wedges systems were driven using stepper motors. In summary, four temperature sensors, two for each stage, one laser profiler sensor, for the part quality, and one distance sensor for each punch position were used. Figure 7 shows the sensor and actuator concept developed in this study. 5.4 Hardware Architecture The different sensors and actuators were connected, so that the process data could be collected and the actuators could be operated via a central processing hardware. The sensors where first connected to Amplifiers and measuring hardware. Then, the data of the Amplifiers were sent to the control hardware, which was an Arduino Mega 2560. The control hardware was also connected to the motors of the wedge systems through the appropriate drivers. The function of the control hardware was to collect the data, process them and compute the right positions of the punches, so that the part quality requirements are met. 5.5 Data processing and control software The data processing and control software in this application was implemented according to the second approach presented in the section 4.3. That means, there was an offline data processing and an online data processing. 5.5.1 Offline data processing The offline data processing consisted of every step that is necessary in order to build a basis ML-model for the process control. Those steps include the data collection, the data preprocessing and the model building. A total of seven features was considered for the data processing and each feature was extracted from the data of the sensors mentioned in the section 5.3. The inputs for the ML-model were the maximum temperatures of every press stroke as well as the part head height and the targets were the positions of the punches. Table 1 gives an overview of the features considered in this use case. Table 1 features considered for the model training Features Unit Maximum temperature 1 °C Maximum temperature 2 °C Maximum temperature 3 °C Maximum temperature 4 °C Position of first punch mm Position of second punch mm Part head height mm for the basis ML-model, a total of 6 production runs at a stroke rate of 25 min − 1 was recorded, which corresponds to a total of 1735 strokes. Figure 8 shows the recorded temperature data and the corresponding part head height. The different runs were recorded at different punch positions and the maximal number of strokes achieved during a production run was 396. Every recorded production run was started at a die temperature of around 26°C. The highest recorded temperature after 396 strokes was 80°C and for every run, the highest temperature increase was identified after the first 40 strokes, corresponding to a maximum die temperature of around 70°C. For the ML-model design, the data had to be restructured according to the number of process stages and the moment of the part quality measurement. The head height of every part was measured two strokes after it was first formed. Because one part was in the press for two strokes, it seems rightful to assume that the part quality was affected by two strokes and hence consider the corresponding data in order to control the process. The issue with this approach is the necessity to wait for two strokes before the ML-model can generate new target positions for the punches. In this case, the process control would no more affect the part, as it would have already been formed. To solve this issue, the situation was considered from another point of view. The question was, how do the actual positions of the punches after each stroke correlate with the temperature measurements of that stroke and the part quality measured two strokes ahead? Based on this reflection, the training data for the model were shifted and structured so that based on the temperature measurements of one stroke, the punches could be operated and the quality of the part inside the press adjusted. After the data collection and structuration, the samples were shuffled and split so that 90% were used for the training and 10% for the validation. The validation data were used in order to evaluate the offline performance of the ML-model during the training and before the online deployment. Then, the training and validation data were standardized with the mean and standard deviation of the training data. The model considered for this use case was a feed forward neural network or multilayer perceptron (MLP) [ 22 ]. The first or basis MLP was built using the hyperparameter optimization algorithm hyperband [ 24 ]. For the hyperparameter optimization a range was defined for each hyperparameter, in which the search algorithm inferred the best value. Table 2 summarizes for each hyperparameter the range for the search algorithm. Table 2 Hyperparameter space for the model building Hyperparameter Value range Number of hidden layers [2, 10] Number of neurons per hidden layer [2, 15] Dropout rate per hidden layer [0.2, 0.5] Learning rate 1e-2, 1e-3, 1e-4, 1e-5 Hidden layer activation functions Relu, sigmoid, tanh Loss function Log-cosh, Mean squared error The best model was chosen within 528 models that were trained for an average of 300 epochs per model. The model architecture which resulted from the hyperparameter optimization is illustrated in Fig. 9 . That model was then trained again for a longer period of time and evaluated using the coefficient of determination (R²-score) and the root mean squared error (RMSE) [ 23 ]. While the lowest RMSE is striven, an R²-score of 1 means a perfect correlation and an R²-score of 0 or below is an indication of insufficient model fit. The performances of the trained ML-model are summarized in Table 3 . These high R²-scores are proof of good process controllability, as this proves a high correlation between the punch positions and the process temperatures as well as the part quality. Figure 10 shows the prediction accuracy of the ML-model on validation data. Table 3 performance of the ML model on train and validation data Score Set Average Position punch 1 Position punch 2 R2-score Training 0.969 0.952 0.985 Validation 0.968 0.948 0.9884 RMSE Training 0.030 mm 0.037 mm 0.021 mm Validation 0.030 mm 0.039 mm 0.019 mm The built ML-model was then converted to fit the requirements of the control hardware and was deployed online [ 25 ]. The model training in this study was done in a python environment, using different ML-libraries like Tensorflow and Scikit-Learn. 5.5.2 Online data processing The online data processing represents the routine that ensures that after a certain number of strokes, the ML-model is updated with the new process experience. This update of the ML-model is supposed to be done automatically parallel to the production or after a downtime. For the update of the ML-model, the same data pre-processing as for the offline data processing steps were applied, then the ML-model was trained as a pretrained model using transfer learning. In the case that the results of the transfer learning would seem unsatisfactory, the ML-model would have been built and trained anew via hyperparameter optimization. The data flow for the periodical online data processing is illustrated in Fig. 11 . 5.5.3 Control strategy The control procedure in this study was based on the hypothesis that the process temperatures are the reference features in order to adjust every stroke. The built ML-model is like a virtual operator that relies on a data-based digital twin of the process and which, given the temperatures data and the required part head height, is able to determine the punch positions so that the required part quality is obtained. That means that the controlling ML-model doesn’t need any information on the actual part quality, as it already indirectly infers that knowledge from the process state, just like an experienced operator would. For the model deployment, the part head height was set to the desired target head height and the temperatures data were recorded at every stroke. Figure 12 shows the control strategy developed in this use-case. 6 Results In order to fulfil the requirements of the intelligent metal forming robot as defined in the section Fehler! Verweisquelle konnte nicht gefunden werden. , the developed system should be able to: Stabilize the process and ensure a certain part quality regardless of the noises and dynamical disturbances Adapt the process in order to fit user defined changes in the quality requirements Control the process even in states that have not yet been modelled in the training data Improve the control precision based on the updated process experience These four aspects of the system were evaluated and validated through real experiments of continuous forming strokes at a press speed of 25 min − 1 . The starting temperature for every continuous run was around 26°C for the four temperature sensors. For the first requirement, several production runs were started and for each of them a different target value for the head height was set. Figure 13 shows the performance of the control system for two different targets in comparison with the case where no process control was applied and a part head height of 11.90 mm was expected. In the non-controlled scenario, the part quality seems to be fluctuating with an average of 11.86 mm and at some point, it even starts diverging. This doesn’t seem to be case when the control is activated. At the begin of the continuous run in the controlled scenario, the controlling ML-model tries to find the process working range, which suits the quality requirements and then it tries to stabilize the process so that, despite the temperature changes and the press vibrations, the part quality fluctuates within ± 0.05 mm according to the target value. An aspect of the process which explains why the fluctuations are inevitable is the volume or mass variation of the billets. For the second and third requirements of the intelligent metal forming robot, different target values were set during a continuous run of 535 strokes, whereby after every 25 strokes, which corresponds to 1 minute, the target head height was changed. Figure 14 shows the results when alternating between two different targets as well as the effects on the maximum temperature measured in the second process stage. Changing the target values after only one minute may not be that relevant for the industrial application, however it shows the faculty of such control systems to quickly react to user defined changes or detected events which could affect the process. The control pattern in Fig. 14 doesn’t seem to indicate any particular disturbance or noise like temperatures increase of vibration, even though they are present. Furthermore, the control was evaluated over 535 strokes, which is beyond the maximum of the continuous runs recorded to train the model. This proves the extrapolation ability of the designed system and shows that, even in yet unknow process states the intelligent control agent can be functional and assure a certain process quality. The intelligent control affects the process quality indirectly through an adjustment of the punch position, which is shown in Fig. 15 . The difference in precision between the quality control and the punch position control is obvious. The control errors seem to be less significant for the punch positions, that are directly controlled by the ML-model. Because of this precision gap in the control, it is assumed that an external factor plays an important role, which was not considered in the data acquisition. This external factor is assumed to be the mass variations of the billet. Another important fact illustrated in Fig. 14 and Fig. 15 is the horizontal control shift between target and actual head height. This shift is explained by the fact that, when a new target was identified by the ML model, the first punch was first positioned in the suitable control range and the second punch was operated. The intelligence of the system as the faculty to learn and update its knowledge basis were evaluated by considering the control performance of the controlling ML-model before and after a retrain. The basis ML-model was first deployed and an attempt was carried out to test its ability to process three different targets as illustrated in the Fig. 16 . The left curve in the Fig. 16 shows how instable the part quality control with three different targets is. At the basis level, the ML-model didn’t seem to be able to handle this task properly. However, after a model upgrade with the new process experience and the past control attempts, the updated controlling ML-model could better switch between different three defined targets and also hold them. These results show the ability of the intelligent metal forming robot to learn and improve itself with new process experience. Also, Fig. 14 and Fig. 16 show that the controlling ML-model, based on its experience, can be used as a generative model not only to stabilize the process in case of fluctuations, but also when new quality targets are desired. During these experiments, the ML-model could compute new targets for the process control in less than 30 ms. 7 Conclusion and outlook The current digitalization solutions allow the development of more complex and capable systems. One of those is presented in this paper and is referred to as the intelligent metal forming robot. The ability of this system to intelligently control the process and learn from past and current experience in order to generate the right control targets for the actuators are the key factors that enable its robustness. Indeed, this system has now been implemented in the scope of bulk metal forming, however its principles could be extended to other metal forming processes and manufacturing technologies. Declarations Conflict of interest The authors have no relevant financial or non-financial interests to disclose. Author Contribution P.T. wrote the main manuscript text T.D. and M.L. reviewed the manuscript. References Lange K (1985) Handbook of Metal Forming , ‎ Society., New York McGraw-Hill c 1986 Allwood JM et al (2016) Closed-loop control of product properties in metal forming. CIRP Ann - Manuf Technol 65(2):573–596 Kott M (2022) Methodik zur Entwicklung eines Bedienerassistenzsystems für das Presswerk , Dr.-Ing. Dissertation, Technische Universität Darmstadt, Darmstadt I. F. of Robotics, Artificial Intelligence in Robotics, no. (2022) Jeong HY, Park J, Kim Y, Shin SY, Kim N (2023) Processing parameters optimization in hot forging of AISI 4340 steel using instability map and reinforcement learning. J Mater Res Technol 23:1995–2009 Ma Y et al (2022) Using Deep Reinforcement Learning for Zero Defect Smart Forging, Adv. Transdiscipl. Eng. , vol. 21, no. January, pp. 701–712 Gamal O, Mohamed MIP, Patel CG, Roth H (2021) Data-Driven Model-Free Intelligent Roll Gap Control of Bar and Wire Hot Rolling Process Using Reinforcement Learning. Int J Mech Eng Robot Res 10(7):349–356 Scheiderer C et al (2020) Simulation-as-a-service for reinforcement learning applications by example of heavy plate rolling processes. Procedia Manuf 51:897–903 Molitor DA, Arne V, Kubik C, Noemark G, Groche P (2024) Inline closed-loop control of bending angles with machine learning supported springback compensation. Int J Mater Form, 17, 1 Liu S, Shi Z, Lin J, Li Z (2019) Reinforcement learning in free-form stamping of sheet-metals, Procedia Manuf. , vol. 50, no. pp. 444–449, 2020 Dornheim J, Link N, Gumbsch P (2020) Model-free Adaptive Optimal Control of Episodic Fixed-horizon Manufacturing Processes Using Reinforcement Learning. Int J Control Autom Syst 18(6):1593–1604 Idzik C, Gerlach J, Bailly D, Hirt G (2023) Advanced automatic pass schedule design for hot rolling by coupling reinforcement learning with a fast rolling model, Mater. Res. Proc. , vol. 28, pp. 601–610 Cao J, Bambach M, Merklein M, Mozaffar M, Xue T (2024) Artificial intelligence in metal forming. CIRP Ann 00:1–27 Fiorentino A, Ceretti E, Feriti GC, Giardini C (2015) Improving accuracy in aluminum Incremental Sheet Forming of complex geometries using Iterative Learning Control. Key Eng Mater, vol. 651–653, pp. 1096–1102 Abdolmohammadi T, Richter-Trummer V, Ahrens A, Richter K, Alibrahim A, Werner M (2023) Virtual Sensor-Based Geometry Prediction of Complex Sheet Metal Parts Formed by Robotic Rollforming, Robotics , vol. 12, no. 2 Mansfield J (2018) Industrial internet of things demystified , vol. 535 Al-Turjman F (ed) (2019) Edge Computing. Springer International Publishing, Cham Bai Y, Roth ZS (2018) Classical and Modern Controls with Microcontrollers: Design, Implementation and Applications Sutton RS, Barto AG (2018) Reinforcement Learning , 2nd ed., Cambridge Birkert A, Haage S, Straub M (2013) Umformtechnische Herstellung komplexer Karosserieteile Dietrich J (2018) Praxis der Umformtechnik. Springer Fachmedien Wiesbaden, Wiesbaden Chollet F (2021) Deep Learning with Python , 2nd ed Géron A (2019) Hands-on Machine Learning whith Scikit-Learing, Keras and Tensorfow. O’Reilly Media Li L, Jamieson K, DeSalvo G, Rostamizadeh A, Talwalkar A (2018) Hyperband: A novel bandit-based approach to hyperparameter optimization. J Mach Learn Res 18:1–52 Estrebou CA, Fleming M, Saavedra MD, Adra F (2021) A Neural Network Framework for Small Microcontrollers, pp. 51–60 Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 04 Nov, 2024 Reviews received at journal 30 Oct, 2024 Reviews received at journal 16 Oct, 2024 Reviewers agreed at journal 10 Oct, 2024 Reviewers agreed at journal 24 Sep, 2024 Reviewers invited by journal 19 Sep, 2024 Editor assigned by journal 19 Sep, 2024 Submission checks completed at journal 19 Sep, 2024 First submitted to journal 18 Sep, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5109889","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":373692831,"identity":"00f48521-5e99-41e8-869c-b865a5b3effa","order_by":0,"name":"Papdo Tchasse","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABS0lEQVRIie2SMUvDQBTHXzlIl5is76ikX+GVQmK+TaSDS3WUDhICglPAtQXxMzgVBIcrgWapdM3WQEAdiqS4VIhikqbqoXQWzG+4PHL55X/v7gBqav4oWAxUlgPUy6cjfSB+U6hSZsi9SkFQWPWvXUrjAjYK7FCs5v0kSiFrW80wSVfXB0hCNeMYXJcWl8HT610GWuh9V2z/pGcPgTq3ft/kozEWikUOBEhTptj+AwGfSTEk+mZLBXLyQmF7Y3SLFDx8F6VCqsjXHEm7QfNlpcwfk5e3qzLFRAfcUulkhbKIJSXapkQOtRrep8IKhSWbFKl9e/ic90Ldzk20NLk/ReSBcporAR9NeybbF12Vz6SFWfpxvmMDo03zoyRdn7mohedjvgZX14JJsloKw9BCuX1QYXv0FeyrVIqLoYIM/Xgj2emOyZqampp/wweNlnSss+psQAAAAABJRU5ErkJggg==","orcid":"","institution":"University of Stuttgart","correspondingAuthor":true,"prefix":"","firstName":"Papdo","middleName":"","lastName":"Tchasse","suffix":""},{"id":373692832,"identity":"4cbb3769-1194-4868-a34f-f8aba05b8f27","order_by":1,"name":"Tahsin Deliktas","email":"","orcid":"","institution":"University of Stuttgart","correspondingAuthor":false,"prefix":"","firstName":"Tahsin","middleName":"","lastName":"Deliktas","suffix":""},{"id":373692833,"identity":"ddd6f1c7-f7c1-4725-bbf6-2cf6bc1d6183","order_by":2,"name":"Mathias Liewald","email":"","orcid":"","institution":"University of Stuttgart","correspondingAuthor":false,"prefix":"","firstName":"Mathias","middleName":"","lastName":"Liewald","suffix":""}],"badges":[],"createdAt":"2024-09-18 12:05:35","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5109889/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5109889/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":69901008,"identity":"caecef23-cc03-4711-919a-ed58717ea231","added_by":"auto","created_at":"2024-11-26 11:57:27","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":193474,"visible":true,"origin":"","legend":"\u003cp\u003ePrinciple of reinforcement learning applied on metal forming processes \u003cem\u003e[20]\u003c/em\u003e \u003cem\u003e[21]\u003c/em\u003e\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-5109889/v1/a0475712de6d7dd32146d0ef.png"},{"id":69902570,"identity":"d88e825d-9390-400b-bcda-670f0750ce1d","added_by":"auto","created_at":"2024-11-26 12:21:27","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":92348,"visible":true,"origin":"","legend":"\u003cp\u003emodel-driven process control loop based on supervised and transfer learning\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-5109889/v1/ed986f7321b8fbe56e815c37.png"},{"id":69901006,"identity":"586c225d-6db7-4680-b7a2-f60f6b24c00d","added_by":"auto","created_at":"2024-11-26 11:57:27","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":20651,"visible":true,"origin":"","legend":"\u003cp\u003eshape of the workpiece before and after each forming stage\u003c/p\u003e","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-5109889/v1/2d83c911c641c32fa241a008.png"},{"id":69901014,"identity":"cafa4050-a957-430e-ba7f-f3327addee0a","added_by":"auto","created_at":"2024-11-26 11:57:27","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":116372,"visible":true,"origin":"","legend":"\u003cp\u003eActive tool components and workpiece shape before and after forward extrusion and upsetting operations\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-5109889/v1/0c276b5194ce70f77077e477.png"},{"id":69901271,"identity":"07391d9b-c148-4f68-8f23-da33fe3f7728","added_by":"auto","created_at":"2024-11-26 12:05:27","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":157220,"visible":true,"origin":"","legend":"\u003cp\u003eExperimental setup consisting of the process tool mounted in a servo mechanical press\u003c/p\u003e","description":"","filename":"floatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-5109889/v1/efb3717b47cfbc66cf865e9d.png"},{"id":69902267,"identity":"d959d639-73e9-486f-b228-a0e5dc19e226","added_by":"auto","created_at":"2024-11-26 12:13:27","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":49422,"visible":true,"origin":"","legend":"\u003cp\u003eQuality measurement of the final part after the two forming stages\u003c/p\u003e","description":"","filename":"floatimage6.png","url":"https://assets-eu.researchsquare.com/files/rs-5109889/v1/40f968578d13983ad67707f4.png"},{"id":69902264,"identity":"f55b9f1b-9180-4c9a-b2d4-5ca951bdf8f7","added_by":"auto","created_at":"2024-11-26 12:13:27","extension":"png","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":109339,"visible":true,"origin":"","legend":"\u003cp\u003eSensor and actuator concept for the process control\u003c/p\u003e","description":"","filename":"floatimage7.png","url":"https://assets-eu.researchsquare.com/files/rs-5109889/v1/d2fc7556684cc2a5e30ca989.png"},{"id":69901273,"identity":"bd82faea-cbe7-4397-8a5c-bc43bc09c672","added_by":"auto","created_at":"2024-11-26 12:05:27","extension":"png","order_by":8,"title":"Figure 8","display":"","copyAsset":false,"role":"figure","size":7925644,"visible":true,"origin":"","legend":"\u003cp\u003eRecorded temperature data and corresponding part quality\u003c/p\u003e","description":"","filename":"floatimage8.png","url":"https://assets-eu.researchsquare.com/files/rs-5109889/v1/3ce050c79916001f49c9efb1.png"},{"id":69901275,"identity":"3402fefe-fc16-4458-b9f1-ee26f33ce4d2","added_by":"auto","created_at":"2024-11-26 12:05:27","extension":"png","order_by":9,"title":"Figure 9","display":"","copyAsset":false,"role":"figure","size":98320,"visible":true,"origin":"","legend":"\u003cp\u003emodel architecture as result of the hyperparameter optimization\u003c/p\u003e","description":"","filename":"floatimage9.png","url":"https://assets-eu.researchsquare.com/files/rs-5109889/v1/e3731109b8553e28a760b6dc.png"},{"id":69902571,"identity":"425ad704-1e5c-4e44-8d15-b0db31f2e62e","added_by":"auto","created_at":"2024-11-26 12:21:27","extension":"png","order_by":10,"title":"Figure 10","display":"","copyAsset":false,"role":"figure","size":228583,"visible":true,"origin":"","legend":"\u003cp\u003ePrediction performance of the ML-model evaluated on validation data\u003c/p\u003e","description":"","filename":"floatimage10.png","url":"https://assets-eu.researchsquare.com/files/rs-5109889/v1/67f976dd816cef7ba8235a65.png"},{"id":69901010,"identity":"f11cca3b-b960-44c9-8252-f2ef90193f7e","added_by":"auto","created_at":"2024-11-26 11:57:27","extension":"png","order_by":11,"title":"Figure 11","display":"","copyAsset":false,"role":"figure","size":156889,"visible":true,"origin":"","legend":"\u003cp\u003eprinciple of periodical model upgrade based on online collected data\u003c/p\u003e","description":"","filename":"floatimage11.png","url":"https://assets-eu.researchsquare.com/files/rs-5109889/v1/87981e4290ab153810f03727.png"},{"id":69901278,"identity":"01d309e6-9e6f-4d51-8931-5de833302bb3","added_by":"auto","created_at":"2024-11-26 12:05:27","extension":"png","order_by":12,"title":"Figure 12","display":"","copyAsset":false,"role":"figure","size":184848,"visible":true,"origin":"","legend":"\u003cp\u003econtrol strategy implemented for the two-stage cold forging process\u003c/p\u003e","description":"","filename":"floatimage12.png","url":"https://assets-eu.researchsquare.com/files/rs-5109889/v1/8c48584ef2600e29e3d111ac.png"},{"id":69902266,"identity":"cf93be27-8d4f-4ae8-b040-cde3316b644a","added_by":"auto","created_at":"2024-11-26 12:13:27","extension":"png","order_by":13,"title":"Figure 13","display":"","copyAsset":false,"role":"figure","size":148986,"visible":true,"origin":"","legend":"\u003cp\u003eComparison of the quality evolution in the non-controlled and the controlled process\u003c/p\u003e","description":"","filename":"floatimage13.png","url":"https://assets-eu.researchsquare.com/files/rs-5109889/v1/4d044572b6694a15e3205b38.png"},{"id":69901011,"identity":"ef7cde51-f522-4761-aa8e-297e6da1c67f","added_by":"auto","created_at":"2024-11-26 11:57:27","extension":"png","order_by":14,"title":"Figure 14","display":"","copyAsset":false,"role":"figure","size":150054,"visible":true,"origin":"","legend":"\u003cp\u003eQuality evolution when alternating the desired target head height over a continuous run\u003c/p\u003e","description":"","filename":"floatimage14.png","url":"https://assets-eu.researchsquare.com/files/rs-5109889/v1/06484a081870a2527d3ae96b.png"},{"id":69902265,"identity":"844eab70-fb0a-4c9c-9422-8dc581af994a","added_by":"auto","created_at":"2024-11-26 12:13:27","extension":"png","order_by":15,"title":"Figure 15","display":"","copyAsset":false,"role":"figure","size":255445,"visible":true,"origin":"","legend":"\u003cp\u003ePunch positions when alternating between two target head heights\u003c/p\u003e","description":"","filename":"floatimage15.png","url":"https://assets-eu.researchsquare.com/files/rs-5109889/v1/04ccb1c9428d7890725e768b.png"},{"id":69901018,"identity":"b9641444-0040-4924-8cb7-e7d08e3bd117","added_by":"auto","created_at":"2024-11-26 11:57:27","extension":"png","order_by":16,"title":"Figure 16","display":"","copyAsset":false,"role":"figure","size":122512,"visible":true,"origin":"","legend":"\u003cp\u003eControl performance before and after a model upgrade\u003c/p\u003e","description":"","filename":"floatimage16.png","url":"https://assets-eu.researchsquare.com/files/rs-5109889/v1/f849ec2f1211cfd0faf42723.png"},{"id":69903649,"identity":"2a236c7b-efea-4632-a2cb-c1696f811abd","added_by":"auto","created_at":"2024-11-26 12:29:30","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":10557196,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5109889/v1/0e24f36f-5002-414b-bce6-7464f061df8e.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Development of an intelligent metal forming robot and application to multi-stage Cold Forging","fulltext":[{"header":"1 Introduction","content":"\u003cp\u003eDuring metal forming processes, the shape of solid metal bodies is plastically changed, while their mass and material cohesion are retained [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. Every metal forming part is the result of a prior thorough and detailed process design, which involves many numerical simulations with varying input parameters. Then, based on the simulations results, try-out experiments and expert knowledge, a process range and the corresponding press settings are deduced, which should enable the production the intended part. During the manufacturing however, metal forming processes experience different phases like ramp-up phases and are subject to many influences related to the part material properties, the tool dynamics and the complex frictional interactions between the forming tool and the workpiece. These aspects of the process often represent sources of failures that can lead to downtimes and the production of defective parts. To solve these issues, many press shops rely on the operator knowledge, which is rather an unsafe and unsystematic approach. Also, many researches addressed this problem by developing different control strategies to ensure the process robustness. However, many of these control strategies have been designed only for sheet metal forming applications [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e] and the control goal was to compensate the difference between the current and a reference value of the control variable, which was determined via either experiments or simulations. That means, the process control was based on a static assumption on the process evolution. Such assumptions have been proven to be limited as different unknown and known factors like temperature changes, tool wear or vibrations can strongly affect the relationship between process parameters and the part quality [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eFor this reason, this work aims to tackle the problem by proposing an inclusive method to automatically improve the resilience of metal forming processes despite their different instability phases and states. The objective here is to develop intelligent systems out of classical metal forming tools, that are able to make use of the past and current process experience in order to overcome dynamical challenges and stabilize the process, reducing the downtimes and the production of defective parts. For this purpose, the forming tool is upgraded into an intelligent robot, which learns over the time from the process environment and is able to use that knowledge to make the optimal decisions, given the process goals. In this work, the concept and implementation steps of such an intelligent metal forming robot were developed and then applied on a multi-stage cold forging process.\u003c/p\u003e"},{"header":"2 Concept definition","content":"\u003cp\u003eAccording to the international federation of robotics and ISO 8373\u0026ndash;20213, a robot is defined as a programmed actuated mechanism with a degree of autonomy to perform locomotion, manipulation or positioning [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e]. Following this definition, a metal forming robot was defined in this study as the result of integrating controllable components in a forming tool or in a press, which gives the possibility to purposefully and automatically affect the process online. Furthermore, such a robot would become intelligent if it is extended with sensors, able to collect process data, and a processing hardware as well as a software agent, that is able to learn from the past and the current process data in order to optimally operate controllable actuators, stabilize the process and fit it to user-defined quality requirements despite process disturbances. This intelligence of the metal forming robot is the main ability, which enhances its robustness and allows it to control the process even through yet unknown process states. Hence, an intelligent metal forming robot is a system comprising:\u003c/p\u003e \u003cp\u003e \u003cul\u003e \u003cli\u003e \u003cp\u003eA metal forming tool or press, equipped with the appropriate sensors for the process observation as well as actuators able to effectively influence the process evolution and the part quality,\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eA control hardware as central information processing unit and\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eA control software agent, which, based on the past and current process experience, is able to automatically determine the actual process working range, given the user-defined quality criteria and targets.\u003c/p\u003e \u003c/li\u003e \u003c/ul\u003e \u003c/p\u003e"},{"header":"3 State of the Art","content":"\u003cp\u003eThere have been many different approaches and studies of the application of Machine Learning (ML) to automatically control and optimize different aspects of metal forming processes.\u003c/p\u003e \u003cp\u003eIn [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e], the authors explored the simulative optimization of processing parameters in hot forging of AISI 4340 steel using instability map and reinforcement learning. In this study, a self-learning algorithm based on Q-Learning was developed in order to find the optimal part temperature and stroke speed, which lead to the most stable material range. Another study, implemented using finite element methods (FEM) is presented in [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]. Here, the authors proposed a simulative approach to develop an optimization strategy for the heating process of a forging line based on a digital twin, the objective being the automation of the development of an optimal control policy that adjusts the power for the heating coils in an induction oven based on temperature data recorded via pyrometers. In this use case, a reinforcement learning agent was trained to optimally adjust the heating power in order the keep the temperature of steel bars in the appropriate range. In Hot rolling Technology, Gamal et al. focused on the bar and wire hot rolling process and proposed a control strategy based on reinforcement learning in order to adaptively control the roll force, the roll torque and the roll gap [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]. For this study, the self-learning agent was trained in a simulative environment and the input parameters were chosen depending on the control parameter. Also, in [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e] Scheiderer et al. considered the heavy plate rolling process and designed a simulation-based process control in order to adaptively control the process parameters for each pass. Using input parameters like the height, the grain size and the temperature of the work piece, a reinforcement learning agent was trained to find the optimal roll gap and the inter-pass time, so that the process result fits to the target settings.\u003c/p\u003e \u003cp\u003eIn the sheet metal forming, some studies also focused on applying ML-based control. In [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e] for example, Molitor et al. focused on the springback compensation in the air bending process and proposed a control approach, for which the input was the part image and the punch force. Based on the force signals and the part image, the bending angle was predicted using a multilayer perceptron, a multioutput linear regression model and a convolutional neural work. Then, the predicted result was processed using a transfer function that provided the required vertical position of the press ram, and thus the punch, in order to compensate the springback. In [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e], Liu et al. explored the use of reinforcement learning in free sheet metal free form stamping. In this study, the authors trained a reinforcement learning algorithm in order to predict the optimal forming route of the hammer in order to achieve the desired part shape. This study was carried out in a simulative environment without providing any kind of prior expertise to the learning agent. Furthermore, [\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e] presents a self-learning optimal control algorithm for episodic fixed-horizon manufacturing processes with time discrete control actions. This optimal control is based on model free reinforcement learning and is applied in deep drawing in order to find the optimal blank holder force for each stroke based on the process experience. In this use case, the process state parameters were the punch position, the blank draw-in, and the position of the blank holder. The control algorithm in this study was developed and validated in a simulative environment.\u003c/p\u003e \u003cp\u003eAccording the state of the art, the practical application of ML Methods in metal forming has mainly been in order to monitor or predict the value of defined quality parameters. Then, the subsequent process control was done either incrementally, with a transfer function or a pre-control based on static or limited assumptions on the process behaviour and without considering short and long-term process instabilities. Furthermore, the few works that investigated a model-driven process control have been mostly implemented in simulative environments, based on input and output parameters that are hardly accessible in the practice like the sizes of finite elements [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]. Hence, it becomes a considerable challenge to evaluate the industrial applicability of the developed approaches. Also, because the process control has been principally developed in FEM Environments, different essential and practical aspects like the tool design, the sensor positioning, the actuator definition and the process speed have been strongly neglected. The ML applications so far also mainly cover the sheet metal forming and cutting technologies, so that practical insights of ML Methods applied to bulk metal forming seem not to be frequent. The focus of artificial Intelligence (AI) or ML in bulk metal forming so far have principally been on wear and maintenance prediction due to the high tool loads [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThis paper addresses the practical and industry-oriented model-driven process control of metal forming processes. Model-driven means that the target values to control actuators are generated by an AI-model, depending on the current process state. In other words, the process knowledge is replaced by an AI-model, that can learn long and short-time patterns and so, adapt to the different dynamical changes occurring during the process evolution. Furthermore, because the control software directly depends on different physical process components, it becomes obvious that in order to successfully deploy the control software, the process environment has to be upgraded. Therefore, this paper introduces the concept of intelligent metal forming robot, which includes the software, the hardware and the physical components required in order to build a practical and effective intelligent model-driven process control. Of course, the integration of Actuators into a metal forming tool or press and the use of sensors in order to monitor different aspects of the process are not a new concept in the field of metal forming. Also, The concept of a robot as a supporting asset for the forming operation is already present in some metal forming technologies like incremental sheet metal forming [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e] or roll forming [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e]. However, this study introduces the robot as the principal active forming component, which explores and interacts with each process stroke while learning from them. The purpose and the novelty of introducing the metal forming tool or machine as an intelligent metal forming robot is to present an integral system, which end-use, the intelligent adaptive process control, should be considered from the early stage of process design. The objective in this approach is to define a systematic methodology which considers the specific manufacturing environment and allows the development of a self-learning system that is able to exploit the extent and the versatility of the modern ML solutions in order to interact with the process and make it more robust in the short and long term. This approach is particularly relevant for one or multi-stage processes like deep drawing, forging or stamping.\u003c/p\u003e"},{"header":"4 System architecture of an intelligent metal forming robot","content":"\u003cp\u003eIn this section, the different system components as well as the implementation steps required to build an intelligent metal forming robot are presented.\u003c/p\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003e4.1 Actuators and sensors in Metal forming\u003c/h2\u003e \u003cp\u003eThe concept of intelligent metal forming robot is compatible not only with modern sensors and actuator technologies but also with sensors and actuators that are already familiar to the metal forming industry. For a process, different sensors can be used before, during and after a forming operation. Before the forming operation, different sensors like eddy current sensors, balances or optical sensors can be used to measure different aspects of the workpieces like the surface quality, the volume and the material properties [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. Then, during the process, force, pressure, energy or material flow sensors can be used to record the evolution of the formed part, which can later, after the forming operation, be measured using product sensors like cameras. The actuators are usually integrated to purposefully influence the forming operation. This can be done via a position, force, speed, temperature or lubrication adjustment [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. Depending on the application scenario, the positioning of the required sensors and actuators for an intelligent system may happen to be a challenge. In this case, FEM Simulations combined with mathematical correlation analysis would allow their appropriate disposition in or around the forming tool.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003e4.2 Control Hardware\u003c/h2\u003e \u003cp\u003eThe further development of digitization solutions for the industry has put the conventional programmable logic controllers (PLC) under a considerable pressure [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e]. To solve these new challenges and allow the deployment of novel software solutions like AI-models, the traditional PLCs have been upgraded in performance and modulated, so that they can be extended with additional hardware through different communication technologies like the open platform communications unified architecture (OPC UA) or the message queuing telemetry transport (MQTT). An example of additional hardware are edge devices, which act as an entry point in the internal process control and allows the extension of the standard control loop with additional software and also enables an access to cloud-based solutions [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e]. Furthermore, due to their low-cost, compared to conventional PLC, micro-controller-based PLC are also being further developed and extended so that they can support the requirements of the latest data processing technologies [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e].\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003e4.3 Control Software\u003c/h2\u003e \u003cp\u003eTwo particular software approaches can be considered for the process control of the intelligent forming robot. The first approach presented in this work is based on reinforcement learning (RL) and relies on the hypothesis that an online adaptive control of metal forming processes using RL would allow not only to take advantage of the process experience in order to guarantee a certain process stability but also to overcome dynamic instabilities like fluctuations of part material properties, tool temperatures changes and tool wear.\u003c/p\u003e \u003cp\u003eIn RL, a software agent makes \u003cem\u003eobservations\u003c/em\u003e and takes \u003cem\u003eactions\u003c/em\u003e within an \u003cem\u003eenvironment\u003c/em\u003e and in return it receives \u003cem\u003erewards\u003c/em\u003e, the objective of the agent being to learn to act in a way that will maximize its expected rewards over the time [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e]. The solution proposed with this approach consists in two steps. The first step consists in using process simulations in order to find and equip the metal forming tools with the appropriate sensors and actuators. Furthermore, the suitable hardware set would be incorporated in the system in order to process the sensor information and control the actuators. Then, the second and most important step is to design and develop the data processing software, which will make a resilient and self-learning system out of the equipped process tool. Thus, in this case:\u003c/p\u003e \u003cp\u003e \u003cul\u003e \u003cli\u003e \u003cp\u003eThe environment is the metal forming process including the tool and the integrated hardware and the process disturbances and influencing factors,\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eThe agent is the control software to be developed,\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eThe observations and the rewards are derived from the process or sensors data,\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eAnd the actions are carried out via actuator integrated in or around the tool.\u003c/p\u003e \u003c/li\u003e \u003c/ul\u003e \u003c/p\u003e \u003cp\u003eA learning step is defined as an action leading to an observation and a reward. The principle of RL applied on metal forming processes is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe training algorithm is the key point in this approach, especially because RL Tasks are known to be complex. Two families of training algorithms are popular for solving RL Tasks, Policy gradients (PG) algorithms and Markov decision processes (MDP) algorithms. In practice, popular and well performing algorithms are based on a combination of PG and MDP. But of course, The righteous choice of the algorithm depends on the considered environment and the specific task.\u003c/p\u003e \u003cp\u003eThe second approach proposed in this work is based on the hypothesis that if a metal forming tool is equipped with sensors able to record the process settings, noise and the part quality, then supervised and transfer learning methods can be used to generate an extensible virtual model of the process environment that allows to precisely control the process. The fundamental idea in this second approach is to use supervised learning methods in order to build a data-based model of the process and its environment and use it to build a virtual process operator. For this purpose, raw data are to be acquired and the suitable features for the specific use case are to be extracted. Then, after the data pre-processing, hyperparameter optimization is to be used in order to generate a ML-model architecture, that minimizes the validation loss between predicted and real targets and allows to avoid overfitting and underfitting [\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e]. An important aspect in this procedure is the choice of the input and output features. Given a system with control parameters, noise parameters and system output, the features of the ML-model are to be chosen so that the input features of the ML-model are the noise parameters and the system output and the targets features are the control parameters. One key aspect of such a ML-model is its ability to be used as a prediction model, which can interpolate between the known process states and even extrapolate, controlling the process even in unknow states, depending on the quality of the training data. Furthermore, as it is shown in the following section of this paper, the resulting virtual process operator doesn\u0026rsquo;t even need to directly monitor the part quality in order to control the actuators like in conventional control loops. Instead, it acts as a generative ML-model that provides new control targets for the actuators based only on the process state and the operator indication of the desired target quality.\u003c/p\u003e \u003cp\u003eOne essential advantage of using supervised learning is the possibility to upgrade the trained model by using domain adaption or transfer learning and hence create a dynamical and extensible ML-model for the process control. Transfer learning is used to fit pretrained ML-models so that they can be extended to new tasks or new data distributions of a given task [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e]. Applying transfer learning means that the data of every stroke is not processed directly after it is done and the software agent is not constantly training like in RL. Instead, each ML-model is trained and redeployed periodically, after a certain amount of process experience has been collected in form of labelled data. This way, the ML-models are trained at regular or irregular time intervals and then deployed online. Updating the ML-model means to apply the following data processing steps after a defined time interval: data acquisition and cleaning, data shuffle and splitting, data standardization, model training and model application. If the validation loss after each periodical model training seems unsatisfactory, then hyperparameter optimization can be used once again in order to build up another ML-model, that will better capture the correlations between the features in the whole dataset. The advantage of this approach is the simplicity of the model training and the achievable process speed when deploying the ML-models, since they are trained offline and deployed only after their training. In Fact, Transfer learning could be combined with the first approach in order to limit the training cost of the RL agents. The Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e illustrates the idea of the control loop for the second approach.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003e4.4 Implementation steps of an intelligent metal forming robot\u003c/h2\u003e \u003cp\u003eDeveloping an intelligent tool for a metal forming process requires implementing the following steps:\u003c/p\u003e \u003cp\u003e \u003col\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eProcess definition and specification of the quality criteria\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eDetermination of the quality influencing factors and quality parameters\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eDevelopment of an appropriate sensor and actuator concept and tool design\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eDesign of a Signal processing model and choice of the applicable processing hardware\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eDevelopment of the signal processing and control software\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003c/ol\u003e \u003c/p\u003e \u003cp\u003eThese implementation steps were applied to a multi-stage cold forging process, which is described in the following sections.\u003c/p\u003e \u003c/div\u003e"},{"header":"5 Application of the intelligent metal forming robot to cold forging","content":"\u003cp\u003eThe first use case for the aforementioned method is a cold forging application, in which the objective is to reduce scrap and hence the loss during the ramp-up phase of a two-stage cold forging process. A ramp-up phase is inevitable every time presses are started after a standstill, downtimes or because of a different process tool mounted in the press. In this phase, defective parts are produced at a higher rate than in the steady phase, which often represents a non-negligible financial loss. Consequently, being able to better understand and intelligently influence the ramp-up phase of the process would lead to the reduction of the amount of scrap and therefore the financial loss.\u003c/p\u003e \u003cdiv id=\"Sec10\" class=\"Section2\"\u003e \u003ch2\u003e5.1 Process Design\u003c/h2\u003e \u003cp\u003eFor this use case, a two-stage cold forging process was defined, consisting of a forward extrusion stage and an upsetting stage. The objective was to form a cylindrical billet of diameter 15 mm and height 20 mm into a screw-like part without thread, drive and tip and with head diameter 20 mm\u0026thinsp;\u0026plusmn;\u0026thinsp;0.15 and head height 6.25 mm\u0026thinsp;\u0026plusmn;\u0026thinsp;0.1 and body diameter 10 mm and body height 20 mm. The shape of the part during the process is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eDuring the first stage, the punch pressure forces the billet to flow in the punch movement direction, which results in the part body being created with a smaller cross section than the part head. Then, this part head is reduced in the second stage in a way that two parallel surfaces, the limits of the part head, are obtained, as show in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e and Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eFor this process, the material considered was 28B2 (1.5510) KGK. the billet was cut from a rod, that was drawn, annealed and trailed. Also, the rod surface was coated with phosphate and was soaped for temperature resistance. The formability study was mainly done using FEM simulations with the software DEFORM. For the simulation to be started, compression tests were carried out in order to acquire the material properties. After the numerical design of the multi-stage forming process, the forming tool shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e was designed with the three tool slots arranged in line for the implementation of three forming dies, although in this case the last tool stage was used as a measuring stage. The tool was installed in a servo mechanical knuckle joint press from the Schuler company with a ram upsetting capacity of 5000 kN and a stroke rate range of 3 to 45 min\u003csup\u003e\u0026minus;\u0026thinsp;1\u003c/sup\u003e. Furthermore, in order to move the workpiece from one stage to the next, the press was equipped with a gripper transfer system. Figure\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e shows the experimental process tool and the press used for the experimental investigations.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003e5.2 Process parameters\u003c/h2\u003e \u003cp\u003eBoth the forward extrusion and the upsetting operation cause a significant increase of the die temperature. This feature is one key factor in cold forging, as its evolution often characterizes the different process phases including the ramp-up and the steady phases of a production run. For every stroke, the die temperature increases are caused mainly by the frictional interactions between the flowing workpiece and the die shoulder, for the forward extrusion, as well as the deformation work. Furthermore, the temperatures of every stroke affect the next ones, and therefore act as a thermal noise during the process evolution. This complex interactions between the die temperature of different strokes strongly affect the production runs, so that they often experience a ramp-up phase and a steady phase. Because of this importance of the die temperature, it was the main process parameter that was considered for this application.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003e5.3 Sensor and Actuator concept\u003c/h2\u003e \u003cp\u003eThe choice and positioning of the sensors and the actuators were deduced from the process design and simulation studies. For the temperature measurements, temperature sensors were integrated in the tool, according to the simulation results, so that they would be as near as possible of the hottest point in the dies during the process. Two temperature sensors were used for each stage. Also, other sensors for the part quality measurement and the process control via actuators were considered. The target quality feature in this use-case was the head height of the final part. Because this study addressed the forming operations involved for the production of screw-like parts, this quality feature was chosen to be industry-relevant. In order to measure the head height, a laser profiler sensor was installed in the tool and was used to capture the difference between the two surfaces, that bound the Part head as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003e.\u003c/p\u003e \u003cp\u003eFor this application, actuators were used to control the vertical position of the punches. Since the main factor during the ramp-up phase is the die temperature evolution, which is caused by frictional and compression forces, vertically adjusting the punch position appeared to be a direct way of controlling the extent of the frictional interactions and the deformation work deployed for the workpiece. For this reason, a wedge system was developed with the objective of converting the rotational motion of a motor into a linear vertical motion of the punches. One wedge system was tied to each punch and distance sensors were integrated in order to measure the actual position of the punches. The wedge systems were conceived so that the punch adjustment range was 0.4 mm for both process stages. The wedges systems were driven using stepper motors.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eIn summary, four temperature sensors, two for each stage, one laser profiler sensor, for the part quality, and one distance sensor for each punch position were used. Figure\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e7\u003c/span\u003e shows the sensor and actuator concept developed in this study.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec13\" class=\"Section2\"\u003e \u003ch2\u003e5.4 Hardware Architecture\u003c/h2\u003e \u003cp\u003eThe different sensors and actuators were connected, so that the process data could be collected and the actuators could be operated via a central processing hardware. The sensors where first connected to Amplifiers and measuring hardware. Then, the data of the Amplifiers were sent to the control hardware, which was an Arduino Mega 2560. The control hardware was also connected to the motors of the wedge systems through the appropriate drivers. The function of the control hardware was to collect the data, process them and compute the right positions of the punches, so that the part quality requirements are met.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec14\" class=\"Section2\"\u003e \u003ch2\u003e5.5 Data processing and control software\u003c/h2\u003e \u003cp\u003eThe data processing and control software in this application was implemented according to the second approach presented in the section 4.3. That means, there was an offline data processing and an online data processing.\u003c/p\u003e \u003cdiv id=\"Sec15\" class=\"Section3\"\u003e \u003ch2\u003e5.5.1 Offline data processing\u003c/h2\u003e \u003cp\u003eThe offline data processing consisted of every step that is necessary in order to build a basis ML-model for the process control. Those steps include the data collection, the data preprocessing and the model building. A total of seven features was considered for the data processing and each feature was extracted from the data of the sensors mentioned in the section 5.3. The inputs for the ML-model were the maximum temperatures of every press stroke as well as the part head height and the targets were the positions of the punches. Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e gives an overview of the features considered in this use case.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003efeatures considered for the model training\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"2\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eFeatures\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eUnit\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMaximum temperature 1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u0026deg;C\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMaximum temperature 2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u0026deg;C\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMaximum temperature 3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u0026deg;C\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMaximum temperature 4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u0026deg;C\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePosition of first punch\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003emm\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePosition of second punch\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003emm\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePart head height\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003emm\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003efor the basis ML-model, a total of 6 production runs at a stroke rate of 25 min\u003csup\u003e\u0026minus;\u0026thinsp;1\u003c/sup\u003e was recorded, which corresponds to a total of 1735 strokes. Figure\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e8\u003c/span\u003e shows the recorded temperature data and the corresponding part head height. The different runs were recorded at different punch positions and the maximal number of strokes achieved during a production run was 396. Every recorded production run was started at a die temperature of around 26\u0026deg;C. The highest recorded temperature after 396 strokes was 80\u0026deg;C and for every run, the highest temperature increase was identified after the first 40 strokes, corresponding to a maximum die temperature of around 70\u0026deg;C.\u003c/p\u003e \u003cp\u003eFor the ML-model design, the data had to be restructured according to the number of process stages and the moment of the part quality measurement. The head height of every part was measured two strokes after it was first formed. Because one part was in the press for two strokes, it seems rightful to assume that the part quality was affected by two strokes and hence consider the corresponding data in order to control the process. The issue with this approach is the necessity to wait for two strokes before the ML-model can generate new target positions for the punches. In this case, the process control would no more affect the part, as it would have already been formed. To solve this issue, the situation was considered from another point of view. The question was, how do the actual positions of the punches after each stroke correlate with the temperature measurements of that stroke and the part quality measured two strokes ahead? Based on this reflection, the training data for the model were shifted and structured so that based on the temperature measurements of one stroke, the punches could be operated and the quality of the part inside the press adjusted.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eAfter the data collection and structuration, the samples were shuffled and split so that 90% were used for the training and 10% for the validation. The validation data were used in order to evaluate the offline performance of the ML-model during the training and before the online deployment. Then, the training and validation data were standardized with the mean and standard deviation of the training data.\u003c/p\u003e \u003cp\u003eThe model considered for this use case was a feed forward neural network or multilayer perceptron (MLP) [\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e]. The first or basis MLP was built using the hyperparameter optimization algorithm hyperband [\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e]. For the hyperparameter optimization a range was defined for each hyperparameter, in which the search algorithm inferred the best value. Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e summarizes for each hyperparameter the range for the search algorithm.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eHyperparameter space for the model building\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"2\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHyperparameter\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eValue range\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNumber of hidden layers\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e[2, 10]\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNumber of neurons per hidden layer\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e[2, 15]\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDropout rate per hidden layer\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e[0.2, 0.5]\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLearning rate\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e1e-2, 1e-3, 1e-4, 1e-5\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHidden layer activation functions\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eRelu, sigmoid, tanh\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLoss function\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eLog-cosh, Mean squared error\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eThe best model was chosen within 528 models that were trained for an average of 300 epochs per model. The model architecture which resulted from the hyperparameter optimization is illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig9\" class=\"InternalRef\"\u003e9\u003c/span\u003e. That model was then trained again for a longer period of time and evaluated using the coefficient of determination (R\u0026sup2;-score) and the root mean squared error (RMSE) [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e]. While the lowest RMSE is striven, an R\u0026sup2;-score of 1 means a perfect correlation and an R\u0026sup2;-score of 0 or below is an indication of insufficient model fit. The performances of the trained ML-model are summarized in Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e. These high R\u0026sup2;-scores are proof of good process controllability, as this proves a high correlation between the punch positions and the process temperatures as well as the part quality. Figure\u0026nbsp;\u003cspan refid=\"Fig10\" class=\"InternalRef\"\u003e10\u003c/span\u003e shows the prediction accuracy of the ML-model on validation data.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eperformance of the ML model on train and validation data\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"5\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eScore\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eSet\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eAverage\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003ePosition punch 1\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003ePosition punch 2\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eR2-score\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eTraining\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.969\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.952\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0.985\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eValidation\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.968\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.948\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0.9884\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eRMSE\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eTraining\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.030 mm\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.037 mm\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0.021 mm\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eValidation\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.030 mm\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.039 mm\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0.019 mm\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eThe built ML-model was then converted to fit the requirements of the control hardware and was deployed online [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e]. The model training in this study was done in a python environment, using different ML-libraries like Tensorflow and Scikit-Learn.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec16\" class=\"Section3\"\u003e \u003ch2\u003e5.5.2 Online data processing\u003c/h2\u003e \u003cp\u003eThe online data processing represents the routine that ensures that after a certain number of strokes, the ML-model is updated with the new process experience. This update of the ML-model is supposed to be done automatically parallel to the production or after a downtime. For the update of the ML-model, the same data pre-processing as for the offline data processing steps were applied, then the ML-model was trained as a pretrained model using transfer learning. In the case that the results of the transfer learning would seem unsatisfactory, the ML-model would have been built and trained anew via hyperparameter optimization. The data flow for the periodical online data processing is illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig11\" class=\"InternalRef\"\u003e11\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec17\" class=\"Section3\"\u003e \u003ch2\u003e5.5.3 Control strategy\u003c/h2\u003e \u003cp\u003eThe control procedure in this study was based on the hypothesis that the process temperatures are the reference features in order to adjust every stroke. The built ML-model is like a virtual operator that relies on a data-based digital twin of the process and which, given the temperatures data and the required part head height, is able to determine the punch positions so that the required part quality is obtained. That means that the controlling ML-model doesn\u0026rsquo;t need any information on the actual part quality, as it already indirectly infers that knowledge from the process state, just like an experienced operator would. For the model deployment, the part head height was set to the desired target head height and the temperatures data were recorded at every stroke. Figure\u0026nbsp;\u003cspan refid=\"Fig12\" class=\"InternalRef\"\u003e12\u003c/span\u003e shows the control strategy developed in this use-case.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003c/div\u003e"},{"header":"6 Results","content":"\u003cp\u003eIn order to fulfil the requirements of the intelligent metal forming robot as defined in the section \u003cb\u003eFehler! Verweisquelle konnte nicht gefunden werden.\u003c/b\u003e, the developed system should be able to:\u003c/p\u003e \u003cp\u003e \u003col\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eStabilize the process and ensure a certain part quality regardless of the noises and dynamical disturbances\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eAdapt the process in order to fit user defined changes in the quality requirements\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eControl the process even in states that have not yet been modelled in the training data\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eImprove the control precision based on the updated process experience\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003c/ol\u003e \u003c/p\u003e \u003cp\u003eThese four aspects of the system were evaluated and validated through real experiments of continuous forming strokes at a press speed of 25 min\u003csup\u003e\u0026minus;\u0026thinsp;1\u003c/sup\u003e. The starting temperature for every continuous run was around 26\u0026deg;C for the four temperature sensors.\u003c/p\u003e \u003cp\u003eFor the first requirement, several production runs were started and for each of them a different target value for the head height was set. Figure\u0026nbsp;\u003cspan refid=\"Fig13\" class=\"InternalRef\"\u003e13\u003c/span\u003e shows the performance of the control system for two different targets in comparison with the case where no process control was applied and a part head height of 11.90 mm was expected.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eIn the non-controlled scenario, the part quality seems to be fluctuating with an average of 11.86 mm and at some point, it even starts diverging. This doesn\u0026rsquo;t seem to be case when the control is activated. At the begin of the continuous run in the controlled scenario, the controlling ML-model tries to find the process working range, which suits the quality requirements and then it tries to stabilize the process so that, despite the temperature changes and the press vibrations, the part quality fluctuates within \u0026plusmn;\u0026thinsp;0.05 mm according to the target value. An aspect of the process which explains why the fluctuations are inevitable is the volume or mass variation of the billets.\u003c/p\u003e \u003cp\u003eFor the second and third requirements of the intelligent metal forming robot, different target values were set during a continuous run of 535 strokes, whereby after every 25 strokes, which corresponds to 1 minute, the target head height was changed. Figure\u0026nbsp;\u003cspan refid=\"Fig14\" class=\"InternalRef\"\u003e14\u003c/span\u003e shows the results when alternating between two different targets as well as the effects on the maximum temperature measured in the second process stage.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eChanging the target values after only one minute may not be that relevant for the industrial application, however it shows the faculty of such control systems to quickly react to user defined changes or detected events which could affect the process. The control pattern in Fig.\u0026nbsp;\u003cspan refid=\"Fig14\" class=\"InternalRef\"\u003e14\u003c/span\u003e doesn\u0026rsquo;t seem to indicate any particular disturbance or noise like temperatures increase of vibration, even though they are present. Furthermore, the control was evaluated over 535 strokes, which is beyond the maximum of the continuous runs recorded to train the model. This proves the extrapolation ability of the designed system and shows that, even in yet unknow process states the intelligent control agent can be functional and assure a certain process quality.\u003c/p\u003e \u003cp\u003eThe intelligent control affects the process quality indirectly through an adjustment of the punch position, which is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig15\" class=\"InternalRef\"\u003e15\u003c/span\u003e. The difference in precision between the quality control and the punch position control is obvious. The control errors seem to be less significant for the punch positions, that are directly controlled by the ML-model. Because of this precision gap in the control, it is assumed that an external factor plays an important role, which was not considered in the data acquisition. This external factor is assumed to be the mass variations of the billet.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eAnother important fact illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig14\" class=\"InternalRef\"\u003e14\u003c/span\u003e and Fig.\u0026nbsp;\u003cspan refid=\"Fig15\" class=\"InternalRef\"\u003e15\u003c/span\u003e is the horizontal control shift between target and actual head height. This shift is explained by the fact that, when a new target was identified by the ML model, the first punch was first positioned in the suitable control range and the second punch was operated.\u003c/p\u003e \u003cp\u003eThe intelligence of the system as the faculty to learn and update its knowledge basis were evaluated by considering the control performance of the controlling ML-model before and after a retrain. The basis ML-model was first deployed and an attempt was carried out to test its ability to process three different targets as illustrated in the Fig.\u0026nbsp;\u003cspan refid=\"Fig16\" class=\"InternalRef\"\u003e16\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe left curve in the Fig.\u0026nbsp;\u003cspan refid=\"Fig16\" class=\"InternalRef\"\u003e16\u003c/span\u003e shows how instable the part quality control with three different targets is. At the basis level, the ML-model didn\u0026rsquo;t seem to be able to handle this task properly. However, after a model upgrade with the new process experience and the past control attempts, the updated controlling ML-model could better switch between different three defined targets and also hold them. These results show the ability of the intelligent metal forming robot to learn and improve itself with new process experience.\u003c/p\u003e \u003cp\u003eAlso, Fig.\u0026nbsp;\u003cspan refid=\"Fig14\" class=\"InternalRef\"\u003e14\u003c/span\u003e and Fig.\u0026nbsp;\u003cspan refid=\"Fig16\" class=\"InternalRef\"\u003e16\u003c/span\u003e show that the controlling ML-model, based on its experience, can be used as a generative model not only to stabilize the process in case of fluctuations, but also when new quality targets are desired. During these experiments, the ML-model could compute new targets for the process control in less than 30 ms.\u003c/p\u003e"},{"header":"7 Conclusion and outlook","content":"\u003cp\u003eThe current digitalization solutions allow the development of more complex and capable systems. One of those is presented in this paper and is referred to as the intelligent metal forming robot. The ability of this system to intelligently control the process and learn from past and current experience in order to generate the right control targets for the actuators are the key factors that enable its robustness. Indeed, this system has now been implemented in the scope of bulk metal forming, however its principles could be extended to other metal forming processes and manufacturing technologies.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e \u003ch2\u003eConflict of interest\u003c/h2\u003e \u003cp\u003eThe authors have no relevant financial or non-financial interests to disclose.\u003c/p\u003e \u003c/p\u003e\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003eP.T. wrote the main manuscript text T.D. and M.L. reviewed the manuscript.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eLange K (1985) \u003cem\u003eHandbook of Metal Forming\u003c/em\u003e, \u0026lrm; Society., New York McGraw-Hill c 1986\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAllwood JM et al (2016) Closed-loop control of product properties in metal forming. CIRP Ann - Manuf Technol 65(2):573\u0026ndash;596\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKott M (2022) \u003cem\u003eMethodik zur Entwicklung eines Bedienerassistenzsystems f\u0026uuml;r das Presswerk\u003c/em\u003e, Dr.-Ing. Dissertation, Technische Universit\u0026auml;t Darmstadt, Darmstadt\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eI. F. of Robotics, Artificial Intelligence in Robotics, no. (2022)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJeong HY, Park J, Kim Y, Shin SY, Kim N (2023) Processing parameters optimization in hot forging of AISI 4340 steel using instability map and reinforcement learning. J Mater Res Technol 23:1995\u0026ndash;2009\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMa Y et al (2022) Using Deep Reinforcement Learning for Zero Defect Smart Forging, \u003cem\u003eAdv. Transdiscipl. Eng.\u003c/em\u003e, vol. 21, no. January, pp. 701\u0026ndash;712\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGamal O, Mohamed MIP, Patel CG, Roth H (2021) Data-Driven Model-Free Intelligent Roll Gap Control of Bar and Wire Hot Rolling Process Using Reinforcement Learning. Int J Mech Eng Robot Res 10(7):349\u0026ndash;356\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eScheiderer C et al (2020) Simulation-as-a-service for reinforcement learning applications by example of heavy plate rolling processes. Procedia Manuf 51:897\u0026ndash;903\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMolitor DA, Arne V, Kubik C, Noemark G, Groche P (2024) Inline closed-loop control of bending angles with machine learning supported springback compensation. Int J Mater Form, 17, 1\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiu S, Shi Z, Lin J, Li Z (2019) Reinforcement learning in free-form stamping of sheet-metals, \u003cem\u003eProcedia Manuf.\u003c/em\u003e, vol. 50, no. pp. 444\u0026ndash;449, 2020\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDornheim J, Link N, Gumbsch P (2020) Model-free Adaptive Optimal Control of Episodic Fixed-horizon Manufacturing Processes Using Reinforcement Learning. Int J Control Autom Syst 18(6):1593\u0026ndash;1604\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eIdzik C, Gerlach J, Bailly D, Hirt G (2023) Advanced automatic pass schedule design for hot rolling by coupling reinforcement learning with a fast rolling model, \u003cem\u003eMater. Res. Proc.\u003c/em\u003e, vol. 28, pp. 601\u0026ndash;610\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCao J, Bambach M, Merklein M, Mozaffar M, Xue T (2024) Artificial intelligence in metal forming. CIRP Ann 00:1\u0026ndash;27\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFiorentino A, Ceretti E, Feriti GC, Giardini C (2015) Improving accuracy in aluminum Incremental Sheet Forming of complex geometries using Iterative Learning Control. Key Eng Mater, vol. 651\u0026ndash;653, pp. 1096\u0026ndash;1102\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAbdolmohammadi T, Richter-Trummer V, Ahrens A, Richter K, Alibrahim A, Werner M (2023) Virtual Sensor-Based Geometry Prediction of Complex Sheet Metal Parts Formed by Robotic Rollforming, \u003cem\u003eRobotics\u003c/em\u003e, vol. 12, no. 2\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMansfield J (2018) \u003cem\u003eIndustrial internet of things demystified\u003c/em\u003e, vol. 535\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAl-Turjman F (ed) (2019) Edge Computing. Springer International Publishing, Cham\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBai Y, Roth ZS (2018) \u003cem\u003eClassical and Modern Controls with Microcontrollers: Design, Implementation and Applications\u003c/em\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSutton RS, Barto AG (2018) \u003cem\u003eReinforcement Learning\u003c/em\u003e, 2nd ed., Cambridge\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBirkert A, Haage S, Straub M (2013) Umformtechnische Herstellung komplexer Karosserieteile\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDietrich J (2018) Praxis der Umformtechnik. Springer Fachmedien Wiesbaden, Wiesbaden\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChollet F (2021) \u003cem\u003eDeep Learning with Python\u003c/em\u003e, 2nd ed\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eG\u0026eacute;ron A (2019) Hands-on Machine Learning whith Scikit-Learing, Keras and Tensorfow. O\u0026rsquo;Reilly Media\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi L, Jamieson K, DeSalvo G, Rostamizadeh A, Talwalkar A (2018) Hyperband: A novel bandit-based approach to hyperparameter optimization. J Mach Learn Res 18:1\u0026ndash;52\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eEstrebou CA, Fleming M, Saavedra MD, Adra F (2021) A Neural Network Framework for Small Microcontrollers, pp. 51\u0026ndash;60\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"production-engineering","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"Learn more about [Production Engineering](https://www.springer.com/journal/11740)","snPcode":"11740","submissionUrl":"https://submission.nature.com/new-submission/11740/3","title":"Production Engineering","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Metal forming, multi-stage cold Forging, self-learning system, intelligent process control, machine learning","lastPublishedDoi":"10.21203/rs.3.rs-5109889/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5109889/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eMetal forming processes often undergo different instability phases due to different factors like tool and part temperature variations, tool vibrations or frictional interactions between workpiece and tool. In the absence of an experienced process operator, these instabilities can induce a very considerable production loss. This study addresses this issue and proposes a method to develop a data-based virtual process operator equipped with the appropriate hardware and physical components that allow it to constantly monitor and if necessary regulate the process. The resulting system is introduced as the intelligent metal forming robot. The objective of this self-learning system is first to stabilize the process and ensure a certain part quality despite the noises, dynamical disturbances and user-defined changes of the part quality requirements, then, to control the process even in states that have not yet been experienced and at last to improve the control precision based on the updated process experience. This intelligent metal forming robot has been implemented and applied on a two-stage cold forging process, where the target quality feature was the part head height of a screw-like part. The results showed that, based on a qualitative process experience and on effective actuators, an intelligent self-learning system can significantly increase the robustness of a metal forming process.\u003c/p\u003e","manuscriptTitle":"Development of an intelligent metal forming robot and application to multi-stage Cold Forging","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-11-26 11:57:22","doi":"10.21203/rs.3.rs-5109889/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-11-04T07:38:37+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-10-30T13:32:09+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-10-16T11:26:25+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"196418762188374062413511260118441469780","date":"2024-10-10T07:55:55+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"168467585090624717310880815749136704591","date":"2024-09-24T14:47:38+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-09-19T13:58:24+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-09-19T13:41:04+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-09-19T12:52:48+00:00","index":"","fulltext":""},{"type":"submitted","content":"Production Engineering","date":"2024-09-18T12:02:42+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"production-engineering","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"Learn more about [Production Engineering](https://www.springer.com/journal/11740)","snPcode":"11740","submissionUrl":"https://submission.nature.com/new-submission/11740/3","title":"Production Engineering","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"f1085ebc-3482-4e4f-93da-ab1d29e4d955","owner":[],"postedDate":"November 26th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2025-01-07T10:24:00+00:00","versionOfRecord":[],"versionCreatedAt":"2024-11-26 11:57:22","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5109889","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5109889","identity":"rs-5109889","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}