Predicting Calibration Parameter Values forConstitutive Models using Genetic Programming

Gabriel Kronbergera, Evgeniya Kablimanb, Johannes Kronsteinerc, MichaelKommendaa

aJosef Ressel Center for Symbolic RegressionUniversity of Applied Sciences Upper AustriaSoftwarepark 11, 4232 Hagenberg, Austria

bTechnical University of MunichDepartment of Mechanical Engineering

cLKR Light Metals TechnologiesAustrian Institute of Technology

Ranshofen, Austria

Abstract

In material science, models are derived to describe emergent properties (e.g.elasticity, strength, conductivity, ...) and their relations to the material and pro-cessing conditions. Constitutive models describe the behaviour of materials, forinstance deformation processes through applied forces. We describe two meth-ods for the extension of constitutive models using genetic programming (GP)and demonstrate their application for a model which is used in computationalmechanics to describe the material response to thermo-mechanical loading con-ditions. The model operates with internal material variables such as dislocationdensity and contains a number of parameters, among them three calibrationparameters. Currently, these must be fit to measured data since the relationsto the processing conditions (e.g. deformation temperature, strain rate) are notfully understood. Our proposed GP-based methods identify these relations andgenerate accurate and short expressions which can be plugged into the consti-tutive model instead of the calibration parameters. With the expressions foundby GP it is possible to interpolate which can potentially reduce the numberof experiments and measurements. Additionally, it opens the possibility forextrapolation.

Keywords: Symbolic Regression, Genetic Programming, Material Modelling,Flow Stress2010 MSC: 68T05, 74-04, 74-05, 74-10, 74C99

Email address: gabriel.kronberger@fh-hagenberg.at (Gabriel Kronberger)

Preprint submitted to Journal of Advances in Engineering Software August 4, 2021

arX

iv:2

108.

0159

5v1

[cs

.NE

] 3

Aug

202

1

1. Introduction and Motivation

Mathematical models are at the core of science and engineering and allowus to predict physical phenomena without direct observation. Only throughmodelling and simulation are we able to build extremely complex and safe phys-ical objects (such as air planes, space vehicles, or power plants). In materialscience, models are derived to describe emergent properties (e.g. elasticity,strength, conductivity) and their relations to the material and processing con-ditions. Constitutive models describe the behaviour of materials for instancedeformation processes through applied forces and usually involve physical con-stants and laws but may also be purely empirical.

In empirical modeling one can distinguish into white-box and black-boxmodels with a whole spectrum of grey-box models between the two extremes[1, 2]. White-box models can be derived from physical principles and have inter-pretable parameters with physical meaning (e.g. Planck’s constant, Avogadro’sconstant). The internals of white-box models are known and can be under-stood. Black-box models establish a functional mapping from inputs to outputsby fitting to observations whereby the internals of the model are irrelevant orunknown. Therefore, the internal parameters of black-box models have no phys-ical meaning [1]. Examples of black-box models are non-parametric statisticalmodels (i.e. all kernel-methods including support vector machines, Gaussianprocesses, LOESS), neural networks, and tree ensemble methods (e.g. randomforest, gradient boosted trees). Grey-box models also establish a functionalmapping from inputs to outputs by fitting to observations but have only a fewparameters and simple interpretable equations.

One possible approach for the identification of grey-box or potentially evenwhite-box models is symbolic regression (SR) with genetic programming (GP).The main aim of SR is finding well fitting model equations (structure) as wellas their parameters [3]. SR offers the possibility of interpretability which canbe achieved by including model complexity as an optimization criterion ad-ditionally to model fit. A popular approach for solving SR problems is GPwhich is an evolutionary algorithm that evolves computer programs. It usesa population of solution candidates (programs) which is iteratively improvedby mimicking processes observed in natural evolution, namely survival of thefittest, recombination, and mutation [4]. Programs that are better with respectto an objective function (also called fitness function in GP) are selected with ahigher probability for recombination while bad programs have a low probabilityto be selected. Recombination combines the two programs randomly and relieson the assumption that combining two good programs may produce anothergood and ideally even better program. Different forms for encoding the evolvedprograms in GP are possible including linear encodings, trees, and graphs. ForSR it is convenient to evolve purely functional expressions represented as ex-pression trees. SR models can be easily integrated into mathematical modelsregardless of the modeling software environment because the commonly usedoperators and functions are readily available in most standard libraries.

We have used GP and SR for extending a physics-based constitutive model

2

in [8] and found short formulas for the calibration parameters of the model.We have combined these formulas with the physics-based models and used theextended model for finite-element (FE) simulations of material deformation.The approach we have used in the past involves two steps: first, parametersare optimized using a global optimizer, afterwards formulas for predicting theoptimized parameters are identified with SR.

In this paper we give a detailed description of this approach (which wewill call the explicit method below) and describe a new method which is alsobased on GP. We call the new method implicit because it evolves extensionsto the constitutive model directly. The motivation for the development of theimplicit method is to simplify the approach by reducing the two separate steps– each requiring a set of parameters that have to be tuned – to a single step.Additionally, we hope to improve the predictive accuracy of the evolved model.

2. Related Work

The well-maintained GP bibliography [9] contains many references to priorwork in which GP has been used for constitutive modelling in particular forpredicting stress for various materials. GP for multi-scale material modelling isextensively discussed in [10].

Two early works which describe the application of GP for the identification ofconstitutive models are [11] and [12]. The approach described in [11] is especiallynotable as it uses specific operators to ensure that the resulting models havephysical interpretation as elastic, plastic and viscous components.

Since then GP has been used extensively for constitutive modelling such asfor modelling flow stress for various metallic materials [13, 14, 15] includingaluminium alloys [16], for modelling stress distribution in cold-formed copperalloys [17] and X6Cr13 steel [18], for predicting impact toughness [19], for theidentification of visco-elastic models for rocks [20], and stress-strain behaviour ofsands under cyclic loading [21, 22], for predicting material properties under hotdeformation, in particular for carbon silicon steel [23], and a nickel-based alloy[24], for predicting the presence of cracks in hot rolled steel [25], for modellingtensile strength, electrical conductivity of cold-drawn copper alloys [26], forprediction of shear strength of reinforced concrete beams [27], for formulatingthe stress-strain relationship of materials in [28], and for predicting fatigue for2024 T3 aluminium alloys under load [29].

GP has further been used for predicting non-linear stress-strain curves (e.g.for aluminium and stainless steel alloys) [30, 31, 32], predicting elastic distor-tional buckling stress of cold-formed steel C-sections [33], predicting residualstress in plasma-nitrided tool steel [34], and modelling mechanical strength ofaustenitic stainless steel alloys (SS304) as a function of temperature, strain andstrain rate [35].

An evolutionary method for polynomial regression and the combination withfinite element analysis has been used for constitutive modelling and applied forinstance to predict the behaviour of soils under drained and undrained loadconditions in [36] and [37].

3

Evolutionary algorithms have been used for the optimization of calibrationparameters of constitutive models in [38] and optimizing alloy composition usingGaussian process surrogate models and constitutive models for simulation [39].In [40], different methods for parameter optimization of a constitutive modelfor hot deformation of a titanium alloy have been tested and compared with thepredictions made by artificial neural networks.

All of the papers discussed above describe a from of regression modelling.In those papers, GP is used for supervised learning to establish a free-formconstitutive model using SR (e.g. [10], [11]) or alternatively the model structureis fixed and parameters are optimized using evolutionary algorithms. We arehowever mainly interested in combining or extending physics-based models withmachine learning models. We found only a few papers with a similar focus inthe material science domain.

A hybrid modelling approach using a physics-based model and neuro-fuzzyevolution has been described and applied for modelling thermo-mechanical pro-cessing of aluminium alloys in [41]. The same authors later sketch a similarGP-based approach in [42]. This work is very similar to our own work andtherefore merits a more detailed discussion. While the main motivation is thesame, there are several important differences. Abbod et al. [42] used a simplerphysics-based model for flow stress, they only predict three relevant points inthe stress-strain curves (steady-state flow stress, the relaxation stress, and therelaxation strain) instead of the full curve, and they first fit neuro-fuzzy modelsto the data and only later used GP to find short equations that predict the out-put of those models. In contrast to our approach the resulting GP models arenot directly linked to the physics-based flow stress model. Instead they used pre-calculated features and sub-expressions derived from the physics-based modelwithin GP to produce similar expressions. On the other hand Abbod et al.[42] modelled multiple aluminium alloys by including effects caused by differentalloying elements while we focus only on a single alloy.

Versino et al. [43] have described different methods for physics-informed SRfor modelling flow stress. The methods include addition of artificial data pointsto improve extrapolation, constraints (e.g. to force models to be non-negative),seeding of a GP population with initial solutions based on physics-based models,and user-defined features using building blocks derived from the physics-basedmodels (e.g. non-linear transformations of input variables). Seeding GP withthe physics-based model is similar to our approach but does not guarantee thatthe evolved model has the same structure as the physics-based model. In theirconclusions Versino et al. [43] state: [...] model development can [again] beexpertly guided by choosing appropriate building blocks, avoiding functions thatmight introduce excessive numerical issues. At the same time [...] symbolic re-gression presents clear limits. When no experimental data or expert knowledgeis available, the behavior of obtained models is highly unpredictable, and unlikelyto be rooted in solid physics. [...] Moreover, symbolic regression will probably re-turn completely different models for different materials, limiting the re-usabilityof a result. Additionally, as EAs are stochastic in nature, there is no guaranteethat two runs of the algorithm on the same dataset will provide exactly the same

4

results, introducing reproducibility problems. We try to partially alleviate theseissues by extending the physics-based model with GP. This ensures that at leastthe core of the model remains unchanged.

Scientific machine learning (SciML) [44] is a recently popular area of re-search concerned with the application of machine learning techniques for scien-tific models. The foundational research themes of SciML are domain-awarenessthrough integration of physical principles, interpretability, and robustness [44].SR is particularly suited for the integration of physical principles and offers thepossibility to produce interpretable models which can be integrated easily intoexisting software frameworks making it a particularly interesting approach forSciML tasks.

In our own work [8] we have used GP and SR to produce purely empiricalmodels for the stress-strain curve as well as prediction models for the calibrationparameters of a physics-based mean dislocation density-based model (MD2M)that can be use to calculate the stress-strain curve. The model describes materialflow behavior due to applied thermo-mechanical loading and can be used tosimulate the distribution of stresses within a metallic object when it is deformed(cf. [30]). It has a number of parameters including three calibration parameterswhich allow us to adjust the model so that it matches observed properties ofa certain material. The calibration parameter values must be determined byfitting the model to measurements from an experiment with a sample of thegiven material. Section 4.2 gives a detailed description of the model.

In the current work we use the same data and the same MD2M, give adetailed description of the (explicit) modelling approach that we have usedpreviously, describe an improved (implicit) approach for extending the MD2Musing GP, and compare the results of both approaches.

3. Assumptions and Objectives

We assume that the constitutive model which should be improved has severalknown parameters (including physical constants) as well as calibration param-eters that have to be fit to measurements.

The goal is to develop a machine learning method that shall allow to findshort interpretable formulas through which calibration parameters can be esti-mated from other known parameters. The method shall allow us to extend orimprove the given white-box model by adapting the set of equations automati-cally using measurements from experiments and machine learning.

This may improve the knowledge of the modelled system or process becausethe connection between calibration parameter values and other parameters isexplicitly established and may be interpreted. Another motivation is that thenumber of experiments required for calibration can potentially be reduced. Arequirement is that the identified formulas produce reliable outputs for interpo-lation and ideally even for extrapolation.

5

φ⋅ : 0.001 φ⋅ : 0.01 φ⋅ : 0.1 φ⋅ : 1 φ⋅ : 10

0.00 0.01 0.02 0.03 0.04 0.050.00 0.02 0.04 0.06 0.080.00 0.05 0.10 0.15 0.20 0.0 0.1 0.2 0.0 0.1 0.2 0.3

0

25

50

75

φ

Kf

Temp

350

375

400

425

450

475

500

Figure 1: Stress-strain curves acquired from the hot-compression tests. Only measurementsup to the maximum stress value are used for modelling.

4. Background

4.1. Data Collection

To generate a data set, a series of hot compression tests was conducted forthe aluminum alloy AA6082. The cylindrical samples were compressed up toa strain of 0.7 at various temperatures and average strain rates as summarizedin Table 1. Invalid measurements at the beginning and end of a test when themachine resets were removed. For each set of deformation parameters (T andϕ), two identical tests were performed and the average values were calculated.

The stress and strain values were derived during the measurement using thefollowing calculation formulas:

kfi =Fi

Acsi

, Acsi =

πd20L0

4Li, ϕi = ln

L0

Li, ϕi =

dϕ

dt, (1)

where i indexes subsequent measurements, kfi is the stress, Fi is the measuredforce, Acs

i is the actual cross section of the sample, ϕi is the strain, L0 is theinitial length at the start of the deformation segment, Li is the actual samplelength, ϕ is the strain rate, dt is the time difference between two measurementpoints, and d0 is the initial sample diameter.

Figure 1 shows the processed stress-strain data. The same data were usedin [8] and [32].

4.2. Constitutive Model

Constitutive models describe the material flow behavior due to applied thermo-mechanical loading and are a core of numerical simulations of material defor-mation processes. Typically, a constitutive model is represented by a sum of aninitial yield (threshold) stress, σy, and a strain-dependent part, σp. While thethreshold stress is usually a guess value, the strain-dependent stress might bewritten as a power law-form or in terms of material internal state variables. The

6

latter approach is based on physical laws and therefore easily interpretable. Inthe present work, we use a constitutive model which is based on the evolutionof mean dislocation density, ρ (cf. [45]).

σ = σy +MGb

[√ρ

2+

1

δ

](2)

Here M is the Taylor factor, G is the shear modulus, b is the norm of theBurgers vector and δ is a mean sub-grain size. By deformation at a temperature,T , and a strain rate, ϕ, the dislocation density will change according to thefollowing equation:

dρ =M√ρ

bAϕdt− 2BM

dannb

ρMϕdt− 2CDGb3

kBT

[ρ2 − ρ2eq

]dt (3)

The first term describes the increase of dislocations and the next two termscorrespond to the dislocations reduction. The first recovery process happenswhen two antiparallel dislocations come to a critical distance, dann. The secondrecovery process is thermally-activated and controlled by a self-diffusion alongthe dislocations, D. When the processing conditions allow, the material canrecover down to an equilibrium state described by the equilibrium dislocationdensity, ρeq.

The model contains ten physical parameters (e.g. the Boltzmann constant,kB) and three calibration parameters A,B,C. The calibration parameters de-pend on the material, its state and deformation conditions, but these dependen-cies are not fully understood. Therefore, the parameters A,B and C have tobe fit to measurements which are collected through testing in a lab. However,when the actual manufacturing parameters (e.g. deformation rate) can not bereached in the lab-scale testing environment, the ability to extrapolate thesevalues becomes crucial.

From inspection of the white-box model it can be determined that A,B, andC must be positive and B and C depend on the scale of A because the relativecontributions of the terms must be of similar size. To simplify optimization andprediction of the parameters we use an alternative parameterization u, v, w with

A−1 = exp(u), B = exp(v)A−1, C = exp(w)A−1, (4)

and limit the search space to u ∈ [−15, 0], v ∈ [−15, 15]w ∈ [−15, 0]. Thetransformation stretches the search space non-linearly whereby the space be-comes exponentially larger as A, B, or C approach zero. The value for A cannotreach zero. This guarantees a physically feasible solutions. The domain foru, v, w is set based on the range of values which are plausible for the domainexperts.

5. Methods

We describe and compare two methods – implicit and explicit – for extendingthe MD2M. The explicit method shown in Figure 2 has two phases. First the

7

MD2M parameters are fit to measurements using a global optimizer (CMAES)which produces an optimal parameter vector for each of the hot compressiontests. The optimized parameters for all tests are collected into a data set andused in the second phase for GP to produce expressions for each of the param-eters through SR.

The implicit method shown in Figure 3 directly evolves extensions to theMD2M using GP. Each individual consists of three symbolic expression treeswhich are used instead of the calibration parameters. Fitness evaluation runssimulations to produce predictions for all tests and calculates the error to themeasurements.

The methods are described in detail in Sections 5.2 and 5.3. Prior to that wespecify how the accuracy of the calibrated and the hybrid MD2M is quantified.

5.1. Quantification of Model Accuracy

We measure model accuracy using the sum of mean of squared errors (SMSE)over all tests for the training and testing set separately.

The measurement frequency is the same for tests with different strain rateswhich implies that the data set has a variable number of measurements for eachtest. A simulation run produces outputs kf(ϕ) with a much higher resolutionfor ϕ. From these values we keep only the points with matching measurementskf(ϕ) and sum up the squared errors. The metric for model accuracy is the sumover all tests of the mean of squared errors (SMSE). This puts equal weight oneach test even when the number of measurements differs over the tests.

MSE(kf, kft, ϕt) =1

nt

nt∑i=1

(kf(ϕt,i)− kft(ϕt,i)

)2(5)

SMSE =∑

t∈testsMSE(kf, kft, ϕt) (6)

kf(ϕ) are the filtered points from the simulation and ϕt and kft the measure-ments from test t. All vectors have nt elements. We do not normalize the MSEvalues for the tests because the target values are all on the same scale (seeFigure 1) and we aim to reduce absolute not relative errors of predictions.

We use two subsets of the data for training and testing of models and assigndata from one test completely either to the training set or to the testing setusing a systematic partitioning scheme [32].

5.2. Explicit Method: Parameter Optimization and Symbolic Regression

Figure 2 shows the workflow of the explicit method. The constitutive modelhas three calibration parameters (u, v, w) as well as two known parameters (tem-

perature T , and strain rate ϕ). Simulation produces the stress-strain cruve kf(ϕ)which is returned as a table.

First, we use covariance matrix adaptation evolution strategy (CMAES)[46] to optimize the calibration parameters for each test in the training set. The

8

Table 1: Assignment of tests for training and testing models. The training data is split intofour partitions with six tests for cross-validation.

ϕ0.001 0.01 0.1 1 10

Tem

per

atu

re[◦C

] 350 Test 4 3 Test 2375 1 Test 4 3 Test400 2 1 Test 4 3425 Test 2 1 Test 4450 3 Test 2 1 Test475 4 3 Test 2 1500 1 4 3 Test 2

parameters may depend on each other and we cannot assume that the opti-mization problem is convex which makes parameter fitting difficult [45]. Thus,a derivative-free global optimization method such as CMAES or differentialevolution can be an appropriate choice.

The optimizer may still converge to different solutions. Thus, multiple rep-etitions for the same data set are required. Our goal is to find a functionmapping values of known parameters to calibration parameter values. There-fore, it is important that the global optimizer reliably converges to the samesolution or similar solutions. Otherwise, we will fail to find a solution in thesubsequent steps. The solutions are collected in a data set with best values for(u, v, w) for each test.

Next, we use the data set of optimized (u, v, w) values for SR with GPto produce three formulas for the calculation of u, v, and w from the knownparameters T, ϕ. In this step we recommend to use cross-validation to tune GPparameters. It is essential to find a GP parameterization that reliably producesa good solution because we must select only a single formula for each of u, v,and w. After grid-search for good GP parameters we execute a GP run withthe whole training set for each of the three calibration parameters and combinethe three formulas with the MD2M to produce the hybrid MD2M.

The hybrid MD2M is then used to produce simulation results kf(ϕ) for thetests in the testing set. In Section 6 we analyse the errors for the tests toquantify the predictive accuracy of the hybrid MD2M and compare the explicitwith the implicit method described in the next section.

5.3. Implicit Method: Evolutionary Extension of the MD2M

Figure 3 shows the workflow for the implicit method. In contrast to the ex-plicit method we do not fit the model to training data by parameter optimizationbut instead use tree-based GP to directly evolve the three expression trees rep-resenting the formulas for the calculation of (u, v, w). It is important to notethat the output of the MD2M depends on all three calibration parameters. Thequality of the extended MD2M model can be quantified via the SMSE betweensimulated and the measured stress-strain curves. However, we cannot attributechanges in the SMSE (Eqn. 6) to each of the formulas for (u, v, w). Therefore,

9

Training data T, ϕ, ϕ, kf(Eqn. 1)

CMAES(Table 2)

GP - SymReg(Table 3)

Combine

Physics-based MD2M(Sec. 4.2)

Simulate

Assess Error(Eqn. 5)

Table 5, Figure 5

Test data T, ϕ, ϕ

Test data kf(ϕ)

Data set T, ϕ, u, v, w

Formulas for u, v, w

Hybrid MD2M

Predictions kf(ϕ)

Figure 2: Workflow for the explicit method for a the MD2M with three calibration parame-ters (u, v, w). CMAES is used to optimize parameters for each test and GP is used to findexpressions for the prediction of parameter values.

10

Training data T, ϕ, ϕ, kf(Eqn. 1)

GP(Fig. 4, Tab. 4)

Physics-based MD2M(Sec. 4.2)

Simulate

Assess Error(Eqn. 5)

Table 5, Figure 5

Test data T, ϕ, ϕ

Test data kf

Hybrid model

Predictions kf

Figure 3: Workflow for the implicit method for extending the MD2M. The extensions areencoded as multi-tree GP individuals and fitness evaluation uses simulation of the hybridizedmodel.

we use a multi-tree GP for the implicit method. Each GP individual consists ofthree separate trees for the three formulas. For fitness evaluation the three for-mulas are used to calculate the (u, v, w) values for each test. The SMSE betweensimulation and measurement over all tests is used to determine fitness, wherebyindividuals with smaller error are assigned a higher fitness value. Therefore,individuals with three formulas that work well in combination have a higherfitness and are more likely to be selected.

The pseudo-code for GP algorithm is shown in Figure 4 to highlight howrecombination and mutation act independently on the components of the model.

Crossover between two parent individuals acts on the three components inde-pendently. To produce a child we need to compose three trees from the parents.For each of the three components we first choose randomly whether a crossoveroperation should be performed using the crossover probability parameter. Thechoice is made independently for each component. Without crossover we sim-ply select one of the two trees for this component from the parents randomly.Otherwise a new tree is created using sub-tree crossover. This crossover schemedoes not allow exchange of genetic material between separate components. Werecommend a crossover rate smaller than 100% to allow combination of alreadywell working formulas. Mutation acts on the three components independentlyusing the mutation probability parameter.

The individual with highest fitness for the tests in the training set is selectedas the solution. The solution is then used to produce simulation results kf(ϕ)for the tests in the test set. The predictive error (Eqn. 5) on the test set iscalculated and used for comparing methods (Figure 6).

11

Inputs: Model: MD2M(x, theta) // x: vector of inputs

// theta: parameter vector of length dim

Matrix of inputs: X = (x_t) // t = 1..tests

List of targets: y = (y_t) // t = 1..tests

// each y_t is a time series of measurements

Output: Hybridized model g(x) = MD2M(x, theta: f(x))

P = Init(popSize) // each individual is a vector

// of expressions encoded as trees

for g = 1 .. maxGenerations

fitness = [ Evaluate(MD2M, y, X, individual))

for each individual in P]

P = order P by descending fitness

P_next[1] = P[1]; // copy best individual

for k = 2 .. popSize

p1 = Select(P) // select two parents with tournament selection

p2 = Select(P)

child = new empty vector of dim expressions

for treeIndex = 1 .. dim

if rand() < crossoverProbability

child[treeIndex] = Crossover(p1[treeIndex], p2[treeIndex])

else

child[treeIndex] = rand() < 0.5 ? p1[treeIndex] : p2[treeIndex]

if rand() < mutationProbability

Mutate(child[treeIndex])

end // for all components

P_next[k] = child

end // for population

P = P_next

end

return P[1]; // return individual with best fitness

Figure 4: Pseudo-code for the multi-tree GP algorithm for evolving extensions for the MD2M.Individuals contain multiple expression trees, one for each element of the parameter vector θ.Crossover and mutation act independently on the components of individuals.

Figure 4 shows the pseudo-code for the GP algorithm.

5.4. Algorithm Configuration

5.4.1. Explicit Approach

For the explicit approach we execute 30 independent CMAES runs and use,for each of the three parameters u, v, w, the best values which are then back-transformed to produce values for A,B, and C. The back-transformed variablesare used as target for SR1. The resulting data set has 35 rows, two input vari-ables, and three target variables. The data set is split up into training and testpartitions as shown in Table 1. Parameter settings for CMAES are shown inTable 2.

1We also ran SR experiments to instead predict the transformed parameters u, v, w butfound that the results were significantly worse.

12

Table 2: CMAES parameters

Parameter ValueSearch space u ∈ [−10..0]× v ∈ [−15..15],×w ∈ [−15, 0]Generations 500Pop. size 100Initialization UniformFitness SMSE (Eqn. 6) for simulated stress-strain curveRecombination Log-weighted

The maximum length of symbolic expression trees is selected using 30 inde-pendent repetitions of cross-validation (CV) with four folds on the 24 trainingsamples. The assignment of tests to CV folds is shown in Table 1. The settingwith the smallest median cross-validated root mean squared error (CV-RMSE)is used to train models on the full training set.

The maximum limit for the number of tree nodes is chosen from the set{5, 7, 10, 15, 20, 25, 30, 35, 40}. The settings with best CV-RMSE are 25 nodesfor A, 35 nodes for B, and 20 nodes for C. These are relatively tight limits forGP but grid-search showed that GP started to overfit with larger models. Thiscan be explained by the the small number of data points for training. Modelswith 20 to 35 nodes (before simplification) are relatively easy to interpret2.

For each of the three targets we execute 30 GP runs and return the indi-vidual with highest fitness as the solution. The outputs of all GP models areclamped using target-specific limits as shown in Tables 3 and 4. This ensuresthat the calibration parameter values produced by GP models are physicallyplausible. For instance negative parameter values are physically impossible andthe maximum values depend on the material. For the aluminum alloy used inthis work typical values for A and B are in the range [0..150] and C is in theunit interval. We generate 30 hybrid MD2M by combining the three modelsfrom the i-th SR run for each target and calculate the SMSE for training andtesting sets for Table 5.

5.4.2. Implicit Approach

We use tree-based GP with generational replacement with elitism. As de-scribed above, each individual contains three trees (u, v, w). Trees are limitedto 25 nodes and a maximum depth of 10 for each component. Our GP systeminitializes trees randomly using PTC2 whereby for each leaf it first randomly de-termines the leaf type: variable or parameter. All variable nodes always includea scaling factor sampled randomly from N(0, 1)3. The parameters are sampledrandomly from U(−20, 20). Table 4 shows the GP parameters values.

2In this context it is important to point out that variables nodes in the leafs of treesimplicitly contain a scaling coefficient (ci ∗ xi). This is counted as only one node in this work

13

Table 3: GP parameters for SR as part of the explicit approach.

Parameter ValuePop. size 300Generations 250Max. length A : 25, B : 35, C : 20Max. depth 8Initialization PTC2 [47]Selection Tournament (size 3)Recombination Sub-tree crossover (90% internal nodes)Mutation Probability 15%

Select randomly:For a random parameter: x← x+N(0, 1)For all parameters: x← x+N(0, 1)Change the symbol of a random nodeChange a random variable node

Clamp predictions A : [0..150], B : [0..150], C : [0..1]Fitness Sum of squared errors (for A,B,C predictions)Replacement Generational with one elite.Function set {+,−, ∗,%, exp(x), log(x)}Terminals 50% variables, 50% numeric parameters

Variables: {temp, ϕ, log10(ϕ)}Numeric parameters ∼ U(−20, 20)

6. Results

Figure 5 visualizes the simulation output of the models produced by bothmethods in comparison with the measurements for the eleven hot compressiontests in the testing set. From the 30 independent runs of both methods weselected the model with best training SMSE.

In Figure 5 a systematic deviation for small kf values is apparent for bothmethods. This is a systematic bias of the MD2M. It is not possible to cali-brate the model to improve the simulation results for small kf values even withCMAES. Therefore, this systematic deviation is not caused by GP.

Figure 6 shows Box-plots for the average training and test MSE valuesachieved by both methods over 30 repetitions. The implicit method leads on av-erage to smaller MSE values on the training and testing sets. Additionally, thevariance for the implicit method is much smaller than for the explicit method.The non-parametric Brown-Mood median test (for samples with different vari-ances) indicates to reject the null-hypothesis of equal medians for the training

while it would be counted as three nodes in most other GP systems.3This is the default of the GP system used. This increases the number of parameters in

the SR models and can be helpful or detrimental for fitting. We have not analysed the effectsof removing scaling parameters for variables.

14

Table 4: GP parameter settings for the implicit approach.

Parameter ValuePop. size 5000Generations 250Max. length 25Max. depth 10Initialization PTC2Selection Tournament (size 7)Recombination Probability 30% for each tree of an individual

Sub-tree crossover (90% internal nodes)Mutation Probability 15% for each tree of an individual

Select randomly:For a random parameter: x← x+N(0, 1)For all parameters: x← x+N(0, 1)Change the symbol of a random nodeChange a random variable node

Clamp predictions u : [−15..0], v : [−15..15], w : [−15..0]Fitness SMSE (Eqn. 6)Replacement Generational with one elite.Function set {+,−, ∗,%, exp(x), log(x)}Terminals 50% variables, 50% numeric parameters

Variables: {temp, ϕ, log10(ϕ)}Numeric parameters ∼ U(−20, 20)

●●●

●●

●

●

●

●

●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●

●●

●●

●●

●●●●●●●●●●●●●●

0

20

40

60

80

0 20 40 60 80

kf [MPa] (measured)

kf [M

Pa]

(pr

edic

ted)

●●●

●●

●

●

●

●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●

●●

●●

●●

●●●●●●●●●●●●●●●

0 20 40 60 80

kf [MPa] (measured)

350

400

450

500Temp

Phip

● 0.001

0.01

0.1

1

10

Figure 5: Scatter plots for predictions over measurements for the explicit (left) and the implicit(right) approach. The implicit approach produces a better fit for high stress values.

15

Explicit (test)

Explicit (train)

Implicit (test)

Implicit (train)

0 10 20 30 40 50Avg. MSE

Alg

orith

m

Figure 6: Box-plots for the average MSE over 30 repetitions for training and testing sets.Instead of SMSE we plot the MSE here to compare training and testing performance becausethe number of tests in training and testing sets differ. Not shown: twelve outliers for theexplicit method (training: 2, test: 10).

set (p-value = 0.0001792) and for the testing set (p-value = 1.183 · 10−07).Table 5 shows the median MSE values for each test for the hybrid mod-

els produced by 30 independent runs of the explicit and implicit method. Theimplicit method has smaller MSE values than the explicit method for the train-ing and the testing sets. For comparison the median MSE values achieved byCMAES are shown as a reference. The CMAES runs reliably converged to thesame solutions and give an indication of the best possible MSE values that canbe achieved when using the MD2M. Here we note again that CMAES producesresults only for the measurements that we use as inputs and does not allow usto interpolate directly. However, they can be used for bilinear or bicubic splineinterpolation using the known values as knots.

With the implicit method we found a better model (with on average a smallerSMSE) than with the explicit method. The fit achieved with the explicit methodgets better with increasing strain rates. For the highest strain rate the explicitmethod produces a better fit for 5 of 7 tests.

Equations 7, 8, 9 show the set of equations with best performance on thetest set after algebraic simplification produced by the implicit method. Todetermine A, B, and C the back-transformation as shown in Equations 10, 11,12 is required. Whereby, clamp(x, l, u) returns min(u,max(l, x)). The modelsidentified by GP are non-linear in the input variables T , ϕ and very short. Mostof the parameters are linear which facilitates interpretation of the formula. Theformulas identified by the explicit approach (not shown) have similar complexitybecause the tree size restrictions for both algorithms were similar.

16

Table 5: Median MSE values over 30 runs for the explicit and implicit method for the trainingand testing sets. The MSE values for CMAES are shown as a reference value as they areindicative of the best achievable value that for the MD2M.

Temp log10 ϕ CMAES Explicit Implicit

Tra

inin

g

375 −3 9.44 22.23 11.48400 −3 12.70 21.11 14.63450 −3 14.99 28.55 18.31475 −3 14.04 33.87 17.19500 −3 14.94 26.34 21.54350 −2 7.76 10.44 10.58400 −2 6.23 10.70 8.33425 −2 8.38 12.02 9.64475 −2 10.71 13.92 12.22500 −2 13.37 23.63 16.20350 −1 7.88 10.41 9.79375 −1 9.61 14.73 11.56425 −1 6.31 8.65 8.03450 −1 3.85 5.66 4.79500 −1 7.88 9.63 11.09375 0 7.82 11.27 10.04400 0 6.54 7.88 7.94450 0 4.89 5.25 7.29475 0 4.45 4.76 6.57350 1 43.58 50.92 50.81400 1 38.07 38.54 38.57425 1 38.86 39.14 40.08475 1 31.05 31.41 33.53500 1 18.59 20.49 22.60Avg 14.25 19.23 16.78

Tes

t

350 −3 17.07 41.24 21.59425 −3 14.43 21.81 16.76375 −2 5.92 9.38 9.21450 −2 8.99 11.44 10.50400 −1 5.07 5.83 6.43450 −1 2.38 2.86 3.53350 0 9.94 81.95 15.11425 0 6.12 8.52 7.66500 0 3.78 4.68 6.22375 1 42.98 45.62 44.47450 1 32.77 32.95 34.65Avg 13.59 24.21 16.01

17

Figure 7: Predicted values for the calibration parameters A, B, and C.

u(T, ϕ) =− log(0.11T + 3.734ϕ− 17.34)− 0.069 log(ϕ) (7)

v(T ) =− 4.35 log(T ) + 4.938 (8)

w(T, ϕ) =− 13.708 log(ϕ)T−1 − 10205 log(ϕ)T−2 + 13675T−2

+ 0.777 log(ϕ)− 0.8657(9)

A(T, ϕ) = exp (−clamp(u(T, ϕ),−15, 0)) (10)

B(T, ϕ) = exp (clamp(v(T ),−15, 15))A(T, ϕ)−1 (11)

C(T, ϕ) = exp (clamp(w(T, ϕ),−15, 0))A(T, ϕ)−1 (12)

Figure 7 shows the predicted values for the tested range of temperatures andstrain rates and visualizes the dependencies of the two parameters.

18

7. Discussion

We showed that it is possible to evolve short formulas for the prediction ofcalibration parameters of a physics-based constitutive model using GP. However,there are several caveats of our work that must be considered and should beaddressed in future work.

7.1. Interpolation and Extrapolation

We have carefully selected data sets from hot-compression test for the testingset to systematically cover the whole range of temperatures and strain rates (cf.Table 1). However, from the experiments in the testing set only the dataset withsmallest temperature and strain rate requires extrapolation. For all other exper-iments it is possible to interpolate based on the next-smaller and the next-largervalue. Therefore, our testing results mainly express interpolation capabilities.For future work it would be interesting to explicitly study extrapolation capa-bilities. This can be achieved for instance by using only the inner experimentsin Table 1 for training.

7.2. Comparison to Classical Interpolation Methods

We have used GP because it options the potential to find short non-linearformulas that we can integrate easily into the MD2M. However, if the maininterest is interpolation to reduce the number of experiments required for modelcalibration, well-known classical methods such as bi-linear or bicubic splineinterpolation could be used as well because the calibration parameters onlydepend on two input variables in this example. In our experiments we havenot compared our approach to such classical approaches. Therefore, we do notknow whether the GP-based approaches produce better interpolation resultsthan classical approaches. We leave comparison of our approach to classicalinterpolation approaches for future work.

Subjectively, the equations produced by GP are attractive compared to anon-parametric interpolation map as produced by linear or cubic spline inter-polation because they have only a few parameter and are easily interpretable.

7.3. Runtime and Convergence

For the explicit method the runtime for one CMAES run was approximately6 hours (10 minutes for each of the 35 tests) and the runtime for SR was just afew minutes as there were only 24 training points. The implicit method requiredapproximately 45 days single-core because each GP fitness evaluation requires aseparate simulation run. Considering the differences in runtime, the comparisonbetween the two approaches is not fair as it is not surprising that the methodwith 100 times longer runtime achieves better results.

We cannot make a definitive statement whether more runtime for the ex-plicit approach would improve the results because we have not tested longerruns. However, we analyzed the convergence of CMAES runs and observed thatit reliably converged to the same points with the parameters that we have cho-sen. Therefore, we believe it is unlikely that much larger population sizes or

19

more generations would have improved the results for CMAES. The runtime ofCMAES is the main contributor to the total runtime for the explicit approachso increasing the number of generations for SR would increase total runtimeonly by a small percentage.

Additionally, an assumption for the explicit approach is that there is a globaloptimum for the calibration parameters that can be found reliably with CMAES.If there are many local optima with almost equal qualities scattered in a largepart of the search space then the explicit approach will be limited because itwill be difficult to find a smooth functional mapping with SR which reliablyproduces the parameter values identified by CMAES. In our experiments forthe MD2M we found that CMAES reliably converged to the same optima.

7.4. Combination of Models

The implicit approach directly evolves a set of models that produces a goodprediction in combination while the explicit approach evolves each model inde-pendently. To create 30 hybrid MD2M for evaluation, we used the three modelsfrom the first run for the first combined model, the three models from the secondrun for the second combined model and so on. The finally selected combinedmodel is the combination with the best performance (SNMSE) on the train-ing set. However, from the 3*30 individual models we could also create all 90combinations with minor overhead. This could potentially increase the chanceto build and select a good combination of models but might also increase thechance for overfitting.

7.5. Choice of Fitness Function for SR

One potential reason for the worse performance of the explicit method isthat the chosen fitness function for SR could be inappropriate. The implicitmethod uses the fit for the simulated stress-strain curve as a fitness criterionand therefore directly optimizes the error that we measure for the test set. Withthe explicit method we build formulas to minimize the error to the (u, v, w)values produced by CMAES. In our experiments we used the sum of squarederrors as the fitness function for this SR step based on the assumption that aformula with smaller squared error for (u, v, w) also leads to a better fit for the

simulated stress-strain curve kf(ϕ). Based on our results we believe that thisassumption does not hold. For example it may be necessary to predict a valuemore accurately as it approaches zero while for larger target values we may alsoallow larger errors. In this case it could be better to minimize the relative error.This can be considered an advantage of the implicit approach because it freesus from the burden to select an appropriate fitness function for the intermediateSR step. It would be worthwhile to study different fitness functions in futurework to gain a better understanding about the effect of this choice on the overallperformance of the approach.

20

7.6. Implementation Effort

The explicit method is easier to implement than the implicit method becausethe separate steps in the workflow only require to call readily available and well-tuned software components (CMAES, SR tools). For the implicit method it isnecessary to use a multi-tree GP system or to adapt a GP system accordingly.Additionally, for fitness evaluation the simulation model has to be implementedor connected to the GP system. Most GP systems however allow this kind ofextension.

8. Conclusions

The results for the flow stress model show that the hybrid models producedby the implicit method have significantly better predictive accuracy on averagethan the models produced by the explicit method. Additionally, implicit methodhas smaller variance.

The critical step in the explicit method is the combination of the SR modelsfor the calibration parameters with the physics-based model. Even though theindividual SR models were able to predict the calibration parameters accuratelyfor training and testing partitions, they did not perform well in combination.The implicit method instead directly optimizes the fit of the hybrid MD2M andis able to evolve a combination of short formulas.

Correspondingly, we recommend the implicit approach because it allows end-to-end fitting of the simulation model and does not have multiple intermediatesteps of model selection where an appropriate fitness function has to be chosen.However, the implicit approach comes at a significantly greater computationalexpense because more evaluations of the simulation model are required. To im-prove the runtime, future work could try to either improve the explicit approachor try to incorporate surrogate models instead of the simulation model.

References

[1] J. Sjoberg, Q. Zhang, L. Ljung, A. Benveniste, B. Delyon, P.-Y. Gloren-nec, H. Hjalmarsson, A. Juditsky, Nonlinear black-box modeling in systemidentification: a unified overview, Automatica 31 (1995) 1691–1724.

[2] M. Von Stosch, R. Oliveira, J. Peres, S. F. de Azevedo, Hybrid semi-parametric modeling in process systems engineering: Past, present andfuture, Computers & Chemical Engineering 60 (2014) 86–101.

[3] J. R. Koza, M. A. Keane, J. P. Rice, Performance improvement of machinelearning via automatic discovery of facilitating functions as applied to aproblem of symbolic system identification, in: 1993 IEEE InternationalConference on Neural Networks, volume I, IEEE, San Francisco, USA, 1993,pp. 191–198.

[4] J. R. Koza, Genetic Programming: On The Programming of Computersby Means of Natural Selection, volume 1, MIT press, 1992.

21

[5] R. Poli, W. B. Langdon, N. F. McPhee, A field guide to geneticprogramming, Published via http://lulu.com and freely available athttp://www.gp-field-guide.org.uk, 2008. (With contributions by J. R.Koza).

[6] M. Schmidt, H. Lipson, Distilling free-form natural laws from experimentaldata, Science 324 (2009) 81–85. doi:10.1126/science.1165893.

[7] Y. Wang, N. Wagner, J. M. Rondinelli, Symbolic regression in materialsscience, MRS Communications 9 (2019) 793–805.

[8] E. Kabliman, A. H. Kolody, J. Kronsteiner, M. Kommenda, G. Kron-berger, Application of symbolic regression for constitutive modeling ofplastic deformation, Applications in Engineering Science 6 (2021) 100052.doi:https://doi.org/10.1016/j.apples.2021.100052.

[9] W. B. Langdon, Genetic programming and evolvable machines at 20, Ge-netic Programming and Evolvable Machines 21 (2020) 205–217. doi:10.1007/s10710-019-09344-6.

[10] K. N. Sastry, Genetic Algorithms and Genetic Programming for MultiscaleModeling: Applications in Materials Science and Chemistry and Advancesin Scalability, Ph.D. thesis, University of Illinois at Urbana-Champaign,Urbana IL., USA, 2007.

[11] M. Schoenauer, M. Sebag, F. Jouve, B. Lamy, H. Maitournam, Evolution-ary identification of macro-mechanical models, in: P. J. Angeline, K. E.Kinnear, Jr. (Eds.), Advances in Genetic Programming 2, MIT Press, Cam-bridge, MA, USA, 1996, pp. 467–488. doi:10.7551/mitpress/1109.003.0030.

[12] M. Sebag, M. Schoenauer, H. Maitournam, Parametric and non-parametricidentification of macro-mechanical models, in: D. Quagliarella, J. Periaux,C. Poloni, G. Winter (Eds.), Genetic Algorithms and Evolution Strategiesin Engineering and Computer Sciences, John Wiley, 1997, pp. 327–340.

[13] M. Brezocnik, J. Balic, L. Gusel, Artificial intelligence approach to de-termination of flow curve, Journal for Technology of Plasticity 25 (2000)1–7.

[14] M. Brezocnik, J. Balic, Z. Kampus, Modeling of forming efficiency us-ing genetic programming, Journal of Materials Processing Technology 109(2001) 20–29. doi:10.1016/S0924-0136(00)00783-4.

[15] M. Brezocnik, J. Balic, K. Kuzman, Genetic programming approach todetermining of metal materials properties, Journal of Intelligent Manufac-turing 13 (2002) 5–17. doi:10.1023/A:1013693828052.

22

[16] K. Sastry, D. D. Johnson, D. E. Goldberg, P. Bellon, Genetic program-ming for multiscale modeling, International Journal for Multiscale Com-putational Engineering 2 (2004) 239–256. doi:10.1615/IntJMultCompEng.v2.i2.50.

[17] M. Brezocnik, L. Gusel, Predicting stress distribution in cold-formedmaterial with genetic programming, International Journal of Ad-vanced Manufacturing Technology 23 (2004) 467–474. doi:10.1007/s00170-003-1649-3.

[18] M. Brezocnik, M. Kovacic, L. Gusel, Comparison between genetic algorithmand genetic programming approach for modeling the stress distribution,Materials and Manufacturing Processes 20 (2005) 497–508. doi:10.1081/AMP-200053541.

[19] L. Gusel, M. Brezocnik, Modeling of impact toughness of cold formedmaterial by genetic programming, Computational Materials Science 37(2006) 476–482. doi:10.1016/j.commatsci.2005.11.007.

[20] X.-T. Feng, B.-R. Chen, C. Yang, H. Zhou, X. Ding, Identification ofvisco-elastic models for rocks using genetic programming coupled with themodified particle swarm optimization algorithm, International Journal ofRock Mechanics and Mining Sciences 43 (2006) 789–801. doi:10.1016/j.ijrmms.2005.12.010.

[21] H. Shahnazari, Y. Dehnavi, A. H. Alavi, Numerical modeling of stress-strain behavior of sand under cyclic loading, Engineering Geology 116(2010) 53–72. doi:10.1016/j.enggeo.2010.07.007.

[22] H. Shahnazari, Y. Dehnavi, A. H. Alavi, The next-generation constitutivecorrelations for simulation of cyclic stress-strain behavior of sand, Jour-nal of Civil Engineering and Management 21 (2015) 31–44. doi:10.3846/13923730.2013.802726.

[23] M. Kovacic, M. Brezocnik, R. Turk, Modeling of hot yield stress curves forcarbon silicon steel by genetic programming, Materials and ManufacturingProcesses 20 (2005) 543–551. doi:10.1081/AMP-200053572.

[24] Y. C. Lin, F.-Q. Nong, X.-M. Chen, D.-D. Chen, M.-S. Chen, Mi-crostructural evolution and constitutive models to predict hot deforma-tion behaviors of a nickel-based superalloy, Vacuum 137 (2017) 104–114.doi:10.1016/j.vacuum.2016.12.022.

[25] M. Kovacic, K. Mackosek, A. Mihevc, T. Marolt, Modeling of cracks pres-ence in steel after hot rolling, in: S. Brdarevic (Ed.), 7th Research/ExpertConference with International Participations, QUALITY 2011, Faculty ofMechanical Engineering, University of Zenica, Neum, Bosnia and Herze-govina, 2011, pp. 569–572.

23

[26] L. Gusel, M. Brezocnik, Application of genetic programming for modellingof material characteristics, Expert Systems with Applications 38 (2011)15014–15019. doi:10.1016/j.eswa.2011.05.045.

[27] A. H. Gandomi, G. J. Yun, A. H. Alavi, An evolutionary approach formodeling of shear strength of RC deep beams, Materials and Structures 46(2013) 2109–2119. doi:10.1617/s11527-013-0039-z.

[28] A. H. Gandomi, G. J. Yun, Coupled SelfSim and genetic programming fornon-linear material constitutive modelling, Inverse Problems in Science andEngineering 23 (2015) 1101–1119. doi:10.1080/17415977.2014.968149.

[29] J. R. Mohanty, T. K. Mahanta, A. Mohanty, D. N. Thatoi, Predictionof constant amplitude fatigue crack growth life of 2024 T3 Al alloy withR-ratio effect by GP, Applied Soft Computing 26 (2015) 428–434. doi:10.1016/j.asoc.2014.10.024.

[30] E. Kabliman, A. H. Kolody, M. Kommenda, G. Kronberger, Prediction ofstress-strain curves for aluminium alloys using symbolic regression, AIPConference Proceedings 2113 (2019) 180009. doi:10.1063/1.5112747.

[31] A. Cevik, I. H. Guzelbey, A soft computing based approach for the pre-diction of ultimate strength of metal plates in compression, EngineeringStructures 29 (2007) 383–394. doi:10.1016/j.engstruct.2006.05.005.

[32] G. Schutzeneder, Die Modellierung von Spannungs-Dehnungs-Kurven furdie Aluminiumlegierung 6082 mit Methoden des maschinellen Lernens,Master’s thesis, University of Applied Sciences Upper Austria, School ofInformatics, Communication, and Media, Hagenberg, 2020.

[33] M. Pala, Genetic programming-based formulation for distortional buck-ling stress of cold-formed steel members, Journal of Constructional SteelResearch 64 (2008) 1495–1504. doi:10.1016/j.jcsr.2008.01.018.

[34] B. Podgornik, V. Leskovsek, M. Kovacic, J. Vizintin, Residual stress fieldanalysis and prediction in nitrided tool steel, Materials and ManufacturingProcesses 26 (2011) 1097–1103. doi:10.1080/10426914.2010.525573.

[35] V. Vijayaraghavan, A. Garg, K. Tai, L. Gao, Thermo-mechanical modelingof metallic alloys for nuclear engineering applications, Measurement 97(2017) 242–250. doi:10.1016/j.measurement.2016.11.003.

[36] M. Rezania, Evolutionary Polynomial Regression Based Constitutive Mod-elling and Incorporation in Finite Element Analysis, Ph.d. in GeotechnicalEngineering, School of Engineering, Computing and Mathematics, Univer-sity of Exeter, UK, 2008.

[37] A. Ahangarasr, Application of an Evolutionary Data Mining Technique forConstitutive Modelling of Geomaterials, Ph.D. thesis, University of Exeter,UK, 2012.

24

[38] J. Lin, J. Cao, D. Balint, Development and determination of unified vis-coplastic constitutive equations for predicting microstructure evolution inhot forming processes, International Journal of Mechatronics and Manu-facturing Systems 4 (2015) 387–401.

[39] F. Tancret, Computational thermodynamics, Gaussian processes andgenetic algorithms: combined tools to design new alloys, Modellingand Simulation in Materials Science and Engineering 21 (2013) 045013.doi:10.1088/0965-0393/21/4/045013.

[40] M. Mulyadi, M. A. Rist, L. Edwards, J. W. Brooks, Parameter optimisationin constitutive equations for hot forging, Journal of Materials ProcessingTechnology 177 (2006) 311–314.

[41] M. Abbod, D. Linkens, Q. Zhu, M. Mahfouf, Physically based andneuro-fuzzy hybrid modelling of thermomechanical processing of aluminiumalloys, Materials Science and Engineering: A 333 (2002) 397 – 408.doi:10.1016/S0921-5093(01)01873-1.

[42] M. F. Abbod, M. Mahfouf, D. A. Linkens, C. M. Sellars, Evolutionarycomputing for metals properties modelling, in: THERMEC 2006, volume539 of Materials Science Forum, Trans Tech Publications, Vancouver, 2006,pp. 2449–2454. doi:10.4028/www.scientific.net/MSF.539-543.2449.

[43] D. Versino, A. Tonda, C. A. Bronkhorst, Data driven modeling of plasticdeformation, Computer Methods in Applied Mechanics and Engineering318 (2017) 981–1004.

[44] N. Baker, F. Alexander, T. Bremer, A. Hagberg, Y. Kevrekidis, H. Najm,M. Parashar, A. Patra, J. Sethian, S. Wild, et al., Workshop report onbasic research needs for scientific machine learning: Core technologies forartificial intelligence, Technical Report, USDOE Office of Science (SC),Washington, DC (United States), 2019.

[45] K. Domkin, Constitutive models based on dislocation density : formulationand implementation into finite element codes, Ph.D. thesis, Lulea TekniskaUniversitet, 2005.

[46] N. Hansen, S. D. Muller, P. Koumoutsakos, Reducing the time complexityof the derandomized evolution strategy with covariance matrix adaptation(CMA-ES), Evolutionary Computation 11 (2003) 1–18.

[47] S. Luke, Two fast tree-creation algorithms for genetic programming, IEEETransactions on Evolutionary Computation 4 (2000) 274–283. doi:10.1109/4235.873237.

25