Biotechnology Letters 25: 1837–1842, 2003.© 2003 Kluwer Academic Publishers. Printed in the Netherlands.

1837

Optimization of a fermentation medium using neural networks andgenetic algorithms

Yuko Nagata & Khim Hoong Chu∗Department of Chemical and Process Engineering, University of Canterbury, Private Bag 4800, Christchurch,New Zealand∗Author for correspondence (Fax: +64-3-3642063; E-mail: khim.chu@canterbury.ac.nz)

Received 5 September 2003; Accepted 8 September 2003

Key words: artificial neural network, genetic algorithm, medium optimization, response surface methodology

Abstract

Artificial neural networks and genetic algorithms are used to model and optimize a fermentation medium for theproduction of the enzyme hydantoinase by Agrobacterium radiobacter. Experimental data reported in the literaturewere used to build two neural network models. The concentrations of four medium components served as inputsto the neural network models, and hydantoinase or cell concentration served as a single output of each model.Genetic algorithms were used to optimize the input space of the neural network models to find the optimum settingsfor maximum enzyme and cell production. Using this procedure, two artificial intelligence techniques have beeneffectively integrated to create a powerful tool for process modeling and optimization.

Introduction

The performance of fermentation processes is affectedby numerous factors, including pH, temperature, ionicstrength, and the concentrations of medium com-ponents. Since the effects of these factors are verycomplex with possible interactions among the variousfactors, they are often characterized through experi-mentation. To account for the interactive influencesof different factors and to reduce the number of la-borious experiments, statistical techniques such asresponse surface methodology (RSM) are increasinglybeing used. RSM seeks to identify and optimize sig-nificant factors with the purpose of determining whatlevels of the factors maximize the response (productyield or productivity). It uses statistical experimentaldesigns to develop empirical models that relate aresponse (dependent variable) to some factors (inde-pendent variables). The literature is replete with stud-ies that demonstrate the effectiveness of RSM whichis essentially a collection of statistical and regressiontechniques.

In recent years, a limited number of studies haveinvestigated the possibility of using non-statistical

techniques, such as artificial intelligence, to optim-ize fermentation processes. These studies have beenreviewed by Kennedy & Krouse (1999) and Weuster-Botz (2000). Among the various artificial intelligencetechniques, genetic algorithms, a powerful stochasticsearch and optimization technique, have received con-siderable attention. Genetic algorithms can be usedto optimize fermentation conditions without the needof statistical designs and empirical models. Such anapproach has recently been used to optimize the pro-duction of polyols (Patil et al. 2002), the productionof xylitols (Baishan et al. 2003), and a culture me-dium for fed-batch culture of insect cells (Marteijnet al. 2003). Although the use of genetic algorithmsfor fermentation optimization has proven to be effect-ive, the methodology does not store the informationgenerated at each stage of the optimization process.In contrast, RSM produces a model, albeit empirical,that mathematically describes the relationship existingbetween the independent and dependent variables ofthe process under consideration. The resulting modelcan be used for optimization as well as analysis ofthe sensitivity of the model output against each inputvariable. The most widely used approximating func-

1838

tions in the model building stage of RSM are quadraticpolynomials.

From the perspective of process modeling, neuralnetworks provide a mathematical alternative to thequadratic polynomial for representing data derivedfrom statistically designed experiments. Neural net-works are universal function approximators under cer-tain general conditions (Hornik et al. 1989). Thisability to approximate functions to any desired degreeof accuracy makes them attractive for use as empiricalmodels in response surface analysis. The input spaceof a neural network model may be optimized usinggenetic algorithms. An attractive feature of the geneticalgorithm is that it does not require continuity or dif-ferentiability of the objective function. A recent studyhas investigated the use of neural network and ge-netic algorithm to model and optimize the productionof gluconic acid from glucose (Cheema et al. 2002).However, no comparison with RSM was made as theexperiments were not based on statistical design. Liuet al. (1999) found that neural networks outperformedquadratic polynomials in the modeling of a fermenta-tion process. However, the neural networks were notused in the optimization step. In this paper we reporton a study of the use of neural network and geneticalgorithm to accomplish objectives similar to thoseof RSM. A comparison of the hybrid approach andthe standard RSM approach and their application topredict optimum conditions for a fermentation processreported by Achary et al. (1997) is presented.

Response surface methodology

Response surface methodology combines statisticalexperimental designs and empirical model building byregression for the purpose of process or product optim-ization. Statistical experimental design is a powerfulmethod for accumulating information about a processrapidly and efficiently from a small number of exper-iments, thereby minimizing experimental costs. Anempirical model is then used to relate the responseof the process to some independent variables. Thisusually entails fitting a quadratic polynomial to theavailable data by regression analysis. The general formof the quadratic polynomial is:

y = b0 + �bixi + �bix2i + �bij xixj + e, (1)

where y is the predicted response, the xi and xj termsstand for independent variables, b0 is the intercept, the

Table 1. Independent variables used and experimental designlevels.

Variables (g l−1)

Coded Molasses NH4NO3 NaH2PO4 MnCl2level (x1) (x2) (x3) (x4)

−2 7.5 0.75 7.5 0

−1 10 1 10 0.025

0 12.5 1.25 12.5 0.05

1 15 1.5 15 0.075

2 17.5 1.75 17.5 0.1

bi and bij terms are regression coefficients, and e is arandom error component.

A near-optimum point can then be deduced by cal-culating the derivatives of Equation (1) or by mappingthe response of the model onto a surface contour plot.There are numerous commercial software packagesthat facilitate the use of the quadratic polynomial forprocess modeling and optimization.

Achary et al. (1997) fitted Equation (1) to theirexperimental data obtained from a central compositedesign for the production of the enzyme hydantoinaseby Agrobacterium radiobacter. The independent vari-ables are the concentrations of four medium compon-ents (x1, x2, x3, and x4). The experimental designlevels and concentration ranges of the four independ-ent variables are listed in Table 1. The dependentvariables are the enzyme and biomass concentrations(y1 and y2). The best fit regression equations for thesetwo dependent variables are listed in Table 2. Thegoodness of fit of the quadratic polynomial is ex-pressed by the coefficient of determination, R2. Thecloser the value of R2 is to 1, the better is the correl-ation between the observed and predicted values. Thevalues of R2 listed in Table 2 indicate a fair degreeof correlation between the observed and predicted val-ues; about 80% of the variability in the two responsescan be explained by the quadratic polynomial model.Contour plots obtained from the regression equationsindicate a local optimum exists for each response inthe area experimentally investigated; a set of valueson the four independent variables that leads to max-imum enzyme or cell mass production. The location ofthis optimum can be obtained by differentiating Equa-tion (2) or (3) with respect to x1-x4 and solving theresulting sets of algebraic equations. The maximumenzyme and cell concentrations reported by Acharyet al. (1997) are 35.39 U ml−1 and 1.69 mg ml−1,

1839

respectively. The combinations of the four independ-ent variables giving these maximum concentrationsare listed in Table 3. Also shown in Table 3 are theoptimum conditions identified by the proposed neuralnetwork-genetic algorithm approach using the samedata set reported by Achary et al. (1997), and theseare discussed later.

Neural network-genetic algorithm approach

A neural network is a mathematical representation ofthe neurological functioning of a brain. It simulatesthe brain’s learning process by mathematically mod-eling the network structure of interconnected nervecells. Because neural networks operate directly oninput–output data, the essential requirement of neuralnetwork modeling is sufficient numbers of data. Aneural network is thus a purely data driven modelmade up of interconnected processing elements calledneurons that are organized in layers. A typical neuralnetwork has an input layer, one or more hidden layer,and an output layer. The neurons in the hidden layer,which are linked to the neurons in the input and out-put layers by adjustable weights, enable the networkto compute complex associations between the inputand output variables. The inputs of each neuron in thehidden and output layers are summed and the result-ing summation is processed by an activation function.The most common choice of activation function isthe sigmoid function. The process of determining theadjustable weights is known as training and it is ana-logous to the process of determining the coefficientsof a polynomial by regression. The weights are ini-tially selected in random and an iterative algorithm isthen used to find the weights that minimize differencesbetween the network-calculated and actual outputs.

The most commonly used algorithm is the back-propagation algorithm. In this training algorithm, theerror between the results of the output neurons andthe actual outputs is calculated and propagated back-ward through the network. The algorithm adjusts theweights in each successive layer to reduce the error.This procedure is repeated until the error between theactual and network-calculated outputs satisfies a pre-specified error criterion. Thus, neural network mod-eling is essentially a curve fit in multidimensionalspace. The text by Baughman & Liu (1995) providesa comprehensive description of the neural networkmodeling approach and its applications in biopro-cessing. In this study two separate neural network

models were constructed to model the fermentationprocess reported by Achary et al. (1997). Each ofthe two neural networks consisted of a single outputneuron (hydantoinase or biomass concentration) andfour input neurons (concentrations of the four mediumcomponents).

The neural network models can be considered asobjective functions for the purpose of optimization.However, using conventional optimization techniquessuch as gradient-based methods to optimize a neuralnetwork model is not a simple task because it is diffi-cult to calculate the derivatives of the model. Geneticalgorithms, which are based on the principles of evol-ution through natural selection, i.e., the survival ofthe fittest strategy, have established themselves as apowerful search and optimization technique to solveproblems with objective functions that are not continu-ous or differentiable. The genetic algorithm exploresall regions of the solution space using a populationof individuals (solutions). Each individual representsa set of independent variables. Initially, a populationof individuals is formed randomly. The fitness of eachindividual is evaluated using an objective function. Inthis work the objective function is the neural networkmodels. Upon completion of the fitness evaluation, ge-netic operations such as mutation and crossover areapplied to individuals selected according to their fit-ness to produce the next generation of individuals forfitness evaluation. This process continues until a nearoptimum solution is found. A complete description ofthe implementation of genetic algorithms and their useas a problem-solving and function optimization tech-nique can be found in the books by Holland (1975) andGoldberg (1989). All of the neural network modelsand genetic algorithms described in this study wereimplemented in Matlab v6.5 and run under the Mi-crosoft Windows NT environment. A modified versionof the genetic algorithm of Houck et al. (1995) wasused.

Results and discussion

Neural network modeling

The first step in implementing a neural network mod-eling approach is to design the topology of the net-work. A number of design parameters affect per-formance. These parameters include the choice ofactivation function and training algorithm, trainingparameters such as learning rate and momentum, num-ber of hidden layers, number of neurons in each hidden

1840

Table 2. Quadratic regression equations obtained for dependent variables (Achary et al. 1997).

Dependent variable Best fit equation R2

Hydantoinase y1 = 34.58 + 1.139x1 − 0.409x2 + 1.126x3 + 1.825x4 − 2.962x21 − 1.328x2

2 − 2.356x23 − 2.3082

4+ 0.799

(U ml−1) 1.092x1x2 − 0.365x1x3 − 0.739x1x4 + 0.552x2x3 − 2.033x2x4 − 1.655x3x4 (2)

Biomass y2 = 1.69 + 0.02x1 + 0.048x2 − 0.055x3 − 0.045x4 − 0.144x21 − 0.102x2

2 − 0.11x23 − 0.0692

4+ 0.812

(mg ml−1) 0.066x1x2 + 0.007x1x3 + 0.018x1x4 − 0.007x2x3 + 0.013x2x4 − 0.042x3x4 (3)

Table 3. Maximum hydantoinase and biomass concentrations identified by quadratic polynomialand neural network models and the optimum input sets that result in the maximum output values.

Independent variable (g l−1)

Dependent Molasses NH4NO3 NaH2PO4 MnCl2Model variablea (x1) (x2) (x3) (x4)

Quadratic polynomial y1 = 35.39 12.36 1.04 12.14 0.07

Neural network y1 = 39.29 11.95 0.75 15.99 0.08

Quadratic polynomial y2 = 1.69 12.75 1.3 14.23 0.04

Neural network y2 = 1.92 14.76 1.53 12.25 0.02

ay1 = hydantoinase (U ml−1); y2 = biomass (mg ml−1).

layer, initial weights, and training duration. In gen-eral, feed-forward neural networks with one hiddenlayer containing a sufficiently large number of hiddenneurons have been shown to be capable of providingaccurate approximations to any continuous nonlin-ear function (Hornik et al. 1989). Unfortunately, nospecific guidelines exist for the remaining design para-meters because the topology of a neural network islikely to be problem-specific. The choice of designparameters for a neural network is thus often the resultof empirical rules combined with trial and error. Theconfiguration of the two neural networks developed inthis work (a 4-6-1 structure: four input neurons-sixneurons in one hidden layer-one output neuron) wasdetermined after brief experimentation. To avoid theproblem of overtraining, the data set comprising 30experimental runs reported by Achary et al. (1997)was split into two categories: a training set compris-ing 27 experimental runs was used to optimize theweights of the two neural networks and a validation setcomprising 3 experimental runs was used to evaluatetheir predictive capability. Because empirical modelslike neural networks do not extrapolate data well, datafor network training should be selected carefully if thebest results are to be achieved. In this study the dataselected for network training covered the lower andupper bounds of the two output neurons (y1 and y2).

Fig. 1. Hydantoinase production calculated by neural network andEquation (2) versus actual hydantoinase production.

Figures 1 and 2 show the network-calculated hy-dantoinase and biomass concentrations for the trainingand validation data sets plotted against the correspond-ing experimental data. The solid circles represent thenetwork-trained outputs while the open circles de-note the network-predicted outputs for input variablesbelonging to the validation set. The network mod-els not only fit the training data very well but alsoprovide predictions of the validation data very closeto those measured experimentally. For comparison,

1841

Fig. 2. Biomass production calculated by neural network and Equa-tion (3) versus actual biomass production.

hydantoinase and biomass concentrations calculatedfrom the polynomial regression equations [Equations(2) and (3)] are also shown in Figures 1 and 2 (tri-angles). It is obvious that the neural network predic-tions are much closer to the line of perfect predic-tion than those of the quadratic polynomial equations,confirming the usefulness of the neural networks asempirical models in response surface analysis.

Optimization by genetic algorithms

Once a satisfactory neural network model is createdover the ranges of independent variables of interest,it can be used for optimization. For the fermentationexample examined in this work, the optimum valuesof hydantoinase and biomass concentrations may beobtained by using a genetic algorithm to optimize theinput space of the neural network models developed.As with other artificial intelligence techniques, per-formance of the genetic algorithm is affected by anumber of design parameters. These parameters in-clude the initial population size, parent selection,crossover rate, mutation rate, and number of genera-tions. Brief experimentation indicates that the geneticalgorithm program is robust to parameter variationswith the population size and number of generationshaving the largest effect on performance. Using apopulation of 50–100 the responses of the neural net-works successfully converged to the optimum valuesafter 200–500 generations. The results obtained areshown in Table 3 together with the input conditionsthat result in the maximum output values. The max-imum achievable hydantoinase and cell concentrationsfor this fermentation are 39.29 U ml−1 and 1.92 mg

ml−1, respectively, according to the neural networkmodels. These maximum concentrations identified bythe neural network models are 11–14% higher thanthose identified by the polynomial equations. Thesedifferences indicate that solution obtained from a poly-nomial model with poor modeling capability is notguaranteed to be optimum. The ability of a model toapproximate the true response surface of a processwith a high degree of accuracy is therefore of keyimportance in the optimization step.

The results in Table 3 reveal that different optimumconditions are found from models that have differentmodeling capabilities. To demonstrate that the differ-ences are not due to the type of optimization procedureused (derivative estimation or genetic algorithm), thegenetic algorithm was used to optimize the polyno-mial equations. The results obtained are very similarto the optimum solutions obtained by calculating thederivatives of Equations (1) and (2). This agreementthus confirms that the optimum conditions identifiedby the polynomial equations are not dependent uponthe method of optimization.

Often only quadratic polynomial models arechosen for a wide variety of fermentation optimizationproblems. We have shown that the quadratic polyno-mial is not always accurate enough. Clearly, in order tobuild a good response surface model for the exampleexamined in this study, higher order polynomials orother models such as neural networks are required. Inthe model building stage of any RSM, it is of great im-portance to use an appropriate model to approximatethe true response surface of a fermentation process inorder to avoid arriving at suboptimal conditions.

Conclusions

Empirical model building in the standard RSM ap-proach often entails fitting quadratic polynomials todata derived from statistically designed experiments.In some cases the ability of the quadratic polynomialto approximate the true response surface of a processmay not be adequate. This work found that neural net-works provided better fits to experimental data thanconventional quadratic polynomials. The input spaceof a neural network model can be optimized usinggenetic algorithms which do not require the objectivefunction to be continuous or differentiable. The hybridneural network-genetic algorithm approach describedin this work serves as a viable alternative to the stand-

1842

ard RSM approach for the modeling and optimizationof fermentation processes.

References

Achary A, Hariharan KA, Bandhyopadhyaya S, Ramachandran R,Jayaraman K (1997) Application of numerical modeling for thedevelopment of optimized complex medium for D-hydantoinaseproduction from Agrobacterium radiobacter NRRL B 11291.Biotechnol. Bioeng. 55: 148–154.

Baishan F, Hongwen C, Xiaolan X, Ning W, Zongding H (2003)Using genetic algorithms coupling neural networks in a studyof xylitol production: medium optimization. Proc. Biochem. 38:979–985.

Baughman DR, Liu YA (1995) Neural Networks in Bioprocessingand Chemical Engineering. San Diego: Academic Press.

Cheema JJS, Sankpal NV, Tambe SS, Kulkarni BD (2002) Geneticprogramming assisted stochastic optimization strategies for op-timization of glucose to gluconic acid fermentation. Biotechnol.Prog. 18: 1356–1365.

Goldberg D (1989) Genetic Algorithms in Search, Optimization, andMachine Learning. Reading: Addison-Wesley.

Holland J (1975) Adaptation in Natural and Artificial Systems. AnnArbor: University of Michigan Press.

Hornik K, Stinchcombe M, White H (1989) Multilayer feedforwardnetworks are universal approximators. Neural Networks 2: 359–366.

Houck CR, Joines JA, Kay MG (1995) A Genetic Algorithm forFunction Optimization: A Matlab Implementation. Technical Re-port NCSU-IE TR 95-09. Raleigh, NC: North Carolina StateUniversity.

Kennedy M, Krouse D (1999) Strategies for improving fermentationmedium performance: a review. J. Ind. Microbiol. Biotechnol.23: 456–475.

Liu CH, Hwang CF, Liao CC (1999) Medium optimization for gluta-thione production by Saccharomyces cerevisiae. Proc. Biochem.34: 17–23.

Marteijn RCL, Jurrius O, Dhont J, de Gooijer CD, Tramper J,Martens DE (2003) Optimization of a feed culture mediumfor fed-batch culture of insect cells using a genetic algorithm.Biotechnol. Bioeng. 81: 269–278.

Patil SV, Jayaraman VK, Kulkarni BD (2002) Optimization of me-dia by evolutionary algorithms for production of polyols. Appl.Biochem. Biotechnol. 102: 119–128.

Weuster-Botz D (2000) Experimental design for fermentation me-dia development: statistical design or global random search? J.Biosci. Bioeng. 90: 473–483.