Editorial

On the proper use of tools

The Viewpoint section of this issue of EJLST contains a commentary on the use of Response

Surface Methology (RSM) in research and technology. In that I have applied RSM in my

personal research and also have reviewed many manuscripts that used this method, I appreci-

ate the invitation extended by Professor and Editor Uwe Bornscheuer to comment on the

Viewpoint.

In the ‘Odds and Ends’ drawer of my desk is an old multi-bladed pocket knife, made of

high quality steel. Although its two smallest blades are intact and sharp, its larger main blade is

snapped off, a stub compared to its original length. This stub is a testament to the fact that even

very good knives make very poor prybars.

Similarly, in my gun cupboard sits a shotgun, many of whose screws have badly chipped

and rounded slots for the corresponding screwdriver. These are a permanent testimony to the

day that, as a much younger man, I learned that most screwdrivers and their companion screws

have tapered blades and correspondingly tapered slots, but that firearms screws and screw-

drivers have parallel faces. Regular screwdrivers therefore do not properly fit the screws in

firearms, and when used in this improper application can slip out and destroy the screw head.

These two examples illustrate the fact that a good tool used inappropriately is a recipe for

disaster. In a Viewpoint in this issue, Dr. Albert Dijkstra essentially observes that the same goes

for the tools we use in our research. He makes this observation in regard to response surface

methodology (RSM), which, to paraphrase Wikipedia, is a statistics-based experimental

method that explores the relationships between multiple experimental variables and one or

more response variables. Response Surface Methodology involves the use of a sequence of

designed experiments to identify the conditions that will give an optimal response. The

technique has become popular in some types of research because it can produce a maximum

amount of data from a relatively small number of reactions, and can yield powerful predictive

results. In that it allows identification of interactions between multiple variables in an

experimental design, RSM offers a power that is lacking in the more conventional approach

of changing one variable at a time.

An experienced scientist, Dr. Dijkstra has spent a long and productive career conducting

research and managing research teams, projects and entire research centers. He is very good at

what he does, and he is widely regarded as a wise and accomplished man. In his Viewpoint he

presents a multi-faceted commentary on the use, or rather the misuses, of statistically designed

experimental planning and the consequent application of response surface methodology to

the results.

I have a high regard for the author of the Viewpoint, a skilled and wise researcher with a

multitude of accomplishments in science and technology. Also, I agree with several of the

cautions voiced in his Viewpoint. But, as in looking at a glass with water in it and seeing it as

either half empty or half full, I see some of the points raised in the Viewpoint in a somewhat

different light than does the author. After reading the Viewpoint, please consider the following

comments, whose numbers correspond to the similarly numbered items there:

1. Dubious significance: Here the author of the Viewpoint observes that by addition of

extra terms to a data fitting equation one can reduce the apparent standard error of the

fit to the data. It is correctly noted that to do so without increasing the number of

experimental determinations as well is ill advised. Perhaps RSM has become too easy to

apply, and thus too easy to apply incorrectly, such as in without appreciating this fact.

Some contemporary RSM programs will help recognize such a situation as this,

reporting an ‘adjusted’ R2 value to alert the researcher to the presence of an excessive

number of terms in the equation of fit.

The Viewpoint admonishment to conduct multiple repeat determinations in order to

determine the standard error of the measurements is wise advice, best heeded by all who

Michael J. Haas

Eur. J. Lipid Sci. Technol. 2010, 112, 1287–1289 DOI: 10.1002/ejlt.201000530 Editorial 1287

� 2010 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim www.ejlst.com

implement RSM. Also, the observation is well made that just because your computer will

print results out to seven digits does not mean that the last four or five have any meaning or

should be reported.

2. Reproducibility: This comment alleges that it would be wise to repeat entire RSM

experiments, in order to determine if the original set of data is accurate. It also

speculates that researchers don’t usually do so for fear that their encouraging first

set of results may be thereby proven incorrect. I suspect that this is no greater a reason

for a lack of duplication than with other experimental design methods (a poor

researcher is a poor researcher largely irrespective of his research tools). In fact a well

designed RSM protocol has sufficient internal controls that a single run of the exper-

iment gives a good feeling for the reliability of the data.

3. Repeatability: An argument for the validation of RSM results by the use of multiple labs

to repeat the work. Such validation would add reliability to all research data, not just

that from RSM. I do not see a reason to demand more validation from RSM than we

routinely demand from other types of experimental design.

4. The use of absolute values: An argument for conducting experiments in a varying

order, to minimize the effects of long term fluctuations. Again, this seems to demand

of RSM that which we do not routinely demand of other experimental methods.

In fact, the Viewpoint goes on to acknowledge that the better practitioners of RSM

adopt a random order in conducting their experiments, to minimize the effect of such

fluctuations.

5. The choice of process variables: The author of the Viewpoint observes that that in

applying an RSM approach one may ignore the investigation of some major parameters

affecting the system. I agree, but again cannot see this as a problem unique to RSM.

Failure to correctly perceive and investigate the parameters most likely to impact our

systems is a danger we all face, whether we use RSM or not. Should I choose to explore

whether my tomatoes grow well if I read to them from Shakespeare, Goethe, or Aldo

Leopold, while failing to provide sunlight and water, neither RSM nor any other

experimental design approach will prevent the world from judging me a fool.

6. The value of process variables: And yes, if I should choose to study the effects of water,

but investigate the effects of only two levels, 0 and 25 liters per day per two liter pot, I

should not be surprised if the plants die under both treatment regimes and I fail to

identify water as a requirement in tomato agriculture. The Viewpoint caution here that

it is very important to have a feel for the process before selecting the variables and their

settings is excellent. RSM provides the world no more insulation against shabby

researchers than do other experimental approaches.

7. RSM is superficial: This section faults RSM methods for being applicable in cases

where one has no hypothetical mechanism from which to identify variables, inter-

actions, and relationships. Exercise more of the familiarization with your system that is

suggested in item no. 6 and you may come up with a hypothesis to guide a better RSM

examination. Even so, I am not convinced that the ability to function in the absence of

hypothesis is necessarily fatal. If one designs and conducts an RSM experiment for

which major effecting variables are not examined, the fit of the results to the data will be

poor, and the serious investigator will immediately see the need for more study to flesh

out the real impacting variables in the system.

8. Non-elimination of outliers/ rogue values: Here the author of the Viewpoint relates his

past improvement of the quality of a data set by deleting a data point that his experience

led him to recognize as an outlier. He faults RSM for lacking this capability. In fact,

some RSM programs can conduct outlier searches and alert the experimenter of their

presence. More importantly, RSM does not remove the researcher from the dance, does

not allow him or her to retire their powers of observation from the field of battle. In fact,

I am unaware of any experimental approach in which this would be advisable.

9. No improvement over established methods: The ‘methods’ referred to here are estab-

lished and wise ways of doing research and technology: attention to your system,

attention to feedback from those implementing the reaction at the bench or plant,

1288 Michael J. Haas Eur. J. Lipid Sci. Technol. 2010, 112, 1287–1289

� 2010 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim www.ejlst.com

and coming to an understanding of the process being studied. RSM does not invalidate

these. I suspect it speeds up the integration of seemingly incongruous observations into

a more complete understanding of the system.

10. Limited usefulness: Here the Viewpoint describes the author’s past use, in industrial

operations, of simple tables that made it easy to optimize the output of a process in the

face of variations in the feedstock. Surely such tables have value; and RSM is not meant

to be all things to all technologies and do away with all else. However, the construction

of those simple guiding tables might have been much easier if guided by the results of an

RSM investigation. The assertion that RSM experiments are done in laboratories and

thus are irrelevant to industrial scale operations is one on which I’d like to have more

data. Taking the assertion as true, one is left wondering why so much of industry would

bother with a method of no practical utility.

11. A method or a methodology? A question of the term, or is it terminology, by which to

refer to RSM. I will leave this decision to those with a stronger sense of English language

usage than I.

In my own research I have found RSM to be a powerful tool. It economizes on the thing I

have the least of: technician time. It also economizes on the use of reagents, which can be in

short supply or of high cost. Most appealing to me, it identifies the existence and magnitude of

interactions between multiple variables in a way that the conventional method of changing one

variable at a time, holding all others at constant values, never can. It also gives one predictive

powers that allow an estimation of how a reaction or system will respond at a chosen set of

conditions not yet examined.

As a journal editor, I have the opportunity to see firsthand the results of the misuse of RSM,

and to make decisions on whether the scientific community should be burdened by the bland

resulting fruits. I am most put off by examples of the poor choice of the ranges of one or more

variables in an experimental design when allegedly seeking to optimize the yield of a reaction.

Poor choice in selecting the ranges of variables leads to predictive surfaces that rise as they

approach their edges, and authors who call these edges ‘maxima’. Local maxima perhaps, but

the rising surface makes it clear that the real optimal conditions lie somewhere off in a space

that the authors were too inattentive or too lazy to search.

Observe any craftsperson, any quality worker in any trade who is able to make a living at

what they do, and you will probably see that success depends on a desire and ability to apply

tools effectively and efficiently, and in knowing when to apply the proper amount of detail and

care. To be a researcher is to be a craftsperson. Success lies in knowing when and how to do a

good job, and in using the tools in ones’ toolbox properly to create a product recognized by

oneself and ones’ peers as of high quality. Do anything less and you will eventually create

faulty, weak data with poor predictive capability. Then you will have the opportunity to ponder

one of the profound questions that all poor craftsmen eventually face: ‘If you don’t have time

to do it right, where will you ever find the time to do it over?’

Michael J. Haas

Eur. J. Lipid Sci. Technol. 2010, 112, 1287–1289 DOI: 10.1002/ejlt.201000530 Editorial 1289

� 2010 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim www.ejlst.com