dissecting doe software

S tatistical techniques not only contribute to many disciplines throughout the business world, but they are also widely used in Six Sigma.1 Design of experiments (DoE) is a key tool in the Six Sigma method. But this technique cannot take hold if experimenters, reluctant to use mathematics related to this method, lack adequate tools to develop and correctly analyze experiments.

The application of designed experiments has grown far beyond its origins in agriculture when Ronald Fischer introduced DoE in 1920. There is not a single area of science and engineering that has not successfully employed sta-tistically designed experiments. Many people who apply Six Sigma, however, have little or no formal training in statistics,2 which makes the application of DoE much more difficult.

Despite the economic advantages for industries, DoE was a latecomer to the desktop computer phenomenon. Now, software allows every user to design and analyze experiments. Moreover, modern software packages are comprehensive, flexible, easy to use and capable of producing excellent information and graphics.

Most packages help us meet common DoE objectives, such as finding important factors, optimizing responses and fitting data into mathematical equations. As a sign of the advances made in this field, most software pack-ages have been updated many times. For example, Minitab is now in its 15th version.

Keep the following questions in mind when you are selecting software for your situation:

How will this software be used?

What type of design is most appropriate for your research area?

What is the most friendly and easy-to-use software?

Is the software able to perform proper analysis?

How much does it cost?Our intention is to analyze the most popular software used for DoE,3 includ-

ing Minitab, JMP, Design-Expert and DoE Pro XL by answering each of these questions. The DoE Pro XL software is included in this comparison because its an Excel add-on program, which is convenient for many Excel users.

How Will the Software Be Used?

We classified the different types of software for experimentation according to the assigned use:

Generic purpose statistical software: These packages include many topics apart from DoE, such as basic statistics, statistical process control (SPC), quality tools and times series. Software of this type are Minitab, JMP-SAS, S-plus, Statistica, Statgraphics and GenStat.

S OFTWARE

Dissecting DoE Software

PINPOINT YOUR

EXACT NEEDS

BEFORE

COMMITTING TO

A SOFTWARE

PACKAGE

By Martn

Tanco, Elisabeth

Viles, Laura

Ilzarbe and

Mara Jesus

lvarez,

University of

Navarra-Tecnun,

Pamplona, Spain

S I X S I G M A F O R U M M A G A Z I N E I M A Y 2 0 0 8 I 25

Dedicated software: These are exclusively dedicat-ed to DoE. Software of this type include Design-Expert (and Design-Ease), Fusion Pro, DoE Pro XL (and DoE KISS), Matrex and WinDex. The last three are examples of Excel add-on soft-ware.

The reasons for choosing a specific package are usually based on the need to obtain greater reach and clarity, and become more efficient.4 On the other hand, generic analytic packages offer more utilities and diversity of analysis.

What Designs Are Most Appropriate?

Use caution when selecting software. Consider the design needs and the possibilities offered by the soft-ware. Because of the broad spectrum of experimen-tal designs noted in literature, we have selected and briefly explained only those included in the analyzed software. Designs have been classified by the varying capabilities and intended uses.5 Table 1 shows the breakdown.

A. Treatment comparisons: The main purpose of this category is to compare several treatments and select the most appropriate design for the user. Minitab and JMP have many tools and enough statistical power for analysis. On the other hand, Design-Expert and DoE Pro lack these virtues because they regard this topic as basic statistics and not DoE. Both of them accomplish comparative tests with factorials for one factor and multiple levels.

B. Screening: If there is a large number of variables in a system, a screening experiment can be conducted to identify the most important. Once the important variables are identified, a follow-up experiment can be conducted. This latter phase falls into the next category. All of our selected software allow for the execution of traditional designs, such as fractional factorial design or PlackettBurman. In addition, JMP and Design-Expert offer other designs with fewer runs, such as supersaturated or Cotter Designs.

C. Characterization: This consists of identifying the effects of a few variables in the designs response. The relationship between the response and these variables is sometimes referred to as a response sur-

face. All selected software programs have basic designs, allowing mixed levels such as full factorials and orthogonal designs of Taguchi. Furthermore, JMP and Design-Expert offer other designs in which fewer runs are needed, such as irregular factorials fractions or other orthogonal arrays (OA).

D. Optimization: For finding an opti-mum design, an intelligent sequential strat-egy can quickly move the experiment to a region containing the optimum settings of the variables. Only in this region is a thor-ough exploration of the response surface warranted. Usual designs for optimization are central composite designs (CCD) or Box-Behnken designs (BBD), which are included in all software except for DoE Pro XL. Design-Expert is the only one that offers other designs with fewer runs, such as small CCD, hybrid and uniform designs.

E. Robust parameter design (RPD): In quality improvement, it is important to have a system that is robust to noise variation. This is often achieved by choosing control factor settings that are less sensitive to sys-

Table 1. Experimental Designs

Object Designs Minitab JMPDoE

PRO XLDesign Expert

A Comparative tests

B

Fractional factorial Placket-Burman designs Other designs

C

Full factorials OA of Taguchi for mixed levels Other designs

D

CCD Fractions of CCD BBD Other designs

E Crossed arrays

F

Mixture designs Combined designs Optimal designs Split-plots designs Other designs Augment design Custom design

Dissect ing DoE Software

26 I M A Y 2 0 0 8 I W W W . A S Q . O R G

tem noise variation. JMP is the only one that offers a specific design for RPD prob-lems, such as crossed arrays, attributed to Taguchi ideas. Some have criticized these designs and analyses.6

The latest approach for this overall objec-tive is to use classical designs and conduct the analysis by manipulating the terms included in the regression analysis. At this point, we might think that all software per-mits us to realize RPD. It must be stressed that prior experience is needed to avoid using crossed arrays.

F. Others: This category contains designs used when restrictions exist. With the exception of DoE Pro XL, all software permits the use of DoE when the factors are components or ingredients of a mix-ture, causing their levels to be dependent. These designs are called mixture experiments and are gen-erally used in chemistry and metallurgic industries.

Besides these designs, Design-Expert offers com-bined designs, allowing experimentation with mix-ture factors and typical controllable factors. In addi-tion, JMP and Design-Expert provide designs that function when restrictions in the inference space or in the experimental conditions are present, opti-mizing them with criterion such as D-optimality. As an enhancement, Minitabs version 15 includes the ability to select an optimal design, reducing the num-ber of experimental runs in a general full-factorial design.

Finally, we would like to consider what we call special design functions. In this section, we include the augmentation of a previous standard design with a criterion of optimality or the guide to create a custom design chosen over some selected points, and also guided with a criterion of optimality. JMP and Design-Expert include both special design func-tions.

Most Easy-to-Use Software

When selecting DoE software, its important not only to look at design possibilities, but also to consider the following items:7

Simple interface thats intuitive and easy to use.

A well-written manual with tutorials to get you off to a quick start.

Spreadsheets flexible enough for data entry and able to deal with missing data and changing levels.

We believe the four software packages we analyzed stand up to these criteria, although individual users might evaluate each according to their own criteria. We must add that Design-Expert offers the most enjoy-able interface, leading users through every step of the process.

It doesnt help that DoE is so infrequently used. Many users work with DoE perhaps twice a year. This makes it quite difficult to develop an intuitive feel for procedures.8

Table 2. Analysis and Plots

Analysis and Plots Minitab JMPDoE PRO

XLDesign Expert

Normal and half-normal plot Residual plots Interaction plots ANOVA Generalized linear models Transformations (Box-Cox) Multiple regression Surface plotscontour plots ANCOVA Steepest ascent

Response optimizercanonical analysis



IF THERE ARE A LARGE NUMBER OF VARIABLES IN A SYSTEM, A SCREENING

EXPERIMENT CAN BE CONDUCTED TO IDENTIFY THE MOST IMPORTANT VARIABLES.

Ana

lysi

sP

lots

Considerable money is wasted on high-end pack-ages by users who not only fail to use a fraction of the softwares statistical capability , but even worse, misuse the statistics. Consequentially, it is essential that all software has good technical support and provides access to consultants with statistical expertise.

With the exception of DoE Pro XL, the standard technical support of most DoE software is sufficient for most users.

Do They Have DoE Analysis?

User friendliness is a welcome characteristic of new software, especially for beginners and those who are not experts in the field. However, it is also necessary to evaluate the type of analysis offered by different statistical packages. As shown in Table 2 (p. 27), we have divided this section into two categories: plots and analysis, although all plots are useful for analysis.

In the plots category, we have included only the most typically used plots that come with all analyzed software. These include:

Normal and half-normal plots to determine, with a lack of replicates, which effects are significant.

Residual plots to check the hypothesis assumed.

Interaction plots to visualize the greater (or less-er) effect of the sum of the effects expected from either factor alone.

The analysis of variance (ANOVA) and regression analysis are classical tools for analyzing experimental designs. Obviously, all software includes both, but only Minitab offers analysis of covariance (ANCOVA) and generalized linear models (GLM), which are very useful when some of the hypotheses assumed in ANOVA are not fulfilled. Furthermore, Minitab and Design-Expert offer transformation when plots show shortcomings.

Finally, contour plots are included in the packages for analyzing the response surface and for allowing the possibility to resolve some multiresponse problems. In addition, the software offers a response optimizer to realize a more formal analysis.

Incredibly, none of them includes the steepest ascent, a very important stage in response surface methodology, although we must clarify that its simple to do by hand.

Economic Resources

Software costs around $1,000 for a single user licence, except for DoE Pro XL, which costs $200, and

the new version of JMP at $1,500. All permit the downloading of a complete trial version for a limited time.

Design-Expert and DoE Pro XL have reduced versions called Design-Ease and DoE KISS, respec-tively. They are simplified versions of the original, but both work with the same interface. For more infor-mation about these versions, con-sult the company websites listed in Table 3.

Table 3. General Features

Software Version Cost Trial Website

Minitab 15 $1,195 30 days www.minitab.com

JMP 7.0 $1,495 30 days www.jmp.com/software/jmp.shtml

Design-Expert 7.1 $995* 45 days www.statease.com/dx7descr.html

DoE Pro XL 2007 $249** 5 days www.sigmazone.com/products.htm***

* The cost of the perpetual single user license of Design-Ease is $495

** The cost of the perpetual single user license of DoE Kiss is $149

*** DoE Pro XL doesnt have an official website.

28 I M A Y 2 0 0 8 I W W W . A S Q . O R G


CONSIDERABLE AMOUNTS OF MONEY ARE WASTED ON HIGH-END PACKAGES BY

USERS WHO NOT ONLY FAIL TO USE A FRACTION OF THE SOFTWARES STATISTICAL

CAPABILITY, BUT EVEN WORSE, MISUSE THE STATISTICS.



WHAT DO YOU THINK OF THIS ARTICLE? Please share

your comments and thoughts with the editor by e-mailing

[email protected].

System Requirements

The minimum requirements for all software are listed in Table 4. Windows XP with 256 MB is recommended for the optimum running of Design-Expert and JMP. Macintosh and Linux con-tain Windows emulator software, which allows users to run Windows programs.

Making the Decision

When deciding which software is most suitable for applying DoE to industry or research, the customer must define exactly what it expects from the product. First and foremost, you must decide whether you need generic or dedicated software.

If you are going to make a one-off investment in a specialized solution, dedicated software might not be appropriate. If designed experiments are a major part of what you do, then it makes sense to focus on dedicated software.9

Always remember to clearly define your real needs before making the decision and spending the money.

REFERENCES

1. J.E. Brady and T.T. Allen, Six Sigma Literature: A Review and Agenda for Future Research, Quality and Reliability Engineering International, Vol. 22, 2006, pp. 335-367.

2. Gerald J. Hahn, Necip Doganaksoy and Christopher. Stanard, Statistical Tools for Six Sigma, Quality Progress, Vol. 34, No. 9, 2001, pp. 78-82.3. J.G. Pigeon, Book Review: Statistics for Experiments: Design, Innovation and Discovery, Technometrics, Vol. 48, No. 2, 2006, pp. 303-304.4. Felix Grant, Planning for the Unexpected, Scientific Computing World, 2006, pp. 24-27.5. C.F.J. Wu and Michael Hamada, eds., Experiments: Planning, Analysis and Parameter Design Optimization, Experiments, Wiley Interscience, 2000.6. V.N. Nair, Taguchis Parameter Design: A Panel Discussion, Technometrics, Vol. 21, No. 2, 1992, pp. 127-161.7. Rich Burnham, How to Select Design of Experiments Software, Quality Digest, Vol. 18, No. 11, 1998, p. 32.8. Grant, Planning for the Unexpected, see reference 5.9. Ibid.

Table 4. Minimum System Requirements

Software Operating SystemMinimum version

RAM CPU

Minitab Windows 2000 256 Mb 300 MHz

JMP Windows, Macintosh, Linux 2000 128 Mb 266 MHz

Design-Expert Windows 98 SE 128 Mb 266 MHz

DoE Pro XL Windows 2000 128 Mb 166 MHz

* Only Windows versions are specified.

CPU = central processing unit

RAM = random access memory

Call for articles

Six Sigma Forum Magazine is seeking articles

for publication. For information on the review

process and types of articles considered, along

with submission requirements, go to www.asq.

org/pub/sixsigma/author.

dissecting doe software

Documents