Available online at www.sciencedirect.com

ScienceDirectProcedia Engineering 00 (2014) 000–000

www.elsevier.com/locate/procedia

“APISAT2014”, 2014 Asia-Pacific International Symposium on Aerospace Technology, APISAT2014

Improving Quality of Wind Tunnel test Data through the MDOE

Zhang Jiang* ,Ma Handong , Qin Yongming China Academy of Aerospace Aerodynamics,17th Yonggang Street,Fentai, Beijing, 100074,China

Abstract

Researching on the improvement of the wind tunnel data quality through Modern Design Of Experiments (MDOE) was carried out. A body with fin model was tested in wind tunnel through the MDOE and traditional OFAT (One Factor at A Time) methods. The uncertainties of data from these two different methods were analyzed, and mechanism of defense against systematic error of Randomization and Repetition in the MDOE method was studied. The defense abilities against the time various systematic errors in wind tunnel tests of the MDOE and the OFAT method were compared, and the mean to improve test data quality using repetition principle in the MDOE was studied with the increasing of cost. Main conclusions: The MDOE method can test and eliminate the time various systematic errors during the wind tunnel tests, while the OFAT test data were polluted by the time various systematic errors; The independent variable changes sequently in the OFAT tests, so the test data are not statistically independent. It is the reason of the disability of the defense against the time various systematic errors in OFAT tests. Randomization can transform the systematic errors into random errors in MDOE tests, which can be eliminated. Repetition in MDOE tests can improve the quality of test data since the average variance in the predicted response can be reduced by adding sample points, and amount of the measured data are still less than the amount in OFAT tests.© 2014 The Authors. Published by Elsevier Ltd.Peer-review under responsibility of Chinese Society of Aeronautics and Astronautics (CSAA).

Keywords: Modern design of experiment（MDOE）; wind tunnel test; data quality; uncertainty; experimental error; randomization; repetition

1. Introduction

As the quality requirement of wind tunnel test data for aircrafts design is more and more high, the experimental testing technology community is focus on how to improve the quality of wind tunnel test data. In wind tunnel tests, aerodynamic data are obtained by measuring the response variable (such as aerodynamic forces, moments and pressure, etc.).At present, most wind tunnel test is based on OFAT (One Factor at A Time) experimental design methods, which changes one independent variable (such as the angle of attack) during one run, and keeps the other

* * Corresponding author. Tel.: +86-13611319903.E-mail address:13611319903@163.com

1877-7058 © 2014 The Authors. Published by Elsevier Ltd.Peer-review under responsibility of Chinese Society of Aeronautics and Astronautics (CSAA).

2 Zhang Jiang / Procedia Engineering 00 (2014) 000–000

independent variables (such as sideslip angle, roll angle, Mach number, etc.) unchanged. OFAT method applied in the field of wind tunnel tests are very successful, which plays an important role in aerodynamic data acquisition, aerodynamic modeling. However, the traditional OFAT method is based on the assumption that only one independent variable changes while the other independent variables are fixed, which is impossible to meet in real circumstances [1,2,3], such as in the experiment is carried out to change the angles of attack, the sideslip angle is also slightly changed (due to the elastic deformation of balance and supporting sting caused by the aerodynamic loads) as the angle of attack. In fact, such undesirable "covariate effect" always happens in the process of the simulation and measurement in the wind tunnel tests, for instance, the change of air total temperature, the elastic modulus change of the balance with temperature, sensors drift, attitude angle mechanism clearance errors, etc [4]. In practice random errors in wind tunnel tests is less than systematic errors than random error, and the former is more difficult to detect and prevent, which often are found in the comparison of the test data measured in different time. Some research organizations and scholars come to realize that it is impossible to keep all potential covariate factors remain unchanged for such a complex system like a wind tunnel test, and the potential to improve data quality traditional OFAT methods cannot meet the needs of the aerodynamic data quality requirement for the advanced aircraft design, and it is important to seek new breakthroughs by the combination of Design of Experimental (DOE) [5].

In 1935 British scholar R.A.Fisher write the book "The Design of Experiments" [6], created the experimental design method, then became an independent discipline as a branch of Applied Mathematics, and has be well developed and the applied. As distinguished from the OFAT method, it is often called MDOE ( Modern Design of Experiments). R.A.Fisher proposed three principles to prevent uncontrollable factors of test conditions leading to the experimental error [7,8]: Repetition, Randomization and Blocking, and the variance analysis method, which has been widely applied in many disciplines and proved to be very effective methods for error prevention and analysis [9].

In 1997, National Aeronautics and Space Administration (NASA) Langley Research Center began to study on the following issues about the application of MDOE in wind tunnel tests through four projects: productivity and cost comparison of OFAT methods and MDOE methods, validation of MDOE to improve the data accuracy [10], the potential to reduce cost and shorten the test time [11,12,13]. The results shows: "This testing methodology has the potential to significantly reduce the amount of data required in wind tunnel tests compared to traditional OFAT test methods, thus reducing test time and test costs." Langley Research Center began actively promoting to use MDOE as an alternative way to traditional OFAT methods [14]. Currently Langley Research Center has applied MDOE method to more than 100 wind tunnel tests [15,16,17,18,19].

In order to study the mechanism and the function of MDOE methods to improve data quality, a wing-body combination model is tested in FD-06 wind tunnel of China Academy of Aerospace Aerodynamics. This paper is focus on the mechanism and ability of improvement of data quality for two MDOE principles that are repetition and randomization.

2. Wind tunnel test model and test technology

The test model is a missile model with wing-body shape (Figure 1), four tails, which has reference data in other wind tunnel [20]. The test Mach is 2.01; the range of angles of attack: α = -2 ° ~ 16 °; the range roll angle: φ = 0 ° ~ 90 °. Both the attitude angles for OFAT and MDOE methods are shown in Figure 2. The total number of measurement points OFAT was 114, while the total number of measurement points for MDOE is 88 (including 20 repeated points). The number of measuring points for MDOE is 77% of OFAT’s. OFAT test is carried by changing the roll angles firstly from 0 ° to 90 ° or 90 ° to 0 ° at each angle of attack, and then increasing the angle of attack subsequently. the measuring points of MDOE test are designed according to IV- optimal design method in the design space of the angle of attack and roll angle. The attack angle and roll angle changed simultaneously in the test, and the order of the measured site is according to the principle of randomization.

Zhang Jiang / Procedia Engineering 00 (2014) 000–000 3

Fig.1 Photograph of No.8 wind tunnel standard model Fig.2 Model actual attitude angles of test points in OFAT and MDOE tests

3. Test results and comparison

OFAT test data are reduced according to the wind tunnel test routine manner, and the uncertainty is analyzed. For MDOE test, response surface model is established by using of the sample data, form which any combination of independent variables in the design space can be predicted with confidence intervals. The response surface of the normal force coefficient CN and rolling moment coefficient mx are given in Figures 3a and 3b respectively, on which the results of OFAT test data are drawn, which shows roughly that results of OFAT and MDOE methods are approximately same.

a ) Comparison of normal force coefficients b) Comparison of normal force coefficients

Fig.3 Response surface of MDOE test vs. OFAT test result

In order to compare the uncertainty of the two test method, 95% prediction intervals of aerodynamic coefficients various with pitch sweep and roll sweep of MDOE tests and OFAT tests are curved in Figure 4a ~ f. Because of the existing of elastic angle, the actual angles of attack of the OFAT are different with the nominal angles of attack slightly. So the data of MDOE are curved correspondingly to the actual angles of attack of the OFAT, which is to benefit from the rich information of MDOE method. The measured values and uncertainty interval of OFAT data point are curved on these figures. And the results of MDOE are curved by the upper and lower limits of the 95% confidence interval. The uncertainty interval of the axial force coefficient of MDOE test is less than OFAT’s. And the uncertainty intervals of the other aerodynamic coefficients for MDOE and OFAT are similar. In Figure 4f, the axial force coefficient decreases more in OFAT test than in MDOE test whenα＞10°(for example, 0.025 less at α=16.3 °), which is caused by the system variation during the run, which is probably due to the temperature drift of the axial force measurement unit of the balance appeared during the test. This systematic error prevented and eliminated in MDOE test by applying random and repetitive strategy (detailed analysis, see 3.1 ).

4 Zhang Jiang / Procedia Engineering 00 (2014) 000–000

a) Normal force coefficients in pitch sweep and roll sweep

b) Pitching moment coefficients in pitch sweep and roll sweep

c) Lateral force coefficients in pitch sweep and roll sweep

Zhang Jiang / Procedia Engineering 00 (2014) 000–000 5

d) Yawing moment coefficients in pitch sweep and roll sweep

e) Rolling moment coefficients in pitch sweep and roll sweep

f) Fore-body axial force coefficients in pitch sweep and roll sweep

Fig.4 Comparisons of 95% prediction intervals in MDOE tests and measured data in OFAT tests

6 Zhang Jiang / Procedia Engineering 00 (2014) 000–000

4. The function of randomization

Since there are 20 duplicate points in the 88 total points in MDOE test, the reproducible data can be compared directly after the test is completed. Figure 5 is drawn using number of the order intervals of measurement for the repeated points as X-axis, and the difference of measured axial force coefficients as Y-axis. As can be seen the relationship between the difference of measured values and the number of order intervals is not in random, while the difference of measured values is increasing with the number of intervals. The least squares method is used to fitting those points, and the F-test results are given in Table 1. There is a linear relationship with -0.0002 in slope. And the determination coefficient is of 0.828 that is relatively close to 1. F statistic value is 86.61 that is significantly greater than the critical value 8.4 as α = 0.01, which means the probability of the linear relationship resulted from the 20 samples is very high. Although the time-varying systematic error in this test is small, MDOE method is still able to conduct quantitative analysis through randomization and statistical analysis.

Fig.5 Differences of the repetition points vs. survey intervals

Table.1 Linear correlation analysis of repetition differences and survey intervals

Slope Intercept R2 F-value Degree of freedom F - critical value（α=0.01）-0.0002 -0.00016 0.828 86.61 18 8.4

5. The function of repetition

A 3rd polynomial response model in two factors model is used in the design space of -2°<α<6°,0°<φ<25°, which has 10 undetermined terms. There are 14 points determined by experimental design methods to build a response surface model, which average predict variance is 0.714σ2. The distribution of the predict standard deviation is shown in Figure 6a. In order to improve the data quality, 6 repeating points are added, which reduce the average predict variance to 0.5σ2. And the distribution of the predict standard deviation for the revised design is shown in Figure 6b. As can be seen with the increase of repeated points, the predict standard deviation of the response surface model is effectively reduced in the whole design space. Figure 7 curves comparison of the rolling moment coefficients (mx) various with angles of attack of with and without the duplicate points in MDOE test, which shows the confidence interval can be significantly reduced through adding the duplicate points. Theoretically, by increasing the number of duplicate points the average variance response surface model can be reduce to an arbitrarily small level. But in practical applications the cost factor is needed to be considered. In this experiment, a satisfactory quality is obtained by 20 data points for the design space in MDOE test, while the number of points in corresponding OFAT test is 24. 17% of measured points have be saved by MODE method. And the measured points should be increased to 48, if the variance of OFAT results is wanted to be reduced to such a level.

Zhang Jiang / Procedia Engineering 00 (2014) 000–000 7

a) Without repetition points

b) With repetition points

Fig.6 Comparisons of predicted standard error between with and without repetition

Fig.7 Comparisons of predicted interval of rolling moment coefficients between with and without repetition points（φ=25°）

8 Zhang Jiang / Procedia Engineering 00 (2014) 000–000

6. Conclusion

By contrast MDOE and OFAT wind tunnel test, the mechanism and effect of MDOE method to improve the quality of test data is the studied. The main conclusions are as follows: MDOE test can effectively detect and eliminate short-term time-varying systematic errors in wind tunnel tests by

repetition and randomization, by which OFAT test results are affected significantly. In OFAT test the measured data do not meet the assumption of statistical independence due to the sequent

change order of the independent variables, for which the time-varying systematic errors cannot be eliminated byc. The variance of the predicted response model of the model is reduced by repetition, which significantly improve

the quality of the test data in MDOE test with less measured points than in OFAT tests.

References

[1] DeLoach.R, Micol.J.R. Analysis of Wind Tunnel Polar Replicates Using the Modern Design of Experiments(Invited). AIAA 2010-4927, 27th AIAA Aerodynamic Measurement Technology and Ground Testing Conference. Chicago, IL, June 28–July 1, 2010

[2] DeLoach.R. Tactical Defenses Against Systematic Variation in Wind Tunnel Testing. AIAA 2002-0540, 40th AIAA Aerospace Sciences Meeting and Exhibit, Reno, Nevada. January 14-17, 2002

[3] DeLoach.R. Propagation of Computational Uncertainty Using the Modern Design of Experiments. Langley Research Center, December 3, 2007

[4] DeLoach.R, Obara.C.J, Goodman,W. A Practical Methodology for Quantifying the Random and Systematic Components of Unexplained Variance in Wind Tunnel Data. AIAA 2012-0764, 50th AIAA Aerospace Sciences Meeting Including the New Horizons Forum and Aerospace Exposition, Nashville, Tennessee. January 9-12, 2012

[5] DeLoach.R. Applications of Modern Experiment Design to Wind Tunnel Testing at NASA Langley Research Center. AIAA 98-0713, 36th AIAA Aerospace Sciences Meeting and Exhibit, Reno, NV, Jan. 1998

[6] Fisher, R. A., The Design of Experiments, 1st Ed., Oliver and Boyd, Edinburgh, 1935[7 ]Fisher, R. A., Statistical Methods for Research Workers, 1st Ed., Oliver and Boyd, Edinburgh, 1925.[8] Fisher, R. A., Statistical Methods and Scientific Inference, 1st Ed., Oliver and Boyd, Edinburgh, 1956.[9] Thomas P. Ryan., Modern Experimental Design, John Wiley & Sons, Inc., Hoboken, New Jersey, 2007.[10] DeLoach.R. Analysis of Variance in the Modern Design of Experiments. AIAA 2010-1111, 48th AIAA Aerospace Sciences Meeting

Including the New Horizons Forum and Aerospace Exposition, Orlando, Florida, January 4-7, 2010.[11] DeLoach.R. Improved Quality in Aerospace Testing Through the Modern Design of Experiments. AIAA 2000-0825, 38th AIAA Aerospace

Sciences Meeting and Exhibit, Reno, Nevada. January 10-13, 2000.[12] DeLoach.R, Marlowe.J.M, Yager.T.J. Uncertainty Analysis for the Evaluation of a Passive Runway Arresting System. AIAA 2009-1156,

47th AIAA Aerospace Sciences Meeting Including The New Horizons Forum and Aerospace Exposition, Orlando, Florida, January 5-8, 2009.

[13] DeLoach.R. Tailoring wind tunnel data volume requirements through the formal design of experiments. AIAA 1998-2884, AIAA Advanced Measurement and Ground Testing Technology Conference, 20th, Albuquerque, NM, June 15-18, 1998.

[14] John R. Micol, NASA Langley Langley Research Center's Unitary Plan Wind Tunnel: Testing Capabilities and Recent Modernization Activities Research Center, AIAA 2001-0456, 39th AIAA Aerospace Sciences Meeting and Exhibit, January 8-11, 2001.

[15] DeLoach.R. The Modern Design of Experiments for Configuration Aerodynamics: A Case Study AIAA 2006-923, 44th AIAA Aerospace Sciences Meeting and Exhibit, Reno, NV, Jan 9-12, 2006.[16] DeLoach.R, Rayos.E.M, Campbell.C.H, et al. Space Shuttle Debris Impact Tool Assessment Using the Modern Design of Experiments.

AIAA 2007-550, 45th AIAA Aerospace Sciences Meeting and Exhibit, Reno, NV, Jan 8-11, 2007.[17] Erickson.G.E, DeLoach.R. Estimation of Supersonic Stage Separation Aerodynamics of Winged-Body Launch Vehicles Using Response

Surface Methods. NASA Center for AeroSpace Information, February 2010.[18] Dowgwillo.R.M, DeLoach.R. Using Modern Design of Experiments to Create a Surface Pressure Database From A Low Speed Wind

Tunnel Test. AIAA 2004-2200, 24th AIAA Aerodynamic Measurement Technology and Ground Testing Conference, Portland, Oregon. June 1-28, 2004.

[19] Danehy.P.M, DeLoach.R, Cutler.A.D. Application of Modern Design of Experiments to CARS Thermometry in a Supersonic Combustor. AIAA 2002-2914, 22nd AIAA Aerodynamic Measurement Technology and Ground Testing Conference, St. Louis, Missouri. June 24-26, 2002.

[20] Jimmie N. Derrick, Donald J. Spring and Gary C. winn Aerodynamic characteristics of a series of bodies with and without tails at Mach numbers from 0.8 to 3.0 and angles of attack from 0 to 45 degrees ADA028324, 16 July 1976.