5 data screening

8/16/2019 5 Data Screening

http://slidepdf.com/reader/full/5-data-screening 1/33

DATA SCREENING &

TRANSFORMATION

STATISTICAL ANALYSIS



Data Screening/CleaningBefore conducting any statistical test, data

need to be checkAlways potential for errors to occur

measurement

transcribing

entering the data




Qualitative Data

Coding is important: recommended to

use numerical-codes to present groups

Easy to check, since each variable can

take one of the numbers of the limited

values

Any values that are not allowable: must

be errors




numerical codes are

recommended compare tostring/text because a small

‘male’ is different from a big

‘Male’.


Qualitative Data



Quantitative Data

Very prone to errors & difficult to check

Out of range numbers should be remove

e.g: decimal point is easily misplaced, any

value which is out of range further checking




Testing For Normality

3 ways:

1)Graphs:

histogram, Q-Q plots

2)Descriptive statistics:

mean,median,skewness,kurtosis

3)Formal statistical test:Kolmogorov Smirnov one Sample test (K-S

test),Shapiro-Wilk test




Graphs: Histogram & Q-Q plots




Skewness & Kurtosis




Formal Statistical Test




Flowchart for Formal Statistical Test





DATA TRANSFORMATION

WHY? Many statistical model based on mean and require it asappropriate central tendency measure (need to treat data asnormal)

Linear least squares regression assumes that the relationshipbetween two variables is linear

Can “straighten” a nonlinear relationship by transforming one orboth of the variables

Often transformations will ‘fix’ problem distributions so that we

can use least-squares regression

When transformations fail: use nonparametric regression,which makes fewer assumptions about the data





TYPICAL TRANFORMATION TYPES

-Logarithmic

-Square Root

-Angular

-Box-Cox

-Reciprocal(inverse)

-Power





LOGARITHMIC TRANFORMATIONS





SQUARE ROOT

TRANSFORMATIONS





RECIPROCAL/INVERSE

TRANSFORMATION





CAN ALL DATA CAN BE

TRANSFORM

May ignore since it’s difficult to interprets afterwardseven though the transformation can be done

If we have a large number of identical observations,usually at zero then any transformation will leave halfthe observations with the same value, at the extreme ofthe distribution. It is impossible to transform these data

to a normal distribution.





can use non-parametric methods( eg:Mann-

Whitney test). These will give us a significancetest but usually no confidence interval

There are some data which should not be

transformed. Sometimes we are interested inthe data in the actual units only: eg cost data


CAN ALL DATA CAN BE

TRANSFORM




Practical SessionOpen SPSS Data: sga





Data transformation





THANK YOU

S S C S S

5 data screening

Documents