correlation & regression (students)

8/16/2019 Correlation & Regression (Students)

http://slidepdf.com/reader/full/correlation-regression-students 1/23

Correlation and Regression Analysis

• Many engineering design and analysis problems involve factors that are

interrelated and dependent. E.g., (1) runoff volume, rainfall; (2) evaporation,temperature, ind speed; (!) pea" discharge, drainage area, rainfall intensity;

(#) crop yield, irrigated ater, fertili$er.

• %ue to inherent comple&ity of system behaviors and lac" of full understanding

of the procedure involved, the relationship among the various relevant factors

or variables are established empirically or semi'empirically.

• egression analysis is a useful and idely used statistical tool dealing ith

investigation of the relationship beteen to or more variables related in a

non'deterministic fashion.

• f a variable * is related to several variables + 1, +2, , +- and their

relationships can be e&pressed, in general, as

* g(+1, +2, , +- )

here g(.) general e&pression for a function;

* %ependent (or response) variable;

+1, +2,, +- ndependent (or e&planatory) variables.



Correlation• /hen a problem involves to dependent random variables, the degree of

linear dependence beteen the to can be measured by the correlation

coefficient ρ(+,*), hich is defined as

here 0ov(+,*) is the covariance beteen random variables X and Y defined

as

here 0ov(+,*) and ≤ ρ(+,*) ≤ .

• arious correlation coefficients are developed in statistics for measuring the

degree of association beteen random variables. 3he one defined above is

called the Pearson product moment correlation coefficient or correlation

coefficient.

• f the to random variables X and Y are independent, then ρ(+,*)

0ov(+,*) . 4oever, the reverse statement is not necessarily true.



Cases of Correlation

5erfectly linearly

correlated in opposite

direction

6trongly 7 positively

correlated in

linear fashion

5erfectly correlated innonlinear fashion, but

uncorrelated linearly.

8ncorrelated in

linear fashion



Calculation of Correlation Coefficient

• 9iven a set of n paired sample observations of to random variables

(&i, yi), the sample correlation coefficient ( r) can be calculated as



Auto-correlation

• 0onsider folloing daily stream flos (in 1::: m!) in une 2::1 at 0hung Mei

8pper 6tation (<1: ha) located upstream of a river feeding to 5lover 0oveeservoir. %etermine its 1'day auto'correlation coefficient, i.e., ρ(=t, =t>1).

• 2? pairs@ A(=t, =t>1)B A(=1, =2), (=2, =!), , (=2?, =!:)B;

elevant sample statistics@ n2?

3he 1'day auto'correlation is :.#!?

Day (t) Flow Q(t) Day (t) Flow Q(t) Day (t) Flow Q(t)

1 8.35 11 313.89 21 20.06

2 6.78 12 480.88 22 17.52

3 6.32 13 151.28 23 116.13

4 17.36 14 83.92 24 68.255 191.62 15 44.58 25 280.22

6 82.33 16 36.58 26 347.53

7 524.45 17 33.65 27 771.30

8 196.77 18 26.39 28 124.20

9 785.09 19 22.98 29 58.00

10 562.05 20 21.92 30 44.08

111C<.22; 2!:.:<; 1CD.#E; 22?.1Dt t t Q t QQ S Q S

++= = = =



Chung Mei Upper Daily Flow

10 20 30

0

100

200

300

400

500

600

700

800

Day

F l o w ( 1 0 0 0 c u b i c m e t e r s )

1 2 3 4 5

1.0

0.8

0.6

0.4

0.2

0.0

0.20.4

0.6

0.8

1.0

F u t o c o r r e l a t i o n

Futocorrelation for une 2::1 %aily Glos at 0hung Mei 8pper, 4-

3ime lags (%ays)

:

1::

2::

!::

#::

::

<::

D::

C::

?::

: 2:: #:: <:: C:: 1:::

=(t), 1::: mH!

= ( t > 1 ) , 1 : : : m H !



Regression Models

• due to the presence of uncertainties a deterministic functional

relationship generally is not very appropriate or realistic.

• 3he deterministic model form can be modified to account for

uncertainties in the model as

* g(+1, +2, , +- ) > ε

here ε model error term ith E(ε):, ar(ε)σ2

.

• n engineering applications, functional forms commonly used for

establishing empirical relationships are

I Fdditive@ * β: > β1+1 > β2+2 > > β- +- >ε

I Multiplicative@ +ε.- 21 J

-

J

2

J

1: +...++J* =



Least Square Method

6uppose that there are n pairs of data, A(& i, yi)B, i1, 2,.. , n and a plot of

these data appears as

/hat is a plausible mathematical model describing & 7 y relationK

&

y



Least Square Method

0onsidering an arbitrary straight line, y β:>β1 &, is to be fitted through thesedata points. 3he Luestion is /hich line is the most representativeNK

1 β1

β:

&i

&

yi

yiH

y β:>β1 &H

ei yi I yi error (residual)H

y

1 β1

β:

&i

&

yi

yiHyiH

y β:>β1 &Hy β:>β1 &Hy β:>β1 &H

ei yi I yi error (residual)Hei yi I yi error (residual)H

y



Least Square Criterion

• /hat are the values of β: and β1 such that the resulting line bestN fits

the data pointsK

• Out, ait PPP /hat goodness'of'fit criterion to use to determine among

all possible combinations of β: and β1 K

• 3he least sLuares (Q6) criterion states that the sum of the sLuares of

errors (or residuals, deviations) is minimum. Mathematically, the Q6

criterion can be ritten as@

• Fny other criteria that can be usedK



Noral !quations for LS Criterion

• 3he necessary conditions for the minimum values of % are@

and

• E&panding the above eLuations

• Rormal eLuations@

::

=∂∂β D :

1

=∂∂β D

( )[ ]( )

( )[ ]( )

=−+−=∂∂

=−+−=∂∂

⇒

∑

∑

=

=

n

i

iii

n

i

ii

x x y D

x y D

1

1:

1

1

1:

:

:2

:12

β β β

β β β

[ ]

[ ]

=−−

=−−⇒

∑

∑

=10

=10

n

i

iii

n

i

ii

x y x

x y

1

1

:

:

β β

β β

=−−

=−−

=

1

=

0

=

=

10

=

:

1

2

11

11

n

i

i

n

i

i

n

i

ii

n

i

i

n

i

in

=

+

=

+

⇒

∑∑∑

∑∑

=1

=0

=

=1

=0

n

i

ii

n

i

i

n

i

i

n

i

i

n

i

i

y x x x

y xn

11

2

1

11

β β

β β



LS Solution "# Un$nowns%

−

−=

−

−=

−=

−

=

∑

∑

∑∑

∑∑∑

∑∑

=

=

==

===

1

11==

0

2

1

2

1

2

11

2

111

11

1

1

S

SS

n

n

n

n

nn

n

i

i

n

i

ii

n

i

i

n

i

i

n

i

i

n

i

i

n

i

ii

n

i

i

n

i

i

β

β β β



Fitting a &olynoial !q' (y LS Method

ni x x x y i

k

ik iii ,,2,1,2

2 ⋅⋅⋅=++⋅⋅⋅+++=

10 ε β β β β

Q6 criterion@

minimi$e % ( )[ ]∑=

210 +⋅⋅⋅+++−n

i

k

iiii x x x y1

22

κ β β β β

κ β β ,, ⋅⋅⋅0

6et k j for D

j,,2,1,:,: ⋅⋅⋅==∂

∂β

⇒ Rormal ELuations are@

=

+⋅⋅⋅+

+

=

=

+⋅⋅⋅+

+

=

+⋅⋅⋅+

+

∑∑∑∑

∑∑∑∑

∑∑∑

===

+1

=0

==

+

=1

=0

===10

n

i

k

ii

n

i

k

i

n

i

k

i

n

i

k

i

n

i

ii

n

i

k

i

n

i

i

n

i

i

n

i

i

n

i

k

i

n

i

i

x y x x x

x y x x x

y x xn

11

2

1

1

1

11

1

1

2

1

111

κ

κ

κ

β β β

β β β

β β β



Fitting a Linear Function of Se)eral *aria+les

ε β β β β κ +++++= 210 k x x x y 21

Rormal eLuations@

=

+⋅⋅⋅+

+

=

=

+⋅⋅⋅+

+

=

+⋅⋅⋅+

+

∑∑∑∑

∑∑∑∑

∑∑∑

===1

=0

===1

=0

===10

n

i

ik i

n

i

ik

n

i

iik

n

i

ik

n

i

ii

n

i

ik i

n

i

i

n

i

i

n

i

i

n

i

ik

n

i

i

x y x x x x

x y x x x x

y x xn

11

2

1

1

1

1

1

1

1

1

2

1

1

111

1

κ

κ

κ

β β β

β β β

β β β

Q6 criterion @

Minimi$e % ( )2

1

1

n

i i k

i

y x x xκ β β β β 0 1 2=

− + + + ⋅ ⋅ ⋅ + ∑

( )k β β β β ,,, 1 0=

6et : , :,1, 2, , j

D for j k β ∂ = = ⋅ ⋅ ⋅∂



Matri, For of Multiple Regression +y LS

+

=

1

0

nk nk nn

k

k

n x x x

x x x

x x x

y

y

y

ε

ε

ε

β

β

β

2

1

21

22221

11211

2

1

1

1

1

(Rote@ ij x ith

observation of the Tth independent variable)

or y > in short

Q6 criterion is@

min( ) ( )J+'yUJ+'yVVU

1

2

∑====

n

ii

D ε

.

6et /. =

∂∂

D , and result in@ /.-y-

0

)'(U =

3he Q6 solutions are@ ( ) y11 .2−=⇒ S



Measure of 3oodness-of-Fit

2

0oefficient of %etermination

( )∑

=

−

∑=−=

n

1i

2y

i

y

n

1i

2i

V

1

1 ' W of variation in the dependent variable, y, une&plained by

the regression eLuation;

W of variation in the dependent variable, y, e&plained by theregression eLuation.



!,aple 2 "LS Method%



LS !,aple



LS !,aple "Matri, Approach%



LS !,aple "+y Minita+ w4 /%



LS !,aple "+y Minita+ w4o /%



LS !,aple "5utput &lots%

correlation & regression (students)

Documents