1-1-11

Research Methodology in FinanceResearch Methodology in FinanceMGMG 522 : Session #1MGMG 522 : Session #1

Charn Soranakom, Ph.D.Charn Soranakom, Ph.D.Lecturer of FinanceLecturer of Finance

College of Management, Mahidol UniversityCollege of Management, Mahidol University

1-1-22

Steps in Writing a Research PaperSteps in Writing a Research Paper1.1. Select a topicSelect a topic

2.2. Review the relevant literatureReview the relevant literature

3.3. Develop a hypothesis to be testedDevelop a hypothesis to be tested

4.4. Gather the needed dataGather the needed data

5.5. Select the appropriate model and toolSelect the appropriate model and tool

6.6. Do the data analysisDo the data analysis

7.7. Interpret the resultsInterpret the results

8.8. ConcludeConclude the paperthe paper

*Note: The first three steps do not have to *Note: The first three steps do not have to

be in that order.be in that order.

1-1-33

1. Select a Topic1. Select a Topic First, you must be familiar with the topic you select.First, you must be familiar with the topic you select. To become familiar with a particular topic, you should To become familiar with a particular topic, you should

read many scholarly (peer-reviewed) papers in academic read many scholarly (peer-reviewed) papers in academic journals.journals.

Use online database to find the papers of your interest. Use online database to find the papers of your interest. Examples areExamples are– ProQuest (access through our e-learning system)ProQuest (access through our e-learning system)– EBSCO (access through our e-learning system)EBSCO (access through our e-learning system)– ScienceDirect (access through MU network)ScienceDirect (access through MU network)– JSTOR (access through MU network)JSTOR (access through MU network)

You should practice your searching technique. For You should practice your searching technique. For example, you should know what keyword to use to get example, you should know what keyword to use to get the results you wanted.the results you wanted.

The keyword should not be too general or too specific.The keyword should not be too general or too specific. Use “AND”, “OR”, “NOT” to limit your search time and Use “AND”, “OR”, “NOT” to limit your search time and

the number of results.the number of results.

1-1-44

You should also observe the trend in research You should also observe the trend in research from studying recently published articles in the from studying recently published articles in the top journals.top journals.

That way you will learn about the trend, i.e., That way you will learn about the trend, i.e., what are the hot topics being studied these what are the hot topics being studied these days.days.

You should not spend your valuable time You should not spend your valuable time studying and writing on the “dead” topics.studying and writing on the “dead” topics.

These are some of the leading journals in the These are some of the leading journals in the field of Finance.field of Finance.– Journal of Finance (JF)Journal of Finance (JF)– Journal of Business (JB)Journal of Business (JB)– Journal of Financial and Quantitative Analysis (JFQA)Journal of Financial and Quantitative Analysis (JFQA)– Review of Financial Studies (RFS), andReview of Financial Studies (RFS), and– Journal of Financial Economics (JFE)Journal of Financial Economics (JFE)– [See, “Finance Journal Rankings”, 2001][See, “Finance Journal Rankings”, 2001]

1-1-55

Other Sources for Journal ArticlesOther Sources for Journal Articles Social Science Research Network (SSRN) has Social Science Research Network (SSRN) has

downloadable working papers, forthcoming downloadable working papers, forthcoming papers, and published papers at papers, and published papers at www.ssrn.comwww.ssrn.com

Journal of Finance gives a free preview of the Journal of Finance gives a free preview of the forthcoming papers at forthcoming papers at www.afajof.org/journal/forthcoming.aspwww.afajof.org/journal/forthcoming.asp

Review of Financial Studies allows you to view Review of Financial Studies allows you to view abstracts of past and current papers at abstracts of past and current papers at rfs.oxfordjournals.orgrfs.oxfordjournals.org

Charles A. Dice Center has a list of journal Charles A. Dice Center has a list of journal websites, which can be found atwebsites, which can be found atwww.cob.ohio-state.edu/fin/journal/jofsites.htmwww.cob.ohio-state.edu/fin/journal/jofsites.htm

1-1-66

Research Areas in FinanceResearch Areas in FinanceThis is a list of possible topics (not comprehensive).This is a list of possible topics (not comprehensive).

Mergers and AcquisitionsMergers and Acquisitions Mutual Fund PerformanceMutual Fund Performance Ownership StructureOwnership Structure Dividend PolicyDividend Policy Share RepurchasesShare Repurchases Earnings and Stock ReturnsEarnings and Stock Returns Capital StructureCapital Structure Efficient Markets (using Event Studies)Efficient Markets (using Event Studies) Investment TechniquesInvestment Techniques DerivativesDerivatives IPOIPO Insider TradingInsider Trading Portfolio ManagementPortfolio Management Behavioral FinanceBehavioral Finance Market MicrostructureMarket Microstructure

1-1-77

2. Literature Review2. Literature Review In this step, you will try to form a In this step, you will try to form a

knowledge base.knowledge base. This step will require a lot of reading.This step will require a lot of reading. After doing a lot of reading, you should After doing a lot of reading, you should

have some ideas about what other authors have some ideas about what other authors have done or discovered.have done or discovered.

With your careful reading, you should be With your careful reading, you should be able to identify what has not been done or able to identify what has not been done or written about in that topic. At this stage, written about in that topic. At this stage, you will come up with your research you will come up with your research question.question.

1-1-88

3. Develop a Hypothesis3. Develop a Hypothesis Once, you identify your research question, you need Once, you identify your research question, you need

to translate your research question into a testable to translate your research question into a testable hypothesis.hypothesis.

Some research questions cannot be tested Some research questions cannot be tested empirically. You might have to work around that empirically. You might have to work around that problem.problem.

A hypothesis is something that can be tested with A hypothesis is something that can be tested with data using a statistical/econometric tool. Examples data using a statistical/econometric tool. Examples areare– Test of a single meanTest of a single mean– Comparison of the means (2 groups or more)Comparison of the means (2 groups or more)– Comparison of the variances (2 groups or more)Comparison of the variances (2 groups or more)– Test of relationship (to see if there is + or - relationship)Test of relationship (to see if there is + or - relationship)– Regression analysis and its variantsRegression analysis and its variants– Time series analysis and forecastingTime series analysis and forecasting

Examples of some hypotheses that CANNOT be tested.Examples of some hypotheses that CANNOT be tested.– Cause and effect (e.g., A causes B)Cause and effect (e.g., A causes B)– Some experiments in human (e.g., human behavior)Some experiments in human (e.g., human behavior)

1-1-99

4. Gather the Needed Data4. Gather the Needed Data Your hypothesis should tell you about what kind Your hypothesis should tell you about what kind

of data (variable) is needed for your analysis.of data (variable) is needed for your analysis. Variables can either be qualitative or Variables can either be qualitative or

quantitative.quantitative. Qualitative type of data needs to be converted Qualitative type of data needs to be converted

into number before performing any analysis (e.g., into number before performing any analysis (e.g., using dummy or binary variable)using dummy or binary variable)

Some variables cannot be measured directly, Some variables cannot be measured directly, hence, need to be proxied.hence, need to be proxied.

Types of variables areTypes of variables are– Nominal scale (e.g., gender or marital status)Nominal scale (e.g., gender or marital status)– Ordinal scale (e.g., grade or education)Ordinal scale (e.g., grade or education)– Interval scale (e.g., year or temperature)Interval scale (e.g., year or temperature)– Ratio scale (e.g., age or salary)Ratio scale (e.g., age or salary)Note:Note: These 4 types of data differ on whether the ratio of, These 4 types of data differ on whether the ratio of,

the difference between, and the order of two variables the difference between, and the order of two variables are meaningful or not .are meaningful or not .

1-1-1010

5. Select Your Tool5. Select Your Tool Almost all of your analysis will be performed Almost all of your analysis will be performed

using a statistical or econometric software using a statistical or econometric software package.package.– SPSS, Minitab, EViews, Limdep, and SASSPSS, Minitab, EViews, Limdep, and SAS

You should be familiar with the software package You should be familiar with the software package you use.you use.

Generally, the software package will not make a Generally, the software package will not make a mistake. It is usually the person who makes the mistake. It is usually the person who makes the mistake.mistake.

Each software package is usually different from Each software package is usually different from the others. If you don’t understand something, the others. If you don’t understand something, you should consult the manual. There is a danger you should consult the manual. There is a danger in using a tool that you don’t know how it in using a tool that you don’t know how it operates.operates.

If you input bad data or input erroneous data, you If you input bad data or input erroneous data, you will not get the correct result. Remember, GIGO will not get the correct result. Remember, GIGO (garbage in, garbage out).(garbage in, garbage out).

1-1-1111

6. Do the Analysis6. Do the Analysis When you have a big data set or many data When you have a big data set or many data

files, you should have or develop a system to files, you should have or develop a system to keep track of your data files.keep track of your data files.

Remember to back up your data.Remember to back up your data. You should make printouts (hard copy) You should make printouts (hard copy)

periodically. Better be safe than sorry). In periodically. Better be safe than sorry). In case you lost your files due to power outage, case you lost your files due to power outage, data corruption, or virus, you still can recover data corruption, or virus, you still can recover your data in the unlikely event that your your data in the unlikely event that your backup files failed.backup files failed.

Keep track of any changes you have made. Keep track of any changes you have made. Dating or numbering your documents.Dating or numbering your documents.

Don’t count on your memory. Write instead.Don’t count on your memory. Write instead.

1-1-1212

7. Interpret the Results7. Interpret the Results The analysis you have performed will need The analysis you have performed will need

to be converted into words that explain to be converted into words that explain what you’ve discovered.what you’ve discovered.

This part requires much of your This part requires much of your communication skills. A good paper communication skills. A good paper explains the results very well to the explains the results very well to the audienceaudience..

Be clear, concise, and complete.Be clear, concise, and complete. Don’t assume that the audience is Don’t assume that the audience is

sophisticated and, hence, will understand sophisticated and, hence, will understand or be able to read your mind.or be able to read your mind.

1-1-1313

8. Conclude the Paper8. Conclude the Paper After your hard work, you have to be able After your hard work, you have to be able

to tell the audience about your research in to tell the audience about your research in overall in writing.overall in writing.

In your conclusion, you should summarize In your conclusion, you should summarize your paper in one or two pages.your paper in one or two pages.

The conclusion section is the part where The conclusion section is the part where interested readers will learn more (than interested readers will learn more (than from reading your abstract) about your from reading your abstract) about your research in a short time. The conclusion research in a short time. The conclusion should stateshould state– Your research hypothesisYour research hypothesis– Your data and tools for the analysisYour data and tools for the analysis– Your major findings and what they meanYour major findings and what they mean

1-1-1414

Writing a PaperWriting a Paper Writing should not occur all at once after Writing should not occur all at once after

the research is completed.the research is completed. Rather, you should write down or update Rather, you should write down or update

your paper periodically, little by little.your paper periodically, little by little. Writing is an art that requires good Writing is an art that requires good

communication skills.communication skills. Remember, you will not be there by the Remember, you will not be there by the

reader’s side to explain your paper to reader’s side to explain your paper to him/her. The words you write that make him/her. The words you write that make up your paper will do the explanation.up your paper will do the explanation.

1-1-1515

Types of Research PapersTypes of Research Papers1.1. Theoretical PaperTheoretical Paper

This type of paper is concerned with developing This type of paper is concerned with developing a model to explain economic or financial a model to explain economic or financial phenomena. It usually requires a set of phenomena. It usually requires a set of assumptions and is involved with mathematical assumptions and is involved with mathematical models. Some of these models can be tested models. Some of these models can be tested empirically, while some cannot.empirically, while some cannot.

2.2. Empirical PaperEmpirical Paper This type of paper requires data for the This type of paper requires data for the

analysis. The relationships and explanations analysis. The relationships and explanations have already been developed or known in have already been developed or known in previous papers. Your job is to gather the data previous papers. Your job is to gather the data and test such relationships.and test such relationships.

1-1-1616

General Format of an Empirical PaperGeneral Format of an Empirical Paper1.1. Introduction and Literature reviewIntroduction and Literature review

– Tell the author’s motivation for doing this research.Tell the author’s motivation for doing this research.– Review the literature (what have been done or found).Review the literature (what have been done or found).– Tell how this paper is different from the others.Tell how this paper is different from the others.

2.2. Hypothesis developmentHypothesis development– This part will usually refer to or restate the previous findings.This part will usually refer to or restate the previous findings.– Tells what is to be tested in this paper and what theory backs up the Tells what is to be tested in this paper and what theory backs up the

hypothesis.hypothesis.

3.3. Data and AnalysisData and Analysis– Tell where the data came from and the tools for data analysis.Tell where the data came from and the tools for data analysis.

4.4. ResultsResults– State your results or what you find in your analysis.State your results or what you find in your analysis.– This part should answer your hypothesis stated earlier in the This part should answer your hypothesis stated earlier in the

“Hypothesis Development” section of your paper.“Hypothesis Development” section of your paper.

5.5. ConclusionConclusion– This is a very important part of your paper.This is a very important part of your paper.– It should briefly summarize your whole paper.It should briefly summarize your whole paper.– It should describe your paper in greater detail than your abstract.It should describe your paper in greater detail than your abstract.– Some authors also provideSome authors also provide

– a direction for future researcha direction for future research– a word of cautions or limitationsa word of cautions or limitations

1-1-1717

General Format of a Theoretical PaperGeneral Format of a Theoretical Paper1.1. Introduction and Literature reviewIntroduction and Literature review

– Similar to that of an empirical paper.Similar to that of an empirical paper.2.2. Assumptions of the modelAssumptions of the model

– There are many factors that influence or explain There are many factors that influence or explain economic behaviors.economic behaviors.

– We need to discard things that are not important and We need to discard things that are not important and focus only on the thing under the study.focus only on the thing under the study.

– Therefore, we need to provide some assumptions to Therefore, we need to provide some assumptions to exclude the things we don’t consider.exclude the things we don’t consider.

3.3. The ModelThe Model– Explain economic behaviors with relationships, Explain economic behaviors with relationships,

mathematical formulas, or theory.mathematical formulas, or theory.4.4. ResultsResults

– State your results. State your results. 5.5. ConclusionConclusion

– similar to that of an empirical papersimilar to that of an empirical paper..

1-1-1818

Types of DataTypes of Data Primary dataPrimary data

– Data you collect by yourself from the interviews or Data you collect by yourself from the interviews or observations.observations.

Secondary dataSecondary data– Data that already exist in print or electronic format.Data that already exist in print or electronic format.– Some are freely available, some are available from data Some are freely available, some are available from data

vendors and require subscription or per-usage fees.vendors and require subscription or per-usage fees.– Examples of data in electronic formatExamples of data in electronic format

CompuStat (from S&P)CompuStat (from S&P) DataStreamDataStream CRSP (from Center for Research in Securities Prices, University of CRSP (from Center for Research in Securities Prices, University of

Chicago)Chicago) SET SMART (from Stock Exchange of Thailand)SET SMART (from Stock Exchange of Thailand)

– Examples of data in print formatExamples of data in print format SET publicationsSET publications NewspaperNewspaper MagazinesMagazines Company annual reports to shareholdersCompany annual reports to shareholders

Data can also be categorized as time-series or cross-Data can also be categorized as time-series or cross-sectional.sectional.

1-1-1919

SET SMARTSET SMART A web-based databaseA web-based database There are two versionsThere are two versions

– Investor version (available for purchase): Non real-time Investor version (available for purchase): Non real-time data, plus limited historical datadata, plus limited historical data

– Enterprise version: Only historical data dated back to Enterprise version: Only historical data dated back to 19751975

Real-time price data are available free of charge Real-time price data are available free of charge at at www.set.or.thwww.set.or.th

Enterprise version can only be accessed from our Enterprise version can only be accessed from our campus at campus at hhttp://set.oz.cmmu.netttp://set.oz.cmmu.net (may require (may require initial set-up)initial set-up)

1-1-2020

Other ConsiderationsOther Considerations PlagiarismPlagiarism Referencing system and formatReferencing system and format Writing a thesis or independent studyWriting a thesis or independent study

– Usually there are five chapters in a thesis or Usually there are five chapters in a thesis or independent study. (The names of the chapters independent study. (The names of the chapters can differ from the names given below.)can differ from the names given below.)

Chapter 1: IntroductionChapter 1: Introduction Chapter 2: Literature ReviewChapter 2: Literature Review Chapter 3: Data and MethodologyChapter 3: Data and Methodology Chapter 4: Results of the StudyChapter 4: Results of the Study Chapter 5: ConclusionChapter 5: Conclusion

Publication considerationsPublication considerations

1-1-2121

An Overview of Regression AnalysisAn Overview of Regression Analysis& Ordinary Least Squares (OLS)& Ordinary Least Squares (OLS)

(Ch. 1 & 2)(Ch. 1 & 2)

1-1-2222

What is Econometrics?What is Econometrics?

Econometrics tries to model and Econometrics tries to model and quantify actual economic quantify actual economic phenomena.phenomena.

Economics provides us with the Economics provides us with the theory that is abstract in nature.theory that is abstract in nature.

Econometrics uses real data and Econometrics uses real data and quantifies the theory.quantifies the theory.

1-1-2323

Uses of EconometricsUses of Econometrics

1.1. To model economic phenomenaTo model economic phenomena– What X’s explain Y?What X’s explain Y?– How X’s affect Y and what are their magnitudes?How X’s affect Y and what are their magnitudes?– For example,For example,

Economic theory: Demand = f(Price, Income)Economic theory: Demand = f(Price, Income) Econometrics: Demand = 100-0.5*Price+5*IncomeEconometrics: Demand = 100-0.5*Price+5*Income

2.2. To test hypotheses about economic theoryTo test hypotheses about economic theory– How strong is the relationship between each X and Y?How strong is the relationship between each X and Y?– Is it significant?Is it significant?– Is it what we expected?Is it what we expected?– Is the model a good fit overall?Is the model a good fit overall?

3.3. To forecast the futureTo forecast the future– What is the value of Y, given X’s?What is the value of Y, given X’s?

1-1-2424

Econometric ApproachesEconometric Approaches

Given a data set to work with, we Given a data set to work with, we could use many different approaches could use many different approaches to come up with a model or equation.to come up with a model or equation.

One of many econometric One of many econometric approaches is regression analysis.approaches is regression analysis.

By far, By far, regressionregression analysis analysis isis the the most widely used most widely used approachapproach..

1-1-2525

What is Regression Analysis?What is Regression Analysis? Regression analysis is a statistical method Regression analysis is a statistical method

that tries to specify a single equation that that tries to specify a single equation that describes what, how, and how much X’s describes what, how, and how much X’s affect Y.affect Y.

Y is called a dependent variable.Y is called a dependent variable. X’s are called independent variables.X’s are called independent variables. Although Y is called a dependent variable, Although Y is called a dependent variable,

it doesn’t mean that Y is dependent on it doesn’t mean that Y is dependent on X’s, or X’s cause Y.X’s, or X’s cause Y.

Regression analysis does not imply cause Regression analysis does not imply cause and effect or causality.and effect or causality.

1-1-2626

Single-Equation Linear ModelsSingle-Equation Linear Models A single-equation linear model takes the form,A single-equation linear model takes the form,

Y = Y = ββ00 + + ββ11XX It is called “single-equation” because there are no It is called “single-equation” because there are no

other equations that specify relationships between other equations that specify relationships between X and Y.X and Y.

It is called “linear” because X is linearly related to It is called “linear” because X is linearly related to Y. In other words, if you were to plot a graph Y. In other words, if you were to plot a graph between X and Y, you would get a straight line.between X and Y, you would get a straight line.

ββ0 0 is called the intercept. It tells the value of Y if X is called the intercept. It tells the value of Y if X is zero. We normally do not care much about is zero. We normally do not care much about ββ00..

Our interest is mainly on Our interest is mainly on ββ11, which is called the , which is called the coefficient. It measures the direction and the coefficient. It measures the direction and the magnitude of change in Y if X changes by one unit.magnitude of change in Y if X changes by one unit.

1-1-2727

What does “linear” mean?What does “linear” mean? A regression equation can be linear in coefficients or A regression equation can be linear in coefficients or

linear in variables.linear in variables. Linear in coefficients: all Linear in coefficients: all ββ’s’s must be in their simplest must be in their simplest

form (sometimes, after a transformation).form (sometimes, after a transformation). Linear in variables: all X’s must be in their simplest Linear in variables: all X’s must be in their simplest

form (sometimes, after a modification). A plot of X form (sometimes, after a modification). A plot of X against Y must be a straight line.against Y must be a straight line.

Q: Which of the followings are linear in coefficients Q: Which of the followings are linear in coefficients and which are linear in variables?and which are linear in variables?

Y = Y = ββ00 + + ββ11XX

Y = Y = ββ00 + + ββ11lnXlnX

lnY= lnY= ββ00 + + ββ11XX

Y = Y = ββ00 + + ββ11XX11 + + ββ22XX22 + + ββ33XX11XX22210

21 XXeY

1-1-2828

What do we mean by “linear” regression?What do we mean by “linear” regression?

We mean that the regression equation We mean that the regression equation must be “linear in coefficients”.must be “linear in coefficients”.

The regression equation DOES NOT have The regression equation DOES NOT have to be “linear in variables.”to be “linear in variables.”

Therefore, our regression equation can be, Therefore, our regression equation can be, ln Y = ln Y = ββ00 + + ββ11XX33, and still satisfies the , and still satisfies the linearity in coefficients condition.linearity in coefficients condition.

Linearity in coefficients condition will Linearity in coefficients condition will always be satisfied if you can write the always be satisfied if you can write the regression equation in the form,regression equation in the form,

f(Y) = f(Y) = ββ00 + + ββ11f(X).f(X).

1-1-2929

Stochastic or Random Error TermStochastic or Random Error Term Usually the relationship between X and Y is not Usually the relationship between X and Y is not

exact.exact. So, there will usually be some variation in Y that So, there will usually be some variation in Y that

cannot be explained completely by X.cannot be explained completely by X. Causes of this random error areCauses of this random error are

– Omitted variables,Omitted variables,– Measurement error,Measurement error,– Incorrect functional form, orIncorrect functional form, or– Purely random error.Purely random error.

Regression analysis only allows an additive Regression analysis only allows an additive random error term like, Y = random error term like, Y = ββ00 + + ββ11X + X + ..

Regression analysis does not allow a Regression analysis does not allow a multiplicative random error term like, Y = multiplicative random error term like, Y = ββ00 + + ββ11X + XX + X..

1-1-3030

About a Regression EquationAbout a Regression Equation A typical regression equation is written as,A typical regression equation is written as,

Y = Y = ββ00 + + ββ11X + X + ββ00 + + ββ11X is the deterministic term.X is the deterministic term. is the stochastic or random error term.is the stochastic or random error term. Y = Y = ββ00 + + ββ11X + X + is the “population” regression is the “population” regression

equation, which is unknown.equation, which is unknown. Given a value of X, we can estimate only the Given a value of X, we can estimate only the

expectedexpected value of Y, not the value of Y, not the truetrue value of Y. value of Y. Our job is to estimate the “population” regression Our job is to estimate the “population” regression

equation with a sample set of data. As a result, equation with a sample set of data. As a result, we will get the “estimated” regression equation we will get the “estimated” regression equation like,like, XY 10

ˆˆˆ

1-1-3131

Estimate a Regression EquationEstimate a Regression Equation

YY1 1 == 00 + + 11*X*X11 + + 11

YY2 2 == 00 + + 11*X*X22 + + 22

YY3 3 == 00 + + 11*X*X33 + + 33

YY4 4 == 00 + + 11*X*X44 + + 44

YY5 5 == 00 + + 11*X*X55 + + 55

…… …… …… ……

…… …… …… ……

YYn n == 00 + + 11*X*Xnn + + nn

1-1-3232

EstimationEstimationEstimated Regression EquationEstimated Regression EquationTrue Regression EquationTrue Regression Equation

0β 00 bor β̂

11 bor β̂1β

ε e

ebbY

or e,Xβ̂β̂Y

10

10

εXββY 10

1-1-3333

Estimation (continue…)Estimation (continue…)

is not observable.is not observable. But we can observe e (called residuals).But we can observe e (called residuals).

There are many ways to come up with the There are many ways to come up with the estimates for estimates for 00 and and 11..

Given data on XGiven data on Xii and Y and Yii..

If we change If we change 00 and/or and/or 11, e, eii will change as will change as well.well.

iii YYe ˆ

iii XYe 10ˆˆ

1-1-3434

Ordinary Least Squares: OLSOrdinary Least Squares: OLS

OLS is a regression estimation method OLS is a regression estimation method that minimizes the sum of the squared that minimizes the sum of the squared residuals.residuals.

OLS is sometimes called an estimator.OLS is sometimes called an estimator. Estimator = Estimation Method.Estimator = Estimation Method.

n

iieOLS

1

2minimizes

n

iii YYOLS

1

2ˆminimizes

1-1-3535

Why use OLS?Why use OLS?

1.1. OLS is quite easy to use.OLS is quite easy to use.

2.2. Minimizing the sum of the squared Minimizing the sum of the squared residuals is a theoretically sound residuals is a theoretically sound objective.objective.* The other objective could be to minimize * The other objective could be to minimize ΣΣeeii or or ΣΣ|e|eii|, but |, but there are problems with this objective.there are problems with this objective.

3.3. OLS estimates possess a number of useful OLS estimates possess a number of useful properties.properties.

– The regression line passes through the means of X & Y.The regression line passes through the means of X & Y.– The sum of the residuals is The sum of the residuals is exactlyexactly zero. zero.– Under some restrictions, OLS is the best estimator.Under some restrictions, OLS is the best estimator.

1-1-3636

How do we estimate How do we estimate 00 and and 11??

The objective function is,The objective function is,

To find To find 00 and and 11 that minimize the objective that minimize the objective function, we take a partial derivative of the function, we take a partial derivative of the objective function with respect to each objective function with respect to each parameter.parameter.

n

iii XY

1

2

10ˆˆMin

01ˆˆ2

ˆ

ˆˆ

110

0

1

2

10

n

iii

n

iii

XYXY

0ˆˆ2

ˆ

ˆˆ

110

1

1

2

10

n

iiii

n

iii

XXYXY

1-1-3737

EstimatesEstimates

n

ii

n

ii XnY

110

1

ˆˆ

n

ii

n

ii

n

iii XXYX

1

21

10

1

ˆˆ

n

ii

i

n

ii

XX

YYXX

1

2

11̂

XY 10ˆˆ

Normal EquationsNormal Equations

1-1-3838

TSS, ESS, and RSSTSS, ESS, and RSS

Total Sum of Squares,Total Sum of Squares,

Explained Sum of Squares,Explained Sum of Squares,

Residual Sum of Squares,Residual Sum of Squares,

n

ii YYTSS

1

2

n

ii YYESS

1

2ˆ

n

ii YYRSS

1

2ˆ

1-1-3939

For a given data set, TSS cannot For a given data set, TSS cannot change and does not depend on the change and does not depend on the estimation method.estimation method.

OLS is the method that guarantees to OLS is the method that guarantees to find the estimates of find the estimates of 00 and and 11 that that minimize RSS.minimize RSS.

We usually delegate the task of We usually delegate the task of finding the estimates of finding the estimates of 00 and and 11 to to the computer, especially for a the computer, especially for a multiple regression analysis.multiple regression analysis.

1-1-4040

Coefficients in a Multiple RegressionCoefficients in a Multiple Regression

The coefficient in a multiple regression The coefficient in a multiple regression measures how much Y will change if the measures how much Y will change if the dependent variable in questiondependent variable in question changes by changes by one unit, one unit, holding constantholding constant the other the other independent variables included in the independent variables included in the regression equation.regression equation.

This is equivalent to a partial derivation of This is equivalent to a partial derivation of Y with respect to each X.Y with respect to each X.

Multiple regression analysis technically Multiple regression analysis technically allows us to hold constant the influences allows us to hold constant the influences of other variables and focus on the of other variables and focus on the influence of just one variable at a time influence of just one variable at a time (where otherwise impossible).(where otherwise impossible).

1-1-4141

Goodness of Fit Test: RGoodness of Fit Test: R22

We want the regression equation to We want the regression equation to fit the data well.fit the data well.

One way to measure the overall One way to measure the overall goodness of fit is to compare ESS to goodness of fit is to compare ESS to TSS.TSS.

That measure is RThat measure is R22.. RR22 = ESS/TSS = 1 – (RSS/TSS) = ESS/TSS = 1 – (RSS/TSS) 0 ≤ R0 ≤ R2 2 ≤ 1≤ 1

1-1-4242

In a time-series type of data, RIn a time-series type of data, R22 of of 0.9 may be considered a good fit.0.9 may be considered a good fit.

On the other hand, in a cross-On the other hand, in a cross-sectional type of data, Rsectional type of data, R22 of 0.5 may of 0.5 may be considered a good fit.be considered a good fit.

How high the value of RHow high the value of R22 is is considered a good fit is very considered a good fit is very subjective.subjective.

1-1-4343

Adj-RAdj-R22

When you add one more variable to the When you add one more variable to the regression equation, your Rregression equation, your R22 value will either stay value will either stay the same or increase. There is no way that the Rthe same or increase. There is no way that the R22 value will decline. value will decline. Why?Why?

Each time we add one independent variable to Each time we add one independent variable to the regression equation, we lose one more the regression equation, we lose one more degree of freedom.degree of freedom.

To adjust for the loss of degrees of freedom as a To adjust for the loss of degrees of freedom as a result of the addition of independent variables, result of the addition of independent variables, we calculate the adjusted Rwe calculate the adjusted R22, or Adj-R, or Adj-R22..

n = # of observations, K = # of independent variables.n = # of observations, K = # of independent variables.

1

111 22

Kn

nRRAdj

1-1-4444

Cautions about RCautions about R22 and Adj-R and Adj-R22

Both RBoth R22 and Adj-R and Adj-R22 measure the measure the overalloverall goodness of fit only.goodness of fit only.

Obtaining a high value of RObtaining a high value of R22 or Adj-R or Adj-R22 for for the regression equation is desirable, but it the regression equation is desirable, but it is not all that matters!is not all that matters!

Comparing two competing regression Comparing two competing regression models on the basis of Rmodels on the basis of R22 or Adj-R or Adj-R22 can be can be misleading and dangerous.misleading and dangerous.

You should evaluate the competing You should evaluate the competing equations with other criteria (which will be equations with other criteria (which will be discussed in later session).discussed in later session).