Applications of Least Squares in EconometricsAuthor(s): Gordon FisherSource: The Canadian Journal of Economics / Revue canadienne d'Economique, Vol. 29, SpecialIssue: Part 2 (Apr., 1996), pp. S548-S550Published by: Wiley on behalf of the Canadian Economics AssociationStable URL: http://www.jstor.org/stable/136104 .

Accessed: 12/06/2014 17:25

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact support@jstor.org.

.

Wiley and Canadian Economics Association are collaborating with JSTOR to digitize, preserve and extendaccess to The Canadian Journal of Economics / Revue canadienne d'Economique.

http://www.jstor.org

This content downloaded from 194.29.185.216 on Thu, 12 Jun 2014 17:25:00 PMAll use subject to JSTOR Terms and Conditions

Applications of least squares in econometrics

GORDON FISHER Concordia University

I. INTRODUCTION

The theory of least squares is taken to encompass any estimation procedure, algorithm or iterative scheme whose solution is obtained either by minimizing the square of a selected euclidean length, or equivalently, by requiring conformity to the corresponding orthogonality condition. Viewed in this way, least squares may accommodate not only those methods that bear its name, but also generalized instrumental variables estimation (GIVE), the generalized method of moments (GMM), generalized minimum chi-square estimation sometimes referred to as summary generalized least squares (Cragg 1993), and certain problems in robust estimation and restricted maximum-likelihood estimation, under the broad heading of iterated re-weighted least squares (IRLS); see e.g. Green (1984). In this broad context, the first aim of this paper is to synthesize various

different methods of econometrics in terms of three characterizations of least squares. A second aim is to illustrate these characterizations with examples, paying particular attention to applications of IRLS to restricted maximum- likelihood estimation in econometrics. A third aim is to stress the importance of the freedom equation approach to restricted estimation and to provide a proof that restricted estimation is never less efficient than unrestricted estimation. Proofs of the efficiency of restricted estimation are standard for linear models in econometrics, but no proofs are available of a more general kind. The principal vehicle of discussion is co-ordinate-free multivariate analysis,

by which is meant that form in which the basic elements (e.g. variables) and the derived elements (e.g. estimators) may be treated as points in vector spaces without explicit bases. The analysis may then be described geometrically and can often usefully be illustrated diagrammatically.

Canadian Journal of Economics Revue canadienne d'Economique, XXIX, Special Issue April avril 1996. Printed in Canada Imprime au Canada

0008-4085 / 96 / S548-550 $1.50 ? Canadian Economics Association

This content downloaded from 194.29.185.216 on Thu, 12 Jun 2014 17:25:00 PMAll use subject to JSTOR Terms and Conditions

Applications of least squares S549

II. SCALAR PRODUCTS

Much of euclidean geometry rests on the possibility of measuring the lengths of straight lines and the angles between them. The way this is done in n-dimensional euclidean (vector) space is by defining a scalar product on the space, i.e. a function obeying certain rules that maps any pair of vectors onto the real line. Of course, measurements can be made in different ways and so, as the scalar product is varied, so are the corresponding measurements of the lengths of vectors and the angles between them. Lengths of vectors are important in defining test-statistics and the squared cosine of an angle corresponds to R2. Thus, in a given practical problem, when the scalar product is clhanged, it is a simple matter to write down corresponding test-statistics and R2, without the need to re-think and re-work each case. In classical least squares, the natural scalar product is used and measurements are thus made in terms of the original data. Generalized least squares (GLS) uses a scalar product depending on the inverse of the error dispersion, because the observations need weighting. Other scalar products represent different ways of weighting the original data. This paper argues that there are three fundamental scalar products in

econometrics. The first corresponds to the GLS scalar product, generalized further to permit the error dispersion to be singular. The second corresponds to the scalar product arising in GIVE which, while obeying certain requirements, is nevertheless not unique. The third corresponds to IRLS and involves only invertible linear transformations. The first scalar product is the most difficult to define, because the subspace spanning the error dispersion is essentially the subspace in which the dependent vector lies and, when the error dispersion is singular, there is no guarantee that the regression manifold lies within it. This implies a need to re-define a least squares solution. Once the issue of singularity is settled the GIVE scalar product can be made precise and it is shown to be well defined only in the case of identifiability. The IRLS scalar product is always well defined.

HI. APPLICATIONS

Examples of applications of the first and second scalar products are given in Fisher (1994). The same paper also gives details of how the IRLS scalar product may be applied in logistic regression, to the estimation of a restricted reduced form using the freedom equation specification, as distinct from the restraint equation specification, and finally to robust regression. All three examples illustrate how least squares may be adapted to handle a broad

This content downloaded from 194.29.185.216 on Thu, 12 Jun 2014 17:25:00 PMAll use subject to JSTOR Terms and Conditions

S550 Gordon Fisher

range of estimation problems. A proof is presented in Fisher (1994) of the relative efficiency of restricted maximum-likelihood estimation. This may be extended to restricted quasi-likelihood estimation.

IV. DISCUSSION

Ways to extend the discussion into non-linear problems and also to GMM estimation are given in Fisher (1992, 1994).

REFERENCES

Cragg J.G. (1993) 'The asymptotic efficiency of summary generalized least squares estimators.' Paper to the Canadian Econometric Study Group Conference, Toronto

Fisher G.R. (1992) 'Teaching econometric theory from the co-ordinate-free viewpoint.' In Proceedings of ICOTS 3, ed. D. Verre-Jones (Voorbug: ISI) 303-13

(1994) 'Applications of the theory of least squares in econometrics.' Concordia University Discussion Paper 9410

Green P.J. (1984) 'Iteratively reweighted least squares for maximum- likelihood estimation, and some robust resistant alternatives' with discussion. Journal of the Royal Statistical Society Series B 46, 149-92

This content downloaded from 194.29.185.216 on Thu, 12 Jun 2014 17:25:00 PMAll use subject to JSTOR Terms and Conditions