Appendix B

Further reading

B.1. Statistics and data analysis

Alan Agresti: An Introduction to Categorical Data Analysis, Wiley, 2nd edn, 2007.

An excellent book which provides a survey of categorical (i.e. qualitative) data. Two chapters

are added in the second edition on the subject of correlated categorical data, which occur in

longitudinal studies with repeated measurements.

T.W. Anderson: An Introduction to Multivariate Statistical Analysis, Wiley-Interscience, 3rd

edn, 2003.

A classic from 1958, updated and still very useful. Not the easiest read, but rigorous and very

comprehensive, it covers clustering, factor analysis (PCA and MCA), classification, boot-

strapping, etc.

Jean-Paul Benz�ecri: Histoire et Pr�ehistoire de l’Analyse des Donn�ees, Dunod, new edn, 1982

(out of print).

A fascinating story, written in a sparkling style by a leading statistician who is also a thinker.

George Casella and Roger L. Berger: Statistical Inference, Duxbury Press, 2nd edn, 2001.

An excellent textbook, comprehensive and rigorous, for advanced students.

Christophe Croux, Jean-Jacques Droesbeke, Pierre-Louis Gonzalez, Christian Gourieroux,

Gentiane Haesbroeck, Michel Lejeune, Gilbert Saporta, and Michel Tenenhaus: Mod�elesStatistiques pour Donn�ees Qualitatives, Editions Technip, 2005.

The proceedings of a very interesting seminar organized by the Soci�et�e Francaise de

Statistique, on classification methods, logistic regression, the log-linear model, counting

models, generalized linear models, and PLS regression, with applications in medicine and

insurance. Worth reading to discover the state of the art on these subjects.

Data Mining and Statistics for Decision Making, First Edition. Stéphane Tufféry.

© 2011 John Wiley & Sons, Ltd. Published 2011 by John Wiley & Sons, Ltd. ISBN: 978-0-470-68829-8

Bradley Efron and Robert J. Tibshirani: An Introduction to the Bootstrap, Chapman &

Hall, 1994.

A reference work on the bootstrap method, clear and comprehensive (the first author devised

the method).

Brigitte Escofier and J�erome Pag�es: Analyses Factorielles Simples et Multiples, Dunod,

4th edn, 2008.

A very comprehensive book, helpful for the reader who wishes to be able to read a

factor analysis.

Stanton A. Glantz and Bryan K. Slinker: Primer of Applied Regression and Analysis of

Variance, McGraw-Hill, 2000.

Avery full description of linear regression and analysis of variance, with an introduction to the

Cox semi-parametric survival models.

Joseph F. Hair, Bill Black, Barry Babin, Rolph E. Anderson, and Ronald L. Tatham:

Multivariate Data Analysis, Prentice Hall, 6th edn, 2005.

A real doorstop of a book (more than 900 pages), but full of excellent and accessible practical

information, without toomany equations. It provides examples of each type of analysis (factor

analysis, canonical analysis, MANOVA, multiple regression, discriminant analysis, conjoint

analysis, structural equations, etc.), with SAS and SPSS syntax, and an interpretation of the

outputs. It also covers data cleaning and missing values.

David J. Hand: Information Generation: How Data Rule Our World, Oneworld

Publications, 2007.

An excellent survey of statistics and its place in the modern world, written in a clear and

attractive style by a leading expert, of interest to statisticians and others.

David J. Hand: Statistics: A Very Short Introduction, Oxford University Press, 2008.

Avery useful introduction to statistics, detailed and concise, covering the basics of the subject,

from data collection to modelling and computing, via probability theory and inference.

David W. Hosmer and Stanley Lemeshow: Applied Logistic Regression, Wiley, 1989;

2nd edn, 2000.

A well-known work on logistic regression, with numerous examples from the field of

biostatistics. Starting from a basic knowledge of the linear model, this book provides a

remarkably clear explanation of the principles and applications of logistic regression.

Ludovic Lebart, Alain Morineau, and Marie Piron: Statistique Exploratoire Multidimension-

nelle: Visualisations et Inf�erences en Fouille de Donn�ees, Dunod, 4th edn, 2006.

One of the best titles on ‘French-style’ data analysis (factor analysis), also covering

developments in clustering, discriminant analysis, log-linear models, decision trees, neural

networks, validation methods, etc.

Kanti V. Mardia, J. T. Kent, and J. M. Bibby: Multivariate Analysis, Academic Press, 1980.

This book is less recent and does not cover subjects such as the bootstrap, but it is still

a standard work on multivariate analysis, noted for its clear and elegant presentation of

the subject.

Peter McCullagh and John A. Nelder: Generalized Linear Models, Chapman & Hall,

2nd edn, 1989.

676 FURTHER READING

A fundamental treatise on generalized linear models. John Nelder was one of the inventors of

this method. Rather a challenging read, but of excellent quality.

R. H. Myers, D. C. Montgomery, and G. C. Vining: Generalized Linear Models with

Applications in Engineering and the Sciences, Wiley-Interscience, 2001.

A clear introduction to generalized linear models.

Patrick Na€ım, Pierre-Henri Wuillemin, Philippe Leray, Olivier Pourret, and Anna Becker:

R�eseaux Bay�esiens, Editions Eyrolles, 3rd edn, 2007.

A reference work on Bayesian networks, written by specialists and practitioners in this field.

Jean-Pierre Nakache and Josiane Confais: Statistique Explicative Appliqu�ee, Editions

Technip, 2003.

A recent book on the main predictive methods: linear discriminant analysis, logistic

regression and decision trees. The theory is described in a concise and detailed way, followed

by a variety of illuminating examples produced with the SAS and SPAD software.

Jean-Pierre Nakache and Josiane Confais: Approche Pragmatique de la Classification: Arbres

Hi�erarchiques et Partitionnements, Editions Technip, 2004.A recent and very comprehensive text which is to descriptive clustering methods as the

previous book is to predictive methods – an excellent reference work in a style characterized

by its thoroughness in the theoretical sections and an educational approach in the examples of

application, with many references to the recent literature, Internet sites and the latest versions

of software, mainly, but not exclusively, SAS and SPAD.

Olivier Pourret, Patrick Na€ım, and Bruce Marcot: Bayesian Networks: A Practical Guide to

Applications, John Wiley & Sons Ltd, 2008.

A general introduction to Bayesian networks, illustrated with 20 case studies in the fields of

medicine, science, engineering, robotics, finance, risk, etc.

Gilbert Saporta: Probabilit�es, Analyse des Donn�ees et Statistique, Editions Technip,

2nd edn, 2006.

This is the standard work (in French), one to be kept handy at all times, offering a precise and

comprehensive treatment of the subject. It contains all the essentials of probability calcula-

tion, multidimensional data analysis (factor analysis, clustering) and statistics for decision

making (tests, estimation, regression and discrimination).

Michel Tenenhaus: La R�egression PLS: Th�eorie et Pratique, Editions Technip, 1998.

Everything you need to know about PLS regression, used ever more widely in industry for

manipulating a large number of strongly collinear independent variables.

Sylvie Thiria, Olivier Gascuel, Yves Lechevallier, and St�ephane Canu: Statistique et

M�ethodes Neuronales, Dunod, 1997.A collection of technical papers providing a very thorough survey of neural networks (several

papers on the multilayer perceptron), their application to problems of classification, predic-

tion and clustering, and Vapnik’s learning theory.

St�ephane Tuff�ery: Etude de Cas en Statistique D�ecisionnelle, Editions Technip, 2009.Based on a data set from the insurance sector, available on the publisher’s website, this book

applies the principles of statistics to a case study covering two classic problems, namely the

construction of customer segmentation, and the creation of a propensity score for the purchase

STATISTICS AND DATA ANALYSIS 677

of a product. The resources of the SAS software, especially SAS/STAT, are used to show that

rigour and efficiency can be combined.

LarryWasserman: All of Statistics: A Concise Course in Statistical Inference, Springer, 2004.

In just over 400 pages, this book provides an excellent overview of probability and statistics,

without assuming a large amount of previous knowledge on the reader’s part, and gives a

concise and clear introduction to the main principles, including the most recent ones such

as the bootstrap, support vector machines, Bayesian inference and Markov chain Monte

Carlo methods.

B.2. Data mining and statistical learning

Michael J. A. Berry and Gordon Linoff: Data Mining Techniques: for Marketing, Sales, and

Customer Relationship Management, John Wiley & Sons, 2nd edn, 2004.

A readable book on data mining, more useful for its examples than its technical content

(it does not cover factor analysis, discriminant analysis, logistic regression, or their more

recent developments).

Michael J. A. Berry and Gordon Linoff: Mastering Data Mining: The Art and Science of

Customer Relationship Management, John Wiley & Sons, 2000.

The second book on data mining by the same authors, with 20 case studies.

L. Breiman, J.H. Friedman, R.A. Olshen, and C.J. Stone:Classification and Regression Trees,

Wadsworth, 1984.

A basic work on decision trees, written by the inventors of CART.

Bertrand Clarke, Ernest Fokoue, and Hao Helen Zhang: Principles and Theory for Data

Mining and Machine Learning, Springer, 2009.

A recent reference work on data mining, methods of selecting variables, clustering,

regression, ensemble methods, etc., with numerous examples for which the R code

is provided.

Richard O. Duda, Peter E. Hart, and David G. Stork: Pattern Classification, Wiley-

Interscience, 2nd edn, 2000.

A new edition of the classic from 1973, also very well illustrated, accompanied by exercises,

covering numerous techniques ranging from neural networks to Markov models, taking in

mixture models for clustering, with interesting sections on the bias–variance dilemma,

overfitting and ensemble methods.

Paolo Giudici and Silvia Figini: Applied Data Mining: for Business and Industry, Wiley, 2nd

edn, 2009.

This book is aimed at a broad spectrum of readers interested in data mining, applied statistics,

databases and econometrics, providing a simple description of data mining in the 150-page

introductory section. The second part presents seven case studies, each ten pages long, in

the following fields: web mining, market basket analysis, credit risk, lifetime value, etc. The

question of software is considered and some of the case studies use R.

David J. Hand, Heikki Mannila, and Padhraic Smyth: Principles of Data Mining, MIT

Press, 2001.

678 FURTHER READING

Avery well-written reference work, by experienced teachers of the subject, with rather fewer

details of the algorithms than the book by Hastie, Tibshirani and Friedman.

Trevor Hastie, Robert Tibshirani, and Jerome H. Friedman: The Elements of Statistical

Learning: Data Mining, Inference and Prediction, 2nd edn, Springer, 2009.

A high-level work on the statistical aspects of data mining, written by renowned statisticians

who have invented several of the major data mining techniques. Both comprehensive and

thorough. A major work of reference. A further advantage of this book is that it can be read

while using the R package called ElemStatLearn which contains functions and databases

described in the text.

Simon Haykin: Neural Networks and Learning Machines, Prentice Hall, 3rd edn, 2008.

Avery comprehensive work on neural networks, multilayer perceptrons, radial basis function

networks, Kohonen maps (SOMs), support vector machines, etc.

Alan Julian Izenman:ModernMultivariate Statistical Techniques: Regression, Classification,

and Manifold Learning, Springer, 2008.

A recent book covering similar subjects to those discussed by Hastie, Tibshirani and

Friedman, but more accessible, suitable for students. It provides numerous examples and

uses R, S-PLUS and Matlab.

Olivia Parr Rud: Data Mining Cookbook, Wiley, 2000.

A practical guide with useful advice, accompanied with numerous examples of modelling

using SAS software. This is not a very recent book, so it does not cover the enhancements in

the latest versions of SAS, the outputs are not particularly attractive and the programming is

not always very sophisticated, but it is accessible and provides full details of the proposed

solutions. There is a version with a CD-ROM of SAS code included.

Brian D. Ripley: Pattern Recognition and Neural Networks, Cambridge University

Press, 2008.

A very good review of the state of the art and the theoretical bases of neural networks.

Vladimir N. Vapnik: The Nature of Statistical Learning Theory, Springer, 2nd edn, 1999.

A ‘historic’ survey of statistical learning theory by one of its main proponents, with a concise

description of his contribution, and an introduction to support vector machines (and to support

vector regression in the second edition). For readers interested in theory.

Vladimir N. Vapnik: Statistical Learning Theory, Wiley-Interscience, 1998.

A much longer and more demanding work than the previous one, dealing with the theoretical

bases of the concepts of learning, VC dimension and structural risk minimization, and

detailing the theory and practice of support vector machines.

Christopher Westphal and Teresa Blaxton: Data Mining Solutions: Methods and Tools for

Solving Real-World Problems, John Wiley & Sons, 1998.

A useful reference work, especially for those interested in visual methods.

Ian H. Witten and Eibe Frank: Data Mining: Practical Machine Learning Tools and

Techniques, Morgan Kaufmann, 2nd edn, 2005.

Much appreciated for its clarity and simplicity as well as its practical aspects.

Xindong Wu and Vipin Kumar: The Top Ten Algorithms in Data Mining, Chapman & Hall/

CRC, 2009.

DATA MINING AND STATISTICAL LEARNING 679

A survey of the ‘top 10’ data mining algorithms. Chosen from the most important methods

identified in December 2006 by the IEEE International Conference on Data Mining

(ICDM),1 they are as follows: C4.5, CART, k-means, support vector machines, Apriori,

EM (expectation–maximization), PageRank, AdaBoost, k nearest neighbours and the naive

Bayesian classifier. Each of these is described and illustrated with practical examples.

B.3. Text mining

Ludovic Lebart and Andr�e Salem: Statistique Textuelle, Dunod, 1994.

Lebart, L., Salem, A., Berry, L. (1998). Exploring textual data, Kluwer, Dordrecht.

An enthralling classic account.

B.4. Web mining

Myl�ene Bazsalicza and Patrick Na€ım: Data mining pour le Web, Eyrolles, 2001.

A book with a strongly educational approach, which starts by discussing the basics of the

Internet and data mining and goes on to describe the statistical processing of web data.

Michael J.A. Berry and Gordon S. Linoff: Mining the Web, John Wiley & Sons, 2002.

The latest title from the well-known consultants of Data Miners Inc., Boston.

B.5. R software

John M. Chambers: Programming with R, Springer, 2008.

Written by one of the creators of the S language (on which R language is based), this book is

unquestionably the best resource for advanced programming in R. It starts with the basics and

ends by giving the reader all the information he needs to create his own packages.

Pierre-Andr�e Cornillon: Statistiques avec R, Presses Universitaires de Rennes, 2008.

This very well-produced book consists of two parts. The first part is a general course on R,

providing the basic principles, showing how data are manipulated and represented, and

outlining basic programming in R. The second part gives examples of the main statistical

analysis and modelling procedures, in the form of separate sections with several pages each.

The book is accompanied by exercises with answers.

Michael J. Crawley: Statistics: An Introduction Using R, John Wiley & Sons Ltd, 2005.

A basic work, very comprehensive, with numerous examples.

Michael J. Crawley: The R Book, John Wiley & Sons Ltd, 2007.

More than 900 pages on R, covering every aspect, from the basics to the standard statistical

tests, and then going on to more advanced models such as time series, survival analyses,

generalized linear models and generalized additive models.

1 There is an interesting article about the same ‘top 10’: XindongWu, Vipin Kumar, J. Ross Quinlan et al. (2008),

‘Top 10 algorithms in data mining’, Knowledge and Information Systems, 14(1), 1–37.

680 FURTHER READING

Robert A. Muenchen: R for SAS and SPSS Users, Springer, 2009.

A well-designed book, useful for those familiar with SAS and SPSS, as it compares the

languages by creating a ‘Rosetta Stone’, with numerous programs written in all three

languages. So it is also useful for anyone wishing to compare SAS and SPSS with each

other. It also features a ‘trilingual’ glossary and further interesting information on the

publisher’s site. A new way of presenting R, which I can heartily recommend.

A full list of literature on R is available from: http://www.r-project.org/doc/bib/R-books.html

B.6. SAS software

Art Carpenter : Carpenter’s Complete Guide to the SAS Macro Language, SAS Publishing,

2nd edition, 2004.

The reference book on the subject, written by a recognized expert.

Ron Cody : Learning SAS by Example: A Programmer’s Guide, SAS Publishing, 2007.

It is a comprehensive book, clear and concise, whose level is higher than the The Little SAS

Book. Some exercises are corrected.

Ron Cody : SAS Functions by Example, SAS Publishing, 2nd edition, 2010.

A comprehensive guide to SAS functions, including what is new in 9.2. For each function,

it gives a brief description of its purpose, the syntax and clear examples with useful

explanations.

Olivier Decourt: Reporting avec SAS: Mettre en forme et diffuser vos r�esultats avec SAS 9 etSAS 9 BI, Dunod, 2008.

A book on the graphic resources of SAS, describing SAS GRAPH, ODS and ODS

GRAPHICS. A very useful source, because graphic functionality has not always been the

strong point of SAS, and it still suffers from a poor reputation in this field, even though the

latest versions can produce excellent results.

Olivier Decourt and H�el�ene Kontchou Kouomegni: SAS: Maıtriser SAS Base et SAS Macro,

SAS 9.2 et versions ant�erieures, Dunod, 2nd edn, 2007.

An excellent presentation, clear and precise, of SAS Base (and its macro language), including

the latest functionality.

Geoff Der, Brian S. Everitt : A Handbook of Statistical Analyses using SAS, Chapman and

Hall/CRC, 3d edition, 2008.

Avery good overview of what can be donewith SAS in statistics, from descriptive statistics to

survival analysis, through regression, analysis of variance, logistic regression, generalized

linear model, longitudinal data analysis, factor analysis and cluster analysis. With accompa-

nying exercises and examples of SAS macros.

Lora D. Delwiche and Susan J. Slaughter: The Little SAS Book: A Primer, SAS Publishing, 4th

edn, 2008.

An accessible, comprehensive book on the resources of SAS BASE, including the recent

functionality in Version 9, such as ODS GRAPHICS. Concise and supported with numerous

examples, it is a pleasure to read. However, it lacks detail in the areas of the macro language,

ODS and graphics (it does not cover GPLOT or GCHART).

SAS SOFTWARE 681

Stephen McDaniel and Chris Hemedinger: SAS for Dummies, John Wiley & Sons Ltd, 2nd

edn, 2010.

A basic reference.

S�ebastien Ringued�e: SAS Version 9.2: Introduction au d�ecisionnel: m�ethode et maıtrise du

langage, Pearson Education, 2008.

In this serious and exhaustive book, S�ebastien Ringued�e, an academic partner of SAS,

provides the essential information (and more!) that will be needed to achieve ‘SAS Base

Programming for SAS�9’ certification, with accompanying exercises (the solutions are

available on the companion website).

B.7. IBM SPSS software

Arthur Griffith: SPSS for Dummies, John Wiley & Sons Ltd, 2010.

A basic reference.

Paul Kinnear and Colin Gray: IBM SPSS Statistics 18 Made Simple, Psychology Press, 2010.

A book that is accessible to novices, leading on to more complex matters such as statistical

tests, experimental design, regression, discriminant analysis and factor analysis, all illustrated

with examples.

Naresh Malhotra, Jean-Marc D�ecaudin, and Afifa Bouguerra: Recherche et �etudes Marketing

avec SPSS, Pearson Education, 2004.

A book with an accompanying CD-ROM, including many detailed examples. It provides an

introduction to new techniques such as conjoint analysis and multidimensional positioning.

B.8. Websites

Modulad magazine, a mine of practical information (on software, events, etc.) and very

interesting articles on statistics: www.modulad.fr/

The website of Philippe Besse, Professor at the University of Toulouse, providing very full

coverage of statistics and data mining: http://www.math.univ-toulouse.fr/�besse/

The website of Gilbert Saporta, Professor at CNAM, with a very rich content including many

study courses: http://cedric.cnam.fr/�saporta/

StatNotes Online Textbook – David Garson’s online resource, with much well-presented

information on all aspects of statistics and data analysis, with details of implementation in

IBM SPSS Statistics: http://www2.chass.ncsu.edu/garson/pa765/statnote.htm

The StatSoft site for statistics and data mining: www.statsoft.com/textbook/stathome.html

The website for the book The Elements of Statistical Learning (Hastie, Tibshirani and

Friedman), with further information, data, R packages, errata, etc.: http://www-stat.stanford.

edu/�tibs/ElemStatLearn/

The website for the book Introduction to Data Mining (Pang-Ning Tan, Michael Steinbach

and Vipin Kumar), Addison-Wesley, offering a wide range of resources (extracts, PowerPoint

slides, etc.): http://www-users.cs.umn.edu/�kumar/dmbook/index.php

682 FURTHER READING

Re-sampling Methods in Statistical Modeling: a very good course devised by Professor

Bontempi, of the Free University of Brussels, on predictive models and contribution of

jackknife and bootstrap techniques, including ensemble methods such as bagging and

boosting: www.ulb.ac.be/di/map/gbonte/Stat104.html

The very comprehensive on-line help for the SAS software: http://support.sas.com/docu-

mentation/onlinedoc/

Olivier Decourt’s website, with a very interesting FAQ section on SAS, statistics and data

mining, including many concise and well-written descriptions of various technical aspects:

www.od-datamining.com/index.htm

Numerous resources on SPSS: www.spsstools.net/

The R software site: www.r-project.org/

The website of Lexicometrica magazine, where articles on text data mining can be down-

loaded: www.cavi.univ-paris3.fr/lexicometrica/index.htm

A course on web mining by Gregory Piatetsky-Shapiro: www.kdnuggets.com/web_mining_

course/

Real data to illustrate statistical methods, sorted by method (University of Massachusetts):

http://www.umass.edu/statdata/statdata/index.html

A very good glossary on statistics: http://dorakmt.tripod.com/mtd/glosstat.html

An eclectic blog written by Arthur Charpentier, Professor at the University of Rennes 1, about

statistics, probability, actuarial science, econometrics, R, and more: http://blogperso.univ-

rennes1.fr/arthur.charpentier/

Eric Weisstein’s World of Mathematics, an on-line mathematical encyclopaedia with more

than 11 000 entries and 5000 diagrams: http://mathworld.wolfram.com/

WEBSITES 683