Enhancing Interactive Visual Data Analysis by StatisticalFunctionality

Jurgen Platzer∗

VRVis Research CenterVienna, Austria

Abstract

Both information visualization and statistics analyse highdimensional data, but these sciences provide differentways to explore datasets. While techniques of the formerfield provide graphics so that the user can visually detectpatterns of interest, statistical algorithms apply the capa-bilities of computers to produce numerical reports fromthe data. Based on this observation a combination of tech-niques of both sciences can help to overcome their draw-backs. For this purpose a library was compiled that con-tains statistical routines, which are of high importance forinformation visualization techniques and allow a fast mod-ification of their results, to integrate possible adaptationsin the interactive visual data mining process.

Keywords: Data Mining, Information Visualization,Clustering, Dimension Reduction, Outlier Detection

1 Introduction

The exploration of high dimensional datasets is a tremen-dously growing working field. With the capabilities of to-day’s computers to handle data containing millions of datapoints and thousands of dimensions it is essential to ap-ply efficient methods to extract the information the user issearching for. Statistical routines and techniques of infor-mation visualization are useful to achieve this goal. But asone method on its own has several shortcomings combi-nations between the different capabilities of these sciencescould be developed to improve the exploration of multi-variate data, the so called data mining process.

Information visualization techniques create graphicsand animations that stress certain structures and aspectsof high dimensional data. The user, who examines thedata, applies his or her pattern recognition skills as well asthe experience and knowledge about the data to draw thecorrect conclusions. This is an efficient approach to de-tect data items of special interest, examine the main trendsin the data or investigate functional dependencies betweenvariables.

In contrast to that statistical routines use the possibilitiesof computers, which execute millions of operations within

∗Jurgen.Platzer@VRVIS.at

milliseconds. This allows the fast calculation of facts andnumerical summaries. Also models that can predict vari-able values or introduce a simplification of the data canbe fitted. Thus coherences within attributes as well assignificant patterns of the data items can be revealed andanalysed.

Because of the usage of different systems that gather theinformation of interest, a combination of those scienceswould introduce a verification of the results of the appliedmethods. Consequently an error detection approach for thedata mining process could be established that decreasesthe probability that wrong conclusions are implied. Butthe intelligent application of the strengths of the disparatetechniques also makes a more efficient data explorationpossible.

To achieve this collaboration, in this work the im-plementation of a library containing statistical routinesadapted for the use in information visualization applica-tions is presented. Because of the vast number of routinesdeveloped in the field of statistics for data analysis andexploration, the basic functionality, that every visual datamining tool should provide, had to be determined. Fur-thermore aspects like robustness, which decreases the oc-currence of distorted results caused by outliers, and fuzzy-ness, which allows soft decision boundaries to describeuncertainties, are considered. A further demand on thelibrary is, that its routines must be able to process largedatasets efficiently.

Additionally a sample application was developed todemonstrate possible combinations of visualization andstatistics. In the focus of this tool are tasks like outlier de-tection, dimension reduction and clustering, where com-putational approaches are combined with visual verifica-tions and user interactions that can manipulate the resultsof the statistical routine. Special attention was paid on aninteractive workflow were the user can determine the orderof the steps of the data mining procedure.

The following section outlines the main statistical ap-proaches, that were considered for this work, and exam-ples of applications that already realize an interactive col-laboration between computational routines and visualiza-tions. In section 3 the possible benefits of this combinationapproach are discussed, followed by a description of theimplemented functionality for the statistics library. After-wards a proof of concept case demonstrates the usefulness

of the integration of statistical methods into informationvisualization applications. Before the paper is concluded,issues considered for the implementation of the library aredepicted in section 6.

2 Related Work

The information visualization techniques to illustrate mul-tivariate datasets are manifold. They can be roughly clas-sified into the four categories geometric projection tech-niques, icon-based and pixel-based approaches and finallyhierarchical visualizations [13]. This work applies graphicrepresentations of the first type, which maps the variablesof the data on the screen space. The most popular ap-proaches of this category are scatterplots and scatterplotmatrices [8] as well as parallel coordinates [14]. For scat-terplots two attributes are mapped on the x and on the yaxis of the visualization space and data items are repre-sented by points in the coordinate system that is spanned.To allow an illustration of all dimensions of a dataset ascatterplot matrix was introduced, which shows all possi-ble tuples of variables by scatterplot visualizations. Theparallel coordinates achieve the representation of all at-tributes by mapping them on axes which are drawn asequidistant vertical lines. The data items are illustrated bypoly lines, which connect the projected dimension values.

Also the field of statistics provides a multitude of analy-sis procedures for data exploration. In the scope of thiswork only tasks are addressed that are of special impor-tance for a visual data mining application. Thereforethe multivariate outlier detection, dimension reduction andclustering techniques are considered.

As outliers strongly influence statistical routines andcause wrong results, an efficient detection of these ob-ject is crucial. For this purpose a variety of heuristics hasbeen developed. The most popular approaches are distancebased, density based and distribution based methods. Rou-tines of the first type consider the distance of each dataitem to itsk nearest neighbour. If this distance exceeds auser specified limit, the object is identified as outlier [17].Density based techniques refer to a volume parameter anda minimum number of data points that has to be locatedwithin this volume to define dense regions. Data itemsthat could not be assigned to such an area are marked asoutlying [7]. The multivariate outlier detection applicationthat is applied in this work uses a distribution based ap-proach, which assumes that the data applies to a multivari-ate elliptic distribution. Therefore for each data item therobust distance is calculated, which is based on the robustestimate of the covariance matrix [21], that describes theshape of the data cloud. If the data objects correspond tothe distribution constraint their distance measures show achi-squared distribution. Consequently a chi-squared dis-tribution quantile can be considered to determine a deci-sion boundary that differentiates between outliers and ac-tual data points.

Clustering approaches group similar data items to intro-duce partitions of the data. The main two methodologiesfor this task are hierarchical and partitional techniques. Ahierarchical clustering based on a merging operations ini-tiates each data item as cluster. Afterwards the two mostsimilar clusters are merged to a new cluster. This proce-dure is iteratively performed until only one cluster rep-resenting the whole dataset remains. This nested groupstructure can be represented by the tree-like dendrogram.In contrast to that partitional approaches assign data itemsto clusters according to an update rule that optimizes aglobal energy function. The most popular algorithm of thistype is thek means clustering [12], wherek indicates theuser defined number of partitions that are created. Whilethese routines assign each data item to exactly one cluster,fuzzy clustering approaches exist, which calculate for eachobject membership values, that indicate to which degree itis associated with each cluster. For this purpose the fuzzyk means algorithm [6] was considered.

To reduce the dimensionality of a dataset three maintechniques were introduced. The self-organizing maps(SOM) [18] are based on an unsupervised machine learn-ing approach, that tries to iteratively fit reference vectorsin data space to the structure of the data items. These vec-tors are connected in a two dimensional lattice which rep-resents the low dimensional projection of the data. Multidimensional scaling (MDS) techniques [19] try to achievea projection of the multivariate data that maintains the dis-tance relationships between pairs of data items. The sim-plest but nevertheless popular dimension reduction tech-nique is the principal component analysis (PCA) [15],which evaluates the directions of the major variances inthe data cloud. These directions are called the principalcomponents, on which a mapping of the data items can beperformed. As the first principal components describe themajority of the variance in the data, a subset of these ar-tificial dimensions can be chosen to span a subspace con-taining the main information of the data space.

Feature subset selection approaches have the same aimas dimension reduction techniques. But a low dimensionalrepresentation of the data is achieved by choosing only themost informative data attributes. As this concept was de-veloped for supervised machine learning routines, it is dif-ficult to apply for data exploration, because no measure forthe quality of an attribute can be intuitively introduced.

The integration of computational routines in informa-tion visualization applications gained importance in thelast 10 years. Therefore mainly clustering and the creationof low dimension data representations were applied. Thereasons why data partitioning has been favoured are thatgroup finding algorithms provide a fast categorization ofthe data and significantly improve the detection and inter-pretation of the main trends. The focus on the reduction ofvariables simply rises from the fact that humans are usedto think in three dimensional spaces, while multivariatedatasets represent their main information in a higher num-ber of attributes. To overcome this discrepancy projec-

tion methods as well as feature subset selection approacheswere applied.

But while simple visualizations of statistical results onlyserve to explore and present them, an interactive combina-tion of statistics and visual techniques is rarely realized.An example therefore is the Visual Hierarchical Dimen-sion Reduction (VHDR) [24] system, which applies a hi-erarchical clustering on the attributes of the data. The in-troduced dimension groups can be investigated and modi-fied by using InterRing [25] a radial visualization tool forhierarchical data. Finally representative dimensions perselected cluster can be chosen. This approach integratesthe user’s knowledge and experience into the feature sub-set selection task for which a starting point is created by astatistical routine.

An interactive visual feature subset selection and clus-tering tool is presented in [11]. By calculating a measurefor the ”goodness of clustering” for each pair of variablesa color coded matrix visualization is established wherebright fields represent attribute combinations that showsignificant cluster structures. As an ordering heuristic isapplied on the attributes the user can identify light regionsin the visualization which represent groups of variablesthat allow a partitioning of the data items. These dimen-sions can be selected and used for a hierarchical clusteringapproach, that detects groups of arbitrary shapes, becauseof the integration of graph and density based clusteringconcepts. The additional parameters, that were introducedby this enhancements, as well as the number of detectedclusters in the data can be steered by interactive visualiza-tions.

A further example describing the power of the combi-nation of visualizations and computationl routines is theHD-Eye approach [13], which adapts the OptiGrid clus-tering [16], so that the user is involved in the group find-ing process. Therefore a density estimation of the clus-tering procedure is used to decide, whether the data spacecan be subdivided by seperators like hyper planes, that arepositioned in sparse regions. To achieve this a set of pro-jections is suggested from the system, for which an icon-based visualization indicates, if a mapping is helpful todecide, where a separator can be introduced. The user canselect the projection that shows the most significant gapsbetween groups of data. A histogram-like view, showingthe agglomerations of data items by high bars is appliedto define a separator. This approach is iteratively applied,until no subspaces can be divided anymore. Consequentlythis approach is an attempt to incorporate the capabilitiesof the human visual system into a clustering routine toachieve better results.

3 Integration of Statistical Function-ality in Visualization

As the sciences visualization and statistics rely on differ-ent systems that analyse the data, their weaknesses andstrengths are mostly dissimilar. Interactive visual applica-tions provide graphics that can be modified by the user toachieve an efficient information drill down process, wherefirstly an overview is given. Afterwards zooming and fil-tering techniques allow the concentration on patterns ordata items of special interest. Finally details-on-demandoperations show numerical summaries or the data valuesof the selected subset themselves. Therefore mainly theuser’s extraordinary pattern recognition skills and knowl-edge about the data guides the exploration process [23].

Contrary to that statistical routines are algorithms andcalculations of formulas that use computers to cope withthe enormous computational effort for large datasets. Thisimplies that the applied procedures have to be well cho-sen for the data that should be analysed. Consequently ifa dataset contains clusters of arbitrary shapes, ak meansclustering may produce low-quality results, because it onlycreates spherical groups. As this example shows, a generalpurpose method may fail on a given dataset and the detec-tion of this failure is difficult to accomplish. Furthermorethe presence of outliers can also significantly distort resultsof statistical routines.

Therefore the following discussions propose possiblecombinations of statistical methods and information visu-alization techniques for clustering, outlier detection anddimension reduction that may compensate the drawbacksof individual approaches.

3.1 Grouping of Data Items

The most popular statistical routine in data mining applica-tions is clustering, that introduces partitions of the dataset.The detected groups can be seen as a simplification of datathat allows an easier interpretation of the main patterns.But clustering results are also used to create clearer visu-alizations and a reduction of data items for time consum-ing calculations. An example is shown in figure 1 werethe dataset UVW [2] containing 149769 data items is il-lustrated in parallel coordinate view. The first visualiza-tion simply plots all data items, which results in a clutterdgraphic, while the second incorporates the information ofthe fuzzy clustering approach and thus reveals the struc-ture of the data.

Consequently a clustering algorithm can introduce ameaningful division of the data into groups. But a visualverification of these partitions is crucial, because the intro-duced clusters may not be appropriate for the structures inthe data. As general purpose clustering algorithms sufferthat the number of clusters has to be set and/or the createdclusters show a specific shape their results could be ma-nipulated to achieve a better fit of the real groups in the

Figure 1: A cluttered parallel coordinate visualization(above) can reveal structure if clustering information is in-tegrated (below).

data.A visualization system that captures both the high di-

mensionality of the data as well as local features has tobe applied to provide a user interface for the explorationand manipulation of clustering results. In the scope ofthis work the use of parallel coordinates and scatterplotsis suggested. While the latter makes the intuitive inves-tigation of two dimensional features possible, a parallelcoordinates view illustrates all dimensions of a dataset.Furthermore dimension reduction techniques are appliedto map the data items in a two dimensional space, which isvisualized by a scatterplot. This allows a validation of thequality of the introduced partitions and represents a userinterface for multivariate modifications of the clusteringresult.

As operations, that adapt the introduced partitions, clus-ters can be split, merged or deleted. Furthermore a clus-ter can be selected for a subclustering procedure, whereonly the data items of the chosen partition are consideredfor a clustering. But also cluster centers can be reposi-tioned and objects can be reassigned to the cluster withthe nearest center. After those interactions took place areclustering based on the adapted clustering result can beinitiated to improve the solution. Thus an interactive infor-

mation exchange between a computational routine and theuser’s interaction is established, which is a significantlyimproved system in comparison to information visualiza-tion applications that only allow the exploration of clus-tering results. Because now the user is not restricted to theinitiation of interactions, that are based on the perceived(mostly lower dimensional) features, also a routine thatconsiders patterns in data space can be interactively ap-plied.

3.2 Dimension Reduction and Feature Sub-set Selection

Based on a user defined similarity measure between at-tributes, a clustering procedure can be initiated to intro-duce groups of similar variables. Therefore a hierarchi-cal clustering approach is adequate, because it allows theinteractive modification of the group number. The estab-lished hierarchy of dimension relationships can be used asstarting point for an interactive feature subset selection ap-plication that can also be combined with dimension reduc-tion techniques. Thus a visualization of the dendrogramstructure allows an interactive exploration of the cluster-ing result. Dimensions that are represented by a selectednode can be illustrated by parallel coordinates and serve asdecision guidance for the feature selection. Additionally iffor a group no representative dimension can be chosen, adimension reduction approach is available. Consequentlythe main information represented by the cluster of dimen-sions is captured by a small number of artificial attributes,which can be selected.

The concept of this approach is shown in figure 2 by asimple dataset containing four attributes, from which twopairs are highly correlated. The parallel coordinates viewon the left side illustrates this issue, while a representa-tion of the cluster tree structure in the upper right cornershows that the clustering divides the attributes into twosubgroups.

Figure 2: An attribute hierarchy representation and a par-allel coordinate visualization showing the dimension rela-tionships.

Because a clustering approach, that is not adapted to aspecific kind of data, can produce arbitrarily bad fits to thestructure of the dimension coherences, it is crucial that theuser examines the achieved grouping. Therefore the mostcharacteristic attributes of the clusters as well as those

variables that can also be assigned to different partitionshave to be determined. Dimensions of the first categorymay be those candidates that are chosen by the user torepresent other dimensions for further operations. Otherattributes may be of minor importance from the cluster-ing point of view. Nevertheless the user can incorporateher/his knowledge in the process and can also choose thoseattributes, if they represent essential information.

Consequently this approach combines a statistical pro-cedure to create an initial solution for the subset selectionproblem by introducing groups of dimensions for which arepresentative attribute can be chosen. But also the inputof the user is required to choose the correct subset by visu-ally validating the quality of the clustering. If the dimen-sion clusters are visually heterogeneous, a new clusteringcan be tested to achieve a better result or a dimension re-duction approach can be applied.

3.3 Multivariate Outlier Detection

In contrast to the detection of clusters, where similar dataitems are grouped, the identification of outliers searchesfor objects that deviate from the main behaviour of thedata. Consequently this subset of data points may beheterogeneous. As browsing and selection techniques ofinformation visualization applications only highlight dataitems showing similar properties, this approach is not ade-quate to detect high dimensional outliers. Therefore a sta-tistical routine could be used again as an initial solution. Ingeneral such a method provides parameters that steer thenumber of identified outlying objects. Thus it is crucial tohave a visual feedback that allows the interactive determi-nation of the optimal parameter settings. To achieve thisdifferent linked visualizations, which are also able to applydimension reduction techniques, could be used. A projec-tion of the data items on a low dimensional subspace to re-alize a scatterplot illustration is especially helpful, becausethis approach allows the verification, whether the detectedobjects are at the border of the data cloud or deviate fromthe main groups in the dataset. Thus a validation of the sta-tistical outlier detection is achieved and data items that arewrongly marked can also be manually deselected, whichenhances the quality of the outlier detection.

The application of multivariate outlier exclusion is cru-cial for non-robust statistical routines, that calculate mis-leading results in the presence of outlying objects. In con-trast to that visualizations of large datasets mainly stressthe major patterns in the data. Thus it is essential to detectpossibly outlying data items to accentuate their represen-tations.

4 Library for Statistical Functionalityfor Visualization

For the determination of statistical functionality that isof high importance for information visualization applica-

tions software packages like SpotFire [5], Miner3D [4] orGGobi [1] were examined. Also publications of recentyears, that discuss the integration of computational ap-proaches in the visual data mining process were analysed.This research showed that the majority of the applied al-gorithms are concerned with clustering and dimension re-duction. But also the use of transformations to prepare thedata for further procedures was demonstrated especiallyin GGobi. Besides of these main tasks standard calcu-lations like statistical moments and correlation measureswere common.

In the scope of this work also a stronger integrationof robust methods should be obtained. This is shown byimplementing robust estimators for the location and thespread of a set of data values as well as by providing thecalculation of robust correlation measures. But the mainapplication which demonstrates the capabilities of robust-ness is the statistical outlier detection, which introduces ameasure of outlyingness for each data item.

Furthermore the concept of fuzzyness is considered, be-cause decisions made in the real world, from where thedata comes from, are rarely reduced to yes/no answers.Therefore the fuzzyk means clustering was implemented,to show that it is not possible to assign each data itemstrictly to one cluster. Thus a degree of uncertainty is in-troduced to differentiate between objects that are near acluster center and those data points that are located at clus-ter boundaries.

After introducing the main categories of functionalitythat is made available by the library, the remainder of thissection gives an overview of the provided routines.

4.1 Transformations and Moments

Transformations can be seen as mappings of the data val-ues to a certain interval or as modifications of the distribu-tion of a set of values. The former application is useful toprepare a dataset for clustering, so that each dimension hasthe same range of values, which avoids that one attributeshas a stronger influence on the distance calculations inthe group finding process. The latter is of importance forstatistical routines like the distribution-based high dimen-sional outlier detection, which can only be applied on datafrom a multivariate elliptical distribution. Therefore thestatistics library provides linear, logarithmic, exponentialand squareroot transformations.

As these mappings are applied on single dimensionsseparately also the statistical moments are in general cal-culated for attributes of the dataset. Therefore classic aswell as robust estimates for the location (arithmetic mean,median,α - trimmed mean) and the spread (standard devi-ation, median of absolute deviations, inter quartile range)are provided.

4.2 Correlations and Covariances

To analyse the coherence between two variables three cor-relation measures are implemented. The classic Pearsoncorrelation, which is biased if outliers are present, and therobust Spearman and Kendall correlations can be calcu-lated. The two robust estimates do not only detect linearrelationships but also exponential and logarithmic depen-dencies between dimensions.

A rough estimate for the shape of the multidimensionaldata cloud is given by the covariance matrix, which is asymmetric matrix holding the variances of the attributes inthe main diagonal and the covariances between the dimen-sions in the off diagonal entries. As the covariance matrixdescribes the data as an hyper-ellipsoid it can be applied tointegrate the shape of the data distribution into the distancecalculation. This is achieved by the Mahalanobis distance.If a robust calculation scheme for the covariance matrixlike the minimum covariance determinant (MCD) [21] es-timator is applied, this concept can be used to calculaterobust distances that assign high values to data items thatstrongly deviate from the majority of data items.

4.3 Clustering and Dimension Reduction

For the division of datasets into partitions the popular clus-tering proceduresk means and fuzzyk means were imple-mented. While the first algorithm introduces a hard clus-ter structure, where each data item is assign to exactly onegroup, the fuzzy approach calculates memberships, thatindicate to which degree an object belongs to a given clus-ter. Thereby the sum of the memberships for a data itemalways accounts 1. Additionally a hierarchical clusteringapproach based on the correlation matrix is realized to in-troduce groups of dimensions. This routine can be used asbasis for a interactive feature subset selection application.

As dimension reduction routine the principal compo-nent analysis (PCA) is provided. It is based on the covari-ance matrix calculation. Thus also a robust PCA can beaccomplished, by using the MCD estimate as covariancematrix.

4.4 Distributions and Statistical Tests

As theoretical distributions the normal, log normal, expo-nential, uniform and chi-squared distribution are realized.For each of these distributions values of the probabilitydensity function (pdf) and of the cumulative distributionfunction (cdf) as well as quantiles and random numbers areavailable. Additionally a Kolmogorov-Smirnov test can beperformed to validate, whether a set of values comes fromthese theoretical distributions. Furthermore the invocationof tests, whether two attributes show the same distribution,is possible.

4.5 Regression

A least squares linear regression that predicts the valuesof a dependent variable based on a set of independent at-tributes is implemented. This model fitting approach isconvenient for the identification of functional dependen-cies between dimensions of a dataset.

5 Proof of Concept Case

To demonstrate the advantages of the combination be-tween computational routines and a visual verification,which also considers interactive modifications of the com-puted results and possible restarts of the algorithm, the in-teractive clustering tool that was realized for the sampleapplication is presented. For this program the graphicaluser interface was based on the GTK+ [3] library, whilethe visualizations were implemented in OpenGL [22].

The interactive clustering applies ak means algorithmand illustrates its results in a scatterplot visualization ofthe data items mapped on the first two principal compo-nents. The cluster centers are also illustrated and used tomanipulate the introduced partitions. Recluster processesare based on the repositioned cluster centers. Additionallyscatterplots and parallel coordinates make the inspectionof the original data possible.

As dataset the letter image recognition data [10] con-taining 20 000 observations and 16 numeric variables,which describe the properties of letters given as black-and-white images is used. For this proof of concept case onlythe letters A, B, C, D, E and F are considered, which re-duces the number of data items to 4640. For the clusteringprocedure all numeric attributes were used except the hor-izontal and vertical position of the bounding box of theletters that do not contain information to discriminate theletters. As the dataset is based on the discretization of theletters by a pixel raster, the measurements show integervalues and are not continuous.

Before the clustering was initiated the value ranges ofall dimensions were mapped to the unit interval. As thefirst and the second principal component capture 48 % ofthe variance in the data, the visualization of dimension re-duction represents a good hint for the structures in the data.Certainly an enormous amount of information is not con-sidered, and thus not illustrated in the scatterplot visual-izations shown in the figures 3, 5 and 7.

The interactive clustering process starts with an ini-tial clustering, which introduces partitions that can be ex-plored and manipulated. The result of this initial step isshown in the figures 3 and 4. While the former illustra-tion in the scatterplot provides an intuitive overview ofthe data and its multidimensional structures, the parallelcoordinates view makes the investigation of the partitionshapes on single attributes possible.

The overview of the data, which is realized by the di-mension reduction, shows that the green cluster, which

Figure 3: Clustering result mapped on principal compo-nents

Figure 4: Clustering result in parallel coordinates

represents measurements of the letter A is very good sepa-rated. The remainder of the data can not be partitioned thateasy. In the parallel coordinates it can be observed thatthe cluster centers show deviations in specific dimensionsfrom the main behaviour of the data. As example the redcluster shows significantly higher values in the first threedepicted dimensions, while the center of the green parti-tion is discriminated by low values in the attributesy.bar,y2barandx2ybr.

To emphasize on the differences between the clusterstheir centers have been manually repositioned in the scat-terplot visualization shown in figure 5 so that the clustersfocus on certain areas in the data. The modifications arealso projected back into the data space and visualized inthe parallel coordinate view (figure 4). There it is obvi-ous that the modifications assigned more one dimensionalextreme values to the cluster centers. To verify if this ma-nipulation results in a better discrimination of the clustersa reclustering, taking the repositioned centers as initial so-

lution, is initiated.

Figure 5: Repositioned cluster centers in principal compo-nents

Figure 6: Repositioned cluster centers in parallel coordi-nates

This group finding process evaluated a similar solutionthan the initial clustering. Nevertheless the energy func-tion of thek means clustering signalizes that a better resultwas found, because the sum of distances of the data itemsto their nearest cluster centers was reduced. The scatter-plot in figure 7 reveals that the regions of overlap betweenthe red, cyan and magenta cluster has been reduced, whilethe yellow and the blue cluster still interweave. To re-solve this visual inseparability further principal compo-nents have to be taken into account. Only little changescan be observed in the parallel coordinates visualization(figure 8). As the clusters still show extreme values in sin-gle dimensions a subspace clustering can be considered, toseparate the measurements of one letter from the remain-der of the data.

Figure 7: Reclustering result mapped on principal compo-nents

Figure 8: Reclustering result in parallel coordinates

6 Implementation

The implementation of the statistics library is aimed tooperate on large data sets holding millions of data items.Therefore special attention was paid to process large ar-rays of data values as fast as possible. To achieve this goalan implementation in the language C++ was chosen, whichprovides efficient pointer operations. The work with C++also demands a concept for a failure safe usage of the li-brary. To accomplish this issue all routines return a boolvariable indicating false, if the functionality could not beexecuted correctly. Furthermore the parameters that arepassed to the methods apply to a scheme so that functioncalls of different procedures have a similar structure andthus are intuitive to use. The first category of parametersconcerns variables into which the results are written. Af-terwards data specific information like dimension valuesor mean vectors have to be set. The final class of parame-ters is concerned with the properties of the applied algo-rithm itself. Examples therefore are the number of clusters

for a k means clustering, or the robustness factor for anMCD covariance matrix estimator.

To ease the integration of statistical routines into infor-mation visualization applications the definition of an ad-equate interface is crucial. Thus so called hooks of in-teraction have been realized. These special function callsenable the immediate recalculation of statistical facts likecorrelations and moments for subsets of the data items.This is important for applications where numerical sum-maries of selected data items are requested. Those sum-maries have to be updated if the selection changes orif details-on-demand actions are initiated. Besides thesestandard adaptations also task specific interface extensionshad to be included. Based on the statistical routine and itsvisualization several interaction techniques can be spec-ified. Consequently the implications of the user actionshave to be translated into parameter settings for the com-putational algorithm in the statistical library to adapt itsresult. This was accomplished on the basis of thek meansclustering. A typical visualization of a result of this par-titioning procedure involves the representation of the dataitems in colors according to their cluster membership andthe emphasis of the cluster centers. The latter can be usedto manipulate the clustering result by repositioning its cen-ters or selecting clusters to initiate merge and splitting op-erations. Those complex adaptations have to be covered bythe hooks of interactions and reformulated into ordinaryfunction calls to allow a reclustering based on the user’sinput.

As functionalities like the principal component analy-sis and the robust distance calculation require matrixinversion and determinant evaluation, implementationsfor these operations were integrated from the numericalrecipes in C [20]. Also correlation computations, the re-alization of theoretic distributions and the Kolmogorov-Smirnov test build up on the fast and stable routines ofthis repository of basic numerical procedures. Because ofthe integration of robust methods, which require the eval-uation of a value, having a specified position in a sample,an efficient routine that returns thek smallest data value ofan array was realized [9]. Empirical tests prove that thismethod is faster than the application of a quick sort proce-dure.

7 Conclusions

While the most applications that consider techniques ofboth information visualization and statistics, concentrateon presentation and exploration of the results of computa-tional routines, the possible benefit of an interactive col-laboration of these fields is higher.

Future work has to concentrate on the translation of in-teractions performed in a visual data mining view into pa-rameter settings for statistical algorithms. Without thiscontribution an efficient combination of the strengths ofcomputational capabilities with the experience and the

knowledge of the user is not possible. Also the interactiveinformation exchange between statistics and user actionshas to be accomplished. Numerical summaries, that areimmediately updated if selections are changed or details-on-demand operations are executed, are necessary to vali-date the percieved patterns in the visualizations.

Furthermore the concept of robustness, which is wellknown in the field of statistics, has to be incorporated intothe visual data mining workflow. This is a crucial task, be-cause outlying data items can significantly distort the re-sults of statistical routines as well as the visual impressionprovided by visualizations, which build up on the outcomeof these algorithms.

8 Acknowledgement

This work has been realized at theVRVis Research Center in Vienna, Austria(http://www.VRVis.at/), which is funded by the Aus-trian research program called Kplus, to support the basicresarch on visualization (http://www.VRVis.at/vis/).

Special thanks are given to Helwig Hauser, who gaveme the opportunity to work on the combination of statis-tical routines and information visualization. His supportand his vision how both sciences could collaborate madethis work possible.

I also want to thank Peter Filzmoser for his input andhelp concerning statistical routines and their integration invisualization applications. His ideas for the improvementof data exploration by visual techniques was a crucial con-tribution to increase the quality of this work.

Additional thanks go to Harald Piringer, whose dedi-cation for improving implementations and whose expertknowledge concerning the work with large datasets helpedto realize the assembly of statistical routines in a C++ li-brary.

References

[1] GGobi - Data visualization system(http://www.ggobi.org/).

[2] Gretar Tryggvason 3-d fluidflow data, (XmdV data archive:http://davis.wpi.edu/xmdv/datasets/uvw.html).

[3] GTK+ The GIMP Toolkit (http://www.gtk.org/).

[4] Miner3D (http://www.miner3d.com/).

[5] SpotFire Decisionsite(http://www.spotfire.com/products).

[6] James Christian Bezdek.Fuzzy mathematics in pat-tern classification. PhD thesis, Cornell University,1973.

[7] Markus M. Breunig, Hans peter Kriegel, Raymond T.Ng, and Jrg Sander. LOF: Identifying density-basedlocal outliers, March 01 2000.

[8] William S. Cleveland. The Elements of GraphingData. Hobart Press, 1994.

[9] T. H. Cormen, C. E. Leiserson, R. L. Rivest, andC. Stein. Introduction to Algorithms. MIT Press,Cambridge, MA, 2001. Second Edition.

[10] Peter W. Frey and David J. Slate. Letter recogni-tion using holland-style adaptive classifiers.MachineLearning, 6:161, 1991.

[11] Diansheng Guo. Coordinating computational and vi-sual approaches for interactive feature selection andmultivariate clustering. Information Visualization,2(4):232–246, 2003.

[12] J. A. Hartigan.Clustering Algorithms. Wiley, NewYork, 1975.

[13] Alexander Hinnenburg, Daniel A. Keim, and MarkusWawryniuk. HD-eye: Visual mining of high-dimensional data.IEEE Computer Graphics and Ap-plications, 19(5):22–31, September 1999.

[14] Alfred Inselberg and Bernard Dimsdale. Parallel co-ordinates: a tool for visualizing multi-dimensionalgeometry. InVIS ’90: Proceedings of the 1st con-ference on Visualization ’90, pages 361–378. IEEEComputer Society Press, 1990.

[15] J. E. Jackson.A User’s Guide to Principal Compo-nents. Wiley, New York, 1991.

[16] Keim and Hinneburg. Optimal grid-clustering: To-wards breaking the curse of dimensionality in high-dimensional clustering. InVLDB: International Con-ference on Very Large Data Bases. Morgan Kauf-mann Publishers, 1999.

[17] Edwin M. Knorr and Raymond T. Ng. Algorithms formining distance-based outliers in large datasets. InProceedings of the 24th International Conference onVery Large Data Bases (VLDB ’98), pages 392–403,East Sussex - San Francisco, August 1998. MorganKaufmann.

[18] Teuvo Kohonen. The self-organizing map.Proc.IEEE, 78(9):1464–1480, September 1990.

[19] J. B. Kruskal and M. Wish.Multidimensional Scal-ing. Sage Piblications, Beverly Hills, CA, 1978.

[20] W. H. Press et al. Numerical recipes in C (secondedition). Cambridge University Press, 1992.

[21] Peter J. Rousseeuw and Katrien Van Driessen. A fastalgorithm for the minimum covariance determinantestimator, 1998.

[22] SGI. OpenGL - reference manual, the official refer-ence documantation for openGL, release 1.AddisonWesley, ISBN 0-201-63276-4, 1992.

[23] Ben Shneiderman. The eyes have it: A task by datatype taxonomy for information visualizations. InVL,pages 336–343, 1996.

[24] Jing Yang, Matthew O. Ward, Elke A. Rundensteiner,and Shiping Huang. Visual hierarchical dimen-sion reduction for exploration of high dimensionaldatasets. pages 19–28.

[25] Jing Yang, Matthew O. Ward, Elke A. Rundensteiner,and Anilkumar Patro. Interring: a visual interfacefor navigating and manipulating hierarchies.Infor-mation Visualization, 2(1):16–30, 2003.