september 4, 2015data mining: concepts and techniques1 data mining: concepts and techniques —...
TRANSCRIPT
April 21, 2023Data Mining: Concepts and
Techniques 1
Data Mining: Concepts and
Techniques
— Chapter 2 —
April 21, 2023Data Mining: Concepts and
Techniques 2
General data characteristics
Basic data description and exploration
Measuring data similarity
What is about Data?
April 21, 2023Data Mining: Concepts and
Techniques 3
What is Data?
Collection of data objects and their attributes
An attribute is a property or characteristic of an object
Examples: eye color of a person, temperature, etc.
Attribute is also known as variable, field, characteristic, or feature
A collection of attributes describe an object
Object is also known as record, point, case, sample, entity, or instance
Tid Refund Marital Status
Taxable Income Cheat
1 Yes Single 125K No
2 No Married 100K No
3 No Single 70K No
4 Yes Married 120K No
5 No Divorced 95K Yes
6 No Married 60K No
7 Yes Divorced 220K No
8 No Single 85K Yes
9 No Married 75K No
10 No Single 90K Yes 10
Attributes
Objects
April 21, 2023Data Mining: Concepts and
Techniques 4
Important Characteristics of Structured Data
Dimensionality Curse of dimensionality
Sparsity Only presence counts
Resolution Patterns depend on the scale
Similarity Distance measure
April 21, 2023Data Mining: Concepts and
Techniques 5
Attribute Values
Attribute values are numbers or symbols assigned to an attribute
Distinction between attributes and attribute values Same attribute can be mapped to different attribute
values Example: height can be measured in feet or
meters
Different attributes can be mapped to the same set of values
Example: Attribute values for ID and age are integers
But properties of attribute values can be different ID has no limit but age has a maximum and
minimum value
April 21, 2023Data Mining: Concepts and
Techniques 6
Types of Attribute Values
Nominal E.g., profession, ID numbers, eye color, zip
codes Ordinal
E.g., rankings (e.g., army, professions), grades, height in {tall, medium, short}
Binary E.g., medical test (positive vs. negative)
Interval E.g., calendar dates, body temperatures
Ratio E.g., temperature in Kelvin, length, time, counts
April 21, 2023Data Mining: Concepts and
Techniques 7
Properties of Attribute Values
The type of an attribute depends on which of the following properties it possesses: Distinctness: = Order: < > Addition: + - Multiplication: * /
Nominal attribute: distinctness Ordinal attribute: distinctness & order Interval attribute: distinctness, order & addition Ratio attribute: all 4 properties
April 21, 2023Data Mining: Concepts and
Techniques 8
Attribute Type
Description Examples Operations
Nominal The values of a nominal attribute are just different names, i.e., nominal attributes provide only enough information to distinguish one object from another. (=, )
zip codes, employee ID numbers, eye color, sex: {male, female}
mode, entropy, contingency correlation, 2 test
Ordinal The values of an ordinal attribute provide enough information to order objects. (<, >)
hardness of minerals, {good, better, best}, grades, street numbers
median, percentiles, rank correlation, run tests, sign tests
Interval For interval attributes, the differences between values are meaningful, i.e., a unit of measurement exists. (+, - )
calendar dates, temperature in Celsius or Fahrenheit
mean, standard deviation, Pearson's correlation, t and F tests
Ratio For ratio variables, both differences and ratios are meaningful. (*, /)
temperature in Kelvin, monetary quantities, counts, age, mass, length, electrical current
geometric mean, harmonic mean, percent variation
April 21, 2023Data Mining: Concepts and
Techniques 9
Discrete vs. Continuous Attributes
Discrete Attribute Has only a finite or countably infinite set of values E.g., zip codes, profession, or the set of words in a
collection of documents Sometimes, represented as integer variables Note: Binary attributes are a special case of
discrete attributes Continuous Attribute
Has real numbers as attribute values Examples: temperature, height, or weight Practically, real values can only be measured and
represented using a finite number of digits Continuous attributes are typically represented as
floating-point variables
April 21, 2023Data Mining: Concepts and
Techniques 10
Types of data sets
Record Data Matrix Document Data Transaction Data
Graph World Wide Web Molecular Structures
Ordered Spatial Data Temporal Data Sequential Data Genetic Sequence Data
April 21, 2023Data Mining: Concepts and
Techniques 11
Important Characteristics of Structured Data
Dimensionality Curse of Dimensionality
Sparsity Only presence counts
Resolution Patterns depend on the scale
April 21, 2023Data Mining: Concepts and
Techniques 12
Record Data
Data that consists of a collection of records, each of which consists of a fixed set of attributes
Tid Refund Marital Status
Taxable Income Cheat
1 Yes Single 125K No
2 No Married 100K No
3 No Single 70K No
4 Yes Married 120K No
5 No Divorced 95K Yes
6 No Married 60K No
7 Yes Divorced 220K No
8 No Single 85K Yes
9 No Married 75K No
10 No Single 90K Yes 10
April 21, 2023Data Mining: Concepts and
Techniques 13
Data Matrix
If data objects have the same fixed set of numeric attributes, then the data objects can be thought of as points in a multi-dimensional space, where each dimension represents a distinct attribute
Such data set can be represented by an m by n matrix, where there are m rows, one for each object, and n columns, one for each attribute
1.12.216.226.2512.65
1.22.715.225.2710.23
Thickness LoadDistanceProjection of y load
Projection of x Load
1.12.216.226.2512.65
1.22.715.225.2710.23
Thickness LoadDistanceProjection of y load
Projection of x Load
April 21, 2023Data Mining: Concepts and
Techniques 14
Document Data
Each document becomes a `term' vector, each term is a component (attribute) of the
vector, the value of each component is the number of
times the corresponding term occurs in the document.
Document 1
season
timeout
lost
win
game
score
ball
play
coach
team
Document 2
Document 3
3 0 5 0 2 6 0 2 0 2
0
0
7 0 2 1 0 0 3 0 0
1 0 0 1 2 2 0 3 0
April 21, 2023Data Mining: Concepts and
Techniques 15
Transaction Data
A special type of record data, where each record (transaction) involves a set of
items. For example, consider a grocery store. The set
of products purchased by a customer during one shopping trip constitute a transaction, while the individual products that were purchased are the items.
TID Items
1 Bread, Coke, Milk
2 Beer, Bread
3 Beer, Coke, Diaper, Milk
4 Beer, Bread, Diaper, Milk
5 Coke, Diaper, Milk
April 21, 2023Data Mining: Concepts and
Techniques 16
Graph Data
Examples: Generic graph and HTML Links
5
2
1
2
5
<a href="papers/papers.html#bbbb">Data Mining </a><li><a href="papers/papers.html#aaaa">Graph Partitioning </a><li><a href="papers/papers.html#aaaa">Parallel Solution of Sparse Linear System of Equations </a><li><a href="papers/papers.html#ffff">N-Body Computation and Dense Linear System Solvers
April 21, 2023Data Mining: Concepts and
Techniques 17
Chemical Data
Benzene Molecule: C6H6
April 21, 2023Data Mining: Concepts and
Techniques 18
Ordered Data
Sequences of transactions
An element of the sequence
Items/Events
April 21, 2023Data Mining: Concepts and
Techniques 19
Ordered Data
Genomic sequence data
GGTTCCGCCTTCAGCCCCGCGCCCGCAGGGCCCGCCCCGCGCCGTCGAGAAGGGCCCGCCTGGCGGGCGGGGGGAGGCGGGGCCGCCCGAGCCCAACCGAGTCCGACCAGGTGCCCCCTCTGCTCGGCCTAGACCTGAGCTCATTAGGCGGCAGCGGACAGGCCAAGTAGAACACGCGAAGCGCTGGGCTGCCTGCTGCGACCAGGG
April 21, 2023Data Mining: Concepts and
Techniques 20
Ordered Data
Spatio-Temporal Data
Average Monthly Temperature of land and ocean
April 21, 2023Data Mining: Concepts and
Techniques 21
General data characteristics
Basic data description and exploration
Measuring data similarity
April 21, 2023Data Mining: Concepts and
Techniques 22
Mining Data Descriptive Characteristics
Motivation
To better understand the data: central tendency, variation and spread
Data dispersion characteristics
median, max, min, quantiles, outliers, variance, etc.
Numerical dimensions correspond to sorted intervals
Data dispersion: analyzed with multiple granularities of precision
Boxplot or quantile analysis on sorted intervals
Dispersion analysis on computed measures
Folding measures into numerical dimensions
Boxplot or quantile analysis on the transformed cube
April 21, 2023Data Mining: Concepts and
Techniques 23
Measuring the Central Tendency
Mean (algebraic measure) (sample vs. population):
Weighted arithmetic mean:
Trimmed mean: chopping extreme values
Median: A holistic measure
Middle value if odd number of values, or average of the
middle two values otherwise
Estimated by interpolation (for grouped data):
Mode
Value that occurs most frequently in the data
Unimodal, bimodal, trimodal
Empirical formula:
n
iix
nx
1
1
n
ii
n
iii
w
xwx
1
1
widthfreq
lfreqNLmedian
median
))(2/
(1
)(3 medianmeanmodemean
N
x
April 21, 2023Data Mining: Concepts and
Techniques 24
Symmetric vs. Skewed Data
Median, mean and mode of symmetric, positively and negatively skewed data
positively skewed
negatively skewed
symmetric
April 21, 2023Data Mining: Concepts and
Techniques 25
Measuring the Dispersion of Data
Quartiles, outliers and boxplots
Quartiles: Q1 (25th percentile), Q3 (75th percentile)
Inter-quartile range: IQR = Q3 – Q1
Five number summary: min, Q1, M, Q3, max
Boxplot: ends of the box are the quartiles, median is marked,
whiskers, and plot outlier individually
Outlier: usually, a value higher/lower than 1.5 x IQR
Variance and standard deviation (sample: s, population: σ)
Variance: (algebraic, scalable computation)
Standard deviation s (or σ) is the square root of variance s2 (or σ2)
n
i
n
iii
n
ii x
nx
nxx
ns
1 1
22
1
22 ])(1
[1
1)(
1
1
n
ii
n
ii x
Nx
N 1
22
1
22 1)(
1
April 21, 2023Data Mining: Concepts and
Techniques 26
Boxplot Analysis
Five-number summary of a distribution:
Minimum, Q1, M, Q3, Maximum Boxplot
Data is represented with a box The ends of the box are at the first and third
quartiles, i.e., the height of the box is IQR The median is marked by a line within the box Whiskers: two lines outside the box extend to
Minimum and Maximum
April 21, 2023Data Mining: Concepts and
Techniques 27
Histogram Analysis
Graph displays of basic statistical class descriptions Frequency histograms
A univariate graphical method Consists of a set of rectangles that reflect the counts
or frequencies of the classes present in the given data
April 21, 2023Data Mining: Concepts and
Techniques 28
Histograms Often Tells More than Boxplots
The two histograms shown in the left may have the same boxplot representation The same values
for: min, Q1, median, Q3, max
But they have rather different data distributions
April 21, 2023Data Mining: Concepts and
Techniques 29
Quantile Plot
Displays all of the data (allowing the user to assess both the overall behavior and unusual occurrences)
Plots quantile information For a data xi data sorted in increasing order, fi
indicates that approximately 100 fi% of the data are below or equal to the value xi
April 21, 2023Data Mining: Concepts and
Techniques 30
Quantile-Quantile (Q-Q) Plot
Graphs the quantiles of one univariate distribution against the corresponding quantiles of another
Allows the user to view whether there is a shift in going from one distribution to another
April 21, 2023Data Mining: Concepts and
Techniques 31
Scatter plot
Provides a first look at bivariate data to see clusters of points, outliers, etc
Each pair of values is treated as a pair of coordinates and plotted as points in the plane
April 21, 2023Data Mining: Concepts and
Techniques 32
Loess Curve
Adds a smooth curve to a scatter plot in order to provide better perception of the pattern of dependence
Loess curve is fitted by setting two parameters: a smoothing parameter, and the degree of the polynomials that are fitted by the regression
April 21, 2023Data Mining: Concepts and
Techniques 33
Positively and Negatively Correlated Data
The left half fragment is positively
correlated
The right half is negative correlated
April 21, 2023Data Mining: Concepts and
Techniques 34
Not Correlated Data
April 21, 2023Data Mining: Concepts and
Techniques 35
Data Visualization and Its Methods
Why data visualization? Gain insight into an information space by mapping data
onto graphical primitives Provide qualitative overview of large data sets Search for patterns, trends, structure, irregularities,
relationships among data Help find interesting regions and suitable parameters for
further quantitative analysis Provide a visual proof of computer representations derived
Typical visualization methods: Geometric techniques Icon-based techniques Hierarchical techniques
April 21, 2023Data Mining: Concepts and
Techniques 36
Geometric Techniques
Visualization of geometric transformations and projections of the data
Methods Landscapes Projection pursuit technique
Finding meaningful projections of multidimensional data
Scatterplot matrices Prosection views Hyperslice Parallel coordinates
April 21, 2023Data Mining: Concepts and
Techniques 37
Scatterplot Matrices
Matrix of scatterplots (x-y-diagrams) of the k-dim. data
Use
d by
erm
issi
on o
f M
. W
ard,
Wor
cest
er P
olyt
echn
ic In
stitu
te
April 21, 2023Data Mining: Concepts and
Techniques 38
news articlesvisualized asa landscape
Use
d by
per
mis
sion
of B
. Wrig
ht, V
isib
le D
ecis
ions
Inc.
Landscapes
Visualization of the data as perspective landscape The data needs to be transformed into a (possibly artificial) 2D spatial
representation which preserves the characteristics of the data
April 21, 2023Data Mining: Concepts and
Techniques 39
Attr. 1 Attr. 2 Attr. kAttr. 3
• • •
Parallel Coordinates
n equidistant axes which are parallel to one of the screen axes and correspond to the attributes
The axes are scaled to the [minimum, maximum]: range of the corresponding attribute
Every data item corresponds to a polygonal line which intersects each of the axes at the point which corresponds to the value for the attribute
April 21, 2023Data Mining: Concepts and
Techniques 40
Parallel Coordinates of a Data Set
April 21, 2023Data Mining: Concepts and
Techniques 41
Icon-based Techniques
Visualization of the data values as features of icons Methods:
Chernoff Faces Stick Figures Shape Coding: Color Icons: TileBars: The use of small icons representing the
relevance feature vectors in document retrieval
April 21, 2023Data Mining: Concepts and
Techniques 42
Chernoff Faces
A way to display variables on a two-dimensional surface, e.g., let x be eyebrow slant, y be eye size, z be nose length, etc.
The figure shows faces produced using 10 characteristics--head eccentricity, eye size, eye spacing, eye eccentricity, pupil size, eyebrow slant, nose size, mouth shape, mouth size, and mouth opening): Each assigned one of 10 possible values, generated using Mathematica (S. Dickson)
REFERENCE: Gonick, L. and Smith, W. The Cartoon Guide to Statistics. New York: Harper Perennial, p. 212, 1993
Weisstein, Eric W. "Chernoff Face." From MathWorld--A Wolfram Web Resource. mathworld.wolfram.com/ChernoffFace.html
April 21, 2023Data Mining: Concepts and
Techniques 43
Hierarchical Techniques
Visualization of the data using a hierarchical partitioning into subspaces.
Methods Dimensional Stacking Worlds-within-Worlds Treemap Cone Trees InfoCube
April 21, 2023Data Mining: Concepts and
Techniques 44
Tree-Map
Screen-filling method which uses a hierarchical partitioning of the screen into regions depending on the attribute values
The x- and y-dimension of the screen are partitioned alternately according to the attribute values (classes)
MSR Netscan Image
April 21, 2023Data Mining: Concepts and
Techniques 45
Tree-Map of a File System (Schneiderman)
April 21, 2023Data Mining: Concepts and
Techniques 46
General data characteristics
Basic data description and exploration
Measuring data similarity (Sec. 7.2)
April 21, 2023Data Mining: Concepts and
Techniques 47
Similarity and Dissimilarity
Similarity Numerical measure of how alike two data objects
are Value is higher when objects are more alike Often falls in the range [0,1]
Dissimilarity (i.e., distance) Numerical measure of how different are two data
objects Lower when objects are more alike Minimum dissimilarity is often 0 Upper limit varies
Proximity refers to a similarity or dissimilarity
April 21, 2023Data Mining: Concepts and
Techniques 48
Data Matrix and Dissimilarity Matrix
Data matrix n data points with
p dimensions Two modes
Dissimilarity matrix n data points, but
registers only the distance
A triangular matrix Single mode
npx...nfx...n1x
...............ipx...ifx...i1x
...............1px...1fx...11x
0...)2,()1,(
:::
)2,3()
...ndnd
0dd(3,1
0d(2,1)
0
April 21, 2023Data Mining: Concepts and
Techniques 49
Example: Data Matrix and Distance Matrix
0
1
2
3
0 1 2 3 4 5 6
p1
p2
p3 p4
point x yp1 0 2p2 2 0p3 3 1p4 5 1
Distance Matrix (i.e., Dissimilarity Matrix) for Euclidean Distance
p1 p2 p3 p4p1 0 2.828 3.162 5.099p2 2.828 0 1.414 3.162p3 3.162 1.414 0 2p4 5.099 3.162 2 0
Data Matrix
April 21, 2023Data Mining: Concepts and
Techniques 50
Minkowski Distance
Minkowski distance: A popular distance measure
where i = (xi1, xi2, …, xip) and j = (xj1, xj2, …, xjp) are
two p-dimensional data objects, and q is the order Properties
d(i, j) > 0 if i ≠ j, and d(i, i) = 0 (Positive definiteness)
d(i, j) = d(j, i) (Symmetry) d(i, j) d(i, k) + d(k, j) (Triangle Inequality)
A distance that satisfies these properties is a metric
pp
jx
ix
jx
ix
jx
ixjid )||...|||(|),(
2211
April 21, 2023Data Mining: Concepts and
Techniques 51
Special Cases of Minkowski Distance
q = 1: Manhattan (city block, L1 norm) distance E.g., the Hamming distance: the number of bits that are
different between two binary vectors
q= 2: (L2 norm) Euclidean distance
q . “supremum” (Lmax norm, L norm) distance. This is the maximum difference between any component
of the vectors Do not confuse q with n, i.e., all these distances are defined
for all numbers of dimensions. Also, one can use weighted distance, parametric Pearson
product moment correlation, or other dissimilarity measures
)||...|||(|),( 22
22
2
11 pp jx
ix
jx
ix
jx
ixjid
||...||||),(2211 pp jxixjxixjxixjid
April 21, 2023Data Mining: Concepts and
Techniques 52
Example: Minkowski Distance
Distance Matrix
point x yp1 0 2p2 2 0p3 3 1p4 5 1
L1 p1 p2 p3 p4p1 0 4 4 6p2 4 0 2 4p3 4 2 0 2p4 6 4 2 0
L2 p1 p2 p3 p4p1 0 2.828 3.162 5.099p2 2.828 0 1.414 3.162p3 3.162 1.414 0 2p4 5.099 3.162 2 0
L p1 p2 p3 p4
p1 0 2 3 5p2 2 0 1 3p3 3 1 0 2p4 5 3 2 0
April 21, 2023Data Mining: Concepts and
Techniques 53
Binary Variables
A contingency table for binary data
Distance measure for symmetric binary
variables:
Distance measure for asymmetric binary
variables:
Jaccard coefficient (similarity measure
for asymmetric binary variables):
A binary variable is symmetric if both of
its states are equally valuable and carry
the same weight. cbaa jisim
Jaccard ),(
dcbacb jid
),(
cbacb jid
),(
pdbcasum
dcdc
baba
sum
0
1
01
Object i
Object j
April 21, 2023Data Mining: Concepts and
Techniques 54
Dissimilarity between Binary Variables
Example
gender is a symmetric attribute the remaining attributes are asymmetric binary let the values Y and P be set to 1, and the value N be set
to 0
Name Gender Fever Cough Test-1 Test-2 Test-3 Test-4
Jack M Y N P N N NMary F Y N P N P NJim M Y P N N N N
75.0211
21),(
67.0111
11),(
33.0102
10),(
maryjimd
jimjackd
maryjackd
April 21, 2023Data Mining: Concepts and
Techniques 55
Nominal Variables
A generalization of the binary variable in that it can take more than 2 states, e.g., red, yellow, blue, green
Method 1: Simple matching m: # of matches, p: total # of variables
Method 2: Use a large number of binary variables creating a new binary variable for each of the M
nominal states
pmpjid ),(
April 21, 2023Data Mining: Concepts and
Techniques 56
Ordinal Variables
An ordinal variable can be discrete or continuous Order is important, e.g., rank Can be treated like interval-scaled
replace xif by their rank
map the range of each variable onto [0, 1] by replacing i-th object in the f-th variable by
compute the dissimilarity using methods for interval-scaled variables
11
f
ifif M
rz
},...,1{fif
Mr
April 21, 2023Data Mining: Concepts and
Techniques 57
Ratio-Scaled Variables
Ratio-scaled variable: a positive measurement on a nonlinear scale, approximately at exponential scale, such as AeBt or Ae-Bt
Methods: treat them like interval-scaled variables—not a
good choice! (why?—the scale can be distorted) apply logarithmic transformation
yif = log(xif)
treat them as continuous ordinal data treat their rank as interval-scaled
April 21, 2023Data Mining: Concepts and
Techniques 58
Variables of Mixed Types
A database may contain all the six types of variables symmetric binary, asymmetric binary, nominal,
ordinal, interval and ratio One may use a weighted formula to combine their
effects
f is binary or nominal:dij
(f) = 0 if xif = xjf , or dij(f) = 1 otherwise
f is interval-based: use the normalized distance f is ordinal or ratio-scaled
Compute ranks rif and Treat zif as interval-scaled
)(1
)()(1),(
fij
pf
fij
fij
pf
djid
1
1
f
if
Mrz
if
April 21, 2023Data Mining: Concepts and
Techniques 59
Vector Objects: Cosine Similarity
Vector objects: keywords in documents, gene features in micro-arrays, …
Applications: information retrieval, biologic taxonomy, ... Cosine measure: If d1 and d2 are two vectors, then
cos(d1, d2) = (d1 d2) /||d1|| ||d2|| ,
where indicates vector dot product, ||d||: the length of vector d Example:
d1 = 3 2 0 5 0 0 0 2 0 0
d2 = 1 0 0 0 0 0 0 1 0 2
d1d2 = 3*1+2*0+0*0+5*0+0*0+0*0+0*0+2*1+0*0+0*2 = 5
||d1||= (3*3+2*2+0*0+5*5+0*0+0*0+0*0+2*2+0*0+0*0)0.5=(42)0.5 = 6.481
||d2|| = (1*1+0*0+0*0+0*0+0*0+0*0+0*0+1*1+0*0+2*2)0.5=(6) 0.5 = 2.245
cos( d1, d2 ) = .3150
April 21, 2023Data Mining: Concepts and
Techniques 60
Correlation Analysis (Numerical Data)
Correlation coefficient (also called Pearson’s product moment coefficient)
where n is the number of tuples, and are the respective means of p and q, σp and σq are the respective standard
deviation of p and q, and Σ(pq) is the sum of the pq cross-product.
If rp,q > 0, p and q are positively correlated (p’s values
increase as q’s). The higher, the stronger correlation. rp,q = 0: independent; rpq < 0: negatively correlated
qpqpqp n
qpnpq
n
qqppr
)1(
)(
)1(
))((,
p q
April 21, 2023Data Mining: Concepts and
Techniques 61
Correlation (viewed as linear relationship)
Correlation measures the linear relationship between objects
To compute correlation, we standardize data objects, p and q, and then take their dot product
)(/))(( pstdpmeanpp kk
)(/))(( qstdqmeanqq kk
qpqpncorrelatio ),(
April 21, 2023Data Mining: Concepts and
Techniques 62
Visually Evaluating Correlation
Scatter plots showing the similarity from –1 to 1.