principles of data management lecture #4 (storage and ... · principles of data management lecture...
TRANSCRIPT
Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1
Principles of Data Management
Lecture #4 (Storage and Indexing Principles)
Instructor: Mike Carey [email protected]
Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 2
Today’s Headlines
v Project 1 was due last night § A bunch of you are still working on it § That’s actually fine (at least I think so... J) § Just get it done by Wed’s “late deadline”
v Go ahead and form your Project 2-4 teams § Sign up online per the Readers’ instructions
v Today’s lecture/coverage plan § System catalogs (end of last time’s plan/notes) § Intro to file and index structures (this time’s plan)
Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 3
Alternative File Organizations
Many alternatives exist. Each one is ideal for some situations, but not so good in others: § Heap (random ordered) files: Suitable when typical
access is a file scan retrieving all record or access comes through a variety of secondary indexes.
§ Sorted Files: Best if records must be retrieved in some order, or only a `range’ of records is needed.
§ Indexes: Data structures to organize records via trees or hashing.
• Like sorted files, they speed up searches for a subset of records, based on values in certain (“search key”) fields.
• Updates are much (!) faster than in sorted files.
Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 4
Indexes v An index on a file speeds up selections on the
search key fields for the index. § Any subset of the fields of a relation can serve as
the search key for an index on the relation. § Search key is not the same as a key (a minimal set of
fields that uniquely identify a record in a relation). v An index contains a collection of data entries,
and supports efficient retrieval of all data entries k* with a given key value k. § Given data entry k*, we can find one record with
key k with just more disk I/O. (Details soon …)
Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 5
B+ Tree Indexes (Overview)
v Leaf pages contain data entries, and are chained (prev & next) v Non-leaf pages have index entries; used only to direct searches:
P 0 K 1 P 1 K 2 P 2 K m P m
index entry
Non-leaf Pages
Pages (Sorted by search key)
Leaf
Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 6
Example B+ Tree
v Find 28*? Find 29*? Find where > 15* and < 30* v Insert/delete: Find data entry in leaf, then
change it. Need to adjust parent sometimes. § Change sometimes bubbles up further (occasionally).
2* 3*
Root
17
30
14* 16* 33* 34* 38* 39*
13 5
7* 5* 8* 22* 24*
27
27* 29*
Entries < 17 Entries >= 17
Notice that data entries at leaf level are sorted
Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 7
Hash-Based Indexes
v Good for equality selections. v Index is a collection of buckets.
§ Bucket = primary page plus zero or more overflow pages (called static hashing).
§ Buckets contain data entries. v Hashing function h: h(r) = bucket in which
(data entry for) record r belongs. h looks at the search key fields of r. § No need to have non-leaf “entries” in this scheme. § Reason is that location is computed (not searched).
Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 8
Alternatives for Data Entry k* in Index
v In a data entry k* we can store: § Actual data record with key value k, or § <k, rid of data record with search key value k>, or § <k, list of rids of data records with search key k>
v Choice of alternative for data entries is orthogonal to the indexing technique used to locate data entries with a given key value k. § Examples of indexing techniques: B+ trees, hash-
based structures, R trees, … § Index’s job: direct searches to desired data entries § A rid alternative in secondary indexes: primary key
Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 9
Alternatives for Data Entries (Contd.)
v Alternative 1: Data Records Live in Index § If this is used, index structure is actually a file
organization for the data records (e.g., instead of a separate Heap file for them to live in).
§ At most one index on a given collection of data records can use Alternative 1. (Otherwise, data records are duplicated, leading to redundant storage and potential inconsistency.)
§ If data records are very large, # of (leaf) pages containing data entries will be high. Implies size of auxiliary information in the index is also large, typically.
Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 10
Alternatives for Data Entries (Contd.)
v Alternatives 2 and 3: Key/Rid or Key/RidList § Data entries typically much smaller than data
records. (Portion of index structure used to direct searches, which depends on size of data entries, is also smaller than in Alternative 1 – fewer leaves!)
§ Alternative 3 more compact than Alternative 2, but leads to variable sized data entries even if the search keys are of fixed length.
§ Can treat each Key/Rid pair in a composite key-like fashion in higher levels of the index to handle case where a (big) RidList could overflow a leaf page.
Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 11
Index Classification
v Primary vs. secondary: If search key contains the primary key, it’s called the primary index. § Unique index: Search key contains a candidate key.
v Clustered vs. unclustered: If order of data records is the same as, or `close to’, the order of stored data records, it’s called a clustered index. § Alternative 1 implies clustered; in practice, clustered
also implies Alternative 1 (as sorted files are rare). § A file can be clustered on at most one search key. § Cost of retrieving data records via an index varies
greatly based on whether it is clustered or not!
Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 12
Clustered vs. Unclustered Index v Suppose that Alternative (2) is used for data entries,
and that the data records are stored in a Heap file. § To build a clustered index, first sort the Heap file (with
some free space left on each page for future inserts). § Overflow pages may be needed for inserts. (Thus, order of
data recs is `close to’, but not identical to, the sort order.)
Index entries
Data entries
direct search for
(Index File) (Data file)
Data Records
data entries
Data entries
Data Records
CLUSTERED UNCLUSTERED
Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 13
A Back of the Envelope Cost Model
We will ignore CPU costs, for simplicity, so: § B: The number of data pages § R: Number of records per page § D: (Average) time to read or write disk page § Counting the number of page I/Os ignores gains of
prefetching a sequence of pages; thus, even the real I/O cost is only roughly approximated for now.
§ Average-case analysis; based on several simplistic (okay, also sloppy J) assumptions.
☛ Good enough to convey the overall trends!
Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 14
Comparison of File Organizations
v Heap files (random order; insert at EOF) v Sorted files, sorted on <age, sal> v Clustered B+ tree file, Alternative (1), search
key <age, sal> v Heap file with unclustered B + tree index on
search key <age, sal> v Heap file with unclustered hash index on
search key <age, sal>
Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 15
Operations to Compare
v Scan: Fetch all records from disk v Equality search v Range selection v Insert a record v Delete a record
Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 16
Assumptions for Our Analysis v Heap Files:
§ Equality selection on key; have exactly one match.
v Sorted Files: § File compacted after a deletion (vs. a deleted bit).
v Indexes (with data in Heap file): § Alt (2), (3): data entry size = 10% size of record
• Implies entry file size = 0.1 * data size
§ Hash: No overflow buckets yet. Reason is • 80% page occupancy à File size = 1.25 * “data” size
(which is 0.125 * data size if “data” is set of all entries)
§ Tree: 67% page occupancy (this is typical). • Implies file size = 1.5 * “data” size (= 0.15 * data size)
Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 17
Assumptions (cont’d.)
v Scans: § Leaf levels of a tree-index are chained. § Index data-entries plus actual file must be
“scanned” for unclustered indexes.
v Range searches: § We can use tree indexes to restrict the set of data
records fetched, but hash indexes are “useless” for range queries.
Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 18
Cost of Operations (a) Scan (b)
Equality (c ) Range (d) Insert (e) Delete
(1) Heap
(2) Sorted
(3) Clustered
(4) Unclustered Tree index
(5) Unclustered Hash index
☛ Several assumptions underlie these (rough) estimates!
Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 19
Cost of Operations (a) Scan (b) Equality (c ) Range (d) Insert (e) Delete
(1) Heap BD 0.5BD BD 2D Search +D
(2) Sorted BD Dlog 2B D(log 2 B + # pgs with match recs)
Search + BD
Search +BD
(3) Clustered
1.5BD Dlog F 1.5B D(log F 1.5B + # pgs w. match recs)
Search + D
Search +D
(4) Unclust. Tree index
BD(R+0.15) D(1 + log F 0.15B)
D(log F 0.15B + # pgs w. match recs)
Search + 2D
Search + 2D
(5) Unclust. Hash index
BD(R+0.125) 2D BD Search + 2D
Search + 2D
☛ Several assumptions underlie these (rough) estimates!
Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 20
Some Unclustered Tree Cost Notes:
v Full unclustered index scans have high cost: § Read each of the (0.15 B) index leaves once § Do a random I/O per record (BR such I/O’s in all) § Thus, cost(scan) = D(0.15B+BR) = BD(R+0.15)
v Insert or delete record now needs 2 writes: § Write back of index leaf, to add/remove k* § Write back of file data page, to add/remove record
v “Number of pages w/matching records” higher by a factor of R in unclustered case! (See why?)
☛ NEXT TIME: Tree-based index structures in detail!