class 18 updates 2 - harvard seasdaslab.seas.harvard.edu/.../cs165fall2016class18.pdf ·...

updates 2.0prof. Stratos Idreos

HTTP://DASLAB.SEAS.HARVARD.EDU/CLASSES/CS165/

class 18

http://daslab.seas.harvard.edu/classes/cs165/

/40CS165, Fall 2016 Stratos Idreos 2

updates

UPDATE table_name SET column1=value1,column2=value2,... WHERE some_column=some_value

INSERT INTO table_name VALUES (value1,value2,value3,...)


traditional applications e.g., banking

how many times per day do you send update queries to your bank account


the world has changed a little bit by now…

updates


still we spy Facebook more than the # of photos we upload or # of our twitter posts, etc…


which kind of update is more common

update, insert, delete


which kind of update is more common

update, insert, delete

so our new challenge is: reads and inserts + variable read/write ratio


monitor CPU utilization

monitor memory hierarchy utilization

monitor clicks (frequency, locations, specific links, sequences)


insert new entry (a,b,c,d,…) on table x

table xA B C D

…

update N columns, K trees, statistics, …

A

B

D


what can go wrong?failures & >1 queries


transaction any database query that can/should be seen as a single task

parse optimize

plan( read X write Z … )


AtomicityConsistencyIsolationDurability

Jim Gray, IBM, Tandem, DEC, Microsoft ACM Turing award ACM SIGMOD Edgar F. Codd Innovations award


AtomicityConsistencyIsolationDurability

>1 queries

all or nothing

correct

persistent


update all rows where A=v1 & B=v2

to (a=a/2,b=b/4,c=c-3,d=d+2)


A B C Dsearch (scan/index) to find row to update

select+project actions


to (a=a/2,b=b/4,c=c-3,d=d+2)




list of rowIDs (positions)

A B C D


to (a=a/2,b=b/4,c=c-3,d=d+2)

(sort)





A B C D


to (a=a/2,b=b/4,c=c-3,d=d+2)

we know what to update but nothing happened yet

(sort)


level 1

CPU

A B C Dlevel 2

read page in L1 update

persist to L2

if problem (power/abort) before we write all pages

we are left with an inconsistent state

update all rows where A=v1 & B=v2 to (a=a/2,b=b/4,c=c-3,d=d+2)


recoveryclassic examplejoe owes mike 100$

both joe and mike have a Bank of Bla account

possible actionsjoe -100 mike + 100

mike + 100 joe - 100

what if there is a failure here?


recoveryclassic examplejoe owes mike 100$

both joe and mike have a Bank of Bla account

possible actionsjoe -100 mike + 100

mike + 100 joe - 100

what if there is a failure here?

actually1)read mike; 2) mike+100; 3) write new mike;


level 1

CPU

A B C Dlevel 2


persist to L2


what do we need to remember (log)

WAL: keep persistent notes as we go so we can resume or undo





A B C D


to (a=a/2,b=b/4,c=c-3,d=d+2)

(sort)

restart: get set of positions again, and either resume from last written page or undo all previously written pages


level 1

CPU

A B C Dlevel 2


persist to L2


(update in place/out of place) problems when failure


memory

disk

buffer updateslog

memory

disk

buffer updates

log

SSD

memory

SSD

buffer updates

log

non volatile memory

disk


ARIES: A Transaction Recovery Method Supporting Fine-Granularity Locking and Partial Rollbacks Using Write-Ahead LoggingC. Mohan, Donald J. Haderle, Bruce G. Lindsay, Hamid Pirahesh, Peter M. Schwarz ACM Transactions on Database Systems, 1992

C Mohan, IBM Research ACM SIGMOD Edgar F. Codd Innovations award 1993

(rumor has it he has his own Facebook server) (also check his noSQL lecture)


query 1

parse optimize

plan( read X write Z … )

query 2

parse optimize

plan( read Z write Z … )


locks

… read X …

… get_rlock(X) read X release_rlock(X) …

… write X …

… get_wlock(X) write X release_wlock(X) … commit

R WR yes noW no no

wait until lock is free


lock granularity

…

databasetable1 table2

…

C1 C2

A survey of B-tree logging and recovery techniquesGoetz Graefe ACM Trans. Database Systems, 2012


2 phase locking1. get all locks

2. do all tasks

3. commit

4. release all locks

(and variations)

be positive! - optimistic concurrency control

HT


2 phase locking1. get all locks

2. do all tasks

3. commit

4. release all locks

(and variations)

be positive! - optimistic concurrency control

HT

3 sections about ACID properties, logging, locking and recovery

online next few days


classic examplejoe owes mike 100$

both joe and mike have a Bank of Bla accountΧwhat about logging and recovery during e-shopping

e.g., shopping cart, wish list, check out

recovery

Quantifying eventual consistency with PBSPeter Bailis, Shivaram Venkataraman, Michael J. Franklin, Joseph M. Hellerstein, Ion Stoica Communications of the ACM, 2014


why noSQLdb noSQL

as apps become more complex

as apps need to be more scalable

complex legacy tuning

expensive …

simple clean

just enough …

newSQL

Mesa: Geo-Replicated, Near Real-Time, Scalable Data WarehousingAshish Gupta et al. International Conference on Very Large Databases (VLDB), 2014

F1: A Distributed SQL Database That ScalesJeff Shute et al. International Conference on Very Large Databases (VLDB), 2013

(more in 265)


remember: it all starts with how we store the data

IDEAL

a data structure that is best for reads, best for writes and does not require too much metadata


Reads

UpdatesMemory

Designing Access Methods: The RUM ConjectureM. Athanassoulis, M. Kester, L. Maas, R. Stoica, Radu, S. Idreos, A. Ailamaki, M Callaghan International Conference on Extending Database Technology (EDBT), 2016

you can’t always get what you want


a perfect access method for reads (point queries)

oracle

x

find(x)



oracle

x

find(x)reads

updates

memory



binary search to find(x)

but with no memory overhead

sorted



binary search to find(x)

reads

updates

memory

but with no memory overhead

sorted


a perfect access method for writes (point writes)

x

update(x)

x xupdate log


a perfect access method for writes (point writes)

x

update(x)

reads

updates

memoryx x

update log


to structure or not to structure

sorted fixed-width & dense

no order fixed-width & dense

sorted fixed-width with holes

insert v, delete v, update v to v’


The Transaction Concept:Virtues and LimitationsJim Gray, Tandem TR 81.3, 1981

in-place updates: the cardinal sin

A(t1) A(t2) A(t3)

…


content vs structure update

insert tuple(a1, b1, c1, …) insert(A,a1), insert(B, b1), …

say there is a secondary index on A

(1) append a1 anywhere to index (any node/buffer) (2) reorganize index to maintain structure

…


great for inserts/updates great for “recent” reads

optimize as much as possible querying “old” data

give read/write cost put(key, value) - get(key, value)

design a key-value store (265 systems project)


L1

L2

all writes go to a small buffer so updates are fast and

reads can find recent updates fast

data is ordered on ingestion order and no in place updates

getting old data is a full scan and we may have duplicates to clean up


L1

L2multi-level L2 structure

push to next level when full remove duplicates with every merge


L1

L2to improve reads:

each level may be sorted, it may be a b-tree

have a bloom filter, etc

worst case scenario: have to scan the whole thing: empty queries


L1

L2to improve reads:

each level may be sorted, it may be a b-tree

have a bloom filter, etc

worst case scenario: have to scan the whole thing: empty queriesmore about this design space in 265


textbook: chapters 16, 17, 18

DATA SYSTEMSprof. Stratos Idreos

class 18

updates 2.0

class 18 updates 2 - harvard seasdaslab.seas.harvard.edu/.../cs165fall2016class18.pdf ·...

Documents