1 what happens when processing storage bandwidth are free and infinite? jim gray microsoft research
TRANSCRIPT
![Page 1: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/1.jpg)
1
What Happens WhenWhat Happens WhenProcessingProcessing
StorageStorageBandwidth Bandwidth
are Free and Infinite?are Free and Infinite?
Jim Gray
Microsoft Research
![Page 2: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/2.jpg)
2
OutlineOutline Hardware CyberBricks
– all nodes are very intelligent Software CyberBricks
– standard way to interconnect intelligent nodes What next?
– Processing migrates to where the power is• Disk, network, display controllers have full-blown OS
• Send RPCs to them (SQL, Java, HTTP, DCOM, CORBA) to them
• Computer is a federated distributed system.
![Page 3: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/3.jpg)
3
A Hypothetical QuestionA Hypothetical QuestionTaking things to the limitTaking things to the limit
Moore’s law 100x per decade:– Exa-instructions per second in 30 years
– Exa-bit memory chips
– Exa-byte disks Gilder’s Law of the Telecosom
3x/year more bandwidth60,000x per decade!
– 40 Gbps per fiber today
![Page 4: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/4.jpg)
4
Grove’s LawGrove’s Law
Link Bandwidth doubles every 100 years! Not much has happened to telephones lately Still twisted pair
![Page 5: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/5.jpg)
5
Gilder’s Telecosom Law: Gilder’s Telecosom Law: 3x bandwidth/year for 25 more years3x bandwidth/year for 25 more years
Today: – 10 Gbps per channel– 4 channels per fiber: 40 Gbps– 32 fibers/bundle = 1.2 Tbps/bundle
In lab 3 Tbps/fiber (400 x WDM) In theory 25 Tbps per fiber 1 Tbps = USA 1996 WAN bisection bandwidth
1 fiber = 25 Tbps
![Page 6: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/6.jpg)
6
ThesisThesisMany little beat few bigMany little beat few big
Smoking, hairy golf ballSmoking, hairy golf ball How to connect the many little parts?How to connect the many little parts? How to program the many little parts?How to program the many little parts? Fault tolerance?Fault tolerance?
$1 $1 millionmillion $100 K$100 K $10 K$10 K
MainframeMainframe MiniMiniMicroMicro NanoNano
14"14"9"9"
5.25"5.25" 3.5"3.5" 2.5"2.5" 1.8"1.8"1 M SPEC marks, 1TFLOP1 M SPEC marks, 1TFLOP
101066 clocks to bulk ram clocks to bulk ram
Event-horizon on chipEvent-horizon on chip
VM reincarnatedVM reincarnated
Multi-program cache,Multi-program cache,On-Chip SMPOn-Chip SMP
10 microsecond ram
10 millisecond disc
10 second tape archive
10 nano-second ram
Pico Processor
10 pico-second ram
1 MM 3
100 TB
1 TB
10 GB
1 MB
100 MB
![Page 7: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/7.jpg)
7
Year 2000 Year 2000 4B4B Machine Machine The Year 2000 commodity PC
Billion Instructions/Sec
.1 Billion Bytes RAM
Billion Bits/s Net
10 B Bytes Disk
Billion Pixel display– 3000 x 3000 x 24
1,000 $
10 GB byte Disk
.1 B byte RAM
1 Bips Processor
1 B
bits
/sec
LA
N/W
AN
![Page 8: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/8.jpg)
8
4 B PC’s: 4 B PC’s: The Bricks of CyberspaceThe Bricks of Cyberspace Cost 1,000 $ Come with
– OS (NT, POSIX,..)
– DBMS
– High speed Net
– System management
– GUI / OOUI
– Tools
Compatible with everyone else CyberBricks
![Page 9: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/9.jpg)
9
Super Server: 4T MachineSuper Server: 4T Machine Array of 1,000 4B machinesArray of 1,000 4B machines
1 b ips processors1 b ips processors1 B B DRAM 1 B B DRAM 10 B B disks 10 B B disks 1 Bbps comm lines1 Bbps comm lines1 TB tape robot1 TB tape robot
A few megabucksA few megabucks Challenge:Challenge:
ManageabilityManageabilityProgrammabilityProgrammabilitySecuritySecurityAvailabilityAvailabilityScaleabilityScaleabilityAffordabilityAffordability
As easy as a single systemAs easy as a single systemFuture servers are CLUSTERSFuture servers are CLUSTERSof processors, discsof processors, discs
Distributed database techniquesDistributed database techniquesmake clusters workmake clusters work
CPU
50 GB Disc
5 GB RAM
Cyber BrickCyber Bricka 4B machinea 4B machine
![Page 10: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/10.jpg)
10
Functionally Specialized CardsFunctionally Specialized Cards Storage
Network
Display
M MB DRAM
P mips processor
ASIC
ASIC
ASIC
Today:
P=50 mips
M= 2 MB
In a few years
P= 200 mips
M= 64 MB
![Page 11: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/11.jpg)
11
It’s Already True of PrintersIt’s Already True of PrintersPeripheral = CyberBrickPeripheral = CyberBrick
You buy a printer You get a
– several network interfaces– A Postscript engine
• cpu, • memory, • software,• a spooler (soon)
– and… a print engine.
![Page 12: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/12.jpg)
12
System On A ChipSystem On A Chip Integrate Processing with memory on one chip
– chip is 75% memory now– 1MB cache >> 1960 supercomputers– 256 Mb memory chip is 32 MB!– IRAM, CRAM, PIM,… projects abound
Integrate Networking with processing on one chip– system bus is a kind of network– ATM, FiberChannel, Ethernet,.. Logic on chip.– Direct IO (no intermediate bus)
Functionally specialized cards shrink to a chip.
![Page 13: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/13.jpg)
13
Tera Byte Backplane
TODAY– Disk controller is 10 mips risc engine
with 2MB DRAM– NIC is similar power
SOON– Will become 100 mips systems
with 100 MB DRAM. They are nodes in a federation
(can run Oracle on NT in disk controller).
Advantages– Uniform programming model– Great tools– Security– economics (cyberbricks)– Move computation to data (minimize traffic)
All Device Controllers will be Cray 1’sAll Device Controllers will be Cray 1’s
CentralProcessor &
Memory
![Page 14: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/14.jpg)
14
With Tera Byte InterconnectWith Tera Byte Interconnectand Super Computer Adaptersand Super Computer Adapters
Processing is incidental to – Networking– Storage– UI
Disk Controller/NIC is – faster than device– close to device– Can borrow device
package & power So use idle capacity for computation. Run app in device.
Tera ByteBackplane
![Page 15: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/15.jpg)
15
ImplicationsImplications
Offload device handling to NIC/HBA
higher level protocols: I2O, NASD, VIA…
SMP and Cluster parallelism is important.
Tera Byte Backplane
Move app to NIC/device controller
higher-higher level protocols: CORBA / DCOM.
Cluster parallelism is VERY important.
CentralProcessor &
Memory
Conventional Radical
![Page 16: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/16.jpg)
16
How Do They Talk to Each Other?How Do They Talk to Each Other? Each node has an OS Each node has local resources: A federation. Each node does not completely trust the others. Nodes use RPC to talk to each other CORBA? DCOM? IIOP? RMI? One or all of the above. Huge leverage in high-level interfaces. Same old distributed system story.
Wire(s)VIAL/VIPL
stre
ams
data
gram
s
RP
C?
Applications
VIAL/VIPL
streams
datagrams
RP
C ?
Applications
![Page 17: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/17.jpg)
18
Objects!Objects!
It’s a zoo ORBs, COM, CORBA,.. Object Relationa1 Databases Objects and 3-tier computing
![Page 18: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/18.jpg)
19
So
lari
sU
NIX
Inte
rnat
ion
al
OSFDCE
Op
en s
oft
war
e F
ou
nd
atio
n (
OS
F)
NT
ODBCXA / TX
Ob
ject
M
anag
emen
t G
rou
p (
OM
G)
CORBAOpenGroup
History and Alphabet SoupHistory and Alphabet Soup
1985
1990
1995
X/O
pen
DCE
RPC
GUIDs
IDL
DNS
Kerber
os
COM
Microsoft DCOM based on OSF-DCE TechnologyDCOM and ActiveX extend it
COM
![Page 19: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/19.jpg)
20
The PromiseThe Promise
Objects are Software CyberBricks– productivity breakthrough (plug ins)
– manageability breakthrough (modules) Microsoft Promises Cairo
distributed objects, secure, transparent, fast invocation
IBM/Sun/Oracle/Netscape promise CORBA + Open Doc + Java Beans +
All will deliver Customers can pick the best one
Both campsShare key goals: Encapsulation: hide implementation Polymorphism: generic ops
key to GUI and reuse Uniform Naming Discovery: finding a service Fault handling: transactions Versioning: allow upgrades Transparency: local/remote Security: who has authority Shrink-wrap: minimal inheritance Automation: easy
![Page 20: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/20.jpg)
21
The OLE-COM ExperienceThe OLE-COM Experience Macintosh had Publish & Subscribe PowerPoint needed graphs:
– plugged MS Graph in as an component. Office adopted OLE
– one graph program for all of office Internet arrived
– URLs are object references, – Office is Web Enabled right away!
Office97 smaller than Office95 because of shared components
It works!!
![Page 21: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/21.jpg)
22
Linking And EmbeddingLinking And EmbeddingObjects are data modules;Objects are data modules;
transactions are execution modulestransactions are execution modules
Link: pointer to object somewhere else– Think URL in Internet
Embed: bytesare here
Objects may be active; can callback to subscribers
![Page 22: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/22.jpg)
23
Objects Meet DatabasesObjects Meet Databasesbasis for basis for universaluniversal data servers, access, & integration data servers, access, & integration
DBMSDBMSengineengine
Object-oriented (COM oriented) interface to data
Breaks DBMS into components Anything can be
a data source Optimization/navigation
“on top of” other data sources
Makes an RDBMS anO-R DBMS assuming optimizer understands objects
DatabaseDatabase
SpreadsheetSpreadsheet
PhotosPhotos
MailMail
MapMap
DocumentDocument
![Page 23: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/23.jpg)
24
The BIG PictureThe BIG PictureComponents and transactionsComponents and transactions
Software modules are objects Object Request Broker (a.k.a., Transaction Processing Monitor)
connects objects (clients to servers)
Standard interfaces allow software plug-ins Transaction ties execution of a “job” into an atomic unit:
all-or-nothing, durable, isolated ActiveX Components are a 250M$/year business.
Object RequestObject Request BrokerBroker
![Page 24: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/24.jpg)
25
Transaction
Object Request Broker (ORB)Object Request Broker (ORB) Orchestrates RPCOrchestrates RPC
Registers Servers Manages pools of servers Connects clients to servers Does Naming, request-level authorization, Provides transaction coordination Direct and queued invocation Old names:
– Transaction Processing Monitor,
– Web server,
– NetWare
Object-Request Broker
![Page 25: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/25.jpg)
26
The OO Points So FarThe OO Points So Far
Objects are software Cyber Bricks Object interconnect standards are emerging Cyber Bricks become Federated Systems. Next points:
– put processing close to data
– do parallel processing.
![Page 26: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/26.jpg)
27
Three Tier ComputingThree Tier Computing
Clients do presentation, gather input
Clients do some workflow (Xscript)
Clients send high-level requests to ORB
ORB dispatches work-flows and business objects -- proxies for client, orchestrate flows & queues
Server-side workflow scripts call on distributed business objects to execute task
Database
Business Objects
workflow
Presentation
![Page 27: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/27.jpg)
28
The Three The Three TiersTiers
Web Client
HTML
VB or Java Script Engine
VB or Java Virt Machine
VBscritptJavaScrpt
VB Javaplug-ins
InternetORB
HTTP+DCOM
ObjectserverPool
MiddlewareORB
TP MonitorWeb Server...
DCOM (oleDB, ODBC,...)
Object & Dataserver.
LU6.2
IBMLegacy Gateways
![Page 28: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/28.jpg)
29
Transaction Processing Transaction Processing Evolution to Three TierEvolution to Three Tier
Intelligence migrated to clientsIntelligence migrated to clients Mainframe Batch processing
(centralized)
Dumb terminals & Remote Job Entry
Intelligent terminals database backends
Workflow SystemsObject Request BrokersApplication Generators
Mainframe
cards
Active
green screen3270
Server
TP Monitor
ORB
![Page 29: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/29.jpg)
30
Web Evolution to Three TierWeb Evolution to Three TierIntelligence migrated to clients (like TP)Intelligence migrated to clients (like TP)
Character-mode clients, smart servers
GUI Browsers - Web file servers
GUI Plugins - Web dispatchers - CGI
Smart clients - Web dispatcher (ORB)pools of app servers (ISAPI, Viper)workflow scripts at client & server
archie ghophergreen screen
WebServer
Mosaic
WAIS
NS & IE
Active
![Page 30: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/30.jpg)
31
PC Evolution to Three TierPC Evolution to Three Tier Intelligence migrated to serverIntelligence migrated to server
Stand-alone PC (centralized)
PC + File & print servermessage per I/O
PC + Database server message per SQL statement
PC + App server message per transaction
ActiveX Client, ORB ActiveX server, Xscript
disk I/OIO request
reply
SQL Statement
Transaction
![Page 31: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/31.jpg)
32
Why Did Everyone Go To Three-Why Did Everyone Go To Three-Tier?Tier?
Manageability– Business rules must be with data
– Middleware operations tools
Performance (scaleability)– Server resources are precious
– ORB dispatches requests to server pools
Technology & Physics– Put UI processing near user
– Put shared data processing near shared data
– Minimizes data moves
– Encapsulate / modularityDatabase
Business Objects
workflow
Presentation
![Page 32: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/32.jpg)
33
DAD’sRaw Data
Customer comes to storeTakes what he wantsFills out invoiceLeaves money for goods
Easy to buildNo clerks
Why Put Business Objects at Why Put Business Objects at Server?Server?
Customer comes to store with list Gives list to clerk Clerk gets goods, makes invoiceCustomer pays clerk, gets goods
Easy to manageClerks controls accessEncapsulation
MOM’s Business Objects
![Page 33: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/33.jpg)
34
The OO Points So FarThe OO Points So Far
Objects are software Cyber Bricks Object interconnect standards are emerging Cyber Bricks become Federated Systems. Put processing close to data Next point:
– do parallel processing.
![Page 34: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/34.jpg)
35
Parallelism: Parallelism: the OTHER half of Super-Serversthe OTHER half of Super-Servers Clusters of machines allow two kinds of parallelism
– Many little jobs: Online transaction processing
• TPC A, B, C,…
– A few big jobs: data search & analysis• TPC D, DSS, OLAP
Both give automatic Parallelism
![Page 35: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/35.jpg)
36
Why Parallel Access To Data?Why Parallel Access To Data?
1 Terabyte
10 MB/s
At 10 MB/s1.2 days to scan
1 Terabyte
1,000 x parallel100 second SCAN.
Parallelism: divide a big problem into many smaller ones
to be solved in parallel.
BANDWID
TH
![Page 36: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/36.jpg)
37
Kinds of Parallel ExecutionKinds of Parallel Execution
Pipeline
Partition outputs split N ways inputs merge M ways
Any Sequential Program
Any Sequential Program
SequentialSequential
SequentialSequential Any Sequential Program
Any Sequential Program
![Page 37: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/37.jpg)
38
Why are Relational OperatorsWhy are Relational OperatorsSuccessful for Parallelism?Successful for Parallelism?
Relational data model uniform operatorson uniform data streamClosed under composition
Each operator consumes 1 or 2 input streamsEach stream is a uniform collection of dataSequential data in and out: Pure dataflow
partitioning some operators (e.g. aggregates, non-equi-join, sort,..)
requires innovation
AUTOMATIC PARALLELISM
![Page 38: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/38.jpg)
39
Database Systems Database Systems “Hide” Parallelism “Hide” Parallelism
Automate system management via tools– data placement– data organization (indexing)– periodic tasks (dump / recover / reorganize)
Automatic fault tolerance– duplex & failover– transactions
Automatic parallelism– among transactions (locking)– within a transaction (parallel execution)
![Page 39: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/39.jpg)
40
SQL SQL a Non-Procedural a Non-Procedural Programming LanguageProgramming Language
SQL: functional programming language describes answer set.
Optimizer picks best execution plan– Picks data flow web (pipeline), – degree of parallelism (partitioning)– other execution parameters (process placement, memory,...)
GUI
Schema
Plan
Monitor
Optimizer
ExecutionPlanning
Rivers
Executors
![Page 40: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/40.jpg)
41
Automatic Data Automatic Data PartitioningPartitioningSplit a SQL table to subset of nodes & disks
Partition within set:Range Hash Round Robin
Shared disk and memory less sensitive to partitioning, Shared nothing benefits from "good" partitioning
A...E F...J K...N O...S T...Z A...E F...J K...N O...S T...Z A...E F...J K...N O...S T...Z
Good for equijoins, range queriesgroup-by
Good for equijoins Good to spread load
![Page 41: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/41.jpg)
42
N x M way ParallelismN x M way Parallelism
A...E F...J K...N O...S T...Z
Merge
Join
Sort
Join
Sort
Join
Sort
Join
Sort
Join
Sort
Merge Merge
N inputs, M outputs, no bottlenecks.
![Page 42: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/42.jpg)
43
Parallel Objects?Parallel Objects? How does all this DB parallelism connect to
hardware/software Cyber Bricks? To scale to large client sets
– need lots of independent parallel execution.– Comes for from from ORB.
To scale to large data sets– need intra-program parallelism (like parallel DBs)– Requires some invention.
![Page 43: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/43.jpg)
44
OutlineOutline Hardware CyberBricks
– all nodes are very intelligent Software CyberBricks
– standard way to interconnect intelligent nodes What next?
– Processing migrates to where the power is• Disk, network, display controllers have full-blown OS
• Send RPCs to them (SQL, Java, HTTP, DCOM, CORBA) to them
• Computer is a federated distributed system.
• Parallel execution is important
![Page 44: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/44.jpg)
45
MORE SLIDESMORE SLIDESbut there is only so but there is only so
much time.much time.
Too bad
![Page 45: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/45.jpg)
46
The Disk Farm On a CardThe Disk Farm On a CardThe Disk Farm On a CardThe Disk Farm On a CardThe 100GB disc cardAn array of discsCan be used as 100 discs 1 striped disc 10 Fault Tolerant discs ....etcLOTS of accesses/second bandwidth
14"
Life is cheap, its the accessories that cost ya.
Processors are cheap, it’s the peripherals that cost ya (a 10k$ disc card).
![Page 46: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/46.jpg)
47
Parallelism: Parallelism: Performance is the GoalPerformance is the Goal
Goal is to get 'good' performance. Trade time for money.
Law 1: parallel system should be faster than serial system
Law 2: parallel system should give near-linear scaleup or
near-linear speedup orboth.
Parallel DBMSs obey these laws
![Page 47: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/47.jpg)
48
Success StoriesSuccess Stories Online Transaction Processing
– many little jobs
– SQL systems support
• 50 k tpm-C (44 cpu, 600 disk 2 node )
Batch (decision support and Utility)– few big jobs, parallelism inside
– Scan data at 100 MB/s
– Linear Scaleup to 1,000 processors
tran
sact
ion
s /
sec
hardware
recs
/ se
c
hardware
![Page 48: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/48.jpg)
49
The New Law of ComputingThe New Law of Computing
Grosch's Law:
Parallel Law: Needs
Linear Speedup and Linear ScaleupNot always possible
1 MIPS1 $
1,000 $
1,000 MIPS
2x $ is 2x performance
1 MIPS1 $
1,000 MIPS 32 $.03$/MIPS
2x $ is 4x performance
![Page 49: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/49.jpg)
50
Clusters being builtClusters being built Teradata 1,000 nodes
(30k$/slice) Tandem,VMScluster 150 nodes (100k$/slice) Intel, 9,000 nodes @ 55M$ (
6k$/slice) Teradata, Tandem, DEC
moving to NT+low slice price IBM: 512 nodes ASCI @ 100m$ (200k$/slice) PC clusters (bare handed) at dozens of nodes
web servers (msn, PointCast,…), DB servers KEY TECHNOLOGY HERE IS THE APPS.
– Apps distribute data– Apps distribute execution
![Page 50: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/50.jpg)
52
BOTH SMP and BOTH SMP and Cluster?Cluster?
SMPSuper Server
DepartmentalServer
PersonalSystem
Grow Up with SMPGrow Up with SMP4xP6 is now standard4xP6 is now standard
Grow Out with ClusterGrow Out with Cluster
Cluster has inexpensive partsCluster has inexpensive parts
Clusterof PCs
![Page 51: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/51.jpg)
53
Clusters Have AdvantagesClusters Have Advantages Clients and Servers made from the same stuff.
Inexpensive: – Built with commodity components
Fault tolerance: – Spare modules mask failures
Modular growth– grow by adding small modules
![Page 52: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/52.jpg)
54
Meta-Message:Meta-Message: Technology Ratios Are Important Technology Ratios Are Important
Meta-Message:Meta-Message: Technology Ratios Are Important Technology Ratios Are Important
If everything gets faster & cheaper at the same rate THEN nothing really
changes.
Things getting MUCH BETTER:
– communication speed & cost 1,000x– processor speed & cost 100x– storage size & cost 100x
Things staying about the same– speed of light (more or less constant)– people (10x more expensive)– storage speed (only 10x better)
![Page 53: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/53.jpg)
55
Storage Ratios ChangedStorage Ratios Changed 10x better access time 10x more bandwidth 4,000x lower media price DRAM/DISK 100:1 to 10:10 to 50:1
Disk Performance vs Time
1
10
100
1980 1990 2000
Year
seek
s p
er s
eco
nd
ban
dw
idth
: MB
/s
0.1
1.
10.
Cap
acity
(GB
)
Disk accesses/second vs Time
1
10
100
1980 1990 2000
Year
Acc
esse
s p
er S
eco
nd
Storage Price vs TimeMegabytes per kilo-dollar
0.1
1.
10.
100.
1,000.
10,000.
1980 1990 2000
Year
MB
/k$
![Page 54: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/54.jpg)
59
Performance = Storage AccessesPerformance = Storage Accesses not Instructions Executed not Instructions Executed
In the “old days” we counted instructions and IO’s Now we count memory references Processors wait most of the time
Where the time goes: clock ticks used by AlphaSort Components
SortDisc Wait SortDisc WaitDisc Wait OS
Memory WaitMemory Wait
D-Cache Miss
I-Cache MissB-CacheB-Cache
Data MissData Miss
70 MIPS“real” apps have worse Icache misses so run at 60 MIPSif well tuned, 20 MIPS if not
![Page 55: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/55.jpg)
60
Storage Latency: Storage Latency: How Far Away is the Data?How Far Away is the Data?
RegistersOn Chip CacheOn Board Cache
Memory
Disk
12
10
100
Tape /Optical Robot
10 9
106
Sacramento
This CampusThis Room
My Head
10 min
1.5 hr
2 Years
1 min
Pluto
2,000 YearsAndromdeda
Clo
ck T
icks
![Page 56: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/56.jpg)
61
Tape Farms for Tertiary StorageTape Farms for Tertiary StorageNot Mainframe SilosNot Mainframe Silos
Tape Farms for Tertiary StorageTape Farms for Tertiary StorageNot Mainframe SilosNot Mainframe Silos
Scan in 27 hours.many independent tape robots(like a disc farm)
10K$ robot 14 tapes500 GB 5 MB/s 20$/GB 30 Maps
100 robots
50TB 50$/GB 3K Maps
27 hr Scan
1M$
![Page 57: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/57.jpg)
62
0.01
0.1
1
10
100
1,000
10,000
100,000
1,000,000
1000 x Disc Farm STC Tape Robot 6,000 tapes, 8 readers
100x DLT Tape Farm
GB/K$
Maps
SCANS/Day
Kaps
The Metrics: The Metrics: Disk and Tape Farms Win Disk and Tape Farms Win
The Metrics: The Metrics: Disk and Tape Farms Win Disk and Tape Farms Win
Data Motel:Data checks in, but it never checks out
![Page 58: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/58.jpg)
63
Tape & Optical: Tape & Optical: Beware of the Beware of the Media MythMedia Myth
Tape & Optical: Tape & Optical: Beware of the Beware of the Media MythMedia Myth
Optical is cheap: 200 $/platter 2 GB/platter => 100$/GB (2x cheaper than disc)
Tape is cheap: 30 $/tape 20 GB/tape => 1.5 $/GB (100x cheaper than disc).
![Page 59: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/59.jpg)
64
Tape & Optical Tape & Optical RealityReality: : Media is 10% of System CostMedia is 10% of System Cost
Tape & Optical Tape & Optical RealityReality: : Media is 10% of System CostMedia is 10% of System CostTape needs a robot (10 k$ ... 3 m$ ) 10 ... 1000 tapes (at 20GB each) => 20$/GB ... 200$/GB
(1x…10x cheaper than disc)
Optical needs a robot (100 k$ ) 100 platters = 200GB ( TODAY ) => 400 $/GB
( more expensive than mag disc ) Robots have poor access times Not good for Library of Congress (25TB) Data motel: data checks in but it never checks out!
![Page 60: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/60.jpg)
65
The Access Time MythThe Access Time MythThe Access Time MythThe Access Time MythThe Myth: seek or pick time dominatesThe reality: (1) Queuing dominates (2) Transfer dominates BLOBs (3) Disk seeks often shortImplication: many cheap servers
better than one fast expensive server– shorter queues– parallel transfer– lower cost/access and cost/byte
This is now obvious for disk arraysThis will be obvious for tape arrays
Seek
Rotate
Transfer
Seek
Rotate
Transfer
Wait
![Page 61: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/61.jpg)
66
Billions Of Clients Billions Of Clients
Every device will be “intelligent” Doors, rooms, cars… Computing will be ubiquitous
![Page 62: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/62.jpg)
67
Billions Of ClientsBillions Of ClientsNeed Millions Of ServersNeed Millions Of Servers
MobileMobileclientsclients
FixedFixedclients clients
ServerServer
SuperSuperserverserver
ClientsClients
ServersServers
All clients networked All clients networked to serversto servers May be nomadicMay be nomadic
or on-demandor on-demand Fast clients wantFast clients wantfasterfaster servers servers
Servers provide Servers provide Shared DataShared Data ControlControl CoordinationCoordination CommunicationCommunication
![Page 63: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/63.jpg)
68
1987: 256 tps Benchmark 1987: 256 tps Benchmark 14 M$ computer (Tandem) A dozen people False floor, 2 rooms of machines
Simulate 25,600 clients
A 32 node processor array
A 40 GB disk array (80 drives)
OS expert
Network expert
DB expert
Performance expert
Hardware experts
Admin expert
Auditor
Manager
![Page 64: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/64.jpg)
69
1988: DB2 + CICS Mainframe1988: DB2 + CICS Mainframe65 tps65 tps
IBM 4391 Simulated network of 800 clients 2m$ computer Staff of 6 to do benchmark
2 x 3725 network controllers
16 GB disk farm4 x 8 x .5GB
Refrigerator-sizedCPU
![Page 65: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/65.jpg)
70
1997: 10 years later1997: 10 years later1 Person and 1 box = 1250 tps1 Person and 1 box = 1250 tps
1 Breadbox ~ 5x 1987 machine room 23 GB is hand-held One person does all the work Cost/tps is 1,000x less
25 micro dollars per transaction4x200 Mhz cpu1/2 GB DRAM12 x 4GB disk
Hardware expertOS expertNet expertDB expertApp expert
3 x7 x 4GB disk arrays
![Page 66: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/66.jpg)
71
What Happened?What Happened? Moore’s law:
Things get 4x better every 3 years (applies to computers, storage, and networks)
New Economics: Commodityclass price/mips software $/mips k$/yearmainframe 10,000 100 minicomputer 100 10microcomputer 10 1
GUI: Human - computer tradeoffoptimize for people, not computers
mainframeminimicro
time
pric
e
![Page 67: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/67.jpg)
72
What Happens NextWhat Happens Next
Last 10 years: 1000x improvement
Next 10 years: ????
Today: text and image servers are free
25 $/hit => advertising pays for them Future:
video, audio, … servers are free“You ain’t seen nothing yet!”
1985 20051995
perf
orm
ance
![Page 68: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/68.jpg)
73
Smart Cards Smart Cards Smart Cards Smart Cards
Bull CP8 two chip card first public demonstration 1979
Then (1979)
EMV card with dynamic authentication(EMV=Europay, MasterCard, Visa standard)
door key, vending machines, photocopiers
Now (1997)
Courtesy of Dennis Roberson NCR.
![Page 69: 1 What Happens When Processing Storage Bandwidth are Free and Infinite? Jim Gray Microsoft Research](https://reader033.vdocuments.mx/reader033/viewer/2022061306/55147649550346b2598b4582/html5/thumbnails/69.jpg)
74
Smart Card Smart Card Memory CapacityMemory Capacity
ApplicationsApplications
Cards will be able to storedata (e.g. medical)books, movies,…money
Source: PIN/Card -Tech/ Courtesy of Dennis Roberson NCR
1990 1992 1996 1998 2000 2002
Mem
ory
Siz
e (B
its) 300 M
1 M
3 K
10 K
You are here
2004
16 KB todaybut growing super-exponentially