1 put everything in future (disk) controllers (it’s not “if”, it’s “when?”) jim gray...
TRANSCRIPT
1
Put Everything in Future (Disk) Controllers(it’s not “if”, it’s “when?”)
Jim Gray
http://www.research.Microsoft.com/~Gray
Acknowledgements:
Dave Patterson explained this to me a year ago
Kim Keeton
Erik Riedel
Catharine Van Ingen
Helped me sharpen these arguments
2
Remember Your Roots
3
Technology Drivers: Disks• Disks on track
• 100x in 10 years 2 TB 3.5” drive
• Shrink to 1” is 200GB
• Disk replaces tape?
• Disk is super computer!
Kilo
Mega
Giga
Tera
Peta
Exa
Zetta
Yotta
4
Data Gravity Processing Moves to Transducers
• Move Processing to data sources
• Move to where the power (and sheet metal) is
• Processor in
»Modem
»Display
»Microphones (speech recognition) & cameras (vision)
»Storage: Data storage and analysis
5
It’s Already True of PrintersPeripheral = CyberBrick
• You buy a printer
• You get a
»several network interfaces
»A Postscript engine • cpu, • memory, • software,• a spooler (soon)
»and… a print engine.
6
Tera Byte Backplane
• TODAY» Disk controller is 10 mips risc engine
with 2MB DRAM
» NIC is similar power
• SOON» Will become 100 mips systems
with 100 MB DRAM.
• They are nodes in a federation(can run Oracle on NT in disk controller).
• Advantages» Uniform programming model» Great tools» Security» economics (CyberBricks)» Move computation to data (minimize traffic)
All Device Controllers will be Cray 1’s
CentralProcessor &
Memory
7
Basic Argument for x-Disks• Future disk controller is a super-computer.
»1 bips processor»128 MB dram»100 GB disk plus one arm
• Connects to SAN via high-level protocols» RPC, HTTP, DCOM, Kerberos, Directory Services,…. »Commands are RPCs»management, security,….»Services file/web/db/… requests» Managed by general-purpose OS with good dev environment
• Move apps to disk to save data movement
»need programming environment in controller
8
The Slippery Slope
• If you add function to server
•Then you add more function to server
•Function gravitates to data.
Nothing = Sector Server
Everything = App Server
Something =
Fixed App Server
9
Why Not a Sector Server?(let’s get physical!)
• Good idea, that’s what we have today.
• But
»cache added for performance
»Sector remap added for fault tolerance
»error reporting and diagnostics added
»SCSI commends (reserve,.. are growing)
»Sharing problematic (space mgmt, security,…)
• Slipping down the slope to a 2-D block server
10
Why Not a 1-D Block Server?Put A LITTLE on the Disk Server• Tried and true design
»HSC - VAX cluster»EMC»IBM Sysplex (3980?)
• But look inside»Has a cache »Has space management»Has error reporting & management»Has RAID 0, 1, 2, 3, 4, 5, 10, 50,…»Has locking»Has remote replication»Has an OS»Security is problematic»Low-level interface moves too many bytes
11
Why Not a 2-D Block Server?Put A LITTLE on the Disk Server• Tried and true design
»Cedar -> NFS»file server, cache, space,..»Open file is many fewer msgs
• Grows to have»Directories + Naming»Authentication + access control»RAID 0, 1, 2, 3, 4, 5, 10, 50,…»Locking»Backup/restore/admin»Cooperative caching with client
• File Servers are a BIG hit: NetWare™»SNAP! is my favorite today
12
Why Not a File Server?Put a Little on the Disk Server
• Tried and true design
»Auspex, NetApp, ...
» Netware
• Yes, but look at NetWare
»File interface gives you app invocation interface
»Became an app server• Mail, DB, Web,….
»Netware had a primitive OS• Hard to program, so optimized wrong thing
13
Why Not Everything?
Allow Everything on Disk Server(thin client’s)
• Tried and true design
»Mainframes, Minis, ...
»Web servers,…
»Encapsulates data
»Minimizes data moves
»Scaleable
• It is where everyone ends up.
• All the arguments against are short-term.
14
The Slippery Slope
• If you add function to server
•Then you add more function to server
•Function gravitates to data.
Nothing = Sector Server
Everything = App Server
Something =
Fixed App Server
15
Disk = Node• has magnetic storage (100 GB?)
• has processor & DRAM
• has SAN attachment
• has execution environment
OS KernelSAN driver Disk driver
File System RPC, ...Services DBMS
Applications
16
Technology Drivers: System on a Chip
• Integrate Processing with memory on one chip»chip is 75% memory now»1MB cache >> 1960 supercomputers»256 Mb memory chip is 32 MB!»IRAM, CRAM, PIM,… projects abound
• Integrate Networking with processing on one chip»system bus is a kind of network»ATM, FiberChannel, Ethernet,.. Logic on chip.»Direct IO (no intermediate bus)
• Functionally specialized cards shrink to a chip.
17
Technology Drivers: What if Networking Was as Cheap As Disk IO?
• TCP/IP
»Unix/NT 100% cpu @ 40MBps
• Disk
»Unix/NT 8% cpu @ 40MBps
Why the Difference?Host Bus Adapter does
SCSI packetizing, checksum,…flow controlDMA
Host doesTCP/IP packetizing, checksum,…flow controlsmall buffers
18
Technology Drivers: The Promise of SAN/VIA:10x in 2 years
http://www.viarch.org/
• Today: »wires are 10 MBps (100 Mbps Ethernet)
»~20 MBps tcp/ip saturates 2 cpus
»round-trip latency is ~300 us
• In the lab»Wires are 10x faster Myrinet, Gbps Ethernet, ServerNet,…
» Fast user-level communication• tcp/ip ~ 100 MBps 10% of each processor
• round-trip latency is 15 us
19
Gbps Ethernet: 110 MBps
SAN: Standard Interconnect
PCI: 70 MBps
UW Scsi: 40 MBps
FW scsi: 20 MBps
scsi: 5 MBps
• LAN faster than memory bus?
• 1 GBps links in lab.
• 100$ port cost soon
• Port is computer
RIPFDDI
RIPATM
RIPSCI
RIPSCSI
RIPFC
RIP?
20
Technology Drivers:100 GBps Ethernet replaces SCSI
• Why I love SCSI
»Its fast (40MBps)
»The protocol uses little processor power
• Why I hate SCSI
»Wires must be short
»Cables are pricy
»pins bend
21
Functionally Specialized Cards• Storage
• Network
• Display
M MB DRAM
P mips processor
ASIC
ASIC
ASIC
Today:
P=50 mips
M= 2 MB
In a few years
P= 200 mips
M= 64 MB
22
Technology DriversPlug & Play Software
• RPC is standardizing: (DCOM, IIOP, HTTP)
»Gives huge TOOL LEVERAGE»Solves the hard problems for you:
• naming, • security, • directory service, • operations,...
• Commoditized programming environments » FreeBSD, Linix, Solaris,…+ tools» NetWare + tools» WinCE, WinNT,…+ tools» JavaOS + tools
• Apps gravitate to data.
• General purpose OS on dedicated ctlr can run apps.
23
Basic Argument for x-Disks• Future disk controller is a super-computer.
»1 bips processor»128 MB dram»100 GB disk plus one arm
• Connects to SAN via high-level protocols» RPC, HTTP, DCOM, Kerberos, Directory Services,…. »Commands are RPCs»management, security,….»Services file/web/db/… requests» Managed by general-purpose OS with good dev environment
• Move apps to disk to save data movement
»need programming environment in controller