introduction to solid state drives
DESCRIPTION
These are the slides from a tutorial I presented at LOPSA-East in 2013. It covers spinning media and and solid state drives in detail. A video of the presentation can be found on YouTube: http://www.youtube.com/watch?v=G3wf1HMr6b0TRANSCRIPT
![Page 1: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/1.jpg)
Introduction to Solid State Drive
Technology
Saturday, November 16, 13
![Page 2: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/2.jpg)
Class Overview
• The Evolution of Storage Technology
• Spinning Disks
• Storage Metrics
• Solid State Technology
Saturday, November 16, 13
better understanding of spinning disks, understand high & low level flash, issues with SSDs
![Page 3: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/3.jpg)
The Evolution of Storage Technology
Saturday, November 16, 13
![Page 4: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/4.jpg)
Pre-History
Density/Time Speed/Time
Saturday, November 16, 13
![Page 5: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/5.jpg)
Spinning Disks
Saturday, November 16, 13
![Page 6: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/6.jpg)
The Parts of a Hard Drive
Saturday, November 16, 13
![Page 7: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/7.jpg)
PlattersThe Parts of a Hard Drive
Saturday, November 16, 13
![Page 8: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/8.jpg)
Actuator Arms and Heads
The Parts of a Hard Drive
Saturday, November 16, 13
![Page 9: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/9.jpg)
Controllerand Interface
The Parts of a Hard Drive
Saturday, November 16, 13
![Page 10: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/10.jpg)
Voltron Force Assemble!
Saturday, November 16, 13
![Page 11: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/11.jpg)
Disk Interface
• Removable (USB/CF)
• SATA1 / II / III
• Nearline SAS
• SAS
• Fibre Channel
• PCI-e
Saturday, November 16, 13
![Page 12: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/12.jpg)
Spinning Disk Removable Media
• Advantages:
• Nigh Universal
• Disadvantages:
• Slower
• Fragile
• Easily lost.
• Abstraction Layer
Disk Interface
Saturday, November 16, 13
USB
![Page 13: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/13.jpg)
SATA I / II / III
• Speeds: 1.5 / 3 / 6Gb/s
• Requires AHCI for things like NCQ
• Subset of SAS
• Shares IDE command set
Disk Interface
Saturday, November 16, 13
AHCI - Advanced Host Controller Interface (IDE is ok for TRIM)NCQ on SSD ensure SSD has things to do while host is latent(Intel can queue 32 requests) - logo from SATA-IO (intnl org)
![Page 14: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/14.jpg)
SATA 3.1
• Approved July 2011
• Universal Storage Module
• mSATA
• QTRIM
Disk Interface
(This time, it’s personal)
Saturday, November 16, 13
QTRIM - queued TRIM commands, USM is a mobile drive standard
![Page 15: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/15.jpg)
SAS / Nearline SAS
• SAS
• Enhanced CRC checking
• 512/520/528 bit blocks
• Low density, high reliability
• Nearline SAS
• ...not so much
Saturday, November 16, 13
Serially Attached SCSI - 16 bits of CRC
![Page 16: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/16.jpg)
Disk Geometry
Saturday, November 16, 13
![Page 17: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/17.jpg)
PlattersDisk
Geometry
Saturday, November 16, 13
![Page 18: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/18.jpg)
TracksDisk
Geometry
Saturday, November 16, 13
![Page 19: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/19.jpg)
Cylinders Disk
Geometry
Saturday, November 16, 13
![Page 20: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/20.jpg)
Cylinders Disk
Geometry
Saturday, November 16, 13
![Page 21: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/21.jpg)
SectorsDisk
Geometry
Saturday, November 16, 13
![Page 22: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/22.jpg)
Logical Block Addressing
• First introduced as an abstraction layer
• Replaced CHS addressing
• Address Space is Linear (block 0 - n)
• Size of address space depends on the standard at time of manufacture.
DiskGeometry
Saturday, November 16, 13
Currently at 48-bit LBA -
![Page 23: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/23.jpg)
Variables Affecting Spinning Disk IO Rate
Saturday, November 16, 13
![Page 24: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/24.jpg)
Platter Rotational Speed
Saturday, November 16, 13
![Page 25: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/25.jpg)
Seek Speed
Saturday, November 16, 13
![Page 26: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/26.jpg)
Data Density
Saturday, November 16, 13
![Page 27: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/27.jpg)
Controller CacheVariables Affecting Spinning
Disk Speed
Saturday, November 16, 13
Size / battery backed / hybrid drives
![Page 28: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/28.jpg)
Spinning Disk Damage Vectors
Saturday, November 16, 13
![Page 29: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/29.jpg)
Movement
• Movement vertical or parallel to platter
• Measured in G forces
• Head Crashes
• Spinning Down
• Head uses “Landing Strip”
• Repeated platter contact causes damage to the read/write head
Spinning Disk Damage Vectors
Saturday, November 16, 13
Used to manually park the heads | Putting your computer to sleep can cause the head to park | nanocoating on the bumpy landing strip
![Page 30: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/30.jpg)
Protection against movement
• “Active Drive Protection”: Free-fall sensor
• Has a lift arm to lift the head away from the platter
• Some protection systems are in the drive, some are in the controller
• Don’t mix the two
Saturday, November 16, 13
Apple: Sudden motion sensor, Lenovo/IBM: Hard Drive Active Protection System
Next slide: “You know, vibrations are movement...”
![Page 31: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/31.jpg)
You know, vibrations are movement...
Saturday, November 16, 13
![Page 32: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/32.jpg)
Yes, vibrations are important, too
Saturday, November 16, 13
![Page 33: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/33.jpg)
Shelf Life
• Oil / Lubricants in bearings
• Temperature fluctuations
• Magnetic “events” (bit rot)
• Outgassing / vapor removal
Spinning Disk Damage Vectors
Saturday, November 16, 13
Long-term “archival quality” drives with long-life lubricant Long term “cold storage” arrays which periodically spin up drives every few weeks to clean & scrub the data
![Page 34: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/34.jpg)
Spinning Disks in RAID
• Redundant Array of Inexpensive Disks
• Common RAID levels:
• 0,1,5,6,10
• Software / Hardware
Saturday, November 16, 13
![Page 35: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/35.jpg)
Important Considerations
• Redundancy
• Capacity
• Speed
• Robust
SpinningDisks
In RAID
Saturday, November 16, 13
Speed: Dedicated hardware? Single point of failure? Parity Calculation? How long to rebuild a drive? NUMBER OF SPINDLES!! Redundancy: How many drive failures? URE errors? Capacity: Parity stripe or mirrors? (harder better faster stronger)
![Page 36: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/36.jpg)
Advantages
• Linear Speed
• Price (Per Gigabyte)
• Well-Understood
SpinningDisks
In RAID
Saturday, November 16, 13
![Page 37: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/37.jpg)
Disadvantages
• Random Speed
• Price (per IOPS)
• Failure Rate
• Rebuild Speed
SpinningDisks
In RAID
Saturday, November 16, 13
![Page 38: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/38.jpg)
Storage Metrics
Saturday, November 16, 13
![Page 39: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/39.jpg)
IOPS
• What are they?
• What aren’t they?
Saturday, November 16, 13
![Page 40: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/40.jpg)
The Simplified Equation
IOPS = 1/(((R+W)/2)/1000) + (L/1000)
RWL
= Average Read Time= Average Write Time= Average Latency
Saturday, November 16, 13
![Page 41: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/41.jpg)
Rule of Thumb Assumptions
RPM IOPS5400 50-807200 80-10010k 130-15015k 180-200
Saturday, November 16, 13
![Page 42: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/42.jpg)
Determining IOPS
• Per Drive
• Manufacturer’s Stated Numbers
• Rule of Thumb
• Per RAID Array
• Write penalty
Saturday, November 16, 13
![Page 43: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/43.jpg)
IO Profiling
• Active Tools
• Bonnie++
• dd
• Intel NAS Toolkit
• Passive Tools
• io(stat/meter/top), atop
• Resource Monitor / Process Explorer
Saturday, November 16, 13
http://www.intel.com/products/server/storage/NAS_Perf_Toolkit.htm
![Page 44: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/44.jpg)
Solid State Drive Technology
Saturday, November 16, 13
![Page 45: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/45.jpg)
NOR Flash
• Reads and writes are atomic single-bit
• Expensive
• Small specific use cases
Saturday, November 16, 13
Won’t talk about NOR much.
![Page 46: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/46.jpg)
NAND Flash
• Reads are based on “read blocks” (4k)
• Writes are based on “erasure blocks”
• Cheap (and getting cheaper)
• Broad use cases
Saturday, November 16, 13
![Page 47: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/47.jpg)
Read / Write Profiles
• Logical addresses abstracted from LBA
• No seek time
• Reads are generally very fast
• Writes are typically slower
Saturday, November 16, 13
Random and Linear IO have identical access timeNext slide: The magic
![Page 48: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/48.jpg)
The Magic
InsulatingBarrier
Pure Silicon
Doped silicon capable of holding an electrical charge
Saturday, November 16, 13
Barrier is a dielectric film (silicon oxide)
![Page 49: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/49.jpg)
Quantum Tunneling
(transmission coefficient for a particle tunneling through a single potential barrier)
Saturday, November 16, 13
Hot Carrier InjectionStorage / Erase uses Fowler-Nordheim Tunnel Injection / Release
![Page 50: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/50.jpg)
Doped Silicon
Single Layer Cell (SLC)
Multi-Layer Cell (MLC)
Triple-Layer Cell (TLC)
Saturday, November 16, 13
Use charge pumps to get through the barrier Each charge level has a binary state - 1 or 0
![Page 51: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/51.jpg)
Gradual Destruction
Energy increaseswith cell layers
Multiple cells needmultiple writes
Barrier accumulateselectrons
Electrical potential difference of barrier and cells disappears
Saturday, November 16, 13
![Page 52: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/52.jpg)
Difficulty Going ForwardTLCTLC
000 100
001 101
010 110
011 111
SLC
0
1
MLC
00
01
10
11
4LC4LC4LC4LC
0000 0100 1000 1100
0001 0101 1001 1101
0010 0110 1010 1110
0011 0111 1011 1111
Saturday, November 16, 13
![Page 53: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/53.jpg)
Density
• 3-Dimensional
• Charge levels
• Size of cells
• “Dot Pitch” (Cells Per Inch)
• 5nm, 3nm, 2nm
• Varies with “level” count
Saturday, November 16, 13
![Page 54: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/54.jpg)
SLC / ESLC
• Low Density
• Single (bit) Level Cell
• Quick: 25µs Read / 200-300µ Write
• More robust & long wear time
• Write endurance near 100,000 cycles
Saturday, November 16, 13
Capacity expensive | only in 5nm / 3nm densities |
![Page 55: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/55.jpg)
MLC / EMLC
• Reasonably High Density
• Two (bit) Level Cell
• Decently fast: 50µs Read / 600-900µs Write
• Medium lifetime
• Write endurance near 3,000 cycles
Saturday, November 16, 13
![Page 56: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/56.jpg)
TLC
• Very High Density
• Three (bit) Level Cell
• Decently fast: 75µs Read / 900-1350µs Write
• Not very robust or durable :-(
• Write endurance ~ 1,000 cycles
Saturday, November 16, 13
![Page 57: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/57.jpg)
Write Amplification and Garbage Collection
Saturday, November 16, 13
![Page 58: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/58.jpg)
Block Sizes
• Read Block
• 4k (aka “page”)
• Erasure Block
• (Large) multiple of 4k
• aka “block” 256KB erasure
block size
Saturday, November 16, 13
e-ink parallel
![Page 59: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/59.jpg)
Write Amplification
Written Data
Empty Cell
Saturday, November 16, 13
next - want to change the data in the upper right quadrant
![Page 60: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/60.jpg)
Write Amplification
Written Data
Empty Cell
New Data
Old Data
Saturday, November 16, 13
next - big chunk of new data to write
![Page 61: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/61.jpg)
Write Amplification
Written Data
Empty Cell
Old Data
New Data
Saturday, November 16, 13
Where does this go? We’re out of empty erasure blocks!
![Page 62: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/62.jpg)
Write Amplification
Written Data
Empty Cell
New data writtenover old cell
w/o TRIM
Saturday, November 16, 13
![Page 63: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/63.jpg)
Write Amplification
Written Data
Empty Cell
New data writtenover old cell
w/ TRIM
Saturday, November 16, 13
![Page 64: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/64.jpg)
Garbage Collection
Saturday, November 16, 13
![Page 65: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/65.jpg)
Garbage Collection
Saturday, November 16, 13
![Page 66: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/66.jpg)
Garbage Collection
Saturday, November 16, 13
![Page 67: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/67.jpg)
Garbage Collection
Saturday, November 16, 13
![Page 68: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/68.jpg)
Garbage Collection
Saturday, November 16, 13
![Page 69: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/69.jpg)
IO Performance Profiles
Saturday, November 16, 13
![Page 70: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/70.jpg)
Remember:
• Spinning Disks
• Linear is fast
• Random is slow
• Read marginally faster than writes (sometimes)
Saturday, November 16, 13
writes slower when switching tracks
![Page 71: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/71.jpg)
With SSDs:
• Reads are fast
• Writes are slow(ish)
• Random or linear doesn’t matter (as much)
Saturday, November 16, 13
![Page 72: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/72.jpg)
SSD Performance Overview
• Depends on
• Number of flash chips in use
• Number of busses from the processor
• Performance of controller CPU
• Contention
• Bus speed
• Number of erasure blocks used
• Number of previous writes to flash cells
Saturday, November 16, 13
![Page 73: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/73.jpg)
• Chips
• IO Busses
• CPU Cores
Saturday, November 16, 13
![Page 74: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/74.jpg)
Causes of Contention
• Legitimate use
• Garbage collection
• Legitimate (but latent) useage
• IO Blender!
(Bender Blender: http://bit.ly/10vc7Sf)Saturday, November 16, 13
Latent: updatedb? atime? app-level garbage collection? (t-shirt at threadless)
![Page 75: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/75.jpg)
Bus Speed
• SATA - 3 or 6 Gb/s?
• IOPS Calc
• Can your controller handle your disks?
Saturday, November 16, 13
![Page 76: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/76.jpg)
Read
• Very fast
• No seek time
• moderately improved over spinning disk (linear - random greatly)
• Causes no damage to the media
• Generally scales up with capacity
Saturday, November 16, 13
![Page 77: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/77.jpg)
Write
• Usually fast (depending on drive usage)
• No seek time
• highly improved over spinning disk
• Causes no damage to the media
• Generally scales up with capacity
Saturday, November 16, 13
![Page 78: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/78.jpg)
Spinning DiskRead/Write Matrix
Read Write
Linear
Random
Saturday, November 16, 13
![Page 79: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/79.jpg)
SSDRead/Write Matrix
Read Write
Linear
Random
Saturday, November 16, 13
![Page 80: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/80.jpg)
Solid State in Practice
Saturday, November 16, 13
![Page 81: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/81.jpg)
Solid State Form Factors
Saturday, November 16, 13
![Page 82: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/82.jpg)
Removable Media
Saturday, November 16, 13
![Page 83: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/83.jpg)
Drives
Saturday, November 16, 13
![Page 84: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/84.jpg)
PCI Cards
Saturday, November 16, 13
Next slide: parts of an SSD
![Page 85: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/85.jpg)
Parts of an SSD
Saturday, November 16, 13
![Page 86: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/86.jpg)
Interface
USB
PCI
SATA/SAS
Saturday, November 16, 13
IDE (sadly?)
![Page 87: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/87.jpg)
ControllerMain
Processor
I/O Bus Lanes
RAMCache
Battery / SuperCapacitor
Saturday, November 16, 13
![Page 88: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/88.jpg)
Flash Chips
Saturday, November 16, 13
If individual chip capacity is finite, how do bigger drives increase capacity? What does this mean for performance?
![Page 89: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/89.jpg)
Flash Controllers
• Flash Translation Layer (FTL)
• Stripe Writes
• Interpret bus instructions
• Wear Leveling
• Garbage Collection
Saturday, November 16, 13
Do the heavy lifting - single largest problem with flash drives, without a doubt.
![Page 90: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/90.jpg)
Flash Translation LayerLBA (0...n blocks)
F L A S H C H I P S
Saturday, November 16, 13
![Page 91: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/91.jpg)
SSD Aspects & Concerns
Saturday, November 16, 13
![Page 92: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/92.jpg)
Longevity
• Primarily determined by the class of flash
• (e)SLC, (e)MLC, TLC
• Related to wear-leveling
• Under-reported capacity
• Short-stroking improves lifetime (not speed)
Saturday, November 16, 13
![Page 93: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/93.jpg)
Partition Alignment
• Performance and longevity
• As big (or bigger) issue than it was in spinning disks
• Native 4k read blocks
• Far larger erasure blocks
• larger than is practical for alignment
Saturday, November 16, 13
![Page 94: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/94.jpg)
TRIM
• As a command, refers to ATA-8 spec
• SCSI equivalent is UNMAP, but both are often referred to as TRIM.
• Does not immediately delete unused blocks
• Allows for GC
Saturday, November 16, 13
Linux calls this “discard” - TRIM refers
![Page 95: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/95.jpg)
Linux TRIM Support
• EXT4 / XFS / JFS / BTRFS - Native using ‘discard’ option
• Consider NOOP or Deadline IO scheduler
• fstrim (part of util-linux) for R/W vols
• zerofree for R/O vols
Saturday, November 16, 13
fstrim & zerofree - userland - important for thin-provisioned volumes on SAN arrays which support it. Check docs on schedulers for details - deadline prefers read queues (under /sys/block)
![Page 96: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/96.jpg)
OSX Trim Support
• Comes by default on factory-installed SSDs
• Trim-Enabler
• http://www.groths.org/trim-enabler/
Saturday, November 16, 13
![Page 97: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/97.jpg)
ZFS and SSDs
• ZFS Intent Log (ZIL)
• Adaptive Replacement Cache (ARC)
• arc_summary can help you decide
Saturday, November 16, 13
ZIL is almost like a journal - ARC is a RAM cache that has disk backing it. SSDs can be L2ARC - https://code.google.com/p/jhell/wiki/arc_summary
![Page 98: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/98.jpg)
Filesystems in General
• Standard journaling filesystems
• Mount options (atime/relatime, etc), /tmp->tmpfs
• Next-Gen
• ZFS / BTRFS
• Distributed Filesystems
• DRBD
Saturday, November 16, 13
ZFS - SSD cache pool | ZFS/BTRFS are COW | DRBD no trim
![Page 99: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/99.jpg)
Monitor Health w/ S.M.A.R.T.
• S.M.A.R.T. information
• vendor-specific
• Includes flash erase count
• smartctl on Linux and Mac
• Dozens of tools on Windows (check wiki)
Saturday, November 16, 13
![Page 100: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/100.jpg)
Forensics
(http://bit.ly/fast11-wei-paper)
...Our results lead to three conclusions:
First, built-in commands are effective, but manufacturers sometimes implement them incorrectly.
Second, overwriting the entire visible address space of an SSD twice is usually, but not always, sufficient to sanitize the drive.
Third, none of the existing hard drive-orientedtechniques for individual file sanitization are effective on SSDs
Reliably Erasing Data From Flash-Based Solid State DrivesMichael Wei∗, Laura M. Grupp∗, Frederick E. Spada†, Steven Swanson∗
∗Department of Computer Science and Engineering, University of California, San Diego†Center for Magnetic Recording and Research, University of California, San Diego
Saturday, November 16, 13
![Page 101: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/101.jpg)
SSD-enhanced RAID Array Considerations
Saturday, November 16, 13
![Page 102: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/102.jpg)
Hardware / Software
• Dedicated CPU Power
• Battery-backed storage
Hardware RAID Controllers
• Trust (eyes on code)
• Excessive cost of HW
Software RAID Controllers
• Commercial Support
• Proprietary Tech
• Portability
• Spare CPU Cycles
Saturday, November 16, 13
single point of failure
![Page 103: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/103.jpg)
TRIM / GC?
• Does the RAID software/device know enough to pass along TRIM?
• Will the array eventually crawl because of ongoing GC issues?
Saturday, November 16, 13
No software RAID that I know of supports it. Intel chipset for RAID0 with TRIM
![Page 104: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/104.jpg)
Access Bandwidth
• How much data can a single drive transmit?
• How many drives are in the array?
• What is the aggregate bus speed to the array controller?
• What is the bus speed to the host(s)?
Saturday, November 16, 13
![Page 105: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/105.jpg)
SSD Throughput Example
From Tech Radar: http://bit.ly/100UhvY
4.15Gb/s
Saturday, November 16, 13
![Page 106: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/106.jpg)
Controller / Bus
• Speed / Ports
• How mature / reliable / tested?
RememberMe?
Saturday, November 16, 13
Just because buses exist in a storage array oesn’t make them magic and infinite in size
![Page 107: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/107.jpg)
Tiering / Caching
Very slow, cheap disks
Faster spinning disks
SSD tier - hot blocks
Very fast SDRAM
Saturday, November 16, 13
![Page 108: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/108.jpg)
Future Technology
Saturday, November 16, 13
![Page 109: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/109.jpg)
Enhanced Capacity
Saturday, November 16, 13
Kowloon Walled City
![Page 110: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/110.jpg)
Enhanced Longevity
Saturday, November 16, 13
Telomeres in chromosomes
![Page 111: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/111.jpg)
Smart SSDs
Saturday, November 16, 13
Active Flash
![Page 112: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/112.jpg)
Saturday, November 16, 13
![Page 113: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/113.jpg)
What should I buy?
Saturday, November 16, 13
![Page 114: Introduction to Solid State Drives](https://reader034.vdocuments.mx/reader034/viewer/2022050722/5481166ab37959582b8b5d04/html5/thumbnails/114.jpg)
Questions?
Saturday, November 16, 13