“storage is cheap” and other lies lance stuchell, university of michigan library curating and...
TRANSCRIPT
““Storage is Cheap” and Other LiesStorage is Cheap” and Other Lies
Lance Stuchell, University of Michigan LibraryCurating and Managing Research Data for Re-UseICPSR, July 31, 2013
“Hard Drives” by Michael Muni CC-BY-NC-ND
Overview
• Instability of storage hardware costs • Check out David Rosenthal’s blog at
http://blog.dshr.org/
• Costs of storing digitized moving images
• Effects of storage costs on digital preservation, formats, etc.
“Storage is Cheap”
• Perception is based on “Kryder's Law”• A 2005 Scientific American article about Mark
Kryder, Seagate's Senior VP of Research • Magnetic disk density increases quickly• Disk density closely tied to pricing • 30-year history of disk prices dropping about
40% per year
• Disk costs were affordable & predictable• 30% of total storage costs
The Party's Over
• Mid 2011: Kryder’s Law slows• Latest projections ~ 20% density growth
• Late 2011: Flooding in Taiwan causes shortfall of 70 million disk drives• Prices remain over 50% higher• Not expected to return to pre-flood levels
until 2014
• Changes in technology David S. H. Rosenthal, “Storage Will Be A Lot Less Free Than It Used To Be,” 2012.http://blog.dshr.org/2012/10/storage-will-be-lot-less-free-than-it.html
“Optimistically, for the rest of this decade the rapid decrease in cost per bit of storage that
has been a constant of the last three decades will be much slower; it might even stop.”
David S. H. Rosenthal, et al., “The Economics of Long-Term Digital Storage” 2012.http://www.lockss.org/locksswp/wp-content/uploads/2012/09/unesco2012.pdf
A/V Digitization at MLibrary
Digitization of Special Collections material
Audio:• Uncompressed BWF master files• 1 original audio object ≈ 5 GB storage
Moving Image:• Uncompressed preservation master• Compressed production master• 1 original video object ≈ 40.4 GB Storage
A/V Storage Estimates
A/V Storage EstimatesTB/Year
Initial Costs Estimates
Value Storage: $250/TB per year
Initial Costs Estimates
Value Storage:Tape Backups:
$250/TB per year$1,825/TB per year
Initial Costs Estimates
Value Storage:Tape Backups:
$250/TB per year$1,825/TB per year
2018: $245,3252023: $485,200
2013: $32,450
Reaction
Current Costs Estimates
Value Storage:Tape Backups
(MLib):
$250/TB per yearEquipment costsCheaper most years
Current Cost Estimates
Current Cost Estimates
Current Cost Estimates
Total 11 Year Storage Cost
$569,368
Current Cost Estimates
Current Cost Estimates
Looking Ahead
• Assuming steady pricing• Instability in per-bit storage
• Will not fall under 20% growth until 2018
• ITS storage costs may not fall at same rate
• Long-term costs of cloud are not known• Amazon costs haven't decreased at HD rate1
• Tape storage for preservation copies?
• Economies of scale• Digital Preservation Network
1Rebecca Pool, “Is cloud storage the answer to preservation?” Research Information, 2/14/2013
Ramifications
• Preservation• Some moving image material is not
retained in uncompressed formats• Still image formats are balance of
preservation and size
• Appraisal and re-appraisal • What are we keeping and for how long?
• Costs of starting up
Storage Takeaways
• HD costs are no longer predictable • Budgets often neglect storage
• Ignoring significant and ongoing costs
• Storage costs are community wide problem• Question of scale• Best practice may not be possible
• Backups can cost more than primary storage
• Archival storage ain’t cheap!
Thanks!!