cni mc donough_preservation
TRANSCRIPT
![Page 1: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/1.jpg)
Preservation-Worthy Digital Video: Hope You Brought Your
Checkbook
Jerome McDonoughNew York University
April 12, 2023
![Page 2: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/2.jpg)
Digital Video Basics
A video signal consists of luminance and chrominance informationLuminance – brightness, varying from white to black (abbreviated as Y)Chrominance – color (hue & saturation), conveyed as a pair of color difference signals: R-Y (hue & saturation for red, without luminance) B-Y (hue & saturation for blue, without
luminance)
![Page 3: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/3.jpg)
Digital Video Basics
Where’s the green?Spectral response of the human eye peaks in the green frequencies. The perceived brightness of an item can be constructed using weighted values for its red, green and blue components: Y = 0.299R + 0.587G +0.114B, or
G Y R B 0 3 0 110 5 9. .
.
![Page 4: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/4.jpg)
Digital Video Basics
4:2:2 sampling 4:2:0 sampling
4:1:1 sampling
4:2:2 – High End DV (DigitalBetacam, DVCPro50)
4:2:0 – MPEG 1 & 24:1:1 – DV and DVCAM
![Page 5: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/5.jpg)
Digital Video Basics
Why not 4:4:4 sampling? 720 x 486 resolution = 349,920 pixels per frame 349,920 pixels x 10 bits/sample x 3 samples/pixel =
10,497,600 bits per frame 10,497,600 bits/frame X 29.97 frames/second =
314,613,072 bits per second 314,613,072 bps x 3600 seconds = ~141.58 GB/hour For 1920x1080 HDTV, more like 840 GB/hour
4:2:2 sampling drops that rate by a third with almost no perceptible difference in quality. 4:2:0 and 4:1:1 drop it in half.
![Page 6: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/6.jpg)
Digital Video Basics
MPEG 2 Compression Further subsampling
down sample to 8 bits/sample down sample to 4:2:0 sampling regime
Discrete Cosine Transformation + Requantizing of coefficients from DCT
Variable Length Encoding & Run Length Encoding Interframe compression (motion compensation)
all of which can take a 209 mbps video rate (for 4:2:2 video) and reduce it to around 8 mbps with no apparent visual loss.
![Page 7: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/7.jpg)
Digital Video Basics
Raw digital video is extremely storage and bandwidth intensive.As a result, almost all digital video processing systems employ a mix of lossless and lossy compression mechanisms.
![Page 8: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/8.jpg)
Preservation-Worthy Digital Video
Desired characteristics for digital video we feel we can preserve include: Content can be migrated to new formats
and new media without introducing artifacts Stored in non-proprietary, standard format
which is openly documented Easy to produce derivatives for end-user
distribution Minimize costs of production, distribution &
migration
![Page 9: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/9.jpg)
Sampling, Migration & Artifacts
As in still image digitization, employing lossy compression can lead to artifacting when you migrate.Unlike still images, lossy compression is assumed in almost all video processing technology today.
![Page 10: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/10.jpg)
One River Media Codec Test Image
Images courtesy of Marco Solorio, One River Media
![Page 11: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/11.jpg)
Black Magic 8-Bit 4:2:2 Uncompressed Codec
Images courtesy of Marco Solorio, One River Media
![Page 12: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/12.jpg)
Black Magic 8-Bit: 10th Generation
Images courtesy of Marco Solorio, One River Media
![Page 13: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/13.jpg)
One River Codec Test
Images courtesy of Marco Solorio, One River Media
![Page 14: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/14.jpg)
Digital Voodoo 10-bit Codec
Images courtesy of Marco Solorio, One River Media
![Page 15: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/15.jpg)
Digital Voodoo 10-bit: 10th Gen.
Images courtesy of Marco Solorio, One River Media
![Page 16: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/16.jpg)
One River Codec Test
Images courtesy of Marco Solorio, One River Media
![Page 17: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/17.jpg)
Apple 4:4:4 “None”: 10th Gen.
Images courtesy of Marco Solorio, One River Media
![Page 18: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/18.jpg)
Sampling, Migration & Artifacts
4:4:4 sampling fulfills the digital promise of perfect copies across generations, butMost video equipment doesn’t actually support it. Most high-end video editing packages on computers do and will store 4:4:4 to disk.Lesson: if you want to store 4:4:4 uncompressed video, prepare to buy a lot of disk (or HSM), and abandon videotape.
![Page 19: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/19.jpg)
Storing 4:4:4 Uncompressed Video
QuickTime – Proprietary, but publicly documented and does the task, and software support is availableMJPEG 2000 – Open Standard (ISO/IEC 15444), supports 4:4:4 uncompressed. Software support iffy, but growing.Material Exchange Format – Open Standard, but software support is weak, and some vendor issues
![Page 20: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/20.jpg)
Storing 4:4:4 Uncompressed Video
Videotape is a non-starter. D1 tape decks for uncompressed video cost $200K, and use 4:2:2.Disk vs. HSM Tape Cost vs. Speed Opportunities to detect bit rot, ability to
migrate, time to produce derivatives
Waiting on grid storage….
![Page 21: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/21.jpg)
Mind Games I: Conversion
NYU has approximately 30,000 hours of moving image material, undigitized, in its special collections. Let’s digitize 1/10 of that.9 Digitization/Editing workstations: $690,0009 conversion staff full time for 1 year: $350,000 425 TB of Disk Storage @ $10k/TB = $4,250,000Grand Total: $5,290,000FYI, according to ARL, that’s nearly half our entire 2002 materials budget
![Page 22: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/22.jpg)
Mind Games II: On-going costs
Assume migration every 10 years. Assume time to migrate = 2x time of source material. 6,000 hours x staff salary = ~$120K / 10 = $12k/year
Assume new derivatives every 5 years, and time to migrate = 2x time of source material 12,000 hours x staff salary = ~$240k / 10 = $24k/year
3% disk loss/year x 425 TB = 12.75 TB replaced/year. Assuming disk prices are halved every two years, for next 10 years we’d have $38,750 total replacement costs, or $3,875/year.Grand Total: ~$40K/year maintenance costs
![Page 23: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/23.jpg)
Mind Games III
On-going maintenance costs for 3,000 hours of video on disk aren’t particularly bad.Initial conversion costs, however, are nightmarish.If you don’t spend the money, however, your digital video is unlikely to prove any more preservation-worthy than analog.
![Page 24: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/24.jpg)
NYU Costs: Capture Hardware
Complete system cost: ~$125,000.00
![Page 25: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/25.jpg)
NYU Costs: Conversion Personnel
Currently conversion takes approximately 8 hours for every hour of tape.Minimum personnel cost of ~$150/hour (staff time + benefits) for conversionHope to lower conversion time with practice (and better equipment), but at best, probably around $100/hour of tape.
![Page 26: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/26.jpg)
NYU Costs: Repository Hardware
Sun Enterprise 15K w/L700 Tape Backup: ~$400K/yearSun T3 Disk Arrays (10 TB): ~$100K ($10K/TB)
![Page 27: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/27.jpg)
NYU Costs: Repository Personnel
Fractional part of NYU ITS Unix SysAdmin, Network Support Specialist, Tape Backup support, equivalent to about 1 FTE~$75K/year
![Page 28: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/28.jpg)
NYU Costs: Summary
About $475K/year to keep our server alive and happy; $10K to add another terabyteAbout $125K to add a new video capture/editing workstationAbout $100-150 per hour of video to pay staff capture/conversion costs
![Page 29: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/29.jpg)
The Good vs. the Perfect
Good: 1 hour of video dubbed to Digital Betacam (w/duplicate master) and converted to DVD & MPEG4 streaming derivatives $150 staff time + $70 (2 digibeta tapes) + $3 (DVD-
R blank w/case) + $24.75 (disk for MPEG4) = $247.75
Perfect: 1 hour of video converted to 4:4:4 uncompressed on disk (w/replicated backup) w/DVD & MPEG4 streaming derivatives $150 staff time + $2,860 (disk for master and
backup) + $3 (DVD-R) + $24.75 (disk for MPEG4) = $3,037.75
![Page 30: Cni mc donough_preservation](https://reader036.vdocuments.mx/reader036/viewer/2022062303/5557a543d8b42a696c8b471f/html5/thumbnails/30.jpg)
Affordable Perfection
Disk prices from 1992 to 2000 fell at about 45% per year.If that holds, by 2010, the 1 TB which costs us $10,000 today will cost $276.Storing 143 GB (one hour of 4:4:4 uncompressed video w/audio) will cost $39.50 ($4.50 more than a 60 minute Digital Betacam tape today).