client server methods for future cinema compositing multiple 4k cinema streams
DESCRIPTION
Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams. Scalable City: Massive Scale Virtual Worlds. Massively multiplayer continuous world Hundreds of thousands of interactive objects Large aggregate bandwidth requirements. Challenges/Issues - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/1.jpg)
Client Server methods for Future Cinema
Compositing Multiple 4K Cinema Streams
1
![Page 2: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/2.jpg)
Scalable City: Massive Scale Virtual Worlds
• Massively multiplayer continuous world
• Hundreds of thousands of interactive objects
• Large aggregate bandwidth requirements
Challenges/IssuesOriginally develop multicore accelerated single user, immersive system, to complex virtual world. Create compute and conceptual ecosystem.
2
![Page 3: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/3.jpg)
The environment can be extensible with endless variations, computed at run time.
![Page 4: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/4.jpg)
![Page 5: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/5.jpg)
![Page 6: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/6.jpg)
![Page 7: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/7.jpg)
3 10gb interfaces to compute accelerators
Development Server Framework 2/2011
IBM Z10 mainframe computer at San Diego Supercomputer Center2- IFL’s with 128mb Ram, zVM virtual OS manager with Linux guests6 tb storage fast local storage – 15K disks4 SR and 2 LR 10gb ethernet interfaces
8HS22 blades- 16 Xeons – 96 cores
4 QS20 blades- 8 Cell CPU’s
2 QS22 blades- 4 Cell CPU’s4 way Xeon Server – 32 core
10gb interface to internet
nVidia Tesla accelerator – 4 GPU’s on linux host
Many Clients
7
![Page 8: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/8.jpg)
Virtual World Director
Sheldon Brown, UCSD Site Director CHMPRKristen Kho, Programmer, UCSD Experimental Game Lab
Future Cinema –
![Page 9: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/9.jpg)
Virtual World Director
Allows Virtual World to be platform for creating Cinema.
Enables very flexible methods for creating cinema.
Allows cinematic production to take place in a broadly distributed and many user environment.
Allows cinematic data to be derived from the experiences of many users.
![Page 10: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/10.jpg)
Interface – Machinima Menu
![Page 11: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/11.jpg)
Interface - Editing
![Page 12: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/12.jpg)
4K Cinema Generation
Automate re-rendering of data into 3D 4K frames.
Sony ZEGO BCU-100 Computing Unit• Cell/B.E. Processor and RSX technology • Prototype system with 4 HD-SDI inputs/outputs
• Use to process real-time 4K video• Procedural 4K animations – (ala Galileo films)
• Initial experiments to use BCU-100 as a 4K video player:• Not fast enough to load 4K frames from disk and maintain a
reasonable frame rate. • Must process input and immediately send to output.• Create procedural 4K movies from data sources – i.e. bitmap
compositing, algorithms, etc.
![Page 13: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/13.jpg)
Ultra high resolution projection technologies 4K – 3840 x 2160 pixel displays today8K and beyond tomorrow Digital Cinema Auditorium
ultra-high resolution cinema, 3-D sound; tele-presence conferencing; stereo imaging, Cine-Grid optical ultra-high bandwidth
Digital Cinema –Galileo Movies, 24 short 4K Movies – Sheldon Brown, Kim Stanley-Robinson
![Page 14: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/14.jpg)
Leveraging global lambda networks to improve performance for extremely high bandwidth and time dependent applications such as cinema, teleconferencing and virtual reality.
![Page 15: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/15.jpg)
Collaboration with University of Sao Paulo, Mackenzie University, Keio University, Sony, Zaxel, IBM, Intel
Stream Ultra High Fidelity Cinema on Global Lambda Grid
![Page 16: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/16.jpg)
![Page 17: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/17.jpg)
17
Stream real-time uncompressed 4K to our compositing and annotation server
Extending TritonSort* systemHybrid Disk/Flash Memory Storage Array
100’s of Terabytes
Multiple 10gb interfaces
*George Porter Center for Networked Systems, UCSD
![Page 18: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/18.jpg)
Data-intensive Cinema• Data-intensive Cinema (DI-Cinema) Project
– Encompasses video, scientific visualization, motion picture production, medical imagery, …
– Higher resolution formats + 3D = big data challenge• 100s of TB to multiple PBs of data per project• Not just streaming
– Subjected to increasing amount of computation– Media is the result of staged, pipeline-oriented computing
![Page 19: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/19.jpg)
Digital Cinema as a Computing ProblemBatch-oriented video
computing:– Color adjustment– Digital watermarks– Compositing (join operation)– Format, resolution, frame-rate
adjustments– Rendering– Captioning– Transcoding
Requires high resolution, and thus large data sizes, at each of these stages as well as in the final product
I/O I/O I/O
Compute
Compute
ComputeCompute
Compute
I/OI/O I/O Streaming
![Page 20: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/20.jpg)
Data-intensive video is a problem today…
• Netflix transcodes each of its movies 50+ times for each type of end-user device and resolution, relying on the cloud for scalability (Adrian Cockcroft, CTO Netflix)
• 4+ million “assets,” or pieces of content, must be orchestrated together to form a single project (“What’s up Prof? Current Issues in the Visual Effects & Post-Production Industry”, Leonardo, vol 43, no 1, Feb 2010).
• On digital intermediate:“…the files are so massive they are difficult to display, edit or color correct”, “Real time playback…is extremely difficult on even the fastest RAIDs”, “This is a lot of data to push around…requiring proprietary software and hardware solutions” (Filmlook, Inc., 2008)
![Page 21: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/21.jpg)
* Image used with permission (GNU Free Documentation License)
…and a bigger problem tomorrow
![Page 22: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/22.jpg)
4K/Stereo High Resolution Cinema development with same asset pipeline as game environment
![Page 23: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/23.jpg)
WAAG, AmsterdamCalit2, UCSD
Calit2 and NSF Optiputer Project Allow for High Speed Networks and Ultra-High Fidelity environments.
![Page 24: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/24.jpg)
Potential DI-Cinema system architecture
1. Supporting efficient computation is focus2. Supporting low-latency streaming3. Supporting efficient integration with networked storage
Themis
Multi-stage,iterative
computationu
NAS/SANLong-term Storage
w
vSoftware remote display emulator
Hardwareremote display
Streaming Support
![Page 25: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/25.jpg)
Storage networking challenges• Giant gather/scatter problem
– Read from ~100 disks and aggregate in-network at 60 fps (100+ MB/frame)
– Disk seek latency of 15ms, yet new frames must be loaded within 16 ms
2. Storage for batch and low-latency
![Page 26: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/26.jpg)
Potential New Industrial Connections• Technicolor:
– Major provider of motion picture toolchain
– Acquiring content, video storage, computing, viewing
• Disney– Major source of content– Large computing
infrastructure; large pipeline-oriented computing platform
• Cisco– Datacenter networking
• HP– Provider of computation
and storage to motion picture industry
– Carries out data-intensive computing research
• Google– World’s largest video
processing system– YouTube– Just announced support
for 4K video
![Page 27: Client Server methods for Future Cinema Compositing Multiple 4K Cinema Streams](https://reader035.vdocuments.mx/reader035/viewer/2022062521/56816783550346895ddc94e0/html5/thumbnails/27.jpg)
Next Steps
• Resolve server/compute system infrastructure
• Develop client interface mixing virtual world 4K and 4K camera based
• Integrate streaming storage system with server/compute