porting after effects to the mercury gpu...
TRANSCRIPT
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Porting After Effects to the Mercury GPU EngineJamie Gjerde | Senior Computer Scientist
Scott Carver | Computer Scientist
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
How We Got Started
▪ Developer kitchen: third parties already using CUDA inside Effects
▪ Customers want more performance (native AE rendering on the GPU)
▪ Mercury GPU Engine: Premiere has full GPU rendering pipeline, many effects already ported
2
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Now….
3
• After Effects now supports GPU rendering in its effects pipeline
• Work is underway to port existing AE effects to GPU: >10 effects have been ported
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Why GPU is Important for After Effects
▪ Significantly more compute/processing power for image processing tasks
▪ GPUs are improving at a faster rate than CPUs
▪ Doing more work on the GPU frees up CPU time
▪ In a mobile / non-desktop future, most compute power is GPU-based
4
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Initial Expectations / Requirements
▪ Faster renders
▪ Performance will scale with faster GPUs
▪ Leverage Mercury Engine and existing effect kernels
▪ Untie AE internals from CPU pixels, CPU implementations
▪ Visual parity is critical!
5
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Architectural challenges
▪ Many rendering tasks will be CPU for a long time.
▪ Render graph must support composite GPU and CPU rendering
▪ Must handle upload / readback transparently.
▪ CPU code cannot be replaced, must co-exist.
▪ Solutions?
▪ Render graph refactored to handle buffers independent of hardware: CPU, CUDA, OpenCL, Metal
▪ If an input buffer is located on one hardware, but rendering is happening on different hardware, buffer is transferred lazily.
6
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Architectural challenges: composite render graph
7
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Architectural challenges
▪ Adapt old rendering architecture
▪ Downside: Most code assumes CPU buffers, direct access, not built for pixel-level parallelization
▪ Downside: AE render graph itself is still basically single threaded / synchronous
▪ Upside: Render graph is very memory conservative (designed for 90’s memory limits)
▪ Upside: Single threaded render means simple GPU interaction, no async spaghetti
▪ Solutions?
▪ GPU and CPU paths share pre-render, frame dependency calculation, and setup code.
▪ GPU-specificity hidden from most components.
▪ Invoke GPU kernels only at lowest possible level.
8
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Architectural challenges
▪ Visual Parity is HARD
▪ Bit depth (8, 16, 32-float)
▪ Algorithm match between SSE implementation and OpenCL/CUDA/Metal code
▪ Sometimes, algo's need some rewrite on CPU as well.
▪ Even with effort: 8 and 16 won't always match due to rounding: float differences can accumulate.
▪ Solutions?
▪ Never mix GPU and CPU frames for an effect in a single comp.
▪ Downside: if we run out of memory for a frame, we can't transparently fall back to CPU
▪ On 8 and 16 bpc: clip to 0..1 after each GPU render node
▪ In some cases, clipping/rouning in kernel is also required
9
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Architectural challenges
▪ Memory
▪ AE frame sizes are large (30000 x 30000 x 32bpc maximum)
▪ Render sizes can vary from frame to frame, not predictable
▪ GPU’s cannot recover easily from out-of-memory (e.g. swap)
▪ Solutions?
▪ AE has existing fractional resolution workflows that can help.
▪ Tiled rendering is extremely complex problem for AE's render graph.
▪ However: some kernels could be tiled at a lower level for GPU renders.
10
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Native GPU Plug-in SDK
▪ Current internal effects are based on existing Premiere GPU SDK
▪ AE render graph handles allocation and CPU <> GPU transfers
▪ Effect receives GPU buffer via SDK call, with correct pixels and parameters for render.
▪ Future: Premiere SDK isn't rich enough to support all AE render features – expansion is a must for both internal effects and third parties.
▪ Internal effect development has illuminated areas where SDK needs iteration
▪ 3rd party developers need an SDK with richer API’s and tools to write efficient plugins for a variety of GPU platforms
11
© 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 12