introducing computer vision and assertive technologies...

Title 44pt sentence case

Affiliations 24pt sentence case

20pt sentence case

© ARM 2016

Introducing Computer Vision and Assertive technologies from the new Imaging and Vision Group

Judd Heape

ARM Tech Symposia Asia 2016

Sr. Director, Marketing, Imaging and Vision Group

© ARM 2016 2

Text 54pt sentence case ARM has acquired Apical – a global leader in HDR, Display, ISP and Vision

© ARM 2016 3


Bullets 24pt sentence case

Sub-bullets 20pt sentence case

Video & Imaging is everywhere

Experience Intelligence

© ARM 2016 4




ARM’s new additions to the Mali Multimedia Suite

Delivering user experience Extracting intelligence from vision

Assertive Display

ARM Computer Vision

Assertive Camera

© ARM 2016 5

Text 54pt sentence case Assertive Display

© ARM 2016 6




Assertive Display

Multimedia can be viewed even in bright sunlight

Display power can be reduced by up to 50%

Automated re-mastering for HDR content to be viewed on mobile displays

© ARM 2016 7




Assertive Display

Saves power and allows you to see screens under bright sunlight

Without Assertive Display in

bright sunny conditions

With Assertive Display under

same conditions

© ARM 2016 8




HDR content viewed on new

ultra bright HDR TVs

HDR content viewed on standard mobile display:

With Assertive Display - HDR Without Assertive Display - HDR

© ARM 2016 9

Text 54pt sentence case Assertive Camera

© ARM 2016 10




ARM Image Signal Processor

Lens /

Image

sensor

MIPI CSI-2

Receiver

Host CPU

ARM ISP

CoreLink Interconnect

DRAM

Controller

DRAM

Mali

V-550

Still/Video

Encode SRAM: Line mems, LUTs

Imaging

SOC

Display

Controller

Display

ARM ISP firmware

10

Lens shading (vignetting) correction (mesh and radial) and Geometric Distortion Correction engine

Advanced spatial (2D) noise reduction (sinter®)

Motion-adaptive temporal (3D) noise reduction (temper™)

Multi-exposure HDR image fusion

Space-variant HDR processing (iridix®)

>500Mpixel/sec

1080p60/4K video

256Mpixel sensor support

• Hardware IP

• Firmware

• Tuning software and support

Features

Performance

Deliverables

© ARM 2016 11




ARM – a world leader in HDR

Leading mobile handset with video HDR function

is based on ARM’s HW technology

ARM’s HDR solution leads the market by a significant margin

Supports industry standard HDR formats

HDR10 and Hybrid Log-Gamma capture support

Camera w/o ARM HDR Camera w/ ARM HDR

(same image sensor)

© ARM 2016 12




Complete GDC engine

Geometric Distortion Correction engine

Full correction for fisheye or lens effects

Ideal for automotive applications – mirror replacement and bird’s-eye view

© ARM 2016 13

Text 54pt sentence case ARM Computer Vision

© ARM 2016 14




ARM Computer Vision

On-device hardware accelerator of

vision tasks

Capable of detecting an unlimited number

of objects in real-time HD video*

High power-efficiency and low silicon

area

Suitable for battery-powered, portable

devices

Licensed by multiple SoC vendors

for next-generation smart cameras

and smart sensors

*Subject to the minimum resolvable object size

© ARM 2016 15




Text 30pt sentence case

ARM Computer Vision – enabling new applications

A low-power object detection engine

Smart crop & zoom

Automatic still capture during video

Smart continuous AF & AE

Real-time video indexing at capture

Region-of-Interest based video encoding

ISP

ARM

CV

Core

HEVC

encoder

3A FW

Still capture

Video

Crop/scaler

App level (user space)

CMOS

sensor

MP4

movie file

ARM CV

metadata

© ARM 2016 16




Text 30pt sentence case

Automatic scene composition during video capture Enabled by ARM Computer Vision

8MP video capture, rescaled to

1080p for video encode

Automatic continuous crop &

rescale from 8MP to 1080p for

Region of Interest (ROI)

© ARM 2016 17




Smart crop & zoom example

Video at www.youtube.com/watch?v=Ed1TkYmCUEM

www.youtube.com/watch?v=NnqGhgzQa_M

Smoothly transition from camera-based stabilization to object-based stabilization

http://www.youtube.com/watch?v=Ed1TkYmCUEM

http://www.youtube.com/watch?v=NnqGhgzQa_M

© ARM 2016 19




Combining the best technologies for VR, AR and CV

Virtual Reality

(VR)

Augmented Reality

(AR)

Computer Vision

(CV)

ARM Cortex-A processors

- Cortex-A73 for premium smartphones

- Cortex-A57 and Cortex-A35 for entry-level devices

ARM Mali Graphics Processors

- Ten years of integration with Cortex-A technology Mali Display Technology

- Already integrates with Apical Assertive Display n/a

Mali Video Processors

- Encode and decode HD video at 120fps Apical Assertive Display

- Needed for any display in different lighting conditions n/a

Apical Assertive Camera

- Improves photographs in low-light conditions n/a

Apical Computer Vision

- Provides PPA-leading hardware acceleration of vision tasks n/a

© ARM 2016 20




Delivering an end-to-end HDR experience on mobile

HDR Capture HDR Encode HDR

Decode HDR Display

Assertive

Camera with

Mali GPU

Mali Encode Mali Decode Mali Display

Processor with

Assertive Display

The trademarks featured in this presentation are registered and/or unregistered trademarks of ARM Limited

(or its subsidiaries) in the EU and/or elsewhere. All rights reserved. All other marks featured may be

trademarks of their respective owners.

Copyright © 2016 ARM Limited

© ARM 2016

introducing computer vision and assertive technologies...

Documents