energy-proportional image sensing for
DESCRIPTION
ENERGY-PROPORTIONAL IMAGE SENSING FOR. CONTINUOUS MOBILE VISION. Robert LiKamWa Bodhi Priyantha Matthai Philipose Victor Bahl Lin Zhong. http://roblkw.com. http://research.microsoft.com. Conversational Face Detection. CONTINUOUS MOBILE VISION. CONTINUOUS. VISION. MOBILE. - PowerPoint PPT PresentationTRANSCRIPT
ENERGY-PROPORTIONAL IMAGE SENSING
FOR
Robert LiKamWa
Bodhi PriyanthaMatthai
PhiliposeVictor BahlLin Zhong
CONTINUOUS MOBILE VISION
http://roblkw.comhttp://research.microsoft.com
The Devil Wears Prada
CONTINUOUS MOBILE VISION
CONTINUOUS
MOBILE
VISION
2 of 33
CONTINUOUS MOBILE VISION
CONTINUOUS
MOBILE
VISION
Gestures
Object Memory
3 of 33
Face RecognitionVictor
Fine-grained Localization
BATTERY LIFEBATTERY DEATH
Google Glass2-3 hours
LooxCie2-3 hours
GoPro Hero2-3 hours
4 of 33
2000 mWh10 h200 mW
5 of 33
Image SensorGoal < 25 mW
Sensors~ 5 mW
Processor
~150 mW
Network Stack~20 mW
6 of 33
Image SensorGoal < 25 mW
7 of 33
Image SensorReality > 250
mW
KEY IDEA: ENERGY α QUALITYPo
wer
Frame rate
Powe
r
Resolution
ENERGY PROFILE OF AN IMAGE SENSOR
1 MP, 5 fps250 mW
0.3 MP, 15 fps245 mW
1 MP, 15 fps295 mW
< 25 mWGoal:
Reality: > 230 mW
0.3 MP, 5 fps232 mW
9 of 38
ENERGY-EFFICIENTIMAGE SENSING
Image Sensor Characterization
Energy Reduction Techniques
Energy vs. Vision Performance
10 of 38
IMAGE SENSOR MEASUREMENT
Camera Module
Programmable Clock (I2C)
NI DAQ Device
Power RailResistors
PowerVDD CLK
* Profiled 5 state-of-the-artimage sensors from 2 manufacturers
*
11 of 38
Camera Module
Programmable Clock (I2C)
NI DAQ Device
Power RailResistors
IMAGE SENSOR MEASUREMENT
12 of 38
IMAGE SENSOR WAVEFORMSActive Period
Idle Period
Analog
DigitalPLL
13 of 38
IMAGE SENSOR WAVEFORMSActive Period
Idle Period
14 of 38
IMAGE SENSOR WAVEFORMSActive Period
Idle Period
Pixel Count divided by
Clock Frequency
Frame Timeminus
Active Time
IMAGE SENSOR PIXEL COUNT (N)
Region-of-Interest(Windowing)
Active Active
Powe
r
TimePo
wer
Time 16 of 38
Scaled Resolution(Pixel Skipping)
IMAGE SENSOR PIXEL COUNT (N)
Active Active
Powe
r
TimePo
wer
Time
Video (30 FPS) Power vs. Resolution
17 of 38
ActiveFrame
Readout
ActivePo
wer
Time
Active
IMAGE SENSOR FRAME RATE (R)
Video (0.1 MP) Power vs. FPS
ActiveFrame
ReadoutActive
Powe
r
Time
Active ActiveActive Active
1.0s
18 of 38
CHARACTERIZATION CONCLUSION:NO ENERGY PROPORTIONALITY
Video (0.1 MP) Power vs. FPS
Video (30 FPS) Power vs. Resolution
19 of 38
ENERGY-EFFICIENTIMAGE SENSING
Image Sensor Characterization
Energy Reduction Techniques
Energy vs. Vision Performance
20 of 38
TECHNIQUE #1: AGGRESSIVE STANDBY
Active Active
Active Active
21 of 38
Active Active
Active Active
CAVEAT TO AGGRESSIVE STANDBY
Active Active
Not enough exposure time
This won’t work for long active periods,i.e., high resolution, high frame rate.
22 of 38
TECHNIQUE #2: CLOCK SCALING (f)
One pixel per clock period
23 of 38
TECHNIQUE #2: CLOCK SCALING (f)
Faster clockLower Active TimeHigher Active PowerHigher Idle Power
Slower clock
Higher Active TimeLower Active PowerLower Idle Power
24 of 38
TECHNIQUE #2: CLOCK SCALING (f)
Active
Active
Low Pixel CountLow Frame Rate
Slowed Clock
Optimal clock frequency depends onPixel Count & Frame Rate
25 of 38
AGGRESSIVE STANDBY + CLOCK OPTIMIZATION
Sped-upClock
Aggressive Standby
ActiveReadout
Active
Readout
Optimal clock frequency depends onPixel Count & Frame Rate
Exposure Time26 of 38
ENERGY α QUALITYFr
ame
rate
(FPS
)
Resolution (MP)
30
25
20
15
10
5
021 3 4 5
Fram
e ra
te (
FPS)
Resolution (MP)
30
25
20
15
10
5
021 3 4 5
Pow
er (
mW
)
350
300
250
200
150
100
500
Aggressive Standby &Clock OptimizationUnoptimized
Pow
er (
mW
)
350
300
250
200
150
100
500
27 of 38
CPU
Driv
er
28 of 37
29 of 38
CURRENT IMAGE SENSOR DESIGN
Image Processor
Pixel ArrayColumn Output Gain, ADC
Analog Signal Chain70-85% Power Consumption
30 of 38
Image Processor
Pixel ArrayColumn Output
High-Speed ADC
Mid-Speed ADC
Low-Speed ADC
Gain
HETEROGENEOUS SENSOR DESIGN
Analog Signal ChainHeterogeneous
31 of 38
ENERGY α QUALITY
Default
Clock select
Standby
HW Fix
Power vs. Framerate (at 0.1 MP)
Power vs. Resolution(at 5 FPS)
DefaultClock select
StandbyHW Fix
32 of 36
ENERGY-EFFICIENTIMAGE SENSING
Image Sensor Characterization
Energy Reduction Techniques
Energy vs. Vision Performance
33 of 38
ENERGY vs. VISION: VISION TASKIMAGE REGISTRATION
34 of 36
ENERGY vs. VISION: PERFORMANCE Image
Registration Success
Power Reduction with software
assist
Estimated Power Reduction with hardware assist
Full VGA Resolution0.1 MP, 30 FPS 99.9% 51% 84%Frame Rate Reduction0.1 MP, 3 FPS 95.7% 95% 98%30% Window0.06 MP, 30 FPS 96.5% 63% 91%Subsampled by 20.3 MP, 30 FPS 91.8% 71% 94%
185 mW
TypicalAverage Power
10 mW
With aggr. standby& optimal clock
3 mW
With heterogeneousanalog signal chain
Scalable
Computer Vision
Algorithms
IntegratedSystems Design
DeveloperSupport
EnergyProportional
Image Sensing
36 of 36
Continuous Mobile Vision
ENERGY-PROPORTIONAL IMAGE SENSINGFOR
CONTINUOUS MOBILE VISION
http://roblkw.comhttp://research.microsoft.com
Image sensors are not energy-proportional…
Fram
e ra
te (
FPS)
Resolution (MP)
30
20
10
01 3 5
…but we can make them energy-proportional…
Aggressive Standby
Clock Optimization
Sensor Modifications
… and this is just the beginning.
CMV