Breakthroughs in low-light performance illuminate IP video camera applications

IntroductionVideo cameras are turning up everywhere these

days because they let us see something a place,

a person, an event without being conspicuous.

Camera technology is like the human eye in that

it can only see objects or scenes by taking in and

processing the light that is present. Just as we

struggle to see clearly when the lights in a room

are turned off, many video cameras must cope

with low-light environments. Having a camera on

the scene does not always provide the kind of

insight we want unless the camera is able to

overcome low-light conditions and monitor the

scene clearly.

New advancements in video camera technology,

specifically low-light noise filtering technology,

have enhanced not only the individual still images

or frames which make up a video stream but

also increase the fidelity of the individual pixels

that make up each frame. Operating on three

dimensions, improved filtering techniques are able

to remove visual anomalies in low-light video

feeds and thereby clarify and sharpen the images.

Cameras based on powerful real-time processors

and equipped with sophisticated compression

algorithms can often see better in low-light

conditions than the human eye. According to the

2010 FBI Uniform Crime Report, 52.8 percent of

violent crimes and 81.7 percent of property crimes

remained unsolved1. Cameras equipped with

good low-light technology would help reduce

that number.

1 2010 FBI Uniform Crime Report: Crime in the United States

Gang HuaImaging Architect

Texas Instruments

Jacob JoseBusiness Development Manager

Video Security BusinessTexas Instruments

W H I T E P A P E R

Shedding some light on the problemVideo security cameras connected to the network (IP cameras) are being deployed in a wide range

of applications. Beyond the common surveillance cameras implemented in security and law enforcement

settings, cameras are now pervasive in computers, tablets and smartphones. Increasingly more vehicles are

also being equipped with video monitoring systems, or car black boxes, to assist with complex maneuvers

in close quarters and enhance the safety of the vehicle. Underlying all of these applications and others is

the fact that IP cameras must frequently provide a video stream despite poor lighting, if additional lighting is

provided at all. The problem can become particularly important in security applications.

Video streams provided by IP surveillance cameras monitoring dark alleys or dimly lit hallways can

often be fuzzy, pixilated, indistinct and lacking in contrast. Poor quality of the video stream will often result

in unrecognizable faces or missing an occurrence altogether, jeopardizing the usefulness of the video

surveillance system. Providing better lighting conditions is usually not feasible as the lights themselves are

expensive, require maintenance and would negate the conspicuous nature of surveillance systems.

In such low light environments, exposure time of the sensor may be increased to let more light into the

camera, but doing that would blur moving objects, creating indistinct images. One could also increase the

aperture size of the lens, but beyond about f/1.4, the cost and size of the optics become prohibitive.

Further, it would reduce the depth of field. A third strategy would be to increase the gain of the signal

from the image sensor: however, this simultaneously amplifies any noise in the scene. Having an effective

method to remove this noise provides an optimal solution to get clear and distinct images in low light. The

technology used is commonly called noise filtering. Texas Instruments Incorporated (TI) noise filters com-

bined with the latest technology high sensitivity sensors can enable cameras to produce clear video at light

levels of less than 0.1Lux. However, even with low cost, lower sensitivity sensors, the low light performance

is significantly improved with TI noise filtering technology.

The images below, one without and the other with low-light noise filtering demonstrate how this

technique can improve the effectiveness of applications like video surveillance systems.

What makes it hard for an IP camera to see clearly in low-light conditions is noise; or that extraneous signaling

data which the camera mistakenly includes with the image because of the low-light conditions. These artifacts

or anomalies in the signal processing by the camera cause the video stream to degrade in quality, become

blurred and lose definition. The problem is further amplified when gain is applied to increase brightness of the

scene and the video encoder becomes overwhelmed by the level of noise present. Improving the quality of a

video IP camera operating in a poorly lit environment thus involves removing as much of this noise as possible.

By improving the signal-to-noise ratio (SNR) in the video data stream, the camera is able to reveal a more distinct

image. With less noise, the signaling derived from the actual objects in the scene is stronger and the video image

is cleaner, reducing compressed bit rate and increasing compressed video sharpness. There are several distinct

sources of noise and each one must be dealt with on some level or another.

Shot noiseShot noise comes about naturally from fluctuations in the rate at which light photons strike the sensor in a video

camera. The number of photons hitting on a sensor pixel varies randomly around a mean rate proportional to the

illumination level of the pixel.

Fixed pattern noiseFixed-pattern noise is caused by slight variations in the pixels that make up the sensors in an IP camera. Each

pixel can react differently when it is struck by photons of light. These differences may be caused by variations

in the pixels and color filters, or differences in the circuitry connected to the pixels.

Readout noiseThe analog information captured by a light sensor in a video camera must be converted into digital data to be

processed by the camera. This takes place through analog-to-digital converters (ADC). Readout noise results

from the imperfections of the conversion process.

Breakthroughs in low-light performance illuminate IP video camera applications November 2012

2 Texas Instruments

Noise in low-light video signals

Figure 1: Video image without low-light performance enhancements

Figure 2: Video image with low-light performance enhancements

Breakthroughs in low-light performance illuminate IP video camera applications November 2012

3Texas Instruments

There are two basic methods for removing or filtering out noise from low-light video feeds. The first method,

spatial filtering, operates along the two dimensions, height and width, that make up an image on a display screen.

The second method, temporal filtering, adds the third dimension of time which is also present in a

video stream. A noise filter that combines both spatial (2D) and temporal filtering is commonly referred to as

3D Noise Filter.

Spatial filteringA video stream is made up of individual still images or frames that are displayed momentarily one after the other.

A real-time video feed typically displays 30 frames per second, but other display rates can be implemented if the

verisimilitude of actual motion is not required. Spatial filtering is an algorithmic analysis that examines each frame

independently of the others and compares each pixel along the X and Y axis of the image to identify noise and

remove it.

Temporal filtering A temporal filtering algorithm adds the dimension of the time to the analysis. Rather than examining the entire

frame as spatial filtering does, temporal filtering analyzes each pixel over the dimension of time. That is, each pixel

is compared from one frame to the next to determine if noise is present. If a pixel moves in a certain way it likely is

noise and is removed. Temporal filtering is more complex than spatial filtering because video streams often contain

motion, such as an object moving or a person walking across the view of the camera. Temporal filtering must be

able to distinguish between real motion that the camera is meant to observe and which should be retained in the

video data stream, and any of several sorts of aberrant motion in the pixels which would indicate that this is noise

and should be removed. Several methods have been developed to accomplish this. They are the motion adaptive

and the motion compensated methods.

The motion adaptive method attempts to identify those regions of the video stream where the motion of an

object is taking place. The signaling in these regions is retained, including any noise that may be present in those

regions. Moving pixels in the remainder of the image are analyzed and if they are determined to be noise, they are

removed.

Motion compensated analysis is significantly more complex than the motion adaptive method. A motion

compensated analysis will establish a frame of reference to allow temporal filtering on pixels that are moving

because of motion of an object. If motion analysis is correct, motion compensated methods can remove more noise

on moving objects than motion adaptive methods. However, motion analysis becomes error-prone at the high noise

level under very low-light conditions, and therefore may fail to remove noise and produce annoying artifacts.

Technology suppliers like TI who are able to provide a blend of both the motion adaptive and the motion

compensated methods will be best able to meet the range of requirements that IP camera manufacturers and

security system integrators will have.

IP cameras that can capture high-quality video in low-light environments can be quite effective in a number of

applications, but especially in security surveillance systems that often operate in poorly lit environments and

employ compression that further degrades the information contained in a video.

All IP cameras use some form of video compression, often using an H.264 high profile (HP) codec to achieve

best in class compression efficiency. Video compression results in loss of information. In order to maintain an

acceptable quality for the compressed video, like being able to recognize the license plate or the face of the driver

of a car passing through a poorly lit alley, the bit rate of the compressed video has to be kept as high as possible.

However, size of storage and the required backup time limits the affordable video compression bit rate.

Removing noise

The benefits of low-light

performance

A noisy video takes more bits to compress at a desired video quality since noise provides lots of unwanted

detail for the codec to compress, thus requiring a larger storage and in turn increasing the cost of installation.

If the storage cannot be increased, then the backup time should be reduced to maintain acceptable quality for

compressed video. Similarly, when the video has to be transmitted across a network and the bandwidth is limited,

a less noisy video can be transmitted at higher quality within the constraints of the available bandwidth. This is

especially important in residential installations where multiple Wi-Fi hotspots compete for the little available

spectrum. Conversely, at lower bit rates there is a significant loss in compressed video detail as the codec is

overwhelmed by the noise.

Compression technologies like H.264 rely on the consistencies between multiple frames of video to compress

the information contained. Noise brings a high level of inconsistency across multiple frames, requiring more bits to

compress the information in the compressed video. Good low-light technology removes the inconsistent noise that

allows the usage of lower bitrates to achieve acceptable video quality. The operator of a security system with an

IP camera with the low-light technology can then afford to spend less on the cost of storage or alternately use the

same storage capacity to record a higher quality of video or increase the backup time.

Further, the operator of a security system with an IP camera monitoring a poorly lit exterior door might consider

installing additional external lighting to better illuminate the area and thereby improve the quality of the video feed.

Since it can compensate for the lighting conditions, a camera with low-light performance eliminates the need for

additional lighting. Instead of installing new lights with the required electrical wiring, the security camera could be

replaced with an IP camera capable of superior low-light performance. Moreover, the incremental cost of electricity

to operate the additional external lighting would be avoided entirely.

Many sophisticated surveillance systems today also incorporate automatic security analytics procedures, which

process the video feeds further in order to supplement the human monitors who are operating the system. These

analytics are more effective when they process cleaner, higher definition video feeds. Better security analytics

might reduce the manpower needed to operate surveillance systems.

TIs low-light performance solutions

TI has a proven track record of providing industry-leading technology for video security processing applications,

such as IP-based video surveillance cameras. TIs superior 3D noise filtering technology allows sensors to be

used at higher gain levels, producing detailed video at lower light levels than needed by competition. DaVinci

video processor platforms are the industrys highest performance video engine that enables video and graphics

accelerators to process three 1080p60 fps streams simultaneously to enable new applications and intuitive user

interfaces. DaVinci users benefit from a scalable product line on a single platform with same core IPs, unique

combinations of high performance and low power and multiple derivatives for specific applications. TIs reference

designs enable quick product development in as little as 6 months.

Breakthroughs in low-light performance illuminate IP video camera applications November 2012

4 Texas Instruments

Figure 3: Highly noisy video image without low-light performance enhancements

Figure 4: Video image after applying TIs low-light noise filter detail emerges out of the noise

In addition to the many supporting devices, TIs best-in-class video processors have long been at the

heart of many IP cameras. The recent introduction of the DaVinci DM385 video processor extends these achieve-

ments by integrating even more powerful low-light performance, including 3D noise filtering, wide

dynamic range processing as well as H.264 SVC-T high profile compression technology, which gives

best-in-class in terms of compression efficiency. This is combined with flexibility that allows each stream to

be set at the optimal codec profile, with the main stream being set at High Profile for maximum encode efficiency

and secondary streams being set at Base Profile for maximum decode compatibility. The DM385 is able to

compress up to four megapixels of real-time and secondary D1 real-time video stream with either the H.264

or the SVC-T high-profile codecs. A variety of features such as 4Kx2K or higher resolution video, simultaneous

multi-profile (base/main/high) compression, face detection, video stabilization, lens distortion correction,

multi-exposure and dual sensor support give camera manufacturers the ability to easily differentiate their products

in the marketplace. Moreover, a DM385 IP-based camera reference design allows manufacturers to quickly

deliver new low-light cameras to the marketplace while taking advantage of the most popular sensors.

For more information on the DM385 and other TI DaVinci solutions for IP cameras and other security applications,

go to www.ti.com/ipcamera.

5Texas Instruments

SPRY224

Important Notice: The products and services of Texas Instruments Incorporated and its subsidiaries described herein are sold subject to TIs standard terms and conditions of sale. Customers are advised to obtain the most current and complete information about TI products and services before placing orders. TI assumes no liability for applications assistance, customers applications or product designs, software performance, or infringement of patents. The publication of information regarding any other companys products or services does not constitute TIs approval, warranty or endorsement thereof.

The platform bar is a trademark of Texas Instruments.All other trademarks are the property of their respective owners.

E010208

2012 Texas Instruments Incorporated

DDR3SDRAM

Auto IrisControlSD

CardUSB

RS485 XCVRUART/Alarm

Sensor Board

SensorLVD8324(optional)

TPS5432TPS63036

TPS23753A

PHY

GMAC GPMC McASP

UART USB SDIO

I2C

ISS

CS/2

GPIO/INT

DM385

EMIFO

DSS

NANDFLASH AIC

12V DC

Microphone

Speaker Out

CompositeVideo Out

VideoAmp

Figure 5: DaVincis DM385 video processor block diagram

IMPORTANT NOTICETexas Instruments Incorporated and its subsidiaries (TI) reserve the right to make corrections, enhancements, improvements and otherchanges to its semiconductor products and services per JESD46, latest issue, and to discontinue any product or service per JESD48, latestissue. Buyers should obtain the latest relevant information before placing orders and should verify that such information is current andcomplete. All semiconductor products (also referred to herein as components) are sold subject to TIs terms and conditions of salesupplied at the time of order acknowledgment.TI warrants performance of its components to the specifications applicable at the time of sale, in accordance with the warranty in TIs termsand conditions of sale of semiconductor products. Testing and other quality control techniques are used to the extent TI deems necessaryto support this warranty. Except where mandated by applicable law, testing of all parameters of each component is not necessarilyperformed.TI assumes no liability for applications assistance or the design of Buyers products. Buyers are responsible for their products andapplications using TI components. To minimize the risks associated with Buyers products and applications, Buyers should provideadequate design and operating safeguards.TI does not warrant or represent that any license, either express or implied, is granted under any patent right, copyright, mask work right, orother intellectual property right relating to any combination, machine, or process in which TI components or services are used. Informationpublished by TI regarding third-party products or services does not constitute a license to use such products or services or a warranty orendorsement thereof. Use of such information may require a license from a third party under the patents or other intellectual property of thethird party, or a license from TI under the patents or other intellectual property of TI.Reproduction of significant portions of TI information in TI data books or data sheets is permissible only if reproduction is without alterationand is accompanied by all associated warranties, conditions, limitations, and notices. TI is not responsible or liable for such altereddocumentation. Information of third parties may be subject to additional restrictions.Resale of TI components or services with statements different from or beyond the parameters stated by TI for that component or servicevoids all express and any implied warranties for the associated TI component or service and is an unfair and deceptive business practice.TI is not responsible or liable for any such statements.Buyer acknowledges and agrees that it is solely responsible for compliance with all legal, regulatory and safety-related requirementsconcerning its products, and any use of TI components in its applications, notwithstanding any applications-related information or supportthat may be provided by TI. Buyer represents and agrees that it has all the necessary expertise to create and implement safeguards whichanticipate dangerous consequences of failures, monitor failures and their consequences, lessen the likelihood of failures that might causeharm and take appropriate remedial actions. Buyer will fully indemnify TI and its representatives against any damages arising out of the useof any TI components in safety-critical applications.In some cases, TI components may be promoted specifically to facilitate safety-related applications. With such components, TIs goal is tohelp enable customers to design and create their own end-product solutions that meet applicable functional safety standards andrequirements. Nonetheless, such components are subject to these terms.No TI components are authorized for use in FDA Class III (or similar life-critical medical equipment) unless authorized officers of the partieshave executed a special agreement specifically governing such use.Only those TI components which TI has specifically designated as military grade or enhanced plastic are designed and intended for use inmilitary/aerospace applications or environments. Buyer acknowledges and agrees that any military or aerospace use of TI componentswhich have not been so designated is solely at the Buyer's risk, and that Buyer is solely responsible for compliance with all legal andregulatory requirements in connection with such use.TI has specifically designated certain components as meeting ISO/TS16949 requirements, mainly for automotive use. In any case of use ofnon-designated products, TI will not be responsible for any failure to meet ISO/TS16949.Products ApplicationsAudio www.ti.com/audio Automotive and Transportation www.ti.com/automotiveAmplifiers amplifier.ti.com Communications and Telecom www.ti.com/communicationsData Converters dataconverter.ti.com Computers and Peripherals www.ti.com/computersDLP Products www.dlp.com Consumer Electronics www.ti.com/consumer-appsDSP dsp.ti.com Energy and Lighting www.ti.com/energyClocks and Timers www.ti.com/clocks Industrial www.ti.com/industrialInterface interface.ti.com Medical www.ti.com/medicalLogic logic.ti.com Security www.ti.com/securityPower Mgmt power.ti.com Space, Avionics and Defense www.ti.com/space-avionics-defenseMicrocontrollers microcontroller.ti.com Video and Imaging www.ti.com/videoRFID www.ti-rfid.comOMAP Applications Processors www.ti.com/omap TI E2E Community e2e.ti.comWireless Connectivity www.ti.com/wirelessconnectivity

Mailing Address: Texas Instruments, Post Office Box 655303, Dallas, Texas 75265Copyright 2012, Texas Instruments Incorporated