A Mixed-Signal CMOS VLSI Image Convolution Circuit using Error Spectrum Shaping

Brent Buchanan

July 2001

School of Electrical and Computer EngineeringGeorgia Institute of Technology

ObjectiveDevelop and demonstrate a CMOS VLSI architecture for performing

general image convolutions using Error Spectrum Shaping

Circuit’sSpatial Noise Model

Discrete 2-D Convolution

y[n, m] = x[n-k, m-l]h[k, l]k,l

y

x

h

ew

Weight

ed

Data

em

From OtherMultipliers

Analog Multiplier

Analog-Computation Convolutions

Resistive SheetsE.g., Mahowald’s Silicon Retina

Variable Pixel ResponseE.g., Funatsu’s Variable Sensitivity Photodetectors

Current Mirrors

Iout

To Neighboring PixelsFrom Neighboring Pixels

Oversampling

Sampled Image

Perfect Intensity Resolution

2x Sampled Image

N-bit Quantized

Image Spectrum Quantization Noise Spectrum

Sampled Image

N-bit Quantized

Total Binary Quantization Noise Energy = f(N)3 dB reduction in Quantization Noise per Doubling of Sampling Rate

W. R. Bennett, “Spectra of Quantized Signals,” 1948

Example: 8-bits/sample at 4x Nyquist 6dB in-band improvement 9-bits/sample at Nyquist

‘Disposal’ Band

Signal Band

Error Spectrum Shaping

2x Sampled Image

N-bit Quantized

2x Sampled Image

ESS N-bit Quantized

Image Spectrum Quantization Noise Spectrum

eh(ti) hq[ni]

Binary Quantizationor other

Representational Inaccuracy

5 5 5 5

5 6 5 6

5 5 5 5

5 6 5 7

5 5 5 5

5 5 5 5

5 5 5 5

5 5 5 5

hq[ni]

e

ef(zi)

h(ti)

‘Disposal’ Band

Convolution and Noise Model

Circuit’sSpatial Noise Model

ew

Weight

ed

Data

em

From OtherMultipliers

Analog Multiplier

Discrete 2-D Convolution

y[n, m] = x[n-k, m-l]h[k, l]k,l

Quantization Examples5122 x 1-bit

Simple Quantization ESS Quantization

Signal-Noise Ratio

10 Log [sij2/(yij – sij)2]

x[n]|2 = 1/N |X(n)|2

10 Log [Sij|2 / |Yij – Sij|2]

Space Based SNR

Parseval’s Relation

Spectrum Based SNR Total in-band noise (dB) vs signal bandwidth3-bit Lenna 5122, 10242, 20482 images

Binary Quantization Noise

Random Gaussian Noise = 12.5% Full Scale

ESSAdditive Noise

Ideal 1st Order ESS1-Bit DOG

Difference of GaussiansBandpass Filter

Convolution w/Corrupting Elements

Binary QuantizationNoise:3-bit 2nd orderESS 642 DOGwith 3-bit 2nd orderESS 10242 image

Ideal result:floating point642 DOGwith 8-bit10242 image

3-bit Binary Quantization

Random Gaussian Noise = 12.5% FS

The Architecture

Summing NodesControl

Data Bus

Data WeightMDAC

Data WeightMDAC

Data WeightMDAC

Data WeightMDAC

Data WeightMDAC

Data WeightMDAC

Data WeightMDAC

Data WeightMDAC

Data WeightMDAC

Analog Multiplications

Digital Data & WeightRegisters

Positive & NegativeSumming Nodes

(extendable to Complex)

Sample Instantiations:Convolution Kernel Support

and Pipelining Patterns1-D 2-DSignal

Kernel

MDAC Schematic

IbiasW4W3W2

W0 W1 D4

D1

D2 D3

D0Iout

Weight DACData DAC

1/4x

Divide by 4 Mirror

1/4x 1/2x

P BalanceP-Channel I Sources

1x 2x 4x

N-Channel I Sinks

1xN Balance

MDAC LayoutNode Switch Weight DAC Data DAC Memory Cell

MDAC, 6-Bit Sign/Weight Register, and Output Switch

56

181

Analog MDACDigital

EquivalentCircuit

Analog vs DigitalRelative AreaComparisons

CellArea

Alone1:69

1:83Routed in

5 Layer Metal(Theoretical)

1:3383LM Actual

Measured Results I: Single MDAC

1 3 5 7 9

11 13 15 17 19 21 23 25 27 29 31

S1

S6

S11

S16S21S26S31

-1.00E-05

0.00E+00

1.00E-05

2.00E-05

3.00E-05

4.00E-05

5.00E-05

6.00E-05

Sink Current

5-Bit Data Word

5-Bit Weight Word

8da0b0: Data-Wise Graphed

Chip #8, MDAC at Row 0 Column 0

Measured Results II: Array MDACsIncreasing Output CurrentIncreasing Data Word Increasing Weight Word

IncreasingRowIndex

IncreasingColumn Index

Focus Portion of Array

-0.0002

-0.00018

-0.00016

-0.00014

-0.00012

-0.0001

-0.00008

-0.00006

-0.00004

-0.00002

01 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

-0.0002

-0.00018

-0.00016

-0.00014

-0.00012

-0.0001

-0.00008

-0.00006

-0.00004

-0.00002

01 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

-0.0002

-0.00018

-0.00016

-0.00014

-0.00012

-0.0001

-0.00008

-0.00006

-0.00004

-0.00002

01 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

-0.0002

-0.00018

-0.00016

-0.00014

-0.00012

-0.0001

-0.00008

-0.00006

-0.00004

-0.00002

01 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

-0.0002

-0.00018

-0.00016

-0.00014

-0.00012

-0.0001

-0.00008

-0.00006

-0.00004

-0.00002

01234567891011121314151617181920212223242526272829303132

-0.0002

-0.00018

-0.00016

-0.00014

-0.00012

-0.0001

-0.00008

-0.00006

-0.00004

-0.00002

01 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

-0.0002

-0.00018

-0.00016

-0.00014

-0.00012

-0.0001

-0.00008

-0.00006

-0.00004

-0.00002

01 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

-0.0002

-0.00018

-0.00016

-0.00014

-0.00012

-0.0001

-0.00008

-0.00006

-0.00004

-0.00002

01 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

-0.0002

-0.00018

-0.00016

-0.00014

-0.00012

-0.0001

-0.00008

-0.00006

-0.00004

-0.00002

01 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

-0.0002

-0.00018

-0.00016

-0.00014

-0.00012

-0.0001

-0.00008

-0.00006

-0.00004

-0.00002

01 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

-0.0002

-0.00018

-0.00016

-0.00014

-0.00012

-0.0001

-0.00008

-0.00006

-0.00004

-0.00002

01 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

-0.0002

-0.00018

-0.00016

-0.00014

-0.00012

-0.0001

-0.00008

-0.00006

-0.00004

-0.00002

01 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

-0.0002

-0.00018

-0.00016

-0.00014

-0.00012

-0.0001

-0.00008

-0.00006

-0.00004

-0.00002

01 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

-0.0002

-0.00018

-0.00016

-0.00014

-0.00012

-0.0001

-0.00008

-0.00006

-0.00004

-0.00002

01 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

-0.0002

-0.00018

-0.00016

-0.00014

-0.00012

-0.0001

-0.00008

-0.00006

-0.00004

-0.00002

01 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

-0.0002

-0.00018

-0.00016

-0.00014

-0.00012

-0.0001

-0.00008

-0.00006

-0.00004

-0.00002

01 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Chip #8, MDACs in Rows 0-4 & Columns 0-4

0 - 200 A

Error Distribution

Data Bits Maximum OutputsD4D0

D1 D2 D3

Variance vs Location

Ideal MDAC Array-Specific1st Order ESS

MDAC-Fit DOG Filter

DOG Convolution Results

Ideal result: floating point 642 DOGconvolved with the 8-bit 10242 image

MDAC-based result: 642 MDAC array-specific 1st orderESS DOG convolved with 10242 1-bit 1st order ESS image

Total In-band Noisefor selected MDAC-based convolutions

Chip #8, middle center-band DOG

Total In-band NoiseMeasured at 1282

Chip #8Nyquist Rate Kernel Size

5122 Image ImageBits 642 322 162

5 23.96 24.02 21.954 22.86 24.95 22.163 22.21 23.93 22.582 23.71 16.76 19.89

Unmodified Kernel &Unmodified Image

1 19.54 20.00 19.875 15.20 13.95 12.484 14.66 13.81 12.513 15.11 14.03 12.922 17.62 16.61 15.98

Best-Fit Kernel &Unmodified Image

1 10.82 10.78 9.675 23.95 23.81 21.814 23.63 25.26 22.173 26.03 24.83 23.572 27.57 25.91 27.08

Unmodified Kernel &1st Order ESS Image

1 20.94 26.45 21.005 25.05 24.62 21.304 26.28 25.42 21.513 25.14 23.88 20.852 27.00 25.43 20.00

1st Order ESS Kernel &1st Order ESS Image

1 40.78 34.57 25.375 26.17 24.86 20.052nd Order ESS Kernel &

1st Order ESS Image 1 31.91 30.74 26.70

Oversampled Kernel Size

10242 Image ImageBits 1282 642 322

Unmodified Kernel & 5 24.01 21.91 23.32Unmodified Image 1 19.56 18.97 21.31

5 25.31 24.13 23.261st Order ESS Kernel &1st Order ESS Image 1 51.56 46.24 37.63

20482 Image ImageBits 2562 1282 642

1st Order ESS Kernel &1st Order ESS Image 1 - 52.39 44.6

ConclusionESS demonstrated to successfully displace noise inherent

in CMOS VLSI computational arrays

Crude ESS algorithms

Assumption about noise in Current-mode summation

Future DirectionsMultidimensional ESS/ APS CMOS ImagerPipelined Convolution ProcessorFast MPEG encoder

PublicationsTHESIS RELATEDB. Buchanan, M. Brooke, “ Error Spectrum Shaping in Analog Image Convolution Circuits“, Submitted to IEEE Transactions on Circuits and Systems

I: Fundamental Theory and ApplicationsB. Buchanan, M. Brooke, “A Mixed-Signal Image Convolution Circuit using Error Spectrum Shaping “, Submitted to IEEE Transactions on Circuits

and Systems II: Analog and Digital Signal ProcessingB. Buchanan, M. Brooke, “Analog CMOS Image Convolution Circuitry using Error Spectrum Shaping,” Proceedings of Philips Research 10th

Seminar on Analogue and Mixed-Signal Design, Eindhoven, Netherlands, June2001

OTHERB. Buchanan, “IC Cell and Library Identification”, Application for United States Letters Patent, Filed 29 June 2001B. Buchanan, V. Madisetti, M. Brooke, "Performance of a Fast Analog VLSI Implementation of the DFT", Proceedings of the 35th Midwest

Symposium on Circuits and Systems, Vol 2, pp 1353-6, August 1992B. Buchanan, C. Camperi-Ginestet, T. Morris, M. Brooke, S. DeWeerth, N. Jokerst, M. Allen, "High Density Focal Plane Signal Processing Using 3-D

Vertical Interconnects", Proceedings of the 37th Midwest Symposium on Circuits and Systems, vol 1, pp 191-4, Aug 1994C. Camperi-Ginestet, B. Buchanan, Y. Wang, N. Jokerst, M. Brooke, M. Allen, "Three Dimensional Smart Pixel Integration of a GaAs-Based Detector

Array Directly on Top of Silicon Circuits", LEOS Summer Topical Meetings 1994: Smart Pixels, pp 56-8, July 1994C. Camperi-Ginestet, B. Buchanan, S. Wilkinson, N. Jokerst, M. Brooke, "Integration of InP-Based Thin Film Emitters and Detectors Onto a Single

Silicon Circuit", Optical Society of America 1995 Spring Topical Meetings , March 1995, Salt Lake City, UtahD. S. Wills, W. S. Lacy, C. Camperi-Ginestet, B. Buchanan, H. H. Cat, S.T. Wilkinson, M. Lee, N. M. Jokerst, M. Brooke, "A Three-Dimensional

High-Throughput Archtecture Using Through-Wafer Optical Interconnect", IEEE-OSA J. L. T., vol 13, pp 1085-92, June 1995J. Cross, A. Lopez-Lagunas, B. Buchanan, L. Carastro, S. C. Wang, N. M. Jokerst, S. Wills, M. Brooke, M. A. Ingram "A Single-Fiber Bidirectional

Optical Link Using Colocated Emitters and Detectors", IEEE Photon. Tech. Lett., vol 8, no 10, pp 1385-7, Oct 1996N. M. Jokerst, M. Brooke, O. Vendier, S.T. Wilkinson, S.M. Fike, M. Lee, B. Buchanan, D. S. Wills, A. Brown, "Manufacturable Mult-Material

Integration Compond Semiconductor Devices Bonded to Silicon Circuitry", SPIE, 1995N. M. Jokerst, M. Brooke, O. Vendier, S.T. Wilkinson, S.M. Fike, M. Lee, E. Twyford, J. Cross, B. Buchanan, D. S. Wills, "Thin Film Mult-Material

Optoelectronic Integrated Circuits", IEEE Trans. on Comp. Pack. and Man. Tech. Part B., vol 19, no 1 pp.97-106, Feb 1996N. M. Jokerst, C. Camperi-Ginestet, B. Buchanan, S.T. Wilkinson, M. Brooke, "Communication Through Stacked Silicon Circuitry Using Integratged

Thin Film InP-based Emitters and Detectors", IEEE Photon. Tech. Lett., vol 7, pp 1028-30, Sept 1995S. M. Fike, B. Buchanan, N. Jokerst, M. Brooke, T. Morris, S. DeWeerth, "8x8 Array of Thin-Film Photodetectors Vertically Electrically

Interconnected to Silicon Circuitry", IEEE Photon. Tech. Lett., vo7, no 10, Oct 1995W. S. Lacy, C. Camperi-Ginestet, B. Buchanan, M. Lee, S. Wilkinson, D. S. Wills, N. M. Jokerst, M. Brooke, "A Fine-Grain, High-Throughput

Architecture Using Through-Wafer Optical Interconnect", Special Issue of the Journal of Light. Tech., Jan 1995W. S. Lacy, M. Grossglauser, C. Camperi-Ginestet, B. Buchanan, D. S. Wills, N. M. Jokerst, M. Brooke, "A Fine-Grain, High-Throughput

Architecture Using Through-Wafer Optical Interconnect", Proceedings of: Workshop on Massively Parallel Processing Using Optical Interconnections, April 1994, Cancun, Mexico