a new era of highly specialized computing...

A New Era of Highly SpecializedComputing Systems

Christopher Batten

Associate Professor @ ECE Department, Cornell UniversityVisiting Scholar @ Computer Laboratory, University of CambridgeVisiting Fellow @ Clare Hall, University of Cambridge

Clare Hall ColloquiumMay 2018

• What is Computer Engineering? • Trends in Computer Engineering Specialized Computer Systems

PowerSystems

ComputerEngineering

ElectricalCircuits

ElectricalDevices

SignalProcessing

Telecomm

Cornell University Christopher Batten 2 / 39


ECE is the Study and Application ofElectricity, Micro-Electronics, and Electro-Magnetism

ECE

Power Systems

Computer Engineering

ElectricalCircuits

Electrical Devices

SignalProcessing

Telecomm

Bio-Electrical Engineering

Micro-Electro-Mechanical Devices

Opto-Electrical Devices

Fusion and Plasma Physics

Image, Audio, VideoProcessing

Information Theory

Smart Grid and Smart Buildings

Systems and Synthetic Biology

Atmospheric Science Control Theory

Robotics

Network Protocolsand Optimization

Analog and DigitalCircuits

Computer-AidedDesign



Cornell was founded because of ECE!

Samuel Morse invented thetelegraph (a digital communicationdevice), but needed help buildingthe network

Ezra Cornell built the firsttelegraph line (the beginning oftelecommunications), and investedin the Western Union Telegraph Co

"What hathGod wrought?"

Ezra Cornell’s investments created the fortunethat eventually enabled the founding of Cornell University




ECE

Power Systems


ElectricalCircuits

Electrical Devices

SignalProcessing

Telecomm

Bio-Electrical Engineering

Micro-Electro-Mechanical Devices

Opto-Electrical Devices

Fusion and Plasma Physics

Image, Audio, VideoProcessing

Information Theory

Smart Grid and Smart Buildings

Systems and Synthetic Biology

Atmospheric Science Control Theory

Robotics

Network Protocolsand Optimization

Analog and DigitalCircuits

Computer-AidedDesign



The Computer Systems Stack

Technology

Application

Gap too large to bridge in one step(but there are exceptions, e.g., a magnetic compass)




Register-Transfer Level

CircuitsDevices

Instruction Set Architecture

Programming LanguageAlgorithm

Microarchitecture

Com

pute

rEng

inee

ring

Technology

Application

Operating System

Gate Level

Sort an array of numbers2,6,3,8,4,5 -> 2,3,4,5,6,8

1. Find minimum number in input array2. Move minimum number into output array3. Repeat steps 1 and 2 until finished

Out-of-place selection sort

void isort( int b[], int a[], int n ) { for ( int idx, k = 0; k < n; k++ ) { int min = 100; for ( int i = 0; i < n; i++ ) { if ( a[i] < min ) { min = a[i]; idx = i; } } b[k] = min; a[idx] = 100; }}

C implementation of selection sort





CircuitsDevices



Microarchitecture

Com

pute

rEng

inee

ring

Technology

Application

Operating System

Gate Level

Mac OS X, Windows, LinuxHandles low-level hardware management

blez $a2, donemove $a7, $zeroli $t4, 99move $a4, $a1move $v1, $zeroli $a3, 99lw $a5, 0($a4)addiu $a4, $a4, 4slt $a6, $a5, $a3movn $v0, $v1, $a6addiu $v1, $v1, 1movn $a3, $a5, $a6

MIPS32 Instruction SetInstructions that machine executes





CircuitsDevices



Microarchitecture

Com

pute

rEng

inee

ring

Technology

Application

Operating System

Gate Level

Silicon processtechnology

Si Si Si

SiSiSi

Transistors and wires

Combining devicesto do useful work

Boolean logic gatesand functions

How data flowsthrough system



Computer Systems: CS vs. EE vs. CE

In its broadest definition, computer engineering is thedevelopment of the abstraction/implementation layers that allow us to

execute information processing applications efficientlyusing available manufacturing technologies


CircuitsDevices



Microarchitecture

Com

pute

rEng

inee

ring

Technology

Application

Operating System

Gate Level

TraditionalComputer Science

TraditionalElectrical Engineering

Computer Engineering is at theinterface between hardware and softwareand considers the entire system


What is Computer Engineering? • Trends in Computer Engineering • Specialized Computer Systems

RTL

Devices

ISA

PL

Algorithm

μArch

Technology

Application

OS

Gates

Circuits

Agenda

What is Computer Engineering?

Trends in Computer EngineeringTrend #1: Bell’s “Law”Trend #2: Moore’s “Law”

Specialized Computer Systems



Trends in Computer Engineering


CircuitsDevices



Microarchitecture

Com

pute

rEng

inee

ring

Technology

Application

Operating System

Gate Level

Bell's "Law" How are applicationrequirements changing?

Moore's "Law"How are technologyconstraints changing?



Gordon Bell’s “Law” of Computer Classes

Effe<ct; of Technologyon Near Term

Computer Structures

Given certaincomponents, hardwareand softwaretechniques, and userdemands an accuratepicture of computerdevelopment in thenear future can beplotted.

by C. Gordon Bell,Robert Chenand Satish Rege

The development of computers hasbeen influenced by three factors: thetechnology (i.e., the componentsfrom which we build); the hardwareand software techniques we havelearned to use; and the user (market).The improvements in technologyseem to dominate in determining thepossible resulting structures. Speci-fically, we can observe the evolution

This is a working paper and may not bequoted or reproduced without written per-mission.This work was supported by the AdvancedResearch Projects Agency of the Office oft h e S e c r e t a r y of D efense(F44620-70-C-0107) and is monitored bythe Air Force Office of ScientificResearch.

of four classes of computers:1. The conventional medium and

large-scale, general purposecomputer (circa 1950). Theprice has remained relativelyconstant and the performancehas increased, thereby increas-ing the effectiveness.

2. The minicomputer (circa 1965).The performance has been rela-tively constant, with only a fac-tor of 10 increase from - 1960to - 1970, and the price hasdecreased.

3. Very low cost, specialized digi-tal systems,e.g., desk calculators(circa 1968). The basic techno-logy cost has decreased to aprice which makes mass produc-tion feasible.

4. New, very large structures basedon a high degree of parallelism(circa 1971+). The packing den-sity and the reliability of thetechnology has increased, there-by making large, parallel com-puter fabrication feasible. Thesehighly specialized structuresoffer significant increase in theperformance/cost ratio for cer-tain, usually large problems.

The following sections will brieflydiscuss the evolution of computingstructures in terms of the technology,and general techniques. Conventionalcomputers and minicomputers willthen be discussed as they representtwo of the common computer struc-tures. The next section will briefly

present desk calculators and othermass production digital systems, andthe final section will outline severalcomputers which utilize some formof parallel computation.

Historical Background

The first generation vacuum tubetechnology (circa 1945 1960) com-puters were built to perform long,tedious arithmetic calculations.Because of their relatively poor cost/performance and high cost they wereused mainly for calculations whichwould otherwise be impossible (e.g.,in ballistic calculations). During thisearly period the standard of compari-sons was desk calculator man years.

By the second generation, withtransistor and better random accessmemory technology (circa 1960), thecost/performance had significantlyimproved. This made current com-puter applications (e.g., business anduniversity computing) more feasible.The development of FORTRAN andother higher level languages alsobroadened the user base and provideddemand for more computing power.User demands began to reach andovertake technology, and new tech-niques had to be adopted to raiseperformance levels beyond what thedevice technology provided. This ledto concurrent use of input/outputwith program execution, which inturn led to more general multi-programming.

COMPUTER/MARCH/APRIL 1972/29

I Vice-President of Engineering atDigital Equipment Corporation

I Helped found Microsoft Research

I 1972 paper in IEEE Computer



Trend #1: Bell’s “Law”Roughly every decade a new, smaller, lower priced computer class formsbased on a new programming platform resulting in entire new industries

103

104

105

106

107

108

102

Pric

e in

Dol

lars

1950 1960 1970 1980 2000 2010

G. Bell. "Bell's Law for the Birth and Death of Computer Classes."CACM, Jan 2008.

1990

Volu

me

in c

m3

2020 1950 1960 1970 1980 2000 20101990 2020

Y. Lee et al. "Modular 1mm3 Die-Stacked Sensing Platform ..."JSSC, Jan 2013.

104

106

108

102

100

107

105

103

101

Mainframes

Mainframes

Minicomputers

Minicomputers

Workstations

WorkstationsPersonalComputers

PersonalComputers

Laptops

LaptopsHandhelds

Handhelds

"Ultra"Embedded "Ultra"

Embedded

Supercomputers

ScalableClusters

CloudComputing

Internetof Things



The “Traditional” Internet

Internetof

People

Human beings mustcollect, enter, publish, and analyze

almost all of the informationthat is transmitted over the Internet



Emerging Trend Towards an Internet of Things

Internetof

Things

Interconnected "things" augmented with inexpensiveembedded controllers, sensors, actuators tocollect information and interact with the world



IoT Example: Spending the Day Hiking



IoT Smart Home



IoT Smart Power Distribution Grid



IoT Early Disaster Warning System



IoT Wildlife Tracking System



IoT Wearable Health Monitor



1950 1960 1970 1980 2000 20101990 2020

104

106

108

102

100

107

105

103

101

Trend #1: Bell’s “Law”

Bell’s “Law” predicts an Internet-of-Things,and IoT cloud and embedded devices areincreasingly demanding more performanceand better efficiency



Gordon Moore’s “Law” of Technology Scaling

Electronics, Volume 38, Number 8, April 19, 1965

The future of integrated electronics is the future of electron-ics itself. The advantages of integration will bring about aproliferation of electronics, pushing this science into manynew areas.

Integrated circuits will lead to such wonders as homecomputersóor at least terminals connected to a central com-puteróautomatic controls for automobiles, and personalportable communications equipment. The electronic wrist-watch needs only a display to be feasible today.

But the biggest potential lies in the production of largesystems. In telephone communications, integrated circuitsin digital filters will separate channels on multiplex equip-ment. Integrated circuits will also switch telephone circuitsand perform data processing.

Computers will be more powerful, and will be organizedin completely different ways. For example, memories builtof integrated electronics may be distributed throughout the

machine instead of being concentrated in a central unit. Inaddition, the improved reliability made possible by integratedcircuits will allow the construction of larger processing units.Machines similar to those in existence today will be built atlower costs and with faster turn-around.

Present and futureBy integrated electronics, I mean all the various tech-

nologies which are referred to as microelectronics today aswell as any additional ones that result in electronics func-tions supplied to the user as irreducible units. These tech-nologies were first investigated in the late 1950ís. The ob-ject was to miniaturize electronics equipment to include in-creasingly complex electronic functions in limited space withminimum weight. Several approaches evolved, includingmicroassembly techniques for individual components, thin-film structures and semiconductor integrated circuits.

Each approach evolved rapidly and converged so thateach borrowed techniques from another. Many researchersbelieve the way of the future to be a combination of the vari-ous approaches.

The advocates of semiconductor integrated circuitry arealready using the improved characteristics of thin-film resis-tors by applying such films directly to an active semiconduc-tor substrate. Those advocating a technology based uponfilms are developing sophisticated techniques for the attach-ment of active semiconductor devices to the passive film ar-rays.

Both approaches have worked well and are being usedin equipment today.

Cramming more componentsonto integrated circuits

With unit cost falling as the number of components per

circuit rises, by 1975 economics may dictate squeezing as

many as 65,000 components on a single silicon chip

By Gordon E. MooreDirector, Research and Development Laboratories, Fairchild Semiconductordivision of Fairchild Camera and Instrument Corp.

The author

Dr. Gordon E. Moore is one ofthe new breed of electronicengineers, schooled in thephysical sciences rather than inelectronics. He earned a B.S.degree in chemistry from theUniversity of California and aPh.D. degree in physicalchemistry from the CaliforniaInstitute of Technology. He wasone of the founders of FairchildSemiconductor and has beendirector of the research anddevelopment laboratories since1959.

The experts look ahead

I Co-founder of FairchildSemiconductor

I Co-founder of Intel Corp

I 1965 paper in ElectronicsMagazine



Gordon Moore’s “Law” of Technology Scaling



Trend #2: Moore’s “Law”

Transistors(Thousands)

MIPSR2K

IntelP4

Data collected by M. Horowitz, F. Labonte, O. Shacham, K. Olukotun, L. Hammond, C. Batten1975 1980 1985 1990 1995 2000 2005 2010 2015

100

101

102

103

104

105

106

107

DECAlpha 21264

TypicalPower (W)"Heat"

ComputerPerformance

~15%/year

Power/EnergyLimitsPerformance



One way to address the power challenge

1980singlesimple

processor

1990 2000singlecomplexprocessor

2010multiplesimple

processors



Transition to Multicore

Transistors(Thousands)

MIPSR2K

IntelP4

Data collected by M. Horowitz, F. Labonte, O. Shacham, K. Olukotun, L. Hammond, C. Batten1975 1980 1985 1990 1995 2000 2005 2010 2015

100

101

102

103

104

105

106

107

DECAlpha 21264

TypicalPower (W)"Heat"

ComputerPerformance

~15%/year

Power/EnergyLimitsPerformance

Numberof Cores

Intel 48-Core Prototype

AMD 4-Core Opteron



1950 1960 1970 1980 2000 20101990 2020

104

106

108

102

100

107

105

103

101

1980 1985 1990 1995 2000 2005 2010

100

101

102

103

104

105

106

107




Moore’s “Law” predicts an exponentialincreasing number of transistors per chip,but power limitations have motivated amove to multicore processors

Unfortunately, multicore processors are notenough. What else can we do to use more

transistors to meet the needs ofIoT devices?


What is Computer Engineering? Trends in Computer Engineering • Specialized Computer Systems •

RTL

Devices

ISA

PL

Algorithm

μArch

Technology

Application

OS

Gates

Circuits

Agenda

What is Computer Engineering?

Trends in Computer EngineeringTrend #1: Bell’s “Law”Trend #2: Moore’s “Law”

Specialized Computer Systems



General Purpose Specialized



Example Application Domain: Image Recognition

Starfish

Dog



Machine Learning: Training vs. Inference

forward "starfish"

Training

many images

Model

=? "dog"

labels

errorbackward

Inferenceforward "dog"

fewimages



ImageNet Large-Scale Visual Recognition ChallengeTo

p 5

Erro

r Rat

e

28%26%

16%

12%

7%

2010 2011 2012 2013 2014

Software: Deep Neural Network

Hardware: Graphics Processing Units0 0 4

60

110

Num

of E

ntrie

s U

sing

GPU

sN

um o

f Ent

ries

Usi

ng G

PUs



Hardware Specialization from Cloud to Embedded

CloudComputing

Ultra-EmbeddedComputing

Inte

rnet

of T

hing

s

Google TPUI Training can take weeksI Inference has strict speed

requirementsI Google TPU is custom chip to

accelerate training andinference

Movidius Myriad 2I Vision processing requires

significant computationI Can easily drain the battery of

embedded IoT devicesI Myriad 2 is custom chip to

accelerate machine learning



The Future of Computer Chips



The Celerity Open-Source 511-Core RISC-VTiered Accelerator Fabric:

Fast Architectures & Design Methodologies for Fast Chips

Scott Davidson, Shaolin Xie, Christopher Torng, Khalid Al-HawajAustin Rovinski, Tutu Ajayi, Luis Vega, Chun Zhao, Ritchie Zhao

Steve Dai, Aporva Amarnath, Bandhav Veluri, Paul Gao, Anuj RaoGai Liu, Rajesh K. Gupta, Zhiru Zhang, Ronald G. Dreslinski

Christopher Batten, Michael B. Taylor.

IEEE Micro, 38(2):3041, Mar/Apr. 2018

• TSMC 16nm FFC• 25mm2 die area (5mm x 5mm)• ~385 million transistors• 511 RISC-V cores

• 5 Linux-capable “Rocket Cores”• 496-core mesh tiled array “Manycore”• 10-core mesh tiled array “Manycore” (low voltage)

• 1 Binarized Neural Network Specialized Accelerator• On-chip synthesizable PLLs and DC/DC LDO

• Developed in-house• 3 Clock domains

• 400 MHz – DDR I/O• 625 MHz – Rocket core + Specialized accelerator• 1.05 GHz – Manycore array

• 672-pin flip chip BGA package• 9-months from PDK access to tape-out

Celerity: Chip Overview• TSMC 16nm FFC• 25mm2 die area (5mm x 5mm)• ~385 million transistors• 511 RISC-V cores






Celerity: Chip Overview

• TSMC 16nm FFC• 25mm2 die area (5mm x 5mm)• ~385 million transistors• 511 RISC-V cores






Celerity: Chip Overview



Celerity System-on-Chip Overview

Target Workload: High-Performance Embedded Computing

I 5⇥ 5mm in TSMC 16 nmI 385 million transistorsI 5 “large” processing coresI 496 “small” processing coresI 1 neural network accelerator

for machine learning



Celerity testing led byProf. Michael Taylor atUniversity of Washington


What is Computer Engineering? Trends in Computer Engineering Specialized Computer Systems

1950 1960 1970 1980 2000 20101990 2020

104

106

108

102

100

107

105

103

101

1980 1985 1990 1995 2000 2005 2010

100

101

102

103

104

105

106

107




Moore’s “Law” predicts an exponentialincreasing number of transistors per chip,but power limitations have motivated amove to multicore processors

Specialized Computing Systems

Hardware specialization can use the wealthof transistors to meet the needs of IoT


What is Computer Engineering? Trends in Computer Engineering Specialized Computer Systems

Shreesha Srinath, Christopher Torng, Berkin Ilbeyi, Moyang WangShunning Jiang, Khalid Al-Hawaj, Tuan Ta, Lin Cheng

and many M.S./B.S. students

Equipment, Tools, and IPIntel, NVIDIA, Synopsys, Cadence, Xilinx, ARM


a new era of highly specialized computing...

Documents