introduction to deep learning andneural networks...a neuron is illustrated below. figure 3....

AUGUST30,2016

1) Deep learning and neural networks are techniquesused in computer science to analyze data. They areespeciallyusefulinpatternandimagerecognition.

2) Neuralnetworksusesoftwaretomimicthefunctionofthemanyneuronsinthehumanbrain.Thesefunctionscanbe tunedovertime;thismeansman-madeneuralnetworkscanlearn.

3) 3Deep learning refers to the use ofmultiple layers ofneuralnetworks.

4) Consumersbenefit from these technologieseverydaywhen they read curated news headlines, researchrentingaroomonlineorconversewithachatbot.

5) Although the emerging artificial intelligence (AI)marketisestimatedatjust$400millionthisyear,itcanpotentially enhance adjacent markets such asmedicine, consumer electronics, robotics, and mostimportantly,autonomouscars.

6) Many global industrial and technology companies arepursuingautonomouscars.

7) Venture capitalists and technology companies areaggressivelyinvestinginandacquiringAIstartups.

Introduction to Deep Learning

and Neural Networks

D E B O R A H W EI N S W I G M a n a g i n g D i r e c t o r ,

F u n g G l o b a l R e t a i l & T e c h n o l o g y d e b o r a h w e i n s w i g @ f u n g 1 9 3 7 . c o m

U S : 6 4 6 . 8 3 9 . 7 0 1 7 H K : 8 5 2 . 6 1 1 9 . 1 7 7 9

C H N : 8 6 . 1 8 6 . 1 4 2 0 . 3 0 1 6

AUGUST30,2016

EXECUTIVESUMMARY

AIispresentlycausingagreatdealofexcitementintechnologyandfinancialcircles.DecadesofresearchintoAIandneuralnetworkshavefinallycreatedbreakthroughsthatcanbeusedinsystemsthatcananalyzedataandmakepredictions.ThesebreakthroughshaveresultedinstunningamountsbeingpaidtoacquireAIcompaniesandinvest instartups.Atthesametime,theexcitement over AI has created buzzwords such as “deep learning” and“neuralnetworks,”buttheyarereallyjustwaystoanalyzeandidentifydatawithintherealmofcomputerscience.

Todevelop these approaches, scientists andprogrammers have turned tothemysterioushumanbrainandthetrillionneuronsinsideitthattransmitand process information, as the model for analyzing data. They havedevelopednetworksof cells thatmimic the functionofneurons.Althoughthis approach has gained and lost favor over time among computerscientists it is currently considered the best approach to accurately andproductively identify patterns. Patterns are extremely important in AI;images of handwritten objects or obstacles in the path of a movingautonomousvehiclearepatternsthatneedtobeidentifiedandhandled.

Source:Shutterstock

AlthoughtheAI industry isestimatedatamere$400million in itscurrentinfant state, the market could encompass tens of billions of dollars asadjacentindustriessuchasautonomouscars,robots,military,medicineandconsumer electronics embrace the technology. As mentioned above,technologythatcanidentifyobstaclesintheroadisextremelyimportanttofirms who offer and guarantee a safe ride in an autonomous vehicle.Industrial and technology companies around the globe are aggressivelypursuingthispotentially importantandhugemarket.TherecentadvancesinAIcombinedwith thepotentially largemarkets thatcouldbeenhancedby the technology,are the likely reason for theavalanche in fundingofAIstartupsandthebiddingwarsthathaveoccurredtoacquirethem.

The good news for us humans is neural nets and deep learning have thepotential to improve our lives.We already use them every daywhenwe

AlthoughtheAIindustryisestimatedatamere$400millioninitscurrentinfantstate,themarketcouldencompasstensofbillionsofdollarsasadjacentindustriessuchasautonomouscars,robots,military,medicineandconsumerelectronicsembracethetechnology.

Thehumanbrainiscomprisedof1quadrillionneuronsthatcommunicatewitheachother.

AUGUST30,2016

read a BuzzFeed article or look up a room on Airbnb. These twotechnologies are purelymethods for analyzing data anddonot appear tolead to computers that become autonomous and self-aware, the stuff ofmanyfrighteningscience-fictionstories.

INTRODUCTION

Neural nets are tools that can be createdwith electronics or software toemulate some of the function of a neuron, which comprise the humanbrain.

Thehumanbrain is comprisedof1quadrillionneurons that communicatewith each other. As in the human brain, many artificial neurons can becombined in a network to communicate with each other. Deep learningreferstotheuseofmultiplelayersofneuralnetworksinordertoincreaseaccuracyandproductivity.

Source:Shutterstock

Specifically,neuralnetsaregoodatidentifyingpatterns—inimages,speech,data, or even stock prices. The neurons contain programmable elementsthatareusedtoevaluatetheir inputs,andsincetheparameterscontainedwithin these neurons can be changed or tuned to make the patternidentificationmoreaccurate,thenetwork“learns”toidentifythoseimages.

One application of neural nets is handwriting recognition. In this case, aneural net is fed a sequence of handwritten digits, and the weights aretuneduntil thenetworkcanrecognizethedigitswithacceptableaccuracy.The Mixed National Institute of Standards and Technology (MNIST)establishedadatabaseof60,000 trainingdigitsand10,000 testing imagesfortuningandtestinghandwritingrecognitionsystems.SomeoftheMNISTdigitsareincludedinthefigurebelow.

Deeplearningreferstotheuseofmultiplelayersofneuralnetworksinordertoincreaseaccuracyandproductivity.

Theparameterscontainedwithintheseneuronscanbechangedortunedtomakethepatternidentificationmoreaccurate,thenetwork“learns”toidentifythoseimages.

AUGUST30,2016

Figure1.MNISTTesting/TrainingDigits

Source:neuralnetworksanddeeplearning.com

Neural nets date back to the 1940s, when neurophysiologist WarrenMcCulloch and mathematician Walter Pitts published a paper thatdescribedhowelectricalcircuitscouldbeusedtocreateaneuralnetwork.In the 1960s, the interest of computer scientists turned to vonNeumannarchitectures that are basedon a CPU, logic andmemory, and interest inneural nets waned. However, in the 1980s, some groundbreakingdevelopments in neural nets reinvigorated interest, which continuesthrough the present day. Many researchers consider deep learning “themostaccurateandproductivetechniqueinAIresearchtoday,”accordingtoCBInsights.

Therearedeeplearningandmachinelearninginproductsandservicesweconsumeanduseeveryday,including:

• ProductrecommendationsfromAmazon

• HeadlinesfromBuzzFeed

• PricesontheAirbnbwebsite

• Facebook’sMassistant

Google has integrated a deep-learning machine with its Street Viewplatform to enable the identification of nearly every location, and thecompanyisalsousingitsownDeepMindAItechnologytoreducetheenergyusedtocoolitsdatacenterby40%.

AIANDNEURALNETWORKS

AI refers to thebranchof computer science thatuses computers toapplyhumanintelligencetosolveproblems.WithinAI,machinelearningreferstothetechniquesusedtoperformcognitivefunctions.Deeplearningreferstotheuseofmultiple layersofneuralnetworks.The relationshipamong thethreeisshownintheVenndiagrambelow.

Figure2.HierarchyandInteractionofAIApproaches

Source:CBInsights

Neuralnetsdatebacktothe1940s,whenneurophysiologistWarrenMcCullochandmathematicianWalterPittspublishedapaperthatdescribedhowelectricalcircuitscouldbeusedtocreateaneuralnetwork.

AIreferstothebranchofcomputersciencethatusescomputerstoapplyhumanintelligencetosolveproblems.WithinAI,machinelearningreferstothetechniquesusedtoperformcognitivefunctions.Deeplearningreferstotheuseofmultiplelayersofneuralnetworks.

AUGUST30,2016

Deeplearningisjustonewaytocreatemachinelearning.Othersinclude:

• AssociationRule

• Bayesian

• Clustering

• DecisionTrees

• DimensionalityReduction

• Ensemble

• Instance-based

• Others

• Regression

• Regularization

NEURALNETWORKS

Thehumanbrain iscomposedofabout100billionbraincells,orneurons,andeachneuroncanconnectto10,000otherneurons.Thus,ahumanbraincancontain1quadrillion,i.e.,1,000trillion,synapticconnections.Aneuronisillustratedbelow.

Figure3.IllustrationofaNeuron

Source:appsychology.com

Anartificialneuron,orperceptron,worksmuchthesamewayasahumanneuron,withinputsandoutputs,anditsfunctionaldiagramappearssimilartoaneuron.Severalinputsconnecttoaneuron,whichcanbeimplementedinelectronicsorinsoftware.Theelectronicdeviceorsoftwarecontainsanalgorithmthatusesamathematicalfunctiontocreatearesult,whichissenttotheoutput.

The figurebelow illustrates anartificial neuron inwhich theoutput is thesumofeachoftheinputs,multipliedbyaweightingfactorviatheobjectivefunction.Theresultisconsideredaoneorzero,dependingonwhethertheoutput exceeds a certain threshold value.Neurons can also use nonlinearandmorecomplexmathematicalfunctions.

Thehumanbrainiscomposedofabout100billionbraincells,orneurons,andeachneuroncanconnectto10,000otherneurons.Thus,ahumanbraincancontain1quadrillion,i.e.,1,000trillion,synapticconnections.

Anartificialneuron,orperceptron,worksmuchthesamewayasahumanneuron,withinputsandoutputs.

AUGUST30,2016

Figure4.DiagramofanArtificialNeuron

Source:Infinitemusings.com/FungGlobalRetail&Technology

DEEP-LEARNINGNEURALNETWORKS

Deeplearningisnotamachine-learningapproachinitself,butratherreferstotheuseofmultiplelayersofprocessing.Deepneuralnetworkshavetwoormorehiddenlayers,asdepictedinthefigurebelow.

Figure5.DeepNeuralNetwork

Source:neuralnetworksanddeeplearning.com

APPLICATIONS

Applicationsforneuralnetsanddeeplearningcenteronimagerecognitionand processing, however the networks can be applied to any data orinformationthatcontainspatterns.Examplesinclude:

• Computervision

• Datamining

• Imageprocessing/compression

• Natural-languageprocessing

• Character/patternrecognition

Inputs

Output

Weights

ObjectiveFunction

Deeplearningisnotamachine-learningapproachinitself,butratherreferstotheuseofmultiplelayersofprocessing.Deepneuralnetworkshavetwoormorehiddenlayers.

AUGUST30,2016

• Robotics

• Stock-marketprediction

Other applications continue to be discovered, such as in medicine,identifyingodors,securityandlawenforcementandloanqualification.

Image processing and identification is a very hot topic these days, sincemany companies in the industrial, automotive and technology sectors areracing to develop autonomous vehicles, i.e., self-driving cars, that canidentifyandavoidtypicalandunforeseenobstaclesontheroad.

APPLICATIONINAUTONOMOUSVEHICLES/SELF-DRIVINGCARS

Self-driving cars need advanced image-processing and identificationplatforms inorder tonavigatearoundandavoidobjectsand impedimentsontheroad.

InadditiontotheworkofTeslaandmajorautomobilemanufacturers,manyglobal industrial, technology and software companies are pursuing thedevelopmentofself-drivingcarsandresearchingdeeplearning.

• TherearenumerousreportsthatdetailApple’sresearchandeffortstodevelopanautonomousvehicle.

• Google is developing autonomous vehicles and is documenting on itsblogthetestsitisrunningnearitsSiliconValleyheadquarters.

• Chip companies such as NVIDIA and Qualcomm are developingmachine-visionchipsforuseinautonomousvehicles.

Chinesetechnologygiantsarealsoaggressivelypursuingtheintroductionofself-drivingcars:

• OnlinemarketplaceAlibabalaunchedaventurewithChinesecarmakerSAIC todevelop Internet-enabledcars.Managementcommented theycanquicklydevelopself-drivingcarsfromtheplatform.

• ChinesewebportalBaiduhastestedaself-drivingcarontheroadsofBeijing.

Self-drivingcarsneedadvancedimage-processingandidentificationplatformsinordertonavigatearoundandavoidobjectsandimpedimentsontheroad.

Chinesetechnologygiantsarealsoaggressivelypursuingtheintroductionofself-drivingcars.

AUGUST30,2016

• TencentannouncedapartnershipwithTaiwan’sFoxconnandaChineseluxuryautodealertoexploreopportunitiesindevelopingsmartelectricvehicles.

• CarmakerChongqingChanganAutomobile Co. launchedaself-drivingcar that uses cameras and radar. The vehicle completed a 1,200-miletripinApril2016.

MARKETS

AlthoughthemarketforAIisestimatedtobefairlysmallthisyear,just$400million,marketresearchersexpectrevenuestogrowatahealthy50%+rate,andsurpass$10billionin2024,asillustratedinthegraphbelow.

Figure6.AIRevenue(USDMil.)

Source:Tractica

Intheabovegraph,weseetheAsia-PacificregionisforecasttopassNorthAmericainmarketsizein2019.

There are several adjacent markets that can benefit from AI, includingrobotics, autonomous cars, unmanned aerial vehicles and healthcare, asdepictedinthefigurebelow.

Figure7.SelectedRelatedAIMarketsin2020(USDBil.)

Source:BostonConsultingGroup/Companyreports/euRobotics/Frost&Sullivan/IHSAutomotive/InternationalFederationofRobotics/JapanRobotAssociation/MinistryofEconomy,TradeandIndustry/TealGroup/FungGlobalRetail&Technology

$5,000

$10,000

$15,000

15 16E 17E 18E 19E 20E 21E 22E 23E 24E

NorthAmerica WesternEurope EasternEurope Asia-PacificLannAmerica MiddleEast Africa

CAGR2016–24E=50.9%

$8.0 $7.5$4.8

Roboncs AutonomousCars(2025E)

UnmannedAerialVehicles

Healthcare

AlthoughthemarketforAIisestimatedtobefairlysmallthisyear,just$400million,marketresearchersexpectrevenuestogrowatahealthy50%+rate,andsurpass$10billionin2024.

ThereareseveraladjacentmarketsthatcanbenefitfromAI,includingrobotics,autonomouscars,unmannedaerialvehiclesandhealthcare.

AUGUST30,2016

STARTUPS,ACQUISITIONSANDVENTURE-CAPITALFUNDING

Startups

Thepotentialandpowerofdeeplearninghascreatedseveralstartupsthatdevelop applications. During the past five years, large technologycompanies have aggressively acquired them. At the same time, venture-capitalfinancinghasfloodedintotheindustryatarapidrate.

Severalinnovativeprivatecompaniesareoutlinedinthetablebelow.

Figure8.SelectedPrivateMachine-LearningCompanies(USDMil.)

Company Description Location TotalFunding

Sentient UsesadvancedAItechnologytosolvecomplexbusinessproblems

SanFrancisco,CA

Ayasdi Machineintelligenceplatformthathelpsorganizationsbenefitfrombigdata

MenloPark,CA

DigitalReasoning CognitiveComputingforenterprises Franklin,TN $74

Vicarious Avisualperceptionsystemthatinterpretsphotographsandvideos

SanFrancisco,CA

DataRobot Machinelearningplatformforbuildingaccuratepredictivemodels

Boston,MA $57

Source:CBInsights/Companies

Acquisitions

There has been an explosion in the acquisition of machine-learning anddeep-learningcompaniesbylargecompanies,totaling31acquisitionssince2011, according to CB Insights. The table below lists selected acquisitionsfromthatlist.

Figure9.RecentMachineLearningAcquisitions

Company Business Acquirer Date

Turi MachinelearningandAI Apple Aug.2016

MagicPony Machinelearningandvisualprocessingtechnology

Twitter Jun.2016

Itseez Computervisionandpatternrecognition

Intel May2016

Emotient Emotion-detectiontechnology Apple Jan.2016

VocalIQ Speech-processingforimprovedhuman-machineinteraction

Apple Oct.2015

Saffron Cognitivecomputingplatform Intel Oct.2015

Thepotentialandpowerofdeeplearninghascreatedseveralstartupsthatdevelopapplications.Duringthepastfiveyears,largetechnologycompanieshaveaggressivelyacquiredthem.Atthesametime,venture-capitalfinancinghasfloodedintotheindustryatarapidrate.

Therehasbeenanexplosionintheacquisitionofmachine-learninganddeep-learningcompaniesbylargecompanies,totaling31acquisitionssince2011.

AUGUST30,2016

Company Business Acquirer Date

Whetlab Technologytomakemachinelearningbetterandfaster

Twitter Jun.2015

Madbits Deeplearning-basedvisualintelligenceplatformtoidentifyimagecontent

Twitter Jul.2014

DeepMind Self-learningalgorithms Google Jan.2014

Indisys Natural-languageprocessing Intel Sep.2013

DNNResearch Deeplearningandneuralnetworksforimagesearch

Google Mar.2013

IQEngines Image-recognitionsoftware Yahoo Aug.2013

LookFlow APIforimagerecognitionandcategorization

Yahoo Oct.2013

SkyPhrase Natural-languageprocessingtechnology

Yahoo Dec.2013

DarkBlueLabs Deeplearning-basedtechnologyforunderstandingnaturallanguage

Google Oct.2014

VisionFactory Objectandtext-recognitionusingdeeplearning

Google Oct.2014

Source:CBInsights/Companies

Venture-CapitalFunding

Venture-capital fundingofAI nearlydoubled in2013andnearly tripled in2014, according to CB Insights. The amount of funds raised and dealscompletedincreasedfurtherin2015,asdepictedinthefigurebelow.

Figure10.GlobalAIVCFinancing(USDMil.)

Source:CBInsights

$282 $415$757

$2,177 $2,388

$1,000

$1,500

$2,000

$2,500

$3,000

2011 2012 2013 2014 2015

berofDealsDisclosedInvestmen

DisclosedInvestment NumberofDeals

Venture-capitalfundingofAInearlydoubledin2013andnearlytripledin2014,accordingtoCBInsights.Theamountoffundsraisedanddealscompletedincreasedfurtherin2015

AUGUST30,2016

CONCLUSION

Although deep learning and neural networks are technology buzzwordstoday, they comprise real products and services we use on a daily basis.Deep learning and neural networks use software to mimic how neuronsfunction in the human brain. They are particularly good at patternrecognitionandimageprocessingbecausetheylearnwhentheirfunctionismodified. These technologies will likely enable many global markets,particularlyautonomouscars.Manyglobalindustrialandtechnologygiantsareaggressively researchingAI,and thiscoincideswithan investmentandacquisitionboominAIstartups.

AUGUST30,2016

DeborahWeinswig,CPAManagingDirectorFungGlobalRetail&TechnologyNewYork:917.655.6790HongKong:852.6119.1779China:86.186.1420.3016deborahweinswig@fung1937.comJohnHarmon,CFASeniorAnalyst

HONGKONG:8thFloor,LiFungTower888CheungShaWanRoad,KowloonHongKongTel:85223004406LONDON:242-246MaryleboneRoadLondon,NW16JQUnitedKingdomTel:44(0)2076168988NEWYORK:1359Broadway,9thFloorNewYork,NY10018Tel:6468397017

FBICGROUP.COM

introduction to deep learning andneural networks...a neuron is illustrated below. figure 3....

Documents

perceptron (c) marcin sydow summary perceptron

neural networks - ixnisansa/classes/01... · neural...

ann perceptron

perceptron (actual lecture) · 2020. 4. 21. · perceptron...

research article a novel single neuron perceptron with...

cs407 neural computation … · perceptron the simplest...

perceptron multicapa

neural networks -...

perceptron - danushka...

the perceptron. 0t afferents v thr v rest t max 0 what does...

ia perceptron

artificial neural networks · artificial neural networks:...

linear classification: the perceptron - penn … ·...

perhitungan perceptron

practica perceptron

artificial neural networks introduction -...

perceptron - snu · •hypothesis space of a perceptron...

pre-lab exercises manuals/ap lab...b. open the illustration...

lecture04 - sl - multi-layer perceptron · dr. patrick chan...

jaringan perceptron