artificial intelligence at fermilab · to learn more: andrew ng's coursera ml course 10 but...
TRANSCRIPT
![Page 1: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/1.jpg)
João CaldeiraFermilab Users Meeting12 June 2019
Artificial Intelligence at Fermilab
![Page 2: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/2.jpg)
João CaldeiraFermilab Users Meeting12 June 2019
Wilson HAL 9000?
![Page 3: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/3.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
Artificial Intelligence (AI) or Machine Learning (ML):
- Algorithms for which rules are not given, but inferred from data
- Power and usage grew enormously in the last decade
- This is due to more powerful computers (GPU -- graphical processing units) and some recently discovered techniques
What is Artificial Intelligence?
3
![Page 4: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/4.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
AI has had a place in physics for many years
Similar systems used in Higgs discovery
AI in physics?
4
![Page 5: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/5.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
“Deep learning” has enabled many developments
Recent progress in AI
5
![Page 6: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/6.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
“Deep learning” has enabled many developments
- Games
Recent progress in AI
6
Go
Starcraft
![Page 7: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/7.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
“Deep learning” has enabled many developments
- Games
- Self-driving cars
Recent progress in AI
7
![Page 8: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/8.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
“Deep learning” has enabled many developments
- Games
- Self-driving cars
- Language translation
- Generating new samples
Recent progress in AI
8
www.thispersondoesnotexist.com
![Page 9: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/9.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab9
But what is Deep Learning?
![Page 10: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/10.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
Features get more and more complex as we move forward in the neural network.
To learn more: Andrew Ng's coursera ML course
10
But what is Deep Learning?
![Page 11: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/11.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
Hard problem: chess
AI for science: an analogy
11
![Page 12: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/12.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
Hard problem: chess
AI for science: an analogy
12
Solution: rule-based system based on human knowledge
![Page 13: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/13.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
Hard problem: chess
AI for science: an analogy
13
Solution: rule-based system based on human knowledge
Harder problem: go
![Page 14: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/14.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
Hard problem: chess
AI for science: an analogy
14
Solution: rule-based system based on human knowledge
Harder problem: goSolution: deep learning
![Page 15: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/15.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
Hard problem: chess
AI for science: an analogy
15
Solution: rule-based system based on human knowledge
Harder problem: goSolution: deep learning
Discovery in past & current instruments
Discovery in current & future instruments
![Page 16: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/16.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
Quarks and gluons interact via strong force and are never seen in isolation: jets of hadrons, measure to test Standard Model
AI for CMS
16
![Page 17: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/17.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
Identify jets from images: arXiv:1511.05190,
Quark/gluon discrimination: arXiv:1612.01551,
Particle tracking: HEP.TrkX,arXiv:1810.06111,
Fast inference on FPGA: JINST 13 P07027, arXiv:1904.08986.
AI for CMS: well-established
17
![Page 18: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/18.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
But decisions have to be fast
Deep learning works well
18
JINST 13 P07027arXiv:1904.08986
![Page 19: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/19.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
But decisions have to be fast
Developed automated tools to translate AI into firmware; Inference of networks with 1000s of parameters to run in < 100 ns!
Deep learning works well
19
JINST 13 P07027arXiv:1904.08986
![Page 20: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/20.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
Cosmic frontier: finding & measuring strong lenses
20
![Page 21: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/21.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab21
![Page 22: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/22.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
Lenses discovered in the Dark Energy Survey
22
MNRAS 484 5330
![Page 23: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/23.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
A Shifting Paradigm
23
Galaxy Quasar SNe
Today (all) 1000 <50 2
DES 2,000 120 5
LSST 120,000 8,000 120
Euclid 170,000 - -
ApJ 827 51; ApJ 811 20; ApJ 677 1046; MNRAS 405 2579
Lens type
Sam
ple
![Page 24: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/24.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
Predicting strong lens parameters
24
Can we not only find lenses, but also measure the relevant parameters?
And more importantly, can we measure uncertainties?
: statistical error
: model error
De Bom, Poh, Nord (submitted)
![Page 25: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/25.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
Predicting strong lens parameters
25
Can we not only find lenses, but also measure the relevant parameters?
And more importantly, can we measure uncertainties?
: statistical error
: model error
See also CMB lensing reconstruction: arXiv:1810.01483, blog (and poster outside)
De Bom, Poh, Nord (submitted)
![Page 26: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/26.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
Identify particles from tracks in MicroBooNE: PRD 99 092001, arXiv:1903.05663,
Similar techniques being developed in MINERvA,
Reducing simulation model bias in MINERvA: JINST 13 P11020,
NOvA event classifier: JINST 11 P09001,
NOvA energy estimator: PRD 99 012011.
AI for neutrino experiments
26
![Page 27: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/27.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
Analyze and identify particle tracks in Liquid Argon Time Projection Chambers or LArTPC.
Need adaptation for sparse data!
AI for neutrinos: MicroBooNE
27
![Page 28: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/28.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
MicroBooNE results (simulation)
28
arXiv:1903.05663
91.8% of reconstructions have purity and efficiency > 95%vs
Other methods: 80-90% purity, 2% efficiency
![Page 29: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/29.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
Can we adapt MicroBooNE code infrastructure?
- Different detector (plastic scintillator), higher energy- This means messier events
AI for neutrinos: MINERvA
29
![Page 30: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/30.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
Can we adapt MicroBooNE code infrastructure?
- Different detector (plastic scintillator), higher energy- This means messier events
AI for neutrinos: MINERvA
30
in progress (sims)
![Page 31: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/31.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
AI for neutrinos: MINERvA
31
in progress (sims)
![Page 32: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/32.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
Liquid scintillator detector. Reconstruct event energy:
Can a neural network, just from these images, return the energy of the original neutrino and of the electron shower?
AI for neutrinos: NOvA
32
PRD 99 012011
![Page 33: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/33.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
Better performance than with previous kinematic methods! (on sims)
AI for neutrinos: NOvA
33
PRD 99 012011
![Page 34: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/34.jpg)
2019-06-12 João Caldeira | Artificial Intelligence at Fermilab
- Machine learning is an exciting and crucial toolset whose utility is just starting to be explored
- Often extreme computing challenges in physics experiments need AI tools to succeed
- Needed tools on the science side can also drive development in AI! (eg, uncertainty quantification)
- What is the bottleneck in your work, and can AI help you?
Outlook
34
Nature 560 41
![Page 35: Artificial Intelligence at Fermilab · To learn more: Andrew Ng's coursera ML course 10 But what is Deep Learning? 2019-06-12 João Caldeira | Artificial Intelligence at Fermilab](https://reader036.vdocuments.mx/reader036/viewer/2022071015/5fce9baa412d0226dd34e719/html5/thumbnails/35.jpg)
João CaldeiraFermilab Users Meeting12 June 2019
Wilson Hall + HAL 9000!