![Page 1: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/1.jpg)
TensorFlow
![Page 2: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/2.jpg)
Agenda● Introduction ● Machine Learning● Artificial Neural Networks● TensorFlow Basics● Image Recognition Example● Conclusion
![Page 3: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/3.jpg)
Introduction● Open source library by Google for Machine Learning
● Successor to DistBelief made by Google Brain
● Scalability and portability core feature.
● Main uses are pattern recognition using Artificial Neural Networks
● Large community with wide range of applications internally and outside
![Page 4: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/4.jpg)
Machine Learning (ML)
● Application of Artificial Intelligence (AI)AI - Broader concept, Machines perform “intelligent” tasksML - Give machines data and make them learn by themselves.
● ML : "A Field of study that gives computers the ability to learn without being explicitly programmed.” - Arthur Samuel, 1959
● Requires a large amount of data and computational power
![Page 5: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/5.jpg)
Different ML tasks SUPERVISED LEARNING
UNSUPERVISED LEARNING
REINFORCEMENT LEARNING
![Page 6: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/6.jpg)
Artificial Neural Networks● Network Architecture
● Forward propagation
● Node activation / how they work
● Mathematical model of Neural network
● Gradient computation
![Page 7: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/7.jpg)
Network Architecture
![Page 8: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/8.jpg)
Forward propagation
![Page 9: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/9.jpg)
Mathematical model
![Page 10: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/10.jpg)
Gradient computation
![Page 11: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/11.jpg)
TensorFlow Basics- Tensors- Computational graph and Sessions- TensorBoard- Placeholders- Variables- Training and Optimization
![Page 12: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/12.jpg)
TensorFlow BasicsWhat is a Tensor?
A multidimensional array Different * ranks* types
Rank Math entity
0 Scalar
1 Vector
2 Matrix
3 3-tensor
n n-tensor
![Page 13: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/13.jpg)
Computational graph and Sessions Computational graph- series of TensorFlow opts / nodes arranged into a graph
Session
- for graph evaluation
![Page 14: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/14.jpg)
TensorBoard● interactive visualization tool
![Page 15: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/15.jpg)
Add name and name scopes for better readability
![Page 16: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/16.jpg)
Placeholders import tensorflow as tf
# create placeholder x = tf.placeholder(dtype=tf.float32)
# define session object in order to evaluate sess = tf.Session()
# run and print place holderprint(sess.run(x)) # will fail since x is not provided with values
# make random numbers with numpy, 4X4 tensorrand_array = np.random.rand(4,4)
print(sess.run(x,feed_dict={x: rand_array})) # will work
● Must be provided with values at a later stage
![Page 17: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/17.jpg)
Variables
● trainable parameters● initial value and explicitly initialized
![Page 18: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/18.jpg)
Training● Adjust the Variables in our model to minimize a cost function● tf.train choose optimization algorithm● Base Class : Optimizer
- provides methods to compute gradients● GradientDecentOptimizer
![Page 19: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/19.jpg)
Image recognition example● Digit recognition example
● Use MNIST database for training and validation data
● Example code
● Showcase model
![Page 20: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/20.jpg)
The MNIST database
![Page 21: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/21.jpg)
![Page 22: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/22.jpg)
![Page 23: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/23.jpg)
![Page 24: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/24.jpg)
![Page 25: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/25.jpg)
![Page 26: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/26.jpg)
![Page 27: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/27.jpg)
First 500 learning steps
![Page 28: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/28.jpg)
10 000 training steps
![Page 29: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/29.jpg)
Final 100 training steps
![Page 30: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/30.jpg)
10 000 training steps
![Page 31: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/31.jpg)
Last 100 training stepsAccuracy plotted for different hidden layers
![Page 32: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/32.jpg)
Final 100 training steps, for more nodes
![Page 33: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/33.jpg)
Different batch sizes
![Page 34: TensorFlow - NTNUfolk.ntnu.no/preisig/HAP_Specials/AdvancedSimulation_files/2017/pr… · What is a Tensor? A multidimensional array Different * ranks * types Rank Math entity 0 Scalar](https://reader033.vdocuments.mx/reader033/viewer/2022043022/5f3e0ee8e1ac1c7025302ab1/html5/thumbnails/34.jpg)
Different numbers of hidden layers