![Page 1: K-armed Bandit Livio Torrero,Olivier Morandi, Pierluigi Rolando,Riccardo Giacomelli](https://reader033.vdocuments.mx/reader033/viewer/2022061304/5513b8df55034646298b45df/html5/thumbnails/1.jpg)
K-armed Bandit
Livio Torrero,Olivier Morandi, Pierluigi Rolando,Riccardo Giacomelli
![Page 2: K-armed Bandit Livio Torrero,Olivier Morandi, Pierluigi Rolando,Riccardo Giacomelli](https://reader033.vdocuments.mx/reader033/viewer/2022061304/5513b8df55034646298b45df/html5/thumbnails/2.jpg)
K-armed Bandit
K slot machines stocastiche (Gaussian)Mean reward Standard deviation
2000 actions per apprendere quale sia la slot machine migliore
Come fare?
![Page 3: K-armed Bandit Livio Torrero,Olivier Morandi, Pierluigi Rolando,Riccardo Giacomelli](https://reader033.vdocuments.mx/reader033/viewer/2022061304/5513b8df55034646298b45df/html5/thumbnails/3.jpg)
K-armed Bandit
Strategie Greedy
Scelgo strategia migliore stimata con probabilità
Scelgo una strategia tra le altre con probabilità uniforme con probabilità
1
![Page 4: K-armed Bandit Livio Torrero,Olivier Morandi, Pierluigi Rolando,Riccardo Giacomelli](https://reader033.vdocuments.mx/reader033/viewer/2022061304/5513b8df55034646298b45df/html5/thumbnails/4.jpg)
Test-1
Mean rewards statici (Gaussian)Varianza=1Stima del reward:
11 1 1
t Nt t
Q rQ Q
N N
![Page 5: K-armed Bandit Livio Torrero,Olivier Morandi, Pierluigi Rolando,Riccardo Giacomelli](https://reader033.vdocuments.mx/reader033/viewer/2022061304/5513b8df55034646298b45df/html5/thumbnails/5.jpg)
Test-1
![Page 6: K-armed Bandit Livio Torrero,Olivier Morandi, Pierluigi Rolando,Riccardo Giacomelli](https://reader033.vdocuments.mx/reader033/viewer/2022061304/5513b8df55034646298b45df/html5/thumbnails/6.jpg)
Test-1
![Page 7: K-armed Bandit Livio Torrero,Olivier Morandi, Pierluigi Rolando,Riccardo Giacomelli](https://reader033.vdocuments.mx/reader033/viewer/2022061304/5513b8df55034646298b45df/html5/thumbnails/7.jpg)
Test-2b (varianza=0)
![Page 8: K-armed Bandit Livio Torrero,Olivier Morandi, Pierluigi Rolando,Riccardo Giacomelli](https://reader033.vdocuments.mx/reader033/viewer/2022061304/5513b8df55034646298b45df/html5/thumbnails/8.jpg)
Test-2a (varianza=10)
![Page 9: K-armed Bandit Livio Torrero,Olivier Morandi, Pierluigi Rolando,Riccardo Giacomelli](https://reader033.vdocuments.mx/reader033/viewer/2022061304/5513b8df55034646298b45df/html5/thumbnails/9.jpg)
Test-3
Stima del reward
1 1*t t t tQ Q LR r Q
![Page 10: K-armed Bandit Livio Torrero,Olivier Morandi, Pierluigi Rolando,Riccardo Giacomelli](https://reader033.vdocuments.mx/reader033/viewer/2022061304/5513b8df55034646298b45df/html5/thumbnails/10.jpg)
Test-3a (LR=0.9,variance=0)
![Page 11: K-armed Bandit Livio Torrero,Olivier Morandi, Pierluigi Rolando,Riccardo Giacomelli](https://reader033.vdocuments.mx/reader033/viewer/2022061304/5513b8df55034646298b45df/html5/thumbnails/11.jpg)
Test-3b (LR=0.9,variance=10)
![Page 12: K-armed Bandit Livio Torrero,Olivier Morandi, Pierluigi Rolando,Riccardo Giacomelli](https://reader033.vdocuments.mx/reader033/viewer/2022061304/5513b8df55034646298b45df/html5/thumbnails/12.jpg)
Test-4
Stima del reward
All’azione numero 300: I valori dei rewards cambiano
1 1*t t t tQ Q LR r Q
![Page 13: K-armed Bandit Livio Torrero,Olivier Morandi, Pierluigi Rolando,Riccardo Giacomelli](https://reader033.vdocuments.mx/reader033/viewer/2022061304/5513b8df55034646298b45df/html5/thumbnails/13.jpg)
Test-4a, (step=0.05)
![Page 14: K-armed Bandit Livio Torrero,Olivier Morandi, Pierluigi Rolando,Riccardo Giacomelli](https://reader033.vdocuments.mx/reader033/viewer/2022061304/5513b8df55034646298b45df/html5/thumbnails/14.jpg)
Test-4a (LR=0.1)
![Page 15: K-armed Bandit Livio Torrero,Olivier Morandi, Pierluigi Rolando,Riccardo Giacomelli](https://reader033.vdocuments.mx/reader033/viewer/2022061304/5513b8df55034646298b45df/html5/thumbnails/15.jpg)
Test-4a (LR=0.5)
![Page 16: K-armed Bandit Livio Torrero,Olivier Morandi, Pierluigi Rolando,Riccardo Giacomelli](https://reader033.vdocuments.mx/reader033/viewer/2022061304/5513b8df55034646298b45df/html5/thumbnails/16.jpg)
Test-4a (LR=0.9)
![Page 17: K-armed Bandit Livio Torrero,Olivier Morandi, Pierluigi Rolando,Riccardo Giacomelli](https://reader033.vdocuments.mx/reader033/viewer/2022061304/5513b8df55034646298b45df/html5/thumbnails/17.jpg)
Test-4b (step=0.1)
![Page 18: K-armed Bandit Livio Torrero,Olivier Morandi, Pierluigi Rolando,Riccardo Giacomelli](https://reader033.vdocuments.mx/reader033/viewer/2022061304/5513b8df55034646298b45df/html5/thumbnails/18.jpg)
Test-4b (LR=0.1)
![Page 19: K-armed Bandit Livio Torrero,Olivier Morandi, Pierluigi Rolando,Riccardo Giacomelli](https://reader033.vdocuments.mx/reader033/viewer/2022061304/5513b8df55034646298b45df/html5/thumbnails/19.jpg)
Test-4c (immediate)
![Page 20: K-armed Bandit Livio Torrero,Olivier Morandi, Pierluigi Rolando,Riccardo Giacomelli](https://reader033.vdocuments.mx/reader033/viewer/2022061304/5513b8df55034646298b45df/html5/thumbnails/20.jpg)
Test-4b (LR=0.1)
![Page 21: K-armed Bandit Livio Torrero,Olivier Morandi, Pierluigi Rolando,Riccardo Giacomelli](https://reader033.vdocuments.mx/reader033/viewer/2022061304/5513b8df55034646298b45df/html5/thumbnails/21.jpg)
Test-4b (LR=0.9)