reinforcement learning in starcraft 2

REINFORCEMENT LEARNING IN STARCRAFT 2 Björn Boyd Isacsson

Upload: doanmien

Post on 01-Jan-2017

235 views

Category:

Documents

4 download

Report

Download

Embed Size (px):

TRANSCRIPT

REINFORCEMENT

LEARNING IN

STARCRAFT 2Björn Boyd Isacsson

AI IN GAMES

When is it used

Improvement needed

Players are too good

Difficult and time-consuming creation

General only

STARCRAFT 2

Why?

What is StarCraft 2?

AI in StarCraft

Default

Human-made

Galaxy Script

REINFORCEMENT LEARNING

Why?

Games

Galaxy limitations

What is Reinforcement Learning?

REINFORCEMENT LEARNING

THE BASICS

States and Actions

Reward-based

Iterations

Optimisation vs Exploration

Put it all together!

Problems?

REINFORCEMENT LEARNING

FIXING THE PROBLEMS

Continuous gameplay

Sampling

Lots of actions

Limiting the actions

“Infinite” states

Table → Function

Q[s][a] → Q(s, a) = α + β * x1 + γ * x2 + …

Find values for α, β, γ…

Page 7: REINFORCEMENT LEARNING IN STARCRAFT 2

THE AI IMPLEMENTATION

Sample time

0.1 seconds

Actions

Move left

Move right

Attack closest target

Function

Distance

Reinforcement Learning AI Health

Enemy Health

Action chosen

Page 8: REINFORCEMENT LEARNING IN STARCRAFT 2

MY OPTIMAL

Page 9: REINFORCEMENT LEARNING IN STARCRAFT 2

ONE UNIT – 10 ITERATIONS

Page 10: REINFORCEMENT LEARNING IN STARCRAFT 2

ONE UNIT – 100 ITERATIONS

Page 11: REINFORCEMENT LEARNING IN STARCRAFT 2

MANY UNITS

Implementing many units

One AI

Multiple AIs

Page 12: REINFORCEMENT LEARNING IN STARCRAFT 2

MANY UNITS – 100 ITERATIONS

Page 13: REINFORCEMENT LEARNING IN STARCRAFT 2

MANY UNITS – 1000 ITERATIONS

LIMITATIONS

Reinforcement Learning

Requires instant reward

Many iterations = Lots of time

No guaranteed optimal

Galaxy Script

Memory!

Page 15: REINFORCEMENT LEARNING IN STARCRAFT 2

CONCLUSION

There is potential

Improvements could expand the possibilities

Doesn’t fit every scenario

Reinforcement Learning Das Reinforcement Learning-Problem Alexander Schmid

Reinforcement Learning for the Beginning of Starcraft II Game

On Reinforcement Learning for Full-length Game of StarCraft

StarCraft II as an Environment for Artificial Intelligence ...on-demand.gputechconf.com/gtc/2018/...learning-with-starcraft-II.pdf · Timo Ewalds - DeepMind Chris Lee - Blizzard StarCraft

Multi-Objective Reinforcement Learning using Sets of Pareto … · 2020. 10. 19. · learning and multi-objective reinforcement learning. 2.1 Reinforcement Learning A reinforcement

Deep Reinforcement Learning - Environments Tour€¦ · Deep Reinforcement Learning - Environments Tour Machine Learning Gdańsk, 02.10.2017 - Adam Wróbel Starcraft 2 DeepMind toolset

Reinforcement Learning and Deep Reinforcement Learningcse.ucdenver.edu/.../Class-22-Reinforcement-learning-DL.pdf · 2018. 11. 28. · Outlines 1 Principles of Reinforcement Learning

Reinforcement Learning - Multi-Agent Reinforcement

Reinforcement Learning Lecture Inverse Reinforcement Learningipvs.informatik.uni-stuttgart.de/mlr/wp-content/uploads/2017/07/09... · Reinforcement Learning Inverse Reinforcement

Bayesian Reinforcement Learning - mlg.eng.cam.ac.ukmlg.eng.cam.ac.uk/rowan/files/BayesianReinforcementLearning.pdf · Introduction Bayesian Reinforcement Learning Bayesian Reinforcement

Playing Starcraft with Deep Q-learning+nature

Introduction to Reinforcement Learning · reinforcement learning are able to keep up and even surpass humans in real-time in games as complex go and starcraft, which was said to be

Deep Learning for Reinforcement Learning in · PDF fileDeep Learning for Reinforcement Learning in ... Deep Learning for Reinforcement Learning in Pacman Deep Learning für ... Während

Replicating DeepMind StarCraft II Reinforcement Learning ......Replicating DeepMind StarCraft II Reinforcement Learning Benchmark with Actor-Critic Methods Bachelor’sthesis RomanRing

Cooperative Inverse Reinforcement Learning...Cooperative Inverse Reinforcement Learning Dylan Hadfield-Menell CS237: Reinforcement Learning May 31, 2017

Reinforcement Learning - 4. Model-free reinforcement Learning

From Reinforcement Learning to Deep Reinforcement …fagostin/assets/files/...Keywords: Machine learning · Reinforcement learning Deep learning · Deep reinforcement learning 1 Introduction

Reinforcement Learning

Deep Learning for Reinforcement Learning in Pacman · Deep Learning for Reinforcement Learning in Pacman Deep Learning für Reinforcement Learning in Pacman Vorgelegte Bachelor-Thesis

Reinforcement Learning - uni-freiburg.degki.informatik.uni-freiburg.de/.../recordings/reinforcement.pdf · Reinforcement Learning 3 What is Reinforcement Learning? Learning from interaction

Reinforcement Learning or Active Inference?karl/Reinforcement Learning or Active... · Reinforcement Learning or Active Inference? ... From the point of view of reinforcement learning

Reinforcement Learning Chapter 13 What is Reinforcement Learning? Q-Learning Examples 1

Reinforcement Learning Introduction Passive Reinforcement Learning Temporal Difference Learning Active Reinforcement Learning Applications Summary

Replicating DeepMind StarCraft II Reinforcement Learning

Tutorial: Deep Reinforcement Learning - Machine Learning ...hunch.net/~beygel/deep_rl_tutorial.pdfTutorial: Deep Reinforcement Learning - Machine Learning

Reinforcement Learning: Learning algorithms

Generalization in Reinforcement Learning: Successful ...papers.nips.cc/paper/1109-generalization-in-reinforcement-learning... · Generalization in Reinforcement Learning: Successful

Adaptive High-Level Strategy Learning in StarCraft

Inverse Reinforcement Learning - Peoplecbfinn/_files/bootcamp_inverserl.pdf · Apprenticeship Learning via Inverse Reinforcement Learning. Good introduction to inverse reinforcement

Reinforcement Learning & Apprenticeship Learning

Eick: Reinforcement Learning. Reinforcement Learning Introduction Passive Reinforcement Learning Temporal Difference Learning Active Reinforcement Learning

Inverse Reinforcement Learning CS885 Reinforcement