© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

김일호, Solutions Architect

17-May-2016

Amazon Machine Learning 게임에서활용해보기

Agenda

Machine Learning이란?

Machine Learning 적용예제(아이템이벤트타겟유저)

Amazon Machine Learning (Amazon ML) 특징,장점

Demo

Amazon Machine Learning 활용아키텍쳐

데이터기반개발의세가지방법

Retrospective

분석또는보고

Amazon Redshift,

Amazon RDS

Amazon S3

Amazon EMR

데이터기반개발의세가지방법

Retrospective

분석또는보고Here-and-now

실시간분석/처리또는대쉬보드

Amazon Kinesis

Amazon EC2

AWS Lambda

Amazon Redshift,

Amazon RDS

Amazon S3

Amazon EMR

데이터기반개발의세가지방법

Retrospective

분석또는보고Here-and-now

실시간분석/처리또는대쉬보드

Predictions

보다스마트한서비스개발

Amazon Kinesis

Amazon EC2

AWS Lambda

Amazon Redshift,

Amazon RDS

Amazon S3

Amazon EMR

Machine learning,보다스마트한서비스

Machine Learning 은데이터에서새로운패턴과의미를발견하고,그예측된값을새로운데이더로활용할수있다.

Machine learning,보다스마트한서비스

Machine Learning 은데이터에서새로운패턴과의미를발견하고,그예측된값을새로운데이더로활용할수있다.

Your data + machine learning = smart applications

Your data + machine learning = +

Machine Learning 를이용한예제

Based on what you

know about the user:

이아이템에사용자가정말매력을느낄까?

Based on what you

know about an order:

전에비슷한아이템을구매했는가?

Based on what you know

about a news article:

이사용자는다른정보가아이템구매와관련이있을까?

Machine Learning 적용아이디어

Fraud detection 정상적인 Transaction인가?,의심가는사용자패턴인식,

위험사용자태깅…

Personalization 유져가좋아할만한아이템,미션추천…

Targeted marketing 유져마다각각다른이벤트제안,다른형식의마케팅진행,크로스셀링,업셀링…

Content classification 개인에맞는스킬셋,장비셋제안…

Churn prediction 예상탈퇴또는미사용자검색, 공짜에서유료사용자로전환타이밍감지…

Customer support 고객이메일,게시판등에서예상될이슈감지및경고…

”이번이벤트는장비셋으로합시다”

모든고객님,

어버이날감사이벤트

$49.99장비셋

….

…

타겟유저를뽑아봅시다.

SELECT c.ID

FROM customers c

LEFT JOIN orders o

ON c.ID = o.customer

GROUP BY c.ID

HAVING o.date > GETDATE() – 30

최근 30일내에아이템을구매한유저분들에게제안한다.

조금좁혀서타겟유저를뽑아봅시다.

SELECT c.ID

FROM customers c

LEFT JOIN orders o

ON c.ID = o.customer

GROUP BY c.ID

HAVING

AND o.date > GETDATE() – 30

무기외에방어구만구매한유저만을뽑아본다.

더좁혀서타겟유저를뽑아봅시다.

SELECT c.ID

FROM customers c

LEFT JOIN orders o

ON c.ID = o.customer

GROUP BY c.ID

HAVING o.category = ‘ARMOR’

AND

(COUNT(*) > 2

AND SUM(o.price) > 200

AND o.date > GETDATE() – 30)

)

방어구중에최근두달간갑옷종류를구매한유저만을뽑아본다….

더좁혀서타겟유저를뽑아봅시다.

SELECT c.ID

FROM customers c

LEFT JOIN orders o

ON c.ID = o.customer

LEFT JOIN products p

ON p.ID = o.product

GROUP BY c.ID

HAVING o.category = ‘toys’

AND ((p.description LIKE ‘% %’

AND o.date > GETDATE() - 60)

OR (COUNT(*) > 2

AND SUM(o.price) > 200

AND o.date > GETDATE() – 30)

)

부츠는?헬멧은?

쿼리조건을다시추가…?

‘% %’

‘% %’

타겟유저가너무적다.더뽑아봅시다.

SELECT c.ID

FROM customers c

LEFT JOIN orders o

ON c.ID = o.customer

LEFT JOIN products p

ON p.ID = o.product

GROUP BY c.ID

HAVING o.category = ‘ARMOR’

AND ((p.description LIKE ‘%PLATE%’

AND o.date > GETDATE() - )

OR (COUNT(*) > 2

AND SUM(o.price) > 200

AND o.date > GETDATE() – 30)

)

60일이아니라 120일전기록까지뽑아본다.

타겟유저가너무적다.더뽑아봅시다.

SELECT c.ID

FROM customers c

LEFT JOIN orders o

ON c.ID = o.customer

LEFT JOIN products p

ON p.ID = o.product

GROUP BY c.ID

HAVING o.category = ‘ARMOR’

AND ((p.description LIKE ‘%PLATE%’

AND o.date > GETDATE() - 120)

OR (COUNT(*) > 2

AND SUM(o.price) > 200

AND o.date > GETDATE() – )

)

$200이상유저는30일에서40일전까지…

타겟유저가정말구입가능성이높은유저인가?

SELECT c.ID

FROM customers c

LEFT JOIN orders o

ON c.ID = o.customer

LEFT JOIN products p

ON p.ID = o.product

GROUP BY c.ID

HAVING o.category = ‘ARMOR’

AND ((p.description LIKE ‘%PLATE%’

AND o.date > GETDATE() - )

OR (COUNT(*) > 2

AND SUM(o.price) > 150

AND o.date > GETDATE() – 40)

)

누군가의미있는연관성과패턴을찾아구매가능성이가장높은유저리스트를뽑아주면정말좋겠다!

Machine Learning!

Machine Learning적용이쉬운문제일까?

1. Machine learning 전문가가적다.

2. 확장가능한 Machine learning 아키텍쳐를구성하기어렵다

3. 모델링과서비스를연동하는게많은비용과시간이든다.

Machine Learning을쉽게적용할수있는방법이없을까?

Amazon Machine Learning

• 간단히생성,쉬운사용법

• AWS에서관리하는제공하는 Machine Learning 서비스

• API로개발자도쉽게사용

• Amazon 내부의시스템위에서빠른 Machine Learning 기술사용

• 이미 AWS 에저장된데이터를바로사용하여데이터모델링

• 바로 Production으로사용할수있도록배포(Endpoint)

개발자들이쉽게접근하여사용가능

쉽게접근및사용할수있는 Console로직접 ML 모델링

• 데이터소스지정및생성

• ML 모델생성, 모델품질측정, 튜닝

• 배포및관리

자동화된관리와 API, SDK를이용하여쉽게활용• Java, Python, .NET, JavaScript, Ruby, Javascript

AWS Mobile SDK를통해 Android, iOS 앱에서도쉽게사용가능

신뢰할수있는 machine learning technology

Amazon 내부의 Data Scientist들이사용한기술과동일하고확장성이높은입증된 Machine Learning을기술을기반으로합니다.

알고리즘외의편리한기능:• 간단한데이터타입변환,스키마생성• 원본데이터와모델에대한품질평가

확장가능한 Pay as use 모델• 100GB데이터모델링가능• 배치를통한예측지원• 실시간예측지원

AWS의다른서비스과연계

S3에저장된데이터를직접접근하여활용하며,

Amazon Redshift에저장된데이터를편리하게가져와모델링가능

예측결과파일을간단히 S3에저장하여활용가능

AWS Identity and Access Management (IAM)를활용하여서비스에접근권한제어가능

AWS에서관리하고제공하는서비스

직접 Machine learning을위한시스템을관리하거나배포할필요가없음

클릭몇번으로 Production 배포가능

API를통해모델링과쿼리에대한Workflow자동화가능

Amazon CloudWatch를통한모니터링제공

Buildmodel

Evaluate andoptimize

Retrieve predictions

1 2 3

Amazon Machine Learning사용하기

Trainmodel

Evaluate andoptimize

Retrieve predictions

1 2 3

Amazon Machine Learning사용하기 1

- 데이터소스로부터 DataSource생성- 데이터내용과스키마확인- 모델링진행

데이터로부터 Datasource생성

>>> import boto

>>> ml = boto.connect_machinelearning()

>>> ds = ml.create_data_source_from_s3(

data_source_id = ’my_datasource',

data_spec= {

'DataLocationS3':'s3://bucket/input/',

'DataSchemaLocationS3':'s3://bucket/input/.schema'},

compute_statistics = True)

데이터내용과스키마확인

데이터모델생성

>>> import boto

>>> ml = boto.connect_machinelearning()

>>> model = ml.create_ml_model(

ml_model_id=’my_model',

ml_model_type='REGRESSION',

training_data_source_id='my_datasource')

Trainmodel

Evaluate andoptimize

Retrieve predictions

1 2 3

Amazon Machine Learning사용하기 2

- 생성된모델의품질확인- 모델에해석상세조절

모델결과및품질확인

모델해석상세조절

모델해석상세조절

Trainmodel

Evaluate andoptimize

Retrieve predictions

1 2 3

Amazon Machine Learning사용하기 3

- 배치를통한예측- 실시간예측

배치(Batch)방식예측

비동기적으로대량의예측결과필요시사용

Console, API를통해요청가능

많은양의데이터를한번에처리하여결과를 S3에저장

>>> import boto

>>> ml = boto.connect_machinelearning()

>>> model = ml.create_batch_prediction(

batch_prediction_id = 'my_batch_prediction’

batch_prediction_data_source_id = ’my_datasource’

ml_model_id = ’my_model',

output_uri = 's3://examplebucket/output/’)

실시간(Real-time)예측

동기적으로빠르게예측결과를바로사용가능

API, SDK를통해요청

각각의데이터를실시간으로서비스에서요청하여사용

>>> import boto

>>> ml = boto.connect_machinelearning()

>>> ml.predict(

ml_model_id=’my_model',

predict_endpoint=’example_endpoint’,

record={’key1':’value1’, ’key2':’value2’})

{

'Prediction': {

'predictedValue': 13.284348,

'details': {

'Algorithm': 'SGD',

'PredictiveModelType': 'REGRESSION’

}

}

}

Demo

“과연사용자가이벤트결제를할까?”

“과연어떤요소가결제에영향을줄까?”

Amazon Machine Learning

활용아키텍쳐

Query for predictions with

Amazon ML batch API

Process data

with EMR

Raw data in S3Aggregated data

in S3

Predictions

in S3 Your Game

EMR을같이이용한배치방식

Query for predictions with

Amazon ML batch API

IDCAggregated data

in S3

Predictions

in S3

Your Game

IDC의데이터를 S3로저장후활용

DashBoard

Structured dataIn Amazon Redshift

Load predictions into Amazon Redshift

-or-Read prediction results

directly from S3

Predictions

in S3

Query for predictions with

Amazon ML batch API

Your Game

Amazon Redshift를활용한배치프로세싱

Your Games

Query for predictions with

Amazon ML real-time API

API를이용한실시간예측

Amazon

DynamoDB

+

Trigger event with Lambda

+

Query for predictions with

Amazon ML real-time API

Your Games

DynamoDB의데이터를 Source로실시간예측

Thank you!