computing at scale

Computing at Scale:Data Exploration

Jerjou Cheng, Barry Brumitt

HELLO

/jur'jō/ɯoɔ˙ǝlƃooƃ@noɾɹǝɾ

MY NAME IS

Developer Programs Engineer(Developer Relations)

Google Storage

Michael PIERROThttp://www.freephotobank.org/v/sky-stars/cloud/Cloud-19.jpg.html

Computing at Scale:Data Exploration

Jerjou Cheng, Barry Brumitt

Overview

• Google App Engine• Google Storage for Developers• Prediction API• BigQuery

Introductions

Who are theseservices for?

A World without Clouds

• Build a web applicationoStartup costsoMaintenance / reliabilityoScaling

Michael Scheltgenflickr.com/mscheltgen/

Google App Engine

Easy to startEasy to maintainEasy to scale

Users

gigy Socialize - traffic

Overview

• Google App Engine• Google Storage for Developers• Prediction API• BigQuery

Overview

• Google App Engine• Google Storage for Developers• Prediction API• BigQuery

A World without Clouds

• Store data• Reliability• Sharing• Large objects

Michael Scheltgenflickr.com/mscheltgen/

Google Storage for Developers

• Google infrastructure• You control access to your data• Store massive data in Google's cloud• Easy interface

Example

Internal use cases

• Content hosting  

  • Sharing

• Data Import  Google BigQuery

Google Prediction API

Overview

• Google App Engine• Google Storage for Developers• Prediction API• BigQuery

Prediction API

• Cloud-hosted machine learning as service• Simple interface over complex analysis• Predict results in real-time

How does it work?

"english" The quick brown fox jumped over the lazy dog.

"english" To err is human, but to really foul things up you need a computer.

"spanish" No hay mal que por bien no venga.

"spanish" La tercera es la vencida.

"english" To be or not to be, that is the question.

"spanish" La fe mueve montañas.

The Prediction APIfinds relevantfeatures in the sample data during training.

The Prediction APIlater searches forthose featuresduring prediction.

Prediction API

1. Upload

2. Train

Upload your training data toGoogle Storage 

Build a model from your data

Make new predictions

prediction/v1.1/training?data={}POST : a training request

prediction/v1.1/training/{}/predictGET : model infoPOST : a prediction request  

Use the API, gsutil or any compatible utility to upload your data to Google Storage

3. Predict

Example

Prediction API

• Google's machine learning algorithms• Available as RESTful HTTP service• Predict results in real-time

Overview

• Google App Engine• Google Storage for Developers• Prediction API• BigQuery

GET /information HTTP/1.0

• To request access and get more information, go to:o http://code.google.com/appengineo http://code.google.com/apis/bigqueryo http://code.google.com/apis/predicto http://code.google.com/apis/storage