CIS 191: Linux and Unix

Class 7March 25, 2015

Practical Cloud Solutions

Outline

Cloud Computing Overview

Amazon AWS Services

A Very Brief HTTP Overview

REST APIs

JSON Format

What is Cloud Computing?

What is Cloud Computing?

• Generally a system of programming on a computer that exists somewhere else

• Possibly (and often!) not even really a computer, but a virtual slot on an existing server

What is Cloud Computing?

• Examples:

What is Cloud Computing?

• Examples:– Amazon AWS Elastic Compute Cloud (EC2)– Google App Engine– Microsoft Azure– Heroku app hosting– Eniac!

Cloud Computing – An Analogy

• Power sources– Everybody used to have their own!– Inconvenient…

Cloud Computing – An Analogy

• Power sources– Everybody used to have their own!– Inconvenient…

• So we scaled and built centralized power plants– Metered usage!– Outsource the problem of building, running, maintaining

generators– Pay-as-you-go model

Why Put Computers in the Cloud?

• Cost effective• Quick prototyping• Little to zero maintenance• Redundancy• Device independence• Easier collaboration

Outline

Cloud Computing Overview

Amazon AWS Services

A Very Brief HTTP Overview

REST APIs

JSON Format

Amazon Web Services

• A number of services– Amazon Elactic Compute Cloud (EC2) – Virtual machines– Amazon Simple Storage Service (S3) – Key-value store– Amazon SimpleDB – Simple distributed database– Amazon Relational Database System (RDS) – Full virtual

distributed database system of your choosing– Amazon Mechanical Turk (MTurk) – Pay pennies for labor (or get

paid pennies for labor)– And much much more!

Amazon AWS Account

• To use Amazon web services, you have to first make an AWS account at aws.amazon.com

• When you do this, you have to input a credit card number they can charge to – don’t worry, you won’t be charged as long as you follow some basic guidelines– And even if you were charged it would probably be a dollar or

two at most

• Many AWS services offer a “free tier” for the first year– This tier tends to be super slow and limited, but good enough

for our purposes

Zooming in: Amazon EC2

• After making an account on Amazon AWS, you can set up your very own EC2 instance– You can even pick the size!

• To do this, you must select “EC2” from the AWS management console, and then select some preferences– I’d suggest selecting the free tier, which is a “micro” instance– Yes it’s as small and limited as it sounds – but that’s fine!– I’d also suggest selecting an ubuntu operating system, since

that’s what we’ve been working with in class

Zooming in: Amazon EC2

• You’ll be asked to download a .pem security key; put this in a safe place

• After downloading the key, you’ll be able to log in to your ec2 instance by running – ssh –i my_key.pem ubuntu@ec2_IP_address

• You can get the IP address by looking at the instance’s details

Setting up EC2

• Follow the same procedure as early 191!• sudo apt-get install ...• Exactly the same as working in the virtual machine

command line!

Security Groups

• By default, your EC2 will only be accessible by you, over ssh– By using the pem key file and ssh -i!

• If you want to host a website, you’ll need to explicitly allow this in security groups by allowing access by everyone to port 80– This is the default internet port

Aside: Ports

• A port on your computer is like a sub-IP address (sort of)• These allow requesters to target particular applications

or services running on a machine!• So, web server applications tend to be running on port

80 of a larger server

Pricing Issues

• Make sure to run the micro instance if you don’t want to be charged!

• Make sure to terminate your instance when you are finished with it!– Shutting down isn’t the same…– Access this by right clicking on the instance in the console!

• Double check you don’t have any more resources allocated when you are finished using EC2

Data Issues – Don’t Lose It!

• If you store data on the virtual hard disk of an instance and the instance stops running for some reason– You terminate it– It fails (rare but it happens!)

• You will LOSE YOUR DATA!!!• If you want more security, you can use Elastic Block Store

– This is basically data replication across EC2 instances, for an extra fee

– This option will ensure data safety

Where Exactly Is My Instance?

• Who knows?• All you know is the general region in which the instance

runs, which you can request• Generally, you should pick an instance near where your

target audience is (in this case, us-east)– Reduce network latency as much as possible

Outline

Cloud Computing Overview

Amazon AWS Services

A Very Brief HTTP Overview

REST APIs

JSON Format

What is HTTP, anyway?

• We use it every day, but it’s more than just the http:// at the beginning of a web address

• HTTP specifies a protocol for sending data• This is really just an agreement between two machines,

and it allows for a friendly discourse between them

HyperText Transfer Protocol (HTTP)

• First specified in 1990, and it runs on top of TCP/IP– This ensures that we are already assured that data will reach the

target machine, and the packets we send will arrive in-order

• Default machine port is 80 for unsecure HTTP and 443 for secure (with secure socket layer technology – “Secure” basically means RSA encrypted traffic

• The protocol is actually very simple!– Choice: 1.0 (old and dumb) or 1.1 (new and shiny)– Always go for 1.1

GET /index.html HTTP/1.1Host: google.com

Basic “Methods” for HTTP – To Server

• GET– Retrieve contents of a URI

• HEAD– Retrieve only metadata of a URI

• PUT– Store information at a URI

• DELETE– Delete information at a URI

• POST– Adds new information– Intended for newsgroup posts; today, used to implement dynamic

content through web forms

HTTP Parameters

• HTTP can carry key-value pairs to the server!– Very useful for communication between client and server

• This typically look like – key=value

• These pairs can either be in the request URI, or in the request body

• The request body follows all http headers and will generally carry key/value pairs– Sometimes carries more information than this, however! Up to

the server how to interpret it– Useful as a communication standard… More on this soon

Zooming In: GET

• This is what your browser performs when you visit a page!

• You can add GET parameters (which the server can read) right to your URI– http://mysite.com/?myname=Spencer&mypet=Mochi

Zooming in: POST

• This is what your browser sends when you submit a form• For example, when you place an order at Amazon.com• Information can be sent via GET requests as well, but

POST is generally more secure– GET variables are included directly in the URI!– POST information is not stored by the browser or cached by the

server – GET requests are, for faster loading and response time– POST also has no restrictions on data length!– In general, POST when you want to send information to be

stored at a website, especially sensitive data

Zooming in: POST

• Also used to add something new to a page• For example, one might add a blog entry using the POST

request format• Especially useful if you want to post a new blog entry,

and don’t care what the resulting identifier is– I want to add a NEW post

Zooming in: POST

POST /order.html HTTP/1.1Host: www.my-shopper.comContent-Length: 114Content-Type: text/html

item1=6173849&item2=7272384&item3=32904234&password=monkeyfarts&username=doodooface&creditcardno=5555555555555555

Zooming in: PUT

• Used to update information at a URI• Pretty much the same as POST, except that the user

names the resource to be saved– Since you are naming an existing resource, you are updating it!

• Can also do different things depending on context…– But this is what it was intended for!

Zooming in: PUT

PUT /blogentries?id=2012 HTTP/1.1Host: www.my-site.comContent-Length: 35Content-Type:Text.html

Today was a terrible day because…

HTTP Status Codes – From Server

• Reports how the request was processed• Some common codes (way too many to list here)

– 200 OK– 301 Moved Permanently– 401 Unauthorized– 403 Forbidden– 404 Not Found– 500 Internal Server Error

• This last one is just a catch-all

HTTP Headers

• Included underneath an HTTP request• Specify more information for your request• These can allow you to set cookies, specify your current

browser, the types of charsets you will accept…

GET /index.html HTTP/1.1Host: 158.130.53.72:80Accept: text/html, application/xhtmlConnection: keep-alive

HTTP Headers

• An important header to remember is the Content-Length header

• This is used when a request (typically post or put) has a body!

HTTP Responses• Sent back to you by the server after you request a page• Include the protocol version and the status code, along with

some headers (additional information) and the content you requested– If you sent a HEAD request, just the headers are returned

HTTP/1.1 200 OKDate: …Server: ……

<html><head><title>…

Media (MIME) Types

• Specify nature of data requested or transferred– Defined in RFC 2046– Included in ‘Content-Type’ header– Includes at least type and subtype

• Examples– text/plain– image/png– image/jpeg– video/quicktime

Telnet

• You can connect to a service via telnet and play around

telnet google.com 80

Outline

Cloud Computing Overview

Amazon AWS Services

A Very Brief HTTP Overview

REST APIs

JSON Format

What is REST?

• Something you should get at night!

What is REST?

• Something you should get at night!– JK

What is REST?

• Something you should get at night!– JK

• REST is a type of Web Service protocol– This allows you to request/send information from/to a web

service (web site)– A very useful way to communicate with an application running

in the cloud!

Representational State Transfer (REST)

• Messaging Protocol• Not quite a standard… more of a style• Main Idea:

– Use different HTTP methods as a means of communicating a message’s intent!

REST - GET

• Used to retrieve a representation of a resource• Generally returns answer in XML or JSON, along with a

response code.– The response code also conveys information!– 200 (OK) if resource is found– 404 (Not Found) if ID is bad, or if resource is not found

GET /people.php?id=4092 HTTP/1.1Host: www.my-api.com

REST - PUT

• Typically used to update a resource at the server• For example to change a person’s name in your API, you

may want to send something like the following…– 200 response indicates successful update– 201 indicates successful creation

PUT /people.php?id=3821Host: www.my-api.com

newname=dude&newsocial=555-55-5555

REST - POST

• Typically used to add a new resource to a server– Particularly to create a subordinate resource– For example adding a new person to a user database– 201 indicates successful creation– Note: Two identical POST requests will probably yield two

identical, yet separate, resources!

POST /users.php HTTP/1.1Host: my-api.com

firstname=harvey&lastname=john&meme=lolcat

REST - DELETE

• Used to delete a resource identified by a URI• Pretty much as simple as that!

– 200 (OK) response indicates successful deletion– Sometimes 204 also indicates successful deletion– 404 indicates nothing to delete

Outline

Cloud Computing Overview

Amazon AWS Services

A Very Brief HTTP Overview

REST APIs

JSON Format

JavaScript Object Notation

• Used very commonly across the web because of its simplicity!

• Lots of REST APIs return JSON on GET requests– and expect it for POST and PUT requests…

Why JavaScript?

• This format is literally the same format used by JavaScript to represent objects

• It’s very compact and easily readable• Also has the benefit that it can be directly copied and

pasted into a javascript interpreter!

Return of the Dictionary

• JSON Objects are just dictionaries, really…• And they look like dictionaries, too!• Because they are just dictionaries at heart, they can be

interpreted easily by many languages and stored natively• Java, Python, and Ruby all have JSON interpreters, for

example• Some open-source bash projects as well!

– http://kmkeen.com/jshon/ parses JSON into native bash arrays

JSON - Example

GET /users?id=6504 HTTP/1.1Host: www.my-api.com

200 OK{ “name” : “Spencer Lee”, “id” : 6504, “interests” : { “programming”: [“Python”, “Bash”,”JavaScript”], “animals” : [“cats”,”dogs”,”meercats”], “Pet” : “Mochi” } …..}

Example of a useful REST-ful API

• PARSE! • A no-sql datastore that is easy to use, with a full REST API• Learn more at https://parse.com/docs/rest• May be useful for your projects!