google app engine's latest features
DESCRIPTION
A review and update presentation on Google App Engine's latest features up through version 1.5.3 and including new experimental features. This presentation was given to the San Diego GTUG on Aug 26, 2011.TRANSCRIPT
Introducing App Engine's Latest FeaturesChris SchalkDeveloper Advocate
Agenda
● What is App Engine?
● Part I: App Engine Production Updates ○ Growth trajectory○ App Engine Success so far○ A Big Production Update: App Engine Leaving
Preview
● Part II: App Engine Technical Updates○ Go Runtime, Backends, Channel API, FTS,
Mapreduce
● Easy to build● Easy to maintain● Easy to scale
What is App Engine?
Focus on building your app, let us wear the pagers!
Build and run your web apps on Google’s infrastructure
IaaS
PaaS
SaaS
Source: Gartner AADI Summit Dec 2009
Cloud landscape as Gartner Sees It
What is App Engine?
● Downloadable SDK● Application runtimes
○ Java, Python, Go● Local development tools
○ Eclipse plugin● Specialized api services ● Cloud based dashboard● Automatically scales● Built in fault tolerance, load
balancing● Continuously evolving...
PaaS Development Platform in a Box
Language Runtime Options
GO JavaExperimental
App Engine Services
BlobstoreImages
Mail XMPP Task Queue
Memcache Datastore URL Fetch
User Service
Agenda
● What is App Engine?
● Part I: App Engine Production Updates ○ Growth trajectory○ App Engine Success so far○ A Big Production Update: App Engine Leaving
Preview
● Part II: App Engine Technical Updates○ Go Runtime, Backends, Channel API, FTS,
Mapreduce
App Engine Evolution Through The Years
App Engine LaunchPython
DatastoreMemcachelogs export
2008 2009 2010
JavaDB Import
cron
Batch write/readHttps
Status-Dashboard
Task QueuesXMPP
incoming email
MultitenancyInstance Console
Always Onhi-perf imag
10 min tasks
BlobstoreAppstats
cursorsMapper
2011
Hi-ReplicationDatastore
Channel APIFiles API
Remote APIProsp Search
100,000+ Active developers per Month
200,000+Active apps per week
1.5B+Pageviews per day
App Engine - By the Numbers
Notable App Engine Customers
Royal Wedding - Scalability Success
Official Royal Wedding blog on App Engine
On Wedding day, served: ● Up to 32k requests per second● 37.7 million pageviews ● 13.7 million visitors
http://goo.gl/F1SGc
Enterprise Success on App Engine
Best Buy Applications on App Engine
WebFilings Financial SaaS on App Engine
http://www.webfilings.com/customers/valued_customers
Enterprise customers:
● Procter & Gamble● Delta Airlines● Valero Energy● Sprint/Nextel● eBay ● United Airlines● Dish Network● ...
Recently signed:● One of the world's largest
banks● The world's largest retailer● One of the largest US
auto manufacturers
App Engine: Certified Security
App Engine is now fully SAS70/SSAE16/ISAE 3402 Certified!
● http://googleappengine.blogspot.com/2011/05/year-ahead-for-google-app-engine.html
One Big Production Update...
.... App Engine is coming out of preview!
Production App Engine Features
● Full Google Product - 3 year deprecation support
● 99.95% SLA for all paid users
● Operational and Developer Support available
● Billing via invoice
● Ongoing feature delivery: custom domain SSL, MapReduce, etc..
App Engine Leaving Preview in 2011!
What Does Leaving Preview Mean?
● Long-term committment by Google to App Engine
● Long-term committment by App Engine to users!
● It's a real product now, Enterprises can trust in it!
● http://googleappengine.blogspot.com/2011/05/year-ahead-for-google-app-engine.html
Production App Engine Pricing
● Usage types:○ Free - similar to today○ Paid - $9/app/mo (+ usage) including SLA○ Premier - $500/account/mo (+ usage) including operational
support
● Add-on fee for special services○ For example: SSL for custom domains will be available for
monthly fee
Production App Engine Further Details
● Instance based charging○ CPU-hours replaced by Instance Hours (IH)○ 1 instance running for 1 hour○ Similar to instances in admin console today
● APIs○ Currently charged per CPU-hour○ Will be charged per operation
● FAQ○ https://groups.google.com/group/google-
appengine/browse_thread/thread/a1bfa432e0c002a7○ https://groups.google.com/group/google-
appengine/browse_thread/thread/1e5ba8835fdade7e
Agenda
● What is App Engine?
● Part I: App Engine Production Updates ○ Growth trajectory○ App Engine Success so far○ A Big Production Update: App Engine Leaving
Preview
● Part II: App Engine Technical Updates○ Go Runtime, Backends, Pull Queues, Channel
API, FTS, MapReduce
App Engine 1.5 Updates
● Introduced at Google IO, May 2011
'Go' added as third language runtime
● Released SDK for Mac & Linux○ Windows port in the works
● Uses tweaked Python SDK tools● Launched for production 1.5.2 (Jul 2011)● http://blog.golang.org/2011/05/go-and-google-app-engine.html
High Replication Datastore now default
● Master/Slave (M/S) original App Engine datastore● High Replication Datastore (HRD) uses Paxos algorithm● Was originally 3x the cost of M/S*● Slightly slower (distributed writes)● Queries eventually consistent● Well-received
○ http://googleappengine.blogspot.com/2011/03/high-replication-datastore-solid-choice.html
HRD as default - cont.
● *Price reduction to discourage M/S (no longer 3x)● Has Delivered ~99.999% uptime since launch in January● M/S price increases when App Engine leaves Preview● Working on better tools
○ To make migration easier○ To estimate updated cost
Backends feature introduced
● Long-running servers● No more 30s or 10m deadline
○ Fully configurable instances with options:■ RAM: 128MB to 1GB■ CPU: 600MHz to 4.8GHz no request deadlines
○ Can run indefinitely ○ Resident or Dynamic - Public or Private○ Automatic restarts ○ Easy to configure (backends.yaml, backends.xml)
● (Demo)● http://www.google.com/events/io/2011/sessions/app-engine-
backends.html
"Pull Queues" introduced
● A new alternative to previous Task Queues, now referred to as "Push Queues"
● Lets users decide when to execute tasks● Uses a REST API● Allows external access
Recalling "Push Queues"
"Pull Queues" introduced
● A new alternative to previous Task Queues, now referred to as "Push Queues"
● Task is merely data, no longer a web request● Workers can pull (lease) tasks from queue● Uses a REST API
○ https://www.googleapis.com/taskqueue/myappid/taskqueues/myqueue
● Can configure ACLs for external access
● http://www.google.com/events/io/2011/sessions/putting-task-queues-to-work.html
Pull Queues vs. (Push)Task Queues
● Push Task Queues take & execute work○ Tasks are "pushed" to application○ Execute at a given rate
● Pull queues offer more flexibility○ Workers can pull (lease) tasks when
appropriate
Other App Engine 1.5.(0) Features
● Mail API updates○ Reduced free recipients from 2000 to 100 for new apps○ ...
● Source code downloading○ Now Owner and Uploader can download
● http://googleappengine.blogspot.com/2011/05/app-engine-150-release.html
App Engine 1.5.1 Features
● Geolocation header○ X-AppEngine-country - can customize web pages based on
location
● Presence support in Channel API○ Can take action when user connects/disconnects (demo)
● WebP support in Images API○ WebP 39% smaller than JPEG!
● Create RPC services with ProtoRPC○ Now part of App Engine
● http://googleappengine.blogspot.com/2011/06/app-engine-151-release.html
App Engine 1.5.2 Features
● Production: adjustable scheduler parameters○ Able to set the minimum pending latency and maximum
number of idle instances
● Namespaced datastore statistics● New option to query datastore stats per namespace
● Tasks○ New Task Queue details page○ Pull Queue task size bump to 1MB○ Pull Queue task lease extensions
● http://googleappengine.blogspot.com/2011/07/app-engine-152-sdk-released.html
App Engine 1.5.3 Features
● Blobstore file uploads no longer limited○ Allows you to serve much larger files: images, videos etc.
● Datastore Admin now easily enable via 1-click○ No longer have to update app.yaml, and java developers
don't have to upload a python version of their app
● New HRD migration tool trusted tester program● Download App
○ Can now download any files that were uploaded into your war directory
● http://googleappengine.blogspot.com/2011/08/app-engine-153-sdk-released.html
Experimental feature updates
● Introduced at Google I/O 2011 talks● Full-text Search
○ Full-text search service coming to App Engine○ Other queries: numeric, geo, date search capabilities, etc.○ http://www.google.com/events/io/2011/sessions/full-text-
search.html
● MapReduce○ Part 1: Mapper has been available for awhile○ Work is ongoing for shuffle and reduce○ http://www.google.com/events/io/2011/sessions/app-engine-
mapreduce.html○ http://www.youtube.com/watch?v=EIxelKcyCC0
Q&A