go on gae (go israel meetup)

Download Go on GAE (Go Israel Meetup)

Post on 19-Aug-2015




4 download

Embed Size (px)


  1. 1. Go Israel First Meetup of 2015 Go on GAE Or Hiltch / CTO Streamrail
  2. 2. Intro How we are using GAE GAE PaaS in a nutshell Go specific GAE goodness Dev and (sort of) CI process on GAE - Small Live Demo Pitfalls My wisthlist for GAE Q&A
  3. 3. How we are using GAE whoami StreamRail develops client side SDKs for mobile and web that accelerate content We do cool P2P shit, neat graph theory stuff (web & native mobile, so no Go here, yet) Clients are ad networks and publishers that want to get a speed boost Our backend is 100% Go, but most of it is not actually GAE
  4. 4. GAE - PaaS in a nutshell Exists since 2008, used by SnapChat, Khan Academy (python), Secret (go) and lots of others Strict PaaS Philosophy (sometimes good, sometimes bad) Service for everything, dont care about under the hood - Memcache, Cron, DB, Task queue, Blobstore, Auth (OpenID, OAuth) and more Cannot do outside the box stuff. If you cant do it inside the box (yet), dont use it Easy sandbox model (app containers gets shipped and scheduled on instances with autoscaling), strong vendor lock-in (logging, sending emails, and even performing http requests is via a GAE API).
  5. 5. GAE - PaaS in a nutshell Almost free to begin with (both $ and time wise) Not just the dev, you can achieve CI with GAE with ease Some PaaS providers dont get it right (AWS Beanstalk- configure ElastiCache - private security groups, VPCs, subnets, cache nodes, etc, That part of your app that needs to just work, not your main thing (your product is an app for web or mobile, not something like a search engine)
  6. 6. How does a GAE Go app look like? app.yaml (app name, version, routes, runtime, etc) cron.yaml (cron jobs schedules) crossdomain.xml (CORS settings)
  7. 7. How does a GAE Go app look like? We like to work with GitHub as source control github.com/streamrail/appengine holds all packages related to GAE We like to reflect the package paths on every devs file system too github.com streamrail appengine admin analytics build go-bitly-gae go-gae-bigquery main qa resource swarm twilio-gae util website
  8. 8. action needs to be performed during the request - parallelize (note memcache and datastore APIs) Concurrency Example
  9. 9. RAM: Highest speed - O(1s) Purged most frequently, not horizontal (per instance cache) May overflow instance memory (cant store gigs) Memcache: High speed - O(1ms) Horizontal , Limited to 1 MB per object May be purged without notice Use large dedicated memcache! (more $$$ but much better) Datastore Slowest of the bunch - O(20ms) Horizontal Persistent (not purged) A Word About Caching on GAE
  10. 10. Dont forget - GAE will kill outstanding API calls once req returned Control API Calls
  11. 11. Use the delay package to defer work that can be deferred Deferring Work
  12. 12. Continuous Integration & Delivery GAE does not use Git publishing like Heroku, but its easily achieved if you have a CI server Simple goapp deploy to deploy a version goapp test - like go test, but establishes almost complete GAE test env on CI server / dev machine (inc. http, memcache, datastore, etc.) Create subversions of your app and traffic split before rolling out Easy rollback in one click We have connected GAE to Circle CI to test and deploy
  13. 13. Circle.yaml file
  14. 14. Traffic Splitting
  15. 15. Live Streaming to BQ Prev solution (downloading logs/datastore using Remote API and inserting to DB) failed No need to store (and maintain!) the logs anywhere (GCS, S3, etc.) Apache logs exist anyway as backup on GAE Near real time data stream (in-memory buffering is used to reduce contention) BQ awesome, connects to Google Spreadsheets Useful packages weve written for this: https://github.com/streamrail/go-gae-remote https://github.com/streamrail/go-gae-bigquery https://github.com/streamrail/bq-schema https://github.com/streamrail/gapps-bqutil
  16. 16. Streaming from a handler
  17. 17. Validating schema
  18. 18. Binding BQ to Spreadsheets
  19. 19. Pitfalls & Bad Use Cases
  20. 20. If you do nothing, you can scale infinitely - Scott Hanselman Long-lived connections Serving static content, CDN style No WebSockets solution (Channel API crap) Datastore is annoying No SSL for versioned apps Limited caching options General Pitfalls GOMAXPROCS=1 is set (force one active thread) Go GAE instance allows 10 concurrent blocking (non-active) threads Not very cost effective Sandboxed - no C imports etc.
  21. 21. Firebase integration, reasonable pricing More features (long lived connections, sticky sessions, web sockets, wider selection of DB and caching) Better performance (global distribution, multithreading for Go) Git publishing (heroku style!) Better out of the box monitoring (alerts, schematic logging) My Wishlist for GAE
  22. 22. THANK YOU! Were hiring - jobs@streamrail.com