Tel Aviv - Oct 2013
Google Cloud Platform
Google Cloud Overview
+GreenIdoDeveloper Advocate, Google
App Engine
Who?
Ido GreenDeveloper Advocate, Googleplus.google.com/+greenido
● ido-green.appspot.com
● greenido.wordpress.com
● github.com/greenido/
Agenda
●Part I: Background & Motivation
●Part II: Google App Engine
●Part III: Cloud Platform Updates
About Scale● Google's search index
○ (2012)
● YouTube
○ (2013)
● YouTube... monthly uniques?
○ (2013)
Innovation
Scaling & Innovation
Image courtesy Google/Connie Zhou
Image courtesy Google Maps
*AND* security
Cloud industry service levels
PaaS
SaaSGoogle Apps Salesforce, SugarCRM, NetSuite
Yahoo!Mail YOUR NEXT APP?
Google Apps Script force.com
Google App Engine Cloud Foundry, OpenShift
Windows Azure, Heroku DotCloud, Engine Yard
Google BigQuery, Cloud SQL, Cloud Datastore, Translate, Prediction
Amazon RDS, DynamoDB; Microsoft SQL (Azure) Database
Amazon EC2, S3 Rackspace, Joyent
Google Compute Engine, Cloud Storage
GoGrid, CloudLayerIaaS
Google Cloud Platform service levels
PaaS
SaaSGoogle Apps Salesforce, SugarCRM, NetSuite
Yahoo!Mail, Hotmail YOUR NEXT APP?
Google Apps Script force.com
Google App Engine Cloud Foundry, OpenShift
Windows Azure, Heroku DotCloud, Engine Yard
Google BigQuery, Cloud SQL, Cloud Datastore, Translate, Prediction
Amazon RDS, DynamoDB; Microsoft SQL (Azure) Database
Amazon EC2, S3 Rackspace, Joyent
Google Compute Engine, Cloud Storage
GoGrid, CloudLayerIaaS
● Comprehensive, integrated platform
● Extension of Google internal infrastructure○ Internal services created for us
SHARED with YOU
● Google makes improvements to their cloud○ ...and who gets them first...
What is the Google Cloud Platform?
Google Cloud Platform services
Google worldwide network
"This is what makes Google Google: its physical network, its thousands of fiber miles, and those many thousands of servers that, in aggregate, add up to the mother of all clouds." -- S. Levy/Wired (Oct 2012)
Our cloud runs our business
Google Cloud Platform services
Agenda
●Part I: Background & Motivation
●Part II: Google App Engine
●Part III: Cloud Platform Updates
➢ Hardware failure? ➢ Traffic spikes? Scaling? ➢ Patches & upgrades? ➢ Network issues? ➢ License management?
Building a traditional web application
Why does App Engine exist?
●Easy to build
●Easy to manage
●Easy to scale
● Enabling greater developer productivity!● Focus on building your app, not sys admin● Let us wear the pagers!● BUT don't confuse App Engine w/IaaS
services!
App Engine to the rescue!!
Python, Java, Go, PHP not enough for you?
Specialized APIs/Services for the Sandbox
Cloud StorageImages
Mail Channel Task Queue
Memcache Datastore URL Fetch
User Service
Hardware failure, traffic spikes, scaling, software patches,
upgrades handled by Google... and yes, we’ll operate it for you!
How does it work?
Administration Console
App Engine - By the Numbers*
● 300,000+ Active developers
● 3,000,000+ Active applications
● Half of world's Internet addressestouch an App Engine server(per week)
* per month except as otherwise noted
App Engine - Larger Numbers*
7,500,000,000+Hits (per day)
4,500,000,000,000+Datastore requests
* per month except as otherwise noted
Some App Engine users
● 3.6MM DAUs on FB● 1.9MM DAUs on MS● Add Orkut, Bebo,
Hi5, Friendster, Hyves, Ning…
Long-term scaling...
Short-term scaling...
Images courtesy Gigya
Official blog & live stream apps hosted on App Engine
On Wedding day...Blog app served:
● Up to 2k requests per second● 15 million pageviews ● 5.6 million visitors
Live stream app served:● Up to 32k requests per second● 37.7 million pageviews ● 13.7 million visitors
goo.gl/F1SGc
“Royal” scaling
App Engine! -> Cloud Endpoints!
Not all apps have web UI
App Engine as Mobile Backend
● Pulse’s mobile app○ Android○ iOS○ Kindle
● Awards (2011)○ Apple’s App Store Hall of Fame○ Time Magazine’s Top 50 iOS apps
● 30MM+ users, 10MM+ stories read/day● Thousands of QPS, 100M daily requests● Use of App Engine cost-effective
● Get started building your mobile backend:○ developers.google.com/cloud/samples/mbs
Gaming... share data, mobile or web!
Multimedia scaling...
Google Cloud Endpoints● APIs for web and mobile backends made easy
● Watch the tutorial & get the code at cloud.google.com/mobilebackendtutorial
Popular App Engine use cases
● Mobile/Tablet○ App backends○ Cloud persistence
● Social/Mobile Games○ Speed, scale○ API integrations○ Personals/dating
● Consumer Web Apps○ Unpredictable traffic○ Scale
● Apps in Academia○ Any course where
students build web or mobile apps
○ Research projects○ IT/Operational apps
● Business Apps○ Enterprise○ Java runtime○ Web or Mobile
“Vendor lock-in”● What is it?
○ Systems which inherently make it very difficult or impossible to migrate (data / logic) to other systems
● Applies to App Engine?○ Yes & no.○ Can’t get something for nothing.○ Goal: take advantage of
Google's infrastructure○ Price: need to write against
Google APIs. Make sense?
● Not in Google’s DNA○ Peter Magnusson post http://goo.
gl/X2UyEz
● “Must use” Google APIs○ Not necessarily
● Web app alternatives○ Java: servlets, JSP○ Python: Django, web2py,
Tipfy, Bottle, Pyramid
● Data Storage○ Cloud SQL (MySQL)○ Python: Django ORM*○ Java: JPA/JDO ORMs
● SDKs: open source
● APIs○ Java: JSR stds, Python: stdlib
● Datastore bulkloader○ Easily up/download your data
● Open source backend systems○ Python: AppScale; TyphoonAE○ Java: CapeDwarf/JBoss - TCK○ "Private cloud"
Fighting lock-in
Pricing
Google Prediction API
Google Storage
PricingTier
Monthly Cost
99.95% SLA
Paid Support
Free $0
Paid pay-per-use *
Premier $150USD+ * *
● cloud.google.com/pricing● Please compare - “Apples to Apples”
● SAS 70, SSAE 16, ISAE 3402, ISO 27001:2005 Certified
● Choose from US- or EU-based service
Security Compliance & Localization
● Integration w/Compute Engine, BigQuery, Cloud Datastore● Java Servlet 3.0 support + Blobstore migration tool● VM Runtime App Engine-managed backends via Compute
Engine● developers.google.com/appengine/features/#roadmap_features
App Engine Roadmap
Getting Started
Google Prediction API
Google Storage
What WhereProduct Info cloud.google.com/products
Docs, downloads, etc. developers.google.com/appengine
Create/manage your apps appengine.google.com
Help from experts on anything technical, including App Engine stackoverflow.com
Cloud Blog googlecloudplatform.blogspot.com
Social google.com/+googlecloudplatform @GoogleCloud
● The docs are great, but what about teaching materials?
Cloud Playground
●Try App Engine in your browser...cloud-playground.appspot.com
Agenda
●Part I: Background & Motivation
●Part II: Google App Engine
●Part III: Cloud Platform Updates
Connect to Google Cloud Platform
BigQuery
Google Cloud Storage
Google App Engine
ComputeEngine Cloud SQL
Prediction
???
CloudDatastore
Translate
Google Compute Engine
● Need to go outside App Engine boundaries?
● On-demand VMs (RHEL/CentOS, Debian)○ 1-8 virtual cores○ 1.8-6.5GB RAM/core○ Private VM network
● 3 storage types○ Local/scratch: 420GB○ Persistent disk: 10TB○ Cloud Storage: unlimited
● 3 interfaces○ Command-line interface○ Web-based interface○ Your code via REST API
Compute Engine performance
Source: Scalr/gigaom.com (May 2013)
Real-world Performance Observation
Google Cloud Platform
Google Cloud Storage● Fast, scalable, highly-available object store (US or EU)● Strong read-your-write consistency● Easy, flexible authentication & sharing (OAuth2, ACLs)● REST API, App Engine API, Web UI, command-line● developers.google.com/storage
Google Cloud SQL● MySQL-compatible relational cloud database● App Engine API or externally via JDBC● developers.google.com/cloud-sql
Google Cloud Platform
Google Cloud Datastore● NoSQL database at Google scale● Low-level interface (access via HTTP/RPC)● Familiar native datastore for App Engine● developers.google.com/datastore
Google Translate● Translate text into other languages programmatically● Build multilingual apps with Google translation algorithms● Familiar RESTful interface● developers.google.com/translate
Cloud Datastore
Translate
Google Cloud Platform
Google BigQuery● Large scale query and analysis cloud service● Query multi-terabyte datasets in seconds● SQL-like query language● Based on Google's internal Dremel project● cloud.google.com/bigquery-tour
Google Prediction● Machine Learning service in the cloud● Supervised learning● Train models, get "predictions"● "Magic?" Possibly.● developers.google.com/prediction
Prediction
Fusion Tables● Think: Google Maps + Excel/Sheets on steroids● Aggregate disparate data in spreadsheet & visualize● google.com/fusiontables
Course Builder MOOC
Course Builder MOOC
● Open-source MOOC software○ Easily customizable○ code.google.com/p/course-builder
● Courses hosted on Google App Engine○ Run your own MOOCs: your code, your control○ No need to rely on commercial vendor(s)
● More information○ googleresearch.blogspot.com/2012/09/helping-world-to-teach.html
● Sample course○ powersearchingwithgoogle.com
CloudCourse class registration
● App Engine app for course offerings & registration● http://goo.gl/o8NRSk
Google Apps Script
● script.google.com
● JavaScript baby! outside the browser, in Google’s cloud.
● Browser based IDE
● Deeply integrated with Google Apps & services
○ Automate repetitive tasks
○ Manipulate data across multiple tools
Apps Script example
Google APIs● Android● Chrome ● Google+ ● Ads● Maps● Drive● YouTube● Commerce ● Google TV
● Ido Green - Developer Advocate, Google
○ +GreenIdo
○ @greenido
○ ido-green.appspot.com
● Google Cloud Platform
○ cloud.google.com
○ plus.google.com/+googlecloudplatform
Thank you!
Q&A Time
Ahh! WAIT, there’s one more thing...
Cloud Platform Starter Pack
● Developers from affiliated partners can
receive up to $2,000USD of credit towards
cloud usage○ Up to $1,000 for Google App Engine
○ Up to $1,000 for Google Compute Engine + others
● Apply at cloud.google.com/starterpack and use promo code: gdg-in
● Ido Green - Developer Advocate, Google○ +GreenIdo
○ @greenido
○ ido-green.appspot.com
● Google Cloud Platform○ cloud.google.com
○ plus.google.com/+googlecloudplatform
Thank you!
Q&A Time