Google Cloud Platform Update

Tel Aviv - Oct 2013

Google Cloud Platform

Google Cloud Overview

+GreenIdoDeveloper Advocate, Google

App Engine

Who?

Ido GreenDeveloper Advocate, Googleplus.google.com/+greenido

● ido-green.appspot.com

● greenido.wordpress.com

● github.com/greenido/

Agenda

●Part I: Background & Motivation

●Part II: Google App Engine

●Part III: Cloud Platform Updates

About Scale● Google's search index

○ (2012)

● YouTube

○ (2013)

● YouTube... monthly uniques?

○ (2013)

Innovation

Scaling & Innovation

Image courtesy Google/Connie Zhou

Image courtesy Google Maps

*AND* security

Cloud industry service levels

PaaS

SaaSGoogle Apps Salesforce, SugarCRM, NetSuite

Yahoo!Mail YOUR NEXT APP?

Google Apps Script force.com

Google App Engine Cloud Foundry, OpenShift

Windows Azure, Heroku DotCloud, Engine Yard

Google BigQuery, Cloud SQL, Cloud Datastore, Translate, Prediction

Amazon RDS, DynamoDB; Microsoft SQL (Azure) Database

Amazon EC2, S3 Rackspace, Joyent

Google Compute Engine, Cloud Storage

GoGrid, CloudLayerIaaS

Google Cloud Platform service levels

PaaS

SaaSGoogle Apps Salesforce, SugarCRM, NetSuite

Yahoo!Mail, Hotmail YOUR NEXT APP?

Google Apps Script force.com

Google App Engine Cloud Foundry, OpenShift

Windows Azure, Heroku DotCloud, Engine Yard

Google BigQuery, Cloud SQL, Cloud Datastore, Translate, Prediction

Amazon RDS, DynamoDB; Microsoft SQL (Azure) Database

Amazon EC2, S3 Rackspace, Joyent

Google Compute Engine, Cloud Storage

GoGrid, CloudLayerIaaS

● Comprehensive, integrated platform

● Extension of Google internal infrastructure○ Internal services created for us

SHARED with YOU

● Google makes improvements to their cloud○ ...and who gets them first...

What is the Google Cloud Platform?

Google Cloud Platform services

Google worldwide network

"This is what makes Google Google: its physical network, its thousands of fiber miles, and those many thousands of servers that, in aggregate, add up to the mother of all clouds." -- S. Levy/Wired (Oct 2012)

Our cloud runs our business

Google Cloud Platform services

Agenda

●Part I: Background & Motivation

●Part II: Google App Engine

●Part III: Cloud Platform Updates

➢ Hardware failure? ➢ Traffic spikes? Scaling? ➢ Patches & upgrades? ➢ Network issues? ➢ License management?

Building a traditional web application

Why does App Engine exist?

●Easy to build

●Easy to manage

●Easy to scale

● Enabling greater developer productivity!● Focus on building your app, not sys admin● Let us wear the pagers!● BUT don't confuse App Engine w/IaaS

services!

App Engine to the rescue!!

Python, Java, Go, PHP not enough for you?

Specialized APIs/Services for the Sandbox

Cloud StorageImages

Mail Channel Task Queue

Memcache Datastore URL Fetch

User Service

Hardware failure, traffic spikes, scaling, software patches,

upgrades handled by Google... and yes, we’ll operate it for you!

How does it work?

Administration Console

App Engine - By the Numbers*

● 300,000+ Active developers

● 3,000,000+ Active applications

● Half of world's Internet addressestouch an App Engine server(per week)

* per month except as otherwise noted

App Engine - Larger Numbers*

7,500,000,000+Hits (per day)

4,500,000,000,000+Datastore requests

* per month except as otherwise noted

Some App Engine users

● 3.6MM DAUs on FB● 1.9MM DAUs on MS● Add Orkut, Bebo,

Hi5, Friendster, Hyves, Ning…

Long-term scaling...

Short-term scaling...

Images courtesy Gigya

Official blog & live stream apps hosted on App Engine

On Wedding day...Blog app served:

● Up to 2k requests per second● 15 million pageviews ● 5.6 million visitors

Live stream app served:● Up to 32k requests per second● 37.7 million pageviews ● 13.7 million visitors

goo.gl/F1SGc

“Royal” scaling

App Engine! -> Cloud Endpoints!

Not all apps have web UI

App Engine as Mobile Backend

● Pulse’s mobile app○ Android○ iOS○ Kindle

● Awards (2011)○ Apple’s App Store Hall of Fame○ Time Magazine’s Top 50 iOS apps

● 30MM+ users, 10MM+ stories read/day● Thousands of QPS, 100M daily requests● Use of App Engine cost-effective

● Get started building your mobile backend:○ developers.google.com/cloud/samples/mbs

Gaming... share data, mobile or web!

Multimedia scaling...

Google Cloud Endpoints● APIs for web and mobile backends made easy

● Watch the tutorial & get the code at cloud.google.com/mobilebackendtutorial

Popular App Engine use cases

● Mobile/Tablet○ App backends○ Cloud persistence

● Social/Mobile Games○ Speed, scale○ API integrations○ Personals/dating

● Consumer Web Apps○ Unpredictable traffic○ Scale

● Apps in Academia○ Any course where

students build web or mobile apps

○ Research projects○ IT/Operational apps

● Business Apps○ Enterprise○ Java runtime○ Web or Mobile

“Vendor lock-in”● What is it?

○ Systems which inherently make it very difficult or impossible to migrate (data / logic) to other systems

● Applies to App Engine?○ Yes & no.○ Can’t get something for nothing.○ Goal: take advantage of

Google's infrastructure○ Price: need to write against

Google APIs. Make sense?

● Not in Google’s DNA○ Peter Magnusson post http://goo.

gl/X2UyEz

● “Must use” Google APIs○ Not necessarily

● Web app alternatives○ Java: servlets, JSP○ Python: Django, web2py,

Tipfy, Bottle, Pyramid

● Data Storage○ Cloud SQL (MySQL)○ Python: Django ORM*○ Java: JPA/JDO ORMs

● SDKs: open source

● APIs○ Java: JSR stds, Python: stdlib

● Datastore bulkloader○ Easily up/download your data

● Open source backend systems○ Python: AppScale; TyphoonAE○ Java: CapeDwarf/JBoss - TCK○ "Private cloud"

Fighting lock-in

Pricing

Google Prediction API

Google Storage

PricingTier

Monthly Cost

99.95% SLA

Paid Support

Free $0

Paid pay-per-use *

Premier $150USD+ * *

● cloud.google.com/pricing● Please compare - “Apples to Apples”

● SAS 70, SSAE 16, ISAE 3402, ISO 27001:2005 Certified

● Choose from US- or EU-based service

Security Compliance & Localization

● Integration w/Compute Engine, BigQuery, Cloud Datastore● Java Servlet 3.0 support + Blobstore migration tool● VM Runtime App Engine-managed backends via Compute

Engine● developers.google.com/appengine/features/#roadmap_features

App Engine Roadmap

Getting Started

Google Prediction API

Google Storage

What WhereProduct Info cloud.google.com/products

Docs, downloads, etc. developers.google.com/appengine

Create/manage your apps appengine.google.com

Help from experts on anything technical, including App Engine stackoverflow.com

Cloud Blog googlecloudplatform.blogspot.com

Social google.com/+googlecloudplatform @GoogleCloud

● The docs are great, but what about teaching materials?

Cloud Playground

●Try App Engine in your browser...cloud-playground.appspot.com

Agenda

●Part I: Background & Motivation

●Part II: Google App Engine

●Part III: Cloud Platform Updates

Connect to Google Cloud Platform

BigQuery

Google Cloud Storage

Google App Engine

ComputeEngine Cloud SQL

Prediction

???

CloudDatastore

Translate

Google Compute Engine

● Need to go outside App Engine boundaries?

● On-demand VMs (RHEL/CentOS, Debian)○ 1-8 virtual cores○ 1.8-6.5GB RAM/core○ Private VM network

● 3 storage types○ Local/scratch: 420GB○ Persistent disk: 10TB○ Cloud Storage: unlimited

● 3 interfaces○ Command-line interface○ Web-based interface○ Your code via REST API

Compute Engine performance

Source: Scalr/gigaom.com (May 2013)

Real-world Performance Observation

Google Cloud Platform

Google Cloud Storage● Fast, scalable, highly-available object store (US or EU)● Strong read-your-write consistency● Easy, flexible authentication & sharing (OAuth2, ACLs)● REST API, App Engine API, Web UI, command-line● developers.google.com/storage

Google Cloud SQL● MySQL-compatible relational cloud database● App Engine API or externally via JDBC● developers.google.com/cloud-sql

Google Cloud Platform

Google Cloud Datastore● NoSQL database at Google scale● Low-level interface (access via HTTP/RPC)● Familiar native datastore for App Engine● developers.google.com/datastore

Google Translate● Translate text into other languages programmatically● Build multilingual apps with Google translation algorithms● Familiar RESTful interface● developers.google.com/translate

Cloud Datastore

Translate

Google Cloud Platform

Google BigQuery● Large scale query and analysis cloud service● Query multi-terabyte datasets in seconds● SQL-like query language● Based on Google's internal Dremel project● cloud.google.com/bigquery-tour

Google Prediction● Machine Learning service in the cloud● Supervised learning● Train models, get "predictions"● "Magic?" Possibly.● developers.google.com/prediction

Prediction

Fusion Tables● Think: Google Maps + Excel/Sheets on steroids● Aggregate disparate data in spreadsheet & visualize● google.com/fusiontables

Course Builder MOOC

Course Builder MOOC

● Open-source MOOC software○ Easily customizable○ code.google.com/p/course-builder

● Courses hosted on Google App Engine○ Run your own MOOCs: your code, your control○ No need to rely on commercial vendor(s)

● More information○ googleresearch.blogspot.com/2012/09/helping-world-to-teach.html

● Sample course○ powersearchingwithgoogle.com

CloudCourse class registration

● App Engine app for course offerings & registration● http://goo.gl/o8NRSk

Google Apps Script

● script.google.com

● JavaScript baby! outside the browser, in Google’s cloud.

● Browser based IDE

● Deeply integrated with Google Apps & services

○ Automate repetitive tasks

○ Manipulate data across multiple tools

Apps Script example

Google APIs● Android● Chrome ● Google+ ● Ads● Maps● Drive● YouTube● Commerce ● Google TV

● Ido Green - Developer Advocate, Google

○ +GreenIdo

○ @greenido

○ ido-green.appspot.com

● Google Cloud Platform

○ cloud.google.com

○ plus.google.com/+googlecloudplatform

Thank you!

Q&A Time

Ahh! WAIT, there’s one more thing...

Cloud Platform Starter Pack

● Developers from affiliated partners can

receive up to $2,000USD of credit towards

cloud usage○ Up to $1,000 for Google App Engine

○ Up to $1,000 for Google Compute Engine + others

● Apply at cloud.google.com/starterpack and use promo code: gdg-in

● Ido Green - Developer Advocate, Google○ +GreenIdo

○ @greenido

○ ido-green.appspot.com

● Google Cloud Platform○ cloud.google.com

○ plus.google.com/+googlecloudplatform

Thank you!

Q&A Time