Learn from AWS 2

向成功的雲服務學習Amazon Web Services

Hank Lin

Hank Lin

@hanklinhttp://hanklin.com/hank9x@gmail.com

My AWS Book ;)

• ISBN： 9789861992792

Topics

Amazon如何滿足網路 startup

Amazon與其它廠的競合關係

You Call This Cloud?

http://www.flickr.com/photos/lennysan/4403695791/

I Say This Is Cloud

Photo by Pierre J. (French Army, July 3, 1970)

Everything Is a Service

Software as a ServicePlatform as a ServiceInfrastructure as a Service * as a Service

Nothing New

http://en.wikipedia.org/wiki/File:The_Earth_seen_from_Apollo_17.jpg

Nothing New

InternetVirtualization, VPSPlatformASPBrowser, RIANoSQLScaling Out/Up

Everything Changes

http://en.wikipedia.org/wiki/File:Giantimpact.gif

Everything Changes

APIsOn-DemandPay As You GoInfinite CapacityLease Instead Of ProvisionReallocate Instead Of RecoverScaling In/Down

Customer-centric

Agile Development2 weeks ~ 1 month iterationWhat customers want

Developer-centric

Eat your own dog foodAPIsAsync. & BatchingStart from simple

Guns, Lots of Guns

(a picture was here)

AWS Products

http://www.flickr.com/photos/dunechaser/4785103816/

AWS ProductsFunction Products

Compute EC2、 EMR、 Auto Scaling

Storage S3、 EBS、 AWS Import/Export、 Public Dataset

Database SimpleDB、 RDS

Messaging SNS、 SQS、 SES

CDN CloudFront

Networking Route 53、 VPC、 ELB

Monitoring CloudWatch

Deployment Elastic Beanstalk、 CloudFormation

Payments FPS、 DevPay

Support Premium Support

E-Commerce FWS、 Mechanical Turk

Web Traffic Alexa Top Sites、 Alexa Web Information Service

EC2

http://opencompute.org/

Template (AMI)

Large Instance

High-CPU Extra Large Instance

High-Memory Double Extra Large Instance

Launch different instance types

32-bitType CPU RAM Storage Arch. IO Name

Small 1 ECU 1.7 GB 150 GB 32-bit Moderate m1.small

Micro Max 2 ECUs

613 MB EBS only 32-bit or 64-bit

Low t1.micro

High-CPU Medium

5 ECUs 1.7 GB 340 GB 32-bit Moderate c1.medium

*EC2 Compute Unit (ECU) – equivalent of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor.

64-bitType CPU RAM Storage Arch. IO Name

Large 4 ECUs 7.5 GB 840 GB 64-bit High m1.large

Extra Large 8 ECUs 15 GB 1680 GB 64-bit High m1.xlarge

High-CPU Extra Large

20 ECUs 7 GB 1680 GB 64-bit High c1.xlarge

High-Memory Extra Large

6.5 ECUs 17.1 GB 420 GB 64-bit Moderate m2.xlarge

High-Memory Double Extra Large

13 ECUs 34.2 GB 840 GB 64-bit High m2.2xlarge

High-Memory Quadruple Extra Large

26 ECUs 68.4 GB 1680 GB 64-bit High m2.4xlarge

High Performance ComputingType Processor RAM Storage Arch

.IO Name

Cluster Compute Quadruple Extra Large

33.5 ECUs(2 Intel Xeon X5570, quad-core Nehalem)

23 GB 1680 GB 64-bit Very high (10 Gbps Ethernet)

cc1.4xlarge

Cluster GPU Quadruple Extra Large 

33.5 ECUs (2 Intel Xeon X5570, quad-core Nehalem), 2 NVIDIA Tesla M2050 Fermi GPUs

22 +1 GB2 x 3 GB (on GPU ECC)

1680 GB 64-bit Very high (10 Gbps Ethernet)

cg1.4xlarge

*Tesla Fermi M2050 contains 448 cores and 3 GB of ECC RAM and are designed to deliver up to 515 gigaflops

NASA JPL

Processing 3.2 Giga-pixel Images In 2 Hours, 30 Instances. Improved an order of magnitude.

20 Gigaflops -> 1.3 Teraflops

MATLAB team at MathWorks

Regions & Availability Zones

就算殺了一個我，還有千千萬萬個我(a picture was here)

(a picture of Agent Smiths was here)

Pricing

TypeRegionOSOn-Demand/Reserved/Spot/TenancyPaid AMIInstance-hoursEBS volumes/snapshots

Elasticity & Agility

EBSEIPVPCEMRELBAuto ScalingCloudWatchBeanstalk

EC2 Customers

ZyngaPlayfishRedditMyspaceNetflixJustin.tvAnimotoHeroku

break;

EC2

Attach

EBS volume

Instance

EBS volume

Instance

EBS snapshot in Amazon S3

Create snapshot

EBS volume

Instance A

EBS snapshot in Amazon S3

Detach

Instance B

Same EBS volume

Attach

Create volume

Instance C

New EBS volume

Attach

Web Servers

App Servers

DB Servers

Authorize access from app server group

Authorize access from DB server group

Amazon EC2 Security Group Firewall

Open 80,443 to internet

Open 22 to corporate network

Open 22 to corporate network

All other traffic denied

S3

Save a picture OK

Save a DVD disc OK

Save a Bluray disc OK

PricingStorage us-east, eu-west, ap-se us-west ap-ne

Standard RRS Standard RRS Standard RRS

First 1 TB $0.140 $0.093 $0.154 $0.103 $0.150 $0.100

1~ 50 TB $0.125 $0.083 $0.139 $0.093 $0.135 $0.090

50 ~ 500 TB

$0.110 $0.073 $0.124 $0.083 $0.120 $0.080

500 ~ 1000 TB

$0.095 $0.063 $0.109 $0.073 $0.105 $0.070

1000 ~ 5000 TB

$0.080 $0.053 $0.095 $0.063 $0.090 $0.060

5000+ TB $0.055 $0.037 $0.070 $0.047 $0.065 $0.044

* Prices are calculated per GB** Requests and Bandwidth also incur charges

Simple & Reliable

Buckets & ObjectsCapacity : ∞High Scalability, Availability, DurabilityREST/SOAP/BTACL/metadata/RRSEncryptionIntegrate with other AWS services

S3 Customers

AmazonDropboxTwitterSmugmugYelpUbuntu OneZmandaTumblr

Distinction

Amazon.com’s ExperiencesReal Developers’ NeedsFull IaaS ProductsEnormous Developer Tools/Libraries

AWS Ecosystem

End UsersWeb Sites/StartupsISVsSystem Integrators

人人有功練(a picture was here)

OS

Red Hat Enterprise LinuxSUSE Linux EnterpriseFedoraWindows ServerAmazon Linux AMIGentoo LinuxOracle Enterprise LinuxUbuntu LinuxDebian

IBM DB2IBM Informix Dynamic ServerIBM Lotus Web Content ManagementIBM WebSphere Portal ServerIBM WebSphere Application ServerIBM sMash

Microsoft SQL Server StandardMicrosoft IIS/ASP.NETWindows Media ServerMySQL EnterpriseOracle Database 11gJboss Enterprise Application PlatformWowza Media Server ProSmartFoxStackIQ Rocks+

Demo

Alestic AMIs

EC2 Instance Initializationmbp:~ hank$ ec2-run-instances ami-7c423c2e -k sg-web -t m1.small -g www -z ap-southeast-1a...... i-fd1fd5a8 ami-7c423c2e pending sg-web 0......

mbp:~ hank$ ec2-describe-instances i-fd1fd5a8...... i-fd1fd5a8 ami-7c423c2e ec2-122-248-220-243.ap-southeast-1.compute.amazonaws.com ......

mbp:~ hank$ ssh -i sg-web.pem ubuntu@122.248.220.243

[ubuntu@ip-10-130-93-73 ~]$ sudo apt-get -y update && apt-get -y install apache2 php5 php5-curl

EC2 Instance Initializationmbp:~ hank$ ec2-run-instances ami-7c423c2e -k sg-web -t m1.small -g www -z ap-southeast-1a...... i-fd1fd5a8 ami-7c423c2e pending sg-web 0......

mbp:~ hank$ ec2-describe-instances i-fd1fd5a8...... i-fd1fd5a8 ami-7c423c2e ec2-122-248-220-243.ap-southeast-1.compute.amazonaws.com ......

mbp:~ hank$ ssh -i sg-web.pem ubuntu@122.248.220.243

[ubuntu@ip-10-130-93-73 ~]$ sudo apt-get -y update && apt-get -y install apache2 php5 php5-curl

LAME! I Want Automatic Initialization

EC2 Instance Initialization#!/usr/bin/env bashset -exsudo apt-get -y updatesudo apt-get -y install apache2 php5 php5-curl# install AWS SDK# install application code# get public-ipv4 from EC2 instance metadatapublic_ipv4 = $(wget -qO- 169.254.169.254/latest/meta-data/public-ipv4)# do something useful with $public_ipv4# more initializations ...

"init-web.sh"

EC2 Instance Initializationmbp:~ hank$ ec2-run-instances ami-7c423c2e -k sg-web -t m1.small -g www -z ap-southeast-1a -f init-web.sh...... i-c10fc594 ami-7c423c2e pending sg-web 0......

mbp:~ hank$ ec2-describe-instances i-c10fc594...... i-c10fc594 ami-7c423c2e ec2-122-248-221-187.ap-southeast-1.compute.amazonaws.com ......

mbp:~ hank$ curl 122.248.221.187<!doctype html><html><head><meta http-equiv="content-type" content="text/html; charset=UTF-8"><title>My Killer App</title>......

I Need More Computing Power, And I Need It NOW

mbp:~ hank$ ec2-run-instances ami-12423c40 -k sg-web -t c1.xlarge -g www -z ap-southeast-1a -f init-web.sh -n 10

...... i-320c424e ami-12423c40 pending sg-web 0............ i-5a2c2519 ami-12423c40 pending sg-web 1......

More Work

Scheduled Back UpMonitor And AlertNotify LB, Controller, etcFailoverMulti-AZ, Multi-RegionAutomatically Scale In/OutCeil, Floor, Cool down

AWS Judgment Day

When The Sky Is Falling

AWS Free Usage TierProduct Quota (for new users, expired in 1 year)

EC2 750 instance-hours. (t1.micro, Linux)

EBS 10GB , 1M I/Os. (snapshot: 10GB , 10K GETs, 1K PUTs)

ELB 750 LoadBalancer-hours, 15GB data processing

S3 5GB standard storage, 20K GETs, 2K PUTs

Network 15GB in and 15GB out. (except CloudFront)

Product Quota (for new users, expired in 1 year)

EC2 750 instance-hours. (t1.micro, Linux)

EBS 10GB , 1M I/Os. (snapshot: 10GB , 10K GETs, 1K PUTs)

ELB 750 LoadBalancer-hours, 15GB data processing

S3 5GB standard storage, 20K GETs, 2K PUTs

Network 15GB in and 15GB out. (except CloudFront)

* Quota are calculated per month

If I have seen further it is by standing on the shoulders of giants

http://en.wikipedia.org/wiki/File:Orion_aveugle_cherchant_le_soleil.jpg

Thank you

(a picture was here)