輕鬆學Google的雲端開發 - Google App Engine入門

Simon Su & Sunny HuJCConf 2015

http://goo.gl/X8YY73

Githubhttps://github.com/peihsinsu/JCConf2015-GAE-Workshop

We are QNAPer, Now!

https://www.facebook.com/groups/GCPUG.TW/

https://plus.google.com/u/0/communities/116100913832589966421

Google Cloud Platform User Group Taiwan我們是Google Cloud Platform Taiwan User Group。在Google雲端服務在台灣地區展露頭角之後，

有許多新的服務、新的知識、新的創意，歡迎大家一起分享，一起了解 Google雲端服務...

GCPUG透過網際網路串聯喜好Google Cloud的使用者，分享與交流使用 GCP的點滴鑑驗。如果您

是Google Cloud Platform的初學者，您應該來聽聽前輩們的使用經驗；如果您是 Google Cloud Platform的Expert，您應該來分享一下寶貴的經驗，並與更多高手互相交流；如果您還沒開始用

Google Cloud Platform，那麼您應該馬上來聽聽我們是怎麼使用 Google Cloud的！

●●●●●●●

Compute

Compute Engine

App Engine Container Engine

Storage

Cloud Storage

Cloud SQL Cloud Datastore

App Services

Cloud EndpointsPrediction API Translate APIBigQuery

Big Data

Pub/Sub Dataflow Bigtable

https://console.developers.google.com/project?utm_referrer=blank

Create your own project

https://console.cloud.google.com/freetrial

Free USD$300 Trail

●

Eclipse version Installation instructions Direct plugin link

Eclipse 4.4 (Luna) Plugin for Eclipse 4.4 (Luna) https://dl.google.com/eclipse/plugin/4.4

Eclipse 4.3 (Kepler) Plugin for Eclipse 4.3 (Kepler) https://dl.google.com/eclipse/plugin/4.3

Eclipse 3.8/4.2 (Juno) Plugin for Eclipse 3.8/4.2 (Juno) https://dl.google.com/eclipse/plugin/4.2

●●●

Java source code

Web source code

GAE setting

Servlet setting

●●●●

…

●●●

●●

基本設定

<async-session-persistence enabled="true" />

…

CallbackPage

ResultPage

extends: AbstractAppEngineAuthorizationCodeServlet

extends: AbstractAppEngineAuthorizationCodeCallbackServlet

1

2

34

整合

web.xml

取得

/get-application-id.jsp

●●●●

●●●

Googler’s wayAppEngine

Traditional Web applications

Web application framework

AppEngine(Java, Python, Go, PHP)

Java, Perl/CGI, PHP, Ruby, Python...

Persistent storageNoSQL

● Datastore● Cloud SQL

RDBMS● MySQL● PostgreSQL● SQL Server● Oracle

Datastore RDBMS

Query language flexibility

SQL-like query language● Limited to simple

filter and sort

Full support of SQL● Table JOIN● Flexible filtering● Subquery

Reliability and Scalability

Highly scalable and reliable

Hard to scale

●○○○

●○○

Datastore RDBMS

Category of object Kind Table

One entry/object Entity Row

Unique identifier of data entry Key Primary Key

Individual data Property Field

Property

Property

Property

PostEntry User

Kinds

Key: blog-1234user: simonsu@xxx.commessage: xxxxxxdate: 3/1/2014

Key: simonsu@xxx.comemail: simonsu@xxx.comfollowees: [user2@xxx.com, user3@xxx.com]followers:

Key: user2@xxx.comemail: user2@xxx.comfollowees:followers: [simonsu@xxx.com]

Entities

Keys

DatastoreService datastore =

DatastoreServiceFactory.getDatastoreService();

Entity employee = new Entity("Employee");

employee.setProperty("name", "Simon Su");

employee.setProperty("hireDate", new Date());

Key empKey = datastore.put(employee);

# DatastoreExample.java

// Use email as key when creating entity

Entity employee = new Entity("Employee", "work-id-D001");

datastore.put(employee);

// Later, use the key to retrieve the entity

Key userKey = KeyFactory.createKey("Employee", "work-id-D001");

Entity user = datastore.get(userKey);

# DatastoreExample.java

DatastoreService ds = DatastoreServiceFactory.getDatastoreService();

Key key = KeyFactory.createKey("User", "simonsu.mail@gmail.com");

try {

Entity ent = ds.get(key);

out.print(ent);

Map m = ent.getProperties();

out.println(m.toString());

} catch (EntityNotFoundException e) {

e.printStackTrace();

}

Query query = new Query("Person");

Query.Filter nameFilter = new Query.FilterPredicate( "name", FilterOperator.EQUAL, "John");

query.setFilter(nameFilter);

PreparedQuery results = datastore.prepare(query);

# DatastoreExample.java

Query q = new Query("Person");

Query.Filter filter1 = new FilterPredicate(...);

Query.Filter filter2 = new FilterPredicate(...);

Query.Filter comboFilter =

CompositeFilterOperator.and(filter1, filter2);

q.setFilter(comboFilter);

q.addSort("name");

# DatastoreExample.java

Query query = new Query("Kind");query.setAncestor(parentKey);

● Manual configure:WEB-INF/datastore-indexes.xml

● System generated: WEB-INF/appengine-generated/datastore-indexes-auto.xml

<?xml version="1.0" encoding="utf-8"?><datastore-indexes> <datastore-index kind="Widget"> <property name="x" direction="asc" /> <property name="date" direction="asc" /> </datastore-index> <datastore-index kind="Widget"> <property name="y" direction="asc" /> <property name="date" direction="asc" /> </datastore-index></datastore-indexes>

<?xml version="1.0" encoding="utf-8"?><datastore-indexes autoGenerate="true"></datastore-indexes>

●●

Query for: first_name = Cathy last_name > Able last_name < Mooney

Query for: first_name > Cathy last_name > Able

Query for: first_name = Cathy last_name > Able sort by last_name Query for:

last_name > Able sort by first_name

DatastoreService datastore = DatastoreServiceFactory.getDatastoreService()Transaction txn = datastore.beginTransaction();try { Key ekey = KeyFactory.createKey("Employee", "Joe"); Entity employee = datastore.get(ekey);

/*... reading and writing on employee ...*/

datastore.put(employee); txn.commit();

} finally { if (txn.isActive()) { txn.rollback(); }}

# DatastoreExample.java

● Datastore basic● Datastore operation● Restrictions● Transaction

DatastoreService ds = DatastoreServiceFactory.getDatastoreService();

Entity job = new Entity("User", "simonsu@mitac.com.tw");

job.setProperty("name", "Simon Su");

job.setProperty("start", "20140103");

job.setProperty("create", new Date());

try {

Key k = ds.put(job);

out.println("Done..." + k.getId());

} catch (Exception e) {

e.printStackTrace();

}

User

simonsu@...

File: DatastoreExample.java

DatastoreService ds = DatastoreServiceFactory.getDatastoreService();

Key userKey = KeyFactory.createKey("User", "simonsu.mail@gmail.com");

Entity job = new Entity("Jobs", userKey);

job.setProperty("name", "engineer");

job.setProperty("start", new Date());

try {

Key k = ds.put(job);

out.println("Done..." + k.getId());

} catch (Exception e) {

e.printStackTrace();

}

User

Jobs

File: DatastoreExample2.java

DatastoreService ds = DatastoreServiceFactory.getDatastoreService();

Query q = new Query("Jobs");

Key ancestor = KeyFactory.createKey("User", "simonsu.mail@gmail.com");

q.setAncestor(ancestor);

PreparedQuery results = ds.prepare(q);

Iterator iter = results.asIterable().iterator();

while(iter.hasNext()) {

Entity ent = (Entity) iter.next();

out.println(ent);

}

File: DatastoreExample2.java

●●●●

Memcache

●●●●●

MemcacheService cache = MemcacheServiceFactory.getMemcacheService();

cache.setErrorHandler(

ErrorHandlers.getConsistentLogAndContinue(Level.INFO));

// read from cache

value = (byte[]) cache.get(key);

// save to cache

if (value == null) {

// ........

cache.put(key, value);

}

# DatastoreExample.java

// Using the asynchronous cache

AsyncMemcacheService asyncCache =

MemcacheServiceFactory.getAsyncMemcacheService();

asyncCache.setErrorHandler(

ErrorHandlers.getConsistentLogAndContinue(Level.INFO));

// read from cache

Future<Object> futureValue = asyncCache.get(key);

// ... do other work in parallel to cache retrieval

value = (byte[]) futureValue.get();

if (value == null) {

// get value from other source

// ........

// asynchronously populate the cache

// Returns a Future<Void> which can be used to block until completion

asyncCache.put(key, value);

}

import java.util.Collections;

import net.sf.jsr107cache.*;

Cache cache;

try {

CacheFactory cacheFactory =

CacheManager.getInstance().getCacheFactory();

cache = cacheFactory.createCache(Collections.emptyMap());

} catch (CacheException e) {

// ...

}

String key; // ...

byte[] value; // ...

// Put the value into the cache.

cache.put(key, value);

// Get the value from the cache.

value = (byte[]) cache.get(key);

●

●

●

●●●

Task Queue

●●●●

○○○○

●○○

●○

■■

○■■■

<queue-entries>

<queue>

<name>Queue-Name</name>

<rate>1/s</rate>

...

</queue>

<queue>

<name>Queue-Name2</name>

<mode>pull</mode>

...

</queue>

</queue-entries>

Push Queue

Pull Queue

Queue queue = QueueFactory.getDefaultQueue(); //named queue: //QueueFactory.getQueue("Qname");

TaskOptions task = TaskOptions.Builder .withUrl("/path-to-my-worker") .param(key, value);

queue.add(task);

Reachable Servlet

●

●●●●

○○

○○○

●

DatastoreService ds = DatastoreServiceFactory.getDatastoreService();Queue queue = QueueFactory.getDefaultQueue();try { Transaction txn = ds.beginTransaction();

// … other operations queue.add(TaskOptions.Builder.withUrl("/path-to-my-worker")); // … other operations

txn.commit();} catch (DatastoreFailureException e) { // … exception handle}

Queue queue =

QueueFactory.getQueue("queue_name");

TaskOptions task =

TaskOptions.Builder

.withMethod(TaskOptions.Method.PULL)

.payload(payload);

queue.add(task);

Queue queue =

QueueFactory.getQueue("queue_name");

tasks = q.leaseTasks(

duration,

TimeUnit.SECONDS,

how-many);

//delete named taskqueue.deleteTask("task_ name");

//purge all tasks from queueQueueFactory.getQueue("queue_name").purge();

<queue>

<name>pull-queue1</name>

<mode>pull</mode>

<acl>

<user-email>me@gmail.com</user-email>

<writer-email>you@gmail.com</writer-email>

<writer-email>she@gmail.com</writer-email>

</acl>

</queue>

Cron

<cronentries>

<cron>

<url>/recache</url>

<description>

Put your description here

</description>

<schedule>every 5 minutes</schedule>

</cron>

</cronentries>

●●●●●●●●

●●

●

●

●

●

●

●

●

●

●●●●

●●

$ gcloud auth login

$ gcloud config set project [my-project-id]

$ gsutil mb gs://[my-bucket]

$ gsutil ls gs://[my-bucket]

$ gcloud sql -h

$ gcloud sql instances create [inst-name]

$ gcloud sql instances list

$ gcloud sql instances import instance-name --uri \

gs://your-bucket/sql-dump-file.gz

$ gcloud sql instances patch --assign-ip [inst-name]

$ gcloud sql instances get [inst-name]

$ gcloud sql instances set-root-password test-sql \

-p [password]

$ gcloud sql instances patch test-sql \

--authorized-networks=[your-ip-address]

http://www.mysql.com/products/workbench/

●●

Using Cloud Storage

BlobstoreService blobstoreService =

BlobstoreServiceFactory.getBlobstoreService();

UploadOptions uploadOptions =

UploadOptions.Builder.withGoogleStorageBucketName(

"BUCKET-NAME"); //Name without “gs://”

String uploadUrl = blobstoreService.createUploadUrl(

"/callback-url", uploadOptions);

resp.getWriter().println("<form action=" + uploadUrl

+ " method=\"post\" enctype=\"multipart/form-data\">");

resp.getWriter().println("<input type=\"file\" name=\"field-name\"");

public void doPost(HttpServletRequest req, HttpServletResponse resp) throws IOException {

BlobstoreService blobstoreService = BlobstoreServiceFactory.getBlobstoreService();

Map<String, List<FileInfo>> blobs = blobstoreService.getFileInfos(req);

List<FileInfo> keys = blobs.get("field-name"); //input file name on form if (keys != null && keys.size() > 0) { FileInfo fileKey = keys.get(0);

String objectUploaded = fileKey.getGsObjectName();…. (Save the objectUploaded mapping to datastore or database)

public void doGet(HttpServletRequest req, HttpServletResponse resp){

BlobstoreService blobstoreService = BlobstoreServiceFactory.getBlobstoreService();

BlobKey blobKey = blobstoreService.createGsBlobKey("/gs/<bucket>/<object>");

try {blobstoreService.serve(blobKey, resp);

} catch (IOException e){e.printStackTrace();

}

….

補充

●●

Connect Cloud SQLSystemProperty.Environment.Value.Production) {

Class.forName("com.mysql.jdbc.GoogleDriver");

String url= "jdbc:google:mysql://instance-name/";

conn = DriverManager.getConnection(url);

try {

conn.createPreparedStatement…

...

●●

○○○

●●

●

<?xml version="1.0" encoding="utf-8"?><appengine-web-app xmlns="http://appengine.google.com/ns/1.0"> <application>mitac-cp300</application> <version>eord</version> …. <use-google-connector-j>true</use-google-connector-j> ….</appengine-web-app>

●●●●

Connection conn = null;if (SystemProperty.environment.value() ==

SystemProperty.Environment.Value.Production) {// app is running in productionClass.forName("com.mysql.jdbc.GoogleDriver");String url= "jdbc:google:mysql://instance-name/";conn = DriverManager.getConnection(url);...

} else {// app is running on localhostClass.forName("com.mysql.jdbc.Driver");String url= "jdbc:mysql://db-ip-address/";conn = DriverManager.getConnection(url);...

}conn.~

DriverManager.registerDriver(new AppEngineDriver());

conn = DriverManager.getConnection(connUrl);

String sql = "SELECT count(*) as total FROM " + tableName + "";

PreparedStatement stmt = conn.prepareStatement(sql);ResultSet results = stmt.executeQuery();

while (results.next()) {int total = results.getInt("total");…

}

●●●●

Q & A