www.geogrid.org

GEO Grid: Federating Geospatial Data andGEO Grid: Federating Geospatial Data and Integrating Services

Satoshi Sekiguchi, Yoshio Tanaka, Isao KojimaGrid Technology Research CenterGrid Technology Research Center

National Institute of Advanced Industrial Science and Technology(AIST), Japan

1

www.geogrid.org

What is the AIST ?What is the AIST ?

One of the largest Nat’l gLabs in Japan

3,500 employee+3,000 staff

Research topics includeEnvironmentMate ialMaterialBio/Life scienceStandards (JIS/ISO)( )GeologyGeologySemiconductor deviceIT/Computer ScienceIT/Computer Science

AIST Tsukuba Main Campus7 other campuses across Japan

IT/Computer ScienceIT/Computer Scienceetc.

NIST+USGS+DOE l b Narita

Tsukuba50km

40km

2

labs+NIH NaritaTokyo

50km

50km

www.geogrid.org

What is the GEO Grid ?What is the GEO Grid ?The GEO (Global Earth Observation) Grid is aiming at providing an E-Science Infrastructure for worldwide Earth Sciences communities to accelerate GEO sciences based on the concept communities to accelerate GEO sciences based on the concept that relevant data and computation are virtually integrated with a certain access control and ease-of-use interface those are enabled by a set of Grid and Web service technologies enabled by a set of Grid and Web service technologies.

AIST: OGF Gold sponsor (a founding member)

AIST: OGC Associate member (since 2007)

Satellite Data GeosciencesGrid

Technologies

Geo* ContentsApplications

Geology Data

GIS dataEnvironmentConservation

RDisaster

3

In-situ data Resources mitigation

www.geogrid.org

ASTER : Main ASTER : Main content content of AIST GEO Gridof AIST GEO GridASTER : Main ASTER : Main content content of AIST GEO Gridof AIST GEO Grid

ASTER was developed by METI with AIST scientific & engineering supportsGlobal land coverage of Digital Elevation Model of 15m spatial resolution Global land coverage of Digital Elevation Model of 15m spatial resolution Excellent geo-location accuracy

Easy to mosaic (or make a seamless image/DEM) Easy to overlay to GIS dataEasy to overlay to GIS data

Powerful spectral analysis VNIR 3 bands (+backward 1band)/SWIR 6 bands/TIR 5 bands

20002000~

4

www.geogrid.org

ASTER Global land coverageASTER Global land coverage

5

ASTER archive 1.4 million images = 140 TBMETI/NASA

www.geogrid.org

Digital Elevation Model of 15m spatial resolution Digital Elevation Model of 15m spatial resolution L d lid l L d lid l ti ti i ASTER DEMti ti i ASTER DEMLandslides volume Landslides volume estimation using ASTER DEMestimation using ASTER DEM

Synthetic bird eyes view with ASTER image and DEM

Volume : 0.06 km3, area : 0.5 km2

at DEM difference > 50 m

6

14NOV2000 11OCT2005

www.geogrid.org

ASTER excellent geoASTER excellent geo--location accuracylocation accuracy

Boulder, CO

7

Easy to overlay GIS data (ex. Road map) without adjustment manually

,

www.geogrid.org

DevelopmentDevelopment--caused change at caused change at SuvarnabhumiSuvarnabhumi AirportAirport(near Bangkok, Thailand)(near Bangkok, Thailand)( ea a g o , a a d)( ea a g o , a a d)

2000/11/02

2002/07/03

2004/10/28

2006/02/042006/02/04

8

Data achieved since 2000

www.geogrid.org

More More effective use of ASTER effective use of ASTER DataDataIncorporate with geospatial information to overlay

Integrated data search service across multiple satellite i d imagery data setsInteroperable with GIS based on OGC services and others

WMS, WFS, WCS, CSWISO 19115, GML, GeoSciML

Compliance with owners’ access control policy of Compliance with owners access control policy of data/serviceGrid Security Infrastructure – AuthN, AuthZ, AccountingAccounting

Integrating with processing (computing) servicesH d/ li ht i l ti WPS G id RPC tHeavy and/or light simulations: WPS, Grid RPC, etcResource management – infra providerComplex workflow support in portals incl. data access,

9

p pp p ,simulation execution, visualization, etc.

www.geogrid.org

ASTER ASTER –– FORMOSATFORMOSAT--2 Integrated Data Search2 Integrated Data SearchGet Get satellite satellite imagery of a particular imagery of a particular location from federated databases location from federated databases Get Get satellite satellite imagery of a particular imagery of a particular location from federated databases location from federated databases

R lt

FORMOSAT-2 OrbitFORMOSAT-2 Orbit Google EarthGoogle EarthYesNo

ASTERASTER FORMOSAT-2FORMOSAT-2

Result

Input Key:

PeriodLat/Lon

GEO G id P t lGEO G id P t lSpot ImageSpot Image

Cloud GEO Grid Portal(ASTER DB)

GEO Grid Portal(ASTER DB)

p g(FORMOSAT-2 DB)

p g(FORMOSAT-2 DB)

Result

Input Key

P i dResult

Date

PeriodLat/LonCloudetc

10

ImageOneImageOne

www.geogrid.org

11

www.geogrid.org

Overlay GISOverlay GISyy

12

www.geogrid.orgEnergy Cone Simulation

GEO Grid use case GEO Grid use case scenarioscenarioVolcano Volcano monitoringmonitoringVolcano Volcano monitoringmonitoring

M l i

←３D+Temp

13

AP photo

MelapiASTER

www.geogrid.org

火砕流火砕流 メラピ山オルソメラピ山オルソ パラメータ設定パラメータ設定

14

www.geogrid.org

火砕流火砕流 メラピ山オルソメラピ山オルソ 処理結果処理結果

15

www.geogrid.org

A Workflow example A Workflow example

““Disaster prevention and mitigation (Volcano)Disaster prevention and mitigation (Volcano)””In-situ observations by

CCOP membersHazard Map for

Evacuation planningMonitoring of crustal

deformation by PALSAR

Disaster prevention and mitigation (Volcano)Disaster prevention and mitigation (Volcano)

GEO

ASTER

16

High resolution DEM provided from ASTER

Simulation of lava and/or pyroclastic flow on GEO Grid

www.geogrid.org

A Workflow example A Workflow example

““Disaster prevention and mitigation (Landslide)Disaster prevention and mitigation (Landslide)””Rain fall

(AMEDAS/Fieldserver)

Early warning systembased on Susceptibility map

Geology map (GSJ)Disaster prevention and mitigation (Landslide)Disaster prevention and mitigation (Landslide)

GEOMET

ASTER

High resolution DEM La ge scale Comp te sim lation

17

High resolution DEM provided from ASTER

Large scale Computer simulation using actual landslide DB

www.geogrid.org

But, it was not so simple …

More sophisticated security architecture

Data access and integration

18

www.geogrid.org

GEO G idGEO Grid

Applications Disastermitigation Land slides, floodmitigation

Environmentmonitoring Global warming, CO2 flux estimation

Full L0 ASTER on disk

Oil, Gas Natural resourceexploration

Contents Satellite Imagery

Geology archives Japan, SE Asia

Full L0 ASTER on diskMODIS on disk (East Asia)

gy p ,

Sensors AsiaFlux, Field server

ITInfrastructure Software

Security, data access, service registry, resource mgmt., Weg GIS, Workflow, U/I Portal, etc.

19

Hardware Storage, ServersCluster computers

www.geogrid.org

GEO Grid Players & ToolsGEO Grid Players & Toolsyy

Player Tools Product

End Users Web browser GEO Grid Client

Portal & VO mgr Project Goal

GEO GridPortal Dev. KitVO t l

Customized Web portalVO Management SystemVO mgr

Data Provider

Scientific data

VO tools

G O G dGEO Grid Data Service

Provider

Resource P id

data

Application, IT

GEO GridService Dev. Kit

GEO Grid Hosting ServiceProvider IT resource

G O G d ost g Se ce

GEO Grid Tool kit

20

GEO Grid Tool kit

www.geogrid.org

Disaster iti ti

Environment it i

Natural resource l tiApplication mitigation

Workflow, Portal, PSE

monitoring exploration Application

Framework

Data Access + ComputingService Integration

R i t

Job Submission

A i

Brokering

D t T f

Storage Services

Open Grid

Structured Data Access

Registry

Info. Services Resource Mgmt.

Accounting Data TransferOpen Grid Service

Data and computing as services Accessg

Service

Security, Messaging, CommunicationConnectivity

21

Computing, Data & Storage Structured DataRelational/XML/RDF/WebDB

Resource

www.geogrid.org

AIST GEO Grid Architecture (IT Infrastructure)AIST GEO Grid Architecture (IT Infrastructure)l

Disas

preven

Ecolo

Agricul

Geolo

Applications

sterntion

ogy

lture

ogy

Visualizationservice

VO management

Std Web Service I/F incl. basic grid service

VO hosting services

Interoperable i GEON

GG US

GML/G-XML/OGCTiled Wall Display

Data Grid Computing Grid

Base Map Application hosting engine

GS

J data

GIS

data

US

GS

data

FieldS

ensors

gfarm

ASTER MODISASTER & MODIS

DSM/DEMPALSAR

hosting engineOverlayable GIS data PALSAR

Geometric, Radiometric &Atmospheric CorrectionMosaic & Composite,

Tsukuba (Japan)

MAP on Demand

AIST Supercluster

210TB /P4 3.5TB(raid5)/node x 60C

AIT(Thailand)

Image Processing

22

L0 or L1L1A data will not be in public

NASA(USA)

AISTAIST Supercluster

(11TFlops)

HATOYAMA (Japan)ASTER

MODIS

PALSAR

www.geogrid.org

Catalogue Service OGC-compliantclient

① Search Catalogue

httpCSW

Web serverRDB (postgres, oracle,…)

client

Java Script +

Viewer(google map, deegree, …)

http

httphttp

pGoogleMap API, OGC Client

Web Client

PC② Query metadata

OGSA-DAI CSW / WMS+

MetadataPC

③ Retrieve Data

http

Web serverGeo-featured DB

(PostGIS, oracle + geo, …)

Grid client

W b Cli t

Portlet

③ Retrieve Datahttp

OGC-compliant Services(WMS, WFC, WCS)

DataWeb Client

PC③ Computation

httpGRAM(WMS, WFC, WCS)

Web serverGeo-featured DB

(PostGIS, oracle + geo, …) Computation

GRAMWeb serverApplications

WPS

23

Data(satellite, GIS, …)

ComputationComputers

www.geogrid.org

Security issuesSecurity issues

24

www.geogrid.org

FOUR categories of Data Access AttributeFOUR categories of Data Access AttributeggALLMany

OPENCat. 2

Open to ManCat. 1

Open to AllOPEN Open to Many(Black list)

Open to All(Public)

CLOSECat. 3

Close to Many(White list)

Cat. 4Close to All

(Private)(White list) (Private)

25

www.geogrid.org

You might need to have like thisYou might need to have like this

local

Private Virtual Organization (VO) enabled by GRID In-Situ

Cat 3 Project/Community PEN

Cat 2 Domain CommonCluster

ASTER

Internet

Cat 1 Public

26

Cat. 1 Public

www.geogrid.org

Security Architecture Design PolicySecurity Architecture Design Policy

Introduces concept of VO (Virtual Organization)Data and computation are provided as “services” via Data and computation are provided as services via standard protocols and APIs.A VO is created dynamically by integrating available services and resources according to the interests and requirements of the VO. User level Authentication and VO level AuthorizationUser-level Authentication and VO-level Authorization

User’s right is managed (assigned) by an administrator of his belonging VO.administrator of his belonging VO.Access control to a service is configured by the service provider according to the publication

li Th ti f th policy. There are some options of the access control

VO-level, Group/Role-based, User-level, etc.

27

VO level, Group/Role based, User level, etc.Scalable architecture for the number of users.

www.geogrid.org

Overview and usage model of the GEO Grid systemOverview and usage model of the GEO Grid system

28

www.geogrid.org

account

Security ArchitectureSecurity Architecture

GAMA

CACA

AccountDB

M P

GEO GridAdmin

login “username /Password”

creation User Aw/o certificate GEO Grid Portal

GAMA

MyProxy X.509 long-livedcertificates

User Bw/ certificate

BB

login by certificate X 509 proxy

credential repository

ＡＡ

AABB

ＡＡ

X.509 proxytifi t

certificate X.509 proxycertificates

Anonymous User

anonymouslogin

AABB

VOMS VOMS serverserver

certificatesw/ VOMS attributes

Project

y

GEO GridMeta Data CatalogueVirtual DB

2 Fetch Meta Data info

1. simple query

blacklistwhitelist

Admin

ASTER

Meta D

ata

2. Fetch Meta Data info

3. Data Access with appropriate method

grid-mapfileconfig file

LCASLCMAPS

Filesystem ACL ASTERa

PDP #1

PDP #2

Filesystem ACLAuthZ

29

ASTER(JPEG)

File ASTER(WMS)

ASTER(WFS)Service

ProviderPDP #n

www.geogrid.org

Data IntegrationData Integration

30

www.geogrid.org

GEO Grid data integration GEO Grid data integration -- at a glanceat a glance

CSW WFSWPS WMS

OGC Standards

Workflow engines OGSA-DQPOGC SQLSpatial querySupport of OGC standards

S i

Distributed JoinsService Coordination

Support of OGC standardswith

Grid-based federated DatabasesOGSA-DAI@AISTServicesServices

OGSA-DAI

D t S i

OGSA-WebDB

Data Services GSJ

ASTERData Resources

GTRC

MODIS

Data Services

31

Parallel SQL processing

www.geogrid.org

Data Integration Requirements(1)Data Integration Requirements(1)g q ( )g q ( )★ Support of Geo-Spatial Standards

OGC standardsCSW for catalog serviceCS o cata og se ceWMS,WFS,WCS for geospatial data servicesWPS for data processingOGC ti l OGC spatial query

Data Format standardsData Format standardsData

GML and other XML data formatM t d t ( h )Metadata(schema)

ISO19115,Dublin Core, ebXML etc.

32

www.geogrid.org

Data Integration Requirements(2)Data Integration Requirements(2)g q ( )g q ( )★ Support of Distributed Database Federation

Support of multi-level distributed database federation

Local = Different Resources under one ServiceEnterpriseGlobal = Distributed Services over the worldGlobal = Distributed Services over the world

Support of Heterogeneous Resourcespp gRelational databasesXML databasesWeb databasesWeb databases

33

www.geogrid.org

Data Integration Requirements(3)Data Integration Requirements(3)g q ( )g q ( )★ Support of the Grid Technology

Support of the VO conceptVO b d d t b th i ti / it VO based database authorization/security management

Support of Grid Standards & MiddlewaresOGF Standards & MiddlewareOGF Standards & Middleware

WS-RF/Globus, MDS,GSI(VOMS), OASYS/W3C/

BPEL, Service Registry(ebXML/UDDI)

34

www.geogrid.org

ApproachApproachpppp

Multi-level-distributed data integrationg→ Service-based Database Integration

MiddlewareMiddleware

S t f VO tSupport of VO concept→ VOMS enabled Database Middleware

OGSA-DAI with VOMS

Support of Grid Standard & MiddlewareSupport of Grid Standard & Middleware→ Grid based Database Middleware

35

www.geogrid.org

A sample of ASTER DEM image on Tiled DisplayA sample of ASTER DEM image on Tiled Display

36

www.geogrid.org

37

www.geogrid.org

38