View
212
Download
0
Category
Preview:
Citation preview
A conceptual model of grid resources and services
Authors:Sergio Andreozzi
Massimo SgaravattoCristina Vistoli
Presenter: Sergio AndreozziINFN-CNAF
Bologna (Italy)sergio.andreozzi@cnaf.infn.it
CHEP 2003 - March, 24 20032
OUTLINE
Short introduction to the GLUE activity GLUE Schema overview
– The conceptual model– The implementation status– Deployment roadmap
Related works Open issues
CHEP 2003 - March, 24 20033
GLUE: WHAT
GLUE: Grid Laboratory Uniform Environment collaboration effort focusing on interoperability
between US and EU HENP Grid middlewares Targeted at core grid services
– Resource Discovery and Monitoring– Authorization and Authentication – Data movement infrastructure– Common software deployment procedures
Preserving coexistence for collective services
CHEP 2003 - March, 24 20034
GLUE: WHO and WHEN
Promoted by DataTAG and iVDGL projects
Activity started in April 2002 considering as first step common information service schema between EU and US HENP Grid projects
Important contributions from the following projects: DataGrid, Globus, GriPhyN, PPDG, SRM
Main contributors (in alph order): J.Gordon, C.Kesselman, P.Kunstz, J.McGee, R.Pordes, A.Shoshani, J.Schopf, B.Tierney
CHEP 2003 - March, 24 20035
GLUE Schema overview 1/2
Conceptual model of grid resources to be used by the Grid Information Service for discovery and monitoring purposes
Based on the experience of DataGrid and Globus schema proposals
CHEP 2003 - March, 24 20036
GLUE Schema overview 2/2
Conceptual model – version 1.0 Finalized in Oct ’02 Model of computing resources (Ref. CE) Model of storage resources (Ref. SE) Model of relationships among them (Ref. Close
CE/SE) Currently working on version 1.1
Extensions Model of network resources
Possible other adjustements will come from experience (e.g. DataGrid v.2.0)
CHEP 2003 - March, 24 20037
GLUE Computing resources requirements
Separation between services and resources that implement it
Needs for both detailed host info (monitoring issue) and aggregate view (discovery issue)
CHEP 2003 - March, 24 20038
GLUE Computing Element
Computing Element: entry point into a queuing system– There is one computing element per queue of
a local resource management system– The information associated with a computing
element is limited only to information relevant to the queue
– All information about the physical resources accessed by a queue is represented by the Cluster information element
CHEP 2003 - March, 24 20039
GLUECluster/Subcluster/Host
Cluster: container that groups together subclusters or hosts. A cluster may be referenced by more than one computing element
Subcluster: collection of “homogeneous” hosts:– All hosts of a subcluster have the same value for the
set of asserted attributes Host: characterizes the configuration of a
computing node (e.g. processor, main memory, software)
CHEP 2003 - March, 24 200310
Computing Resources in GLUE
ComputingElement
ComputingElement
ComputingElement
ComputingElement
ComputingElement
ComputingElement
subcluster2subcluster1
Cluster 1
CHEP 2003 - March, 24 200311
CHEP 2003 - March, 24 200312
CHEP 2003 - March, 24 200313
GLUEStorage Service/Space/Library
Storage Service:– grid service identified by a URI that manages disk and
tape resources in term of Storage Spaces– all hardware details are masked– the Storage Service performs file transfer in or out of
its Storage Spaces using a specified set of third part data movement services (e.g. GridFTP)
– files are managed with respect to the lifetime policy specified for the Storage Space where they are kept
a specific date and time lifetime policy can be specified for each file and this is applied against a compatibility rules table
CHEP 2003 - March, 24 200314
GLUEStorage Service/Space/Library
Storage Space: portion of a logical storage extent identified by: – an association to a directory of the
underlying file system (e.g. /permanent/CMS)
– a set of policies (MaxFileSize, MinFileSize, MaxData, MaxNumFiles, MaxPinDuration, Quota)
– an association to access control base rules
CHEP 2003 - March, 24 200315
GLUEStorage Service/Space/Library
Storage Library: the machine providing for both storage space and storage service
CHEP 2003 - March, 24 200316
GLUEStorage Service/Space/Library
Storage LibraryArchitecture type + file system + files
Storage Serviceprotocol info
Storage SpaceStatus, Policies, Access Rules
Directory
CHEP 2003 - March, 24 200317
CHEP 2003 - March, 24 200318
Computing/Storage Services relationships in Glue
The problem:– Job executed on Computing Elements (CEs)– Job may access files stored in Storage Space– Several replicas of these files can be spread over
the grid– The best replica is CE-dependent– Which strategy to assign the job to a CE and select
the best replica for it? Current solution:
– Computing-Storage Services association statically defined by SiteAdmin’s
– When the network resource model will be in place, this association could be dynamically defined
CHEP 2003 - March, 24 200319
GLUE SchemaImplementation status
Implementation status For Globus MDS 2.x:
LDAP Schema (DataTAG WP 4.1) Info providers for both computing and storage
resources
For EDG R-GMA: Relational model implementation (DataGrid, both
schema and information providers)
For Globus OGSA: XML Schema (Globus)
CHEP 2003 - March, 24 200320
Deployment roadmap
– In DataTAG, a mixed testbed already in place with: EDG 1.4.x (+ Glue) nodes in Italy LCG-0 node at CERN VDT nodes in US (FNAL, Indiana University)
– CMS, ATLAS and ALICE experiment suites already successfully integrated with EDG Resource Broker, Glue schema based
– Grid middleware distributions and the Glue schema Already included in
– VDT 1.1.7– LCG-0
Will be soon included in – EDG 2.0– Globus 2.2.x
CHEP 2003 - March, 24 200321
Related works
EDT-LCG Monitoring collaboration:– development of a Grid monitoring tool in order to
monitor the overall functioning of the Grid– Will be presented here at CHEP
Today, parallel session 1b-Monitoring, 4.50 PM
Network resource modeling: – Definition of a network model that enables an efficient
and scalable way of representing the communication capabilities between grid services for brokering activity
– Will be presented here at CHEP: next Thursday, parallel session 1b–Monitoring, 4.10 PM
CHEP 2003 - March, 24 200322
Main open issues
– Computing: refine aggregated view of a cluster from the
queue viewpoint
– Storage: Gain better understanding of Storage Service,
when mapping a disk space access or an SRM service
– High Level Grid Services, define a general grid service
CHEP 2003 - March, 24 200323
REFERENCE
DataTAG Project– http://www.datatag.org
Grid Laboratory Uniform Environment (GLUE) DataTAG WP4 and iVDGL Interoperability Group version 0.1.2
– http://www.hicb.org/glue/glue-v0.1.2.doc GLUE Schema documents
– http://www.cnaf.infn.it/~sergio/datatag/glue EDT-LCG Monitoring
– http://gridmon.na.infn.it/lcg-edt GGF CIM Grid Schema WG
– http://www.isi.edu/~flon/cgs-wg/
Recommended