Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20042004
Current Status and Future Current Status and Future Direction of UNICORE and OGSADirection of UNICORE and OGSA
NAREGI SymposiumNAREGI SymposiumFebruary 23February 23--24, 200524, 2005Dr. David F. SnellingDr. David F. SnellingFujitsu Labs EuropeFujitsu Labs Europe
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
UniGrids ObjectivesUniGrids Objectives
Develop:Develop:An OGSA compliant Unicore ImplementationA framework and technology for Grid interoperabilityA framework for Grid economics
Create Generic Services forCreate Generic Services forVisualisation and steering of simulationsDevice monitoring and controlAccess to distributed data and databasesApplication instrumentation
Influence selected Grid standards and promote Influence selected Grid standards and promote European research resultsEuropean research results
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
What Does my Title Mean?What Does my Title Mean?
Current Status and Future Direction of Current Status and Future Direction of Unicore and OGSAUnicore and OGSA
1133OGSAOGSA
2244UnicoreUnicore
FutureFutureStatusStatus
Current Current StatusStatus and and FutureFuture Direction of Direction of UnicoreUnicore and and OGSAOGSA
44
22 33
11 22
4433
22
1133
11 44
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
OGSA: The FutureOGSA: The Future
OGSA Architecture V2.0 DocumentOGSA Architecture V2.0 DocumentSame scope, more refined than V1.0Possibly only a V1.5, due to effort on OGSA Profiles
OGSA ProfilesOGSA ProfilesBasic Profile
WSRF, WSN (Base Notification only), & WS-AddressingSecurity Profile
Transport Security, WS-Security, ???Basic Execution Profile
JSDL, Basic Execution Service, WS-Agreement?, WSDM?Naming ProfileCandidate Execution and Data Profiles
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
Introducing Profiles to OGSAIntroducing Profiles to OGSA
ProfilesProfilesRefine and clarify referenced specificationsGoals: Interoperability, Interoperability, InteroperabilityReference point for “claims” of OGSA compliance
No compliance suites plannedProfile TypesProfile Types
Full Profiles: GGF Recommendations TrackBased on stable standards
Candidate Profiles: GGF Informational TrackBased on proposed and evolving specifications
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
Full ProfileFull Profile
Required for Required for ““Proposed RecommendationProposed Recommendation”” StatusStatusContains specifications that are at least on a standards track
E.g. GGF Proposed Recommendation, OASIS Committee Draft, etc.
Commitments from multiple implementers to continue development of evolving specifications.
Required for Required for ““RecommendationRecommendation”” StatusStatusContains completed standard specifications only.
De facto standards permitted, e.g. WSDL 1.1.Two or more independently developed, interoperable implementationsExperience document must published
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
Candidate ProfilesCandidate Profiles
Informational Document in GGFInformational Document in GGF ProcessProcess““CandidateCandidate”” Status isStatus is
A function of status of any referenced specificationsE.g. with respect to the standardization process
Not a function of the importance of the profile to OGSA.
Restrictions on Referenced SpecificationsRestrictions on Referenced SpecificationsOnly references specifications are in standards development.At least one reference prototype of each specification exists.Specifications may be evolving.Commitments for multiple implementations of the final specification.
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
Unicore FutureUnicore Future
Current Status and Future Direction of Current Status and Future Direction of Unicore and OGSAUnicore and OGSA
1133OGSAOGSA
2244UnicoreUnicore
FutureFutureStatusStatus
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
Unicore PrinciplesUnicore Principles
Seamless ComputingSeamless ComputingHigh SecurityHigh SecurityPortable ServerPortable ServerComplex WorkflowsComplex WorkflowsFile ManagementFile ManagementAdvanced Client InterfaceAdvanced Client InterfaceExtensibilityExtensibility
RetainTheseIdeals
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
Unicore: The FutureUnicore: The Future
OGSA ProfilesOGSA ProfilesBasic ProfileSecurity ProfileBasic Execution ProfileNaming Profile
Unicore Grid Programming EnvironmentUnicore Grid Programming EnvironmentVirtual Resource FactoryJob and File ManagementWS Based File Transfer
Globus InteroperabilityGlobus InteroperabilityIssues: WSRF Version and GSI Interfaces
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
Unicore: The Future ContinuedUnicore: The Future Continued
Virtual OrganizationsVirtual OrganizationsDynamic Trust Management
SAML and XACML BasedDistributed Trust Base
New Gateway DesignNew Gateway DesignTentative ideas only, Please feedbackHttps as core protocol, not Java only UPLWS-Addressing enabledGateway “Plug-in” support improved
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
Unicore: GPE ArchitectureUnicore: GPE Architecture
WSRF-enabledServers
WSRF-enabledStorage
WSRF-enabledNetwork
Web Service ResourceFramework (WSRF)
Open Grid Services Architecture
U-GPEHigh-level Grid API
OperationsJob managementFile transfersBrokeringSteering, etc.
DescriptionsResources (CIM)Jobs (JSDL)Workflows (BPEL)
Unicore Atomic Services
Grid Programming Library
Grid SDK Grid Beans Client Framework
Applications
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
Unicore Atomic ServicesUnicore Atomic Services
Atomic Services
Job Management
Service(JMS)
ImplementationW
S-*
Managejobs on target system
Grid Programming LibraryTSS
ClientJMS
ClientSMS
ClientFIS
ClientFES
Client
U-GPE (or other) Clients
Target System Service (TSS)
Implementation
Storage Management
Service(SMS)
Implementation
FileImport Service
(FIS)
Implementation
FileExport Service(FES)
Implementation
Manage target system
Managefiles on storage
Manageimports to storage
Manageexports from storage
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
Unicore GPE Grid ServicesUnicore GPE Grid Services
Target System FactoryTarget System FactoryCreate a virtual compute resource for a VO.
Target System ServiceTarget System ServiceJob Submit (JSDL and AJO)Resource Description Properties
Job Management ServiceJob Management ServiceBasic control and status properties
File Management ServiceFile Management ServiceCopy, Delete, List Directory, ...
File Import/Export ServicesFile Import/Export ServicesWS Based File Transfer Capability
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
Atomic Services DetailAtomic Services DetailOperations Service Resource Type Properties
Job Management Service (JMS)
Extends WS-RP/LTstart()abort()hold()resume()
File Import Service (FIS)Extends WS-RP/LT
putChunk(offset)
File Export Service (FES)
Extends WS-RP/LTgetChunk(offset, size)
Target System Service (TSS)
Extends WS-RP/LTJob-EPR submit(JSDL, initialTT)
Job3Job2
Status=RunningOriginalJSDL=<JSDL..>ExecutionJSDL=<JSDL..>etc.Job1
create
FileImport 1
FileImport 2
Sourcefile=c:/tmp/test.txtDestFile=/tmp/test.txtTransferred=300kbetc.
FileExport 1
FileExport 2
Sourcefile=/tmp/output.txt DestFile=c:/tmp/output.txtTransferred=12kbetc.
create
Target System Factory (TSF)
Target System
Memory=1GbRunning Jobs=23Storage=Temp, Rootetc.
create
Storage Management Service (SMS)
Extends WS-RP/LTlistDirectory(path)listFile(path)createDirectory(path)changePermissions(path, perm)copy(path, path)rename(path, path)delete(path)import(path, protocol, isPipe)export(path, protocol, isPipe)
HomeRoot
TS-Property
Temp
AvailableFileSpace=127kbSupportedProtocols=
{GridFTP, UPL, scp}etc.
Work
create
TSR-EPR create(type, config)
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
OGSA StatusOGSA Status
Current Status and Future Direction of Current Status and Future Direction of Unicore and OGSAUnicore and OGSA
1133OGSAOGSA
2244UnicoreUnicore
FutureFutureStatusStatus
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
OGSA-WG: Current StatusOGSA-WG: Current Status
OGSA Version 1.0 and Glossary PublishedOGSA Version 1.0 and Glossary PublishedOGSA WG Focus on ProfilesOGSA WG Focus on Profiles
Base and Security Profiles - Drafts on GridForgeContribute NOW!
Base Execution Profile Draft after GGF13
OGSA Spawned OGSA Spawned WGsWGsByteIO, Basic Execution Services, and Naming
Attend BoFs at GGF13
OGSA Candidate Profiles Starting Work NowOGSA Candidate Profiles Starting Work NowAdvanced Execution Services (SuperScheduler, Broker, ...)Base and Advanced Data Services Profiles
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
WSRF/WSN/WSDM: StatusWSRF/WSN/WSDM: Status
WSRFWSRFResource Properties and Lifetime ready for Committee DraftService Group in processRenewable References moved out of scope
WSNWSNBase Notification stable & seeking synergy with WS-EventingIncludes “pull” based Notification
Important to NAREGI and UniGrids because of Unicore modelWSDMWSDM
Committee Draft(Based of a different version of WSRF than the CD)
HP/Globus/Apache Incubator AnnouncementHP/Globus/Apache Incubator AnnouncementApollo, Hermes, Muse incubatorshttp://wiki.apache.org/incubator/<name>Proposal
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
JSDL: StatusJSDL: Status
Job Submission Description LanguageJob Submission Description LanguagePre-stage, Execute, Post-stage.Uniformity across batch subsystems.Support for multiple file transfer protocols.
Not a Job Management Interface!Not a Job Management Interface!A job description document with a processing semantics
CapabilitiesCapabilitiesOptional data sources/sinks, with different credentials possibleSeamless or restricted resource descriptionSuitable for resource brokers and schedulers
Act as JSDL to JSDL processorsRestrictionsRestrictions
No support for “Multi-Jobs” or work-flow.
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
Schema OverviewSchema Overview
<<JobDefinitionJobDefinition>>
<<JobDescriptionSectionJobDescriptionSection>>
<<JobIdentification JobIdentification ... />?... />?
<User <User id="xsdid="xsd::uriuri””... /> ?... /> ?
<Application <Application id="xsdid="xsd::uri" uri" ... /> ?... /> ?
<Resource <Resource id="xsdid="xsd::uriuri””... />*... />*
<<DataStaging id="xsdDataStaging id="xsd::uriuri””... />*... />*
<Limits <Limits id="xsdid="xsd::uriuri””... />?... />?
<</JobDescriptionSection/JobDescriptionSection>>
<Profile><Profile>
<Description ... />?<Description ... />?
<User <User id="xsdid="xsd::uriuri””... />?... />?
<Application <Application id="xsdid="xsd::uri" uri" ... />?... />?
<Resource <Resource id="xsdid="xsd::uriuri””... />*... />*
<<DataStaging id="xsdDataStaging id="xsd::uriuri””... />*... />*
<Limits <Limits id="xsdid="xsd::uriuri””... />?... />?
</Profile>*</Profile>*
<</JobDefinition/JobDefinition>>
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
JSDL ExampleJSDL Example<<JobDefinitionJobDefinition>>
<<JobDescriptionSectionJobDescriptionSection>>
<<JobIdentification/JobIdentification/>>
<Application id=<Application id=””Daves Daves App">App">
<<ApplicationName ApplicationName ““NanoCodesRSNanoCodesRS””/>/>
<<ApplicationVersion ApplicationVersion ““V1.7aV1.7a””/>/>
</Application></Application>
<</JobDescriptionSection/JobDescriptionSection>><Profile id=<Profile id=““Profile1Profile1””>>
<Resource id=<Resource id=”” LargeSystemLargeSystem””>>
<<ResourceCount ResourceCount ““6464””/>/>
<<CPUCount CPUCount ““88””/>/>
</Resource></Resource>
</Profile></Profile>
<Profile id=<Profile id=““Profile2Profile2””>>
<Resource id=<Resource id=”” SmallSystemSmallSystem””>>
<<ResourceCount ResourceCount ““6464””/>/>
<<CPUCount CPUCount ““88””/>/>
</Resource></Resource>
</Profile></Profile>
<</JobDefinition/JobDefinition>>
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
OGSA Specification OverviewOGSA Specification Overview
SYSTEMSMANAGEMENT
UTILITYCOMPUTING
GRIDCOMPUTING
WS-Addressing
Privacy
WS-Base Notification
CIM/JSIM
WSRF-RAP
WSDM
WS-Security
Naming
OGSA-EMSOGSA Self Mgmt
GFD-C.16
GGF-UR
Data Model
HTTP(S)/SOAP
GRID Computing, Distributed Computing and Utility Computingare different views of the same important problem domain.
Discovery
SAML/XACML
WSDL
WSRF-RL
Trust
WS-DAI
VO Management
Information
Distributed query processing
ASP
Data CentreUse Cases &Applications Collaboration Multi MediaPersistent Archive
Data Transport
WSRF-RP
X.509
Core Services
InfrastructureSpecifications
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
Status LegendStatus Legend
StandardStandardExisting, adopted specification from a recognized Standards Development Organization.
EvolvingEvolvingEmerging specifications with reference implementations and standardization in process
GapGapIdentified architectural component with no existing specifications complete, but work is in progress somewhere.
HoleHoleIdentified architectural component without apparent activity.
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
OGSA Architecture: StatusOGSA Architecture: Status
SYSTEMSMANAGEMENT
UTILITYCOMPUTING
GRIDCOMPUTING
Core Services
InfrastructureSpecifications
WS-Addressing
Privacy
WS-BaseNotification
CIM/JSIM
WSRF-RAP
WSDM
WS-Security
Naming
OGSA-EMSOGSA Self Mgmt
GFD-C.16
GGF-UR
Data Model
HTTP(S)/SOAP
Discovery
SAML/XACML
WSDL
WSRF-RL
Trust
WS-DAI
VO Management
Information
Distributed query processing
ASP
Data CentreUse Cases &Applications Collaboration Multi MediaPersistent Archive
Data Transport
WSRF-RP
X.509
StandardEvolvingGapHole
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
Unicore StatusUnicore Status
Current Status and Future Direction of Current Status and Future Direction of Unicore and OGSAUnicore and OGSA
1133OGSAOGSA
2244UnicoreUnicore
FutureFutureStatusStatus
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
Unicore: Current StatusUnicore: Current Status
Security InfrastructureSecurity InfrastructureExplicit Trust Delegation Framework - Done
WS InfrastructureWS InfrastructurePrototype WSRF Plus job and file management interfaces
Shipped in NovemberV1.0 WSRF by end March 2005V1.0 Unicore Atomic Services by end March 2005
Includes JSDL based job description
WS Based File TransferWS Based File TransferV1.0 by mid AprilV2.0 next year
Fujitsu Laboratories of Europe Fujitsu Laboratories of Europe ©© 20052005
ConclusionsConclusions
Progress is GoodProgress is GoodOGSA is moving aggressively, backed by GGF.WSRF is approaching completionNAREGI/UniGrids WSRF Infrastructure ready soonJSDL’s progress is due to NAREGI/UniGrids Team
Aim for a repeat performance on Basic Execution ServiceNew Tools TooNew Tools Too
Basic execution services for NAREGI requirementsWS Based File Transfer and New Gateway DesignVO support coming (Tell us your requirements)
High Degree of CollaborationHigh Degree of CollaborationNAREGI, UniGrids, Globus, EGEE, Condor, ...