Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 0
dCache meets SARA Patrick Fuhrmann
With contributions by Gerd Behrmann Tigran Mkrtchyan Mattias Wadenstein
These slides are a result of a meeting at SARA, NL with SARA team, BioMed, Long Term Storage and LOFAR
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 1
Content
The dCache organization
dCache spec’s
dCache deployment
dCache plans
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 2
The dCache Organization
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 3
What is dCache.ORG ?
Developers
DESY
NDGF
FERMIlab
IN2P3
Review Board
Code Repository
Testing Building
Support Web
Docs Wiki
Download
Ticket System Mailing lists (user-forum) Workshop organization Phone Conferences CERN gLite repository contact
8 Tier I’s Tier II’s (else)
Tier II’s US
Tier II’s Noridic
Tier II’s Germany
HGF NDGF OSG First level support
Hosted and funded by DESY
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 4
2.5
1
2
3
1.5 0.5
Fermi
NDGF
DESY (IT)
HGF@DESY
D-‐Grid@DESY
EGEE III @ DESY
dCache team by funding agency
About 10 team members in total.
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 5
dCache Specification
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 6
dCache BOX View Storage Control
SRM
Wide AreaTransport (gsi)Ftp
http(s) / WebDav
Posix LIKE Access (gsi)dCap
xRoot
Posix native Access NFS 4.1
Optional Tape Systems
Chimera Name and Metadata System
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 7
dCache Idea
Optional Tape Systems
Chimera Name space
FOO = 7826
7826
7826
7826
The same file, with a single entry in the file- system, can be located at various locations inside and outside of dCache. dCache takes care of all locations and manages necessary transitions, completely transparent to the user.
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 8
The consequence
Data is automatically replicated on detection of access hotspots.
Data can be replicated on arrival. (second copy prior to tape backup)
Data is migrated to tape if configured and restored if necessary.
Data can be scheduled for replication for maintenance operations.
Configuration can enforce a second or third copy of each file.
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 9
Basic file life cycle (all protocols) File written to dCache
After awhile (file is flushed to tape)
Space is running short (File is removed from disk)
User reads file
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 10
Basic file life cycle (technical view)
User reads file while still On write disk.
Expensive raid system.
Cheap JBOD
User reads file after it has Already been removed from Write disk.
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 11
Reliability Checksums are calculated on all transfers (except for reading)
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 12
What is storage control ?
dCache supports both : manual and automatic storage control
Data is directed to pool-groups based on directory, client IP, protocol …
Data can be directed to disk-only or disk-tape (Storage attributes)
Directory based storage attributes for all protocols
File based attributes for SRM only (Storage Resource Manager)
Files can be pinned to disk (forever or for a fixed time) using SRM.
Files can be restored to disk to schedule subsequent access.
Automatic restore (tape -> disk) can be protected to avoid tape disaster.
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 13
Basic file life cycle and storage control (User)
User requests the file for subsequent processing
BRING-ONLINE
User processes data after all files are ONLINE.
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 14
Another example for User-Storage-Control
MySpace MyTape
MyDisk
Disk Tape
User may specify whether a file should end up on tape or on disk only.
/users/x/MySpace/MyTape/Foo
/users/x/MySpace/MyDisk/Foo
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 15
Security
Authentication
X509 Certificates https / WebDav(s)
Proxies FQAN (Group/ Role)
SRM
gsiFtp
gsidCap Kerberos
NFS 4.1 Translator
User <password>
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 16
Security
Authorization
File system, all protocols : full NFS 4.1 ACLs
Tape Protection : simple FQAN/DN based
Space tokens : indirect through file system and link groups
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 17
The dCache Customers
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 18
dCache is in production at :
5 Tier I’s in Europe 3 Tier I’s in North America
40 Tier II’s worldwide
WLCG (Europe plus OSG)
HEP Hera Tier 0 ILC
Other communities Bio Med (NDGF) Photon Science (DESY)
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 19
The largest dCache (as far as I know)
Replicas 300 TBytes Tape
5.4 PBytes
4.5 PBytes
284 Tbytes/day
720 Tbytes/day max
73Tbytes/day
95Tbytes/day max
1350 Pools on 212 Nodes
(Information provided by Jon Bakken, FEMILab)
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 20
Slide stolen from Mattias Wadenstein, NDGF
The most complex dCache (for sure)
The 7 biggest Nordic Computer centers form the NDGF Tier I Resources are scattered (CPU & Storage) Services can be centralized Advantages in redundancy Especially in 7*24 hour data talking
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 21
Further roadmap (Sysadmin only)
Integrated monitoring
Information provided in xml format
Already done for all GLUE values.
Simplified component location configuration
Single file replaces node/pool config
Easy parameter setting per domain/host
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 22
Further roadmap (Sysadmin & User)
Unifying of ‘User Representation’ (May workshop)
File system, tape protection and space tokens will us the
same user representation.
Improved data distribution on bulk transfers
Already done for pool to pool transfer
Next for write into dCache
Moving from manual to automatic redistribution of data
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 23
Further roadmap (User)
https : User/Password authentication
https : support of Proxy/FQAN/Groups/Roles
ACL’s : setting ACLs by user and not only sysadmin
NFS 4.1 : secure (Kerberos, Certs by modified KDC)
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 24
Further roadmap : Going standard
Already supported standards :
gsiFtp (IETF)
SRM (OGF)
Unsecure http (IETF)
In beta testing
NFS 4.1
WebDav (s)
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 25
Further roadmap : NFS 4.1 Why not already NFS 2/3 for data access ?
dCache uses NFS 2/3 for name space operations (ls,mv..) only, as it doesn’t support data of a single instance being distributed among different storage hosts. NFS 4.1 (with parallel NFS) is the first standard posix access protocol allowing this.
Who is supporting NFS 4.1 (pNFS)
All major vendors : EMC, IBM, Linux, NetApp, Panasas, Solaris server. Coming soon : Windows client.
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 26
Further roadmap : NFS 4.1 (pNFS) in dCache Name server and i/o protocol fully implemented.
No security yet Soon : Kerberos.
X509 unlikely : Solution : modified KDC
No automatic recall from tape to protect tape system.
Soon : part of the standard tape protection mech.
Full support of NFS Access Control List (ACLs)
Right now only by system administrator
Soon : through NFS4 ‘setacl’ call by all users. (NFS4 is already part of SL5 dist)
Fully supports storage control (tape/disk) on directory bases.
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 27
Roadmap : NFS 4.1 (pNFS) linux clients NFS 4.1 and the linux kernel
NFS 4 already in SL5
NFS 4.1 in 2.6.32
NFS 4.1 plus pNFS in 2.6.34
Kernel 2.6.34 will be in Fedora 13 and RH6 Enterprise (summer)
Windows Client expected 4Q10.
We are testing with :
SL5 and 2.6.34 plus some special RPM. (mount tools)
See our wiki for further information
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 28
Roadmap : WebDav (s) Requested by
Bio Grid and other communities at NDGF
Light sources (Petra3 and XFEL) at DESY
Beta release in 1.9.6 (3)
Tested with Max OS, Windows(XP), SuSE11.2 (Gnome, KDE)
Supports read and write
Write via ‘redirect’ or if not supported by client via ‘proxy’.
Security
Plain or x509
On redirect, only control line is encrypted.
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 29
Further Reading
www.dCache.org
16 Feb 2010 Sara, Amsterdam dCache meets SARA patrick.fuhrmann @ dCache.ORG 30
Antennas (Europe) Local noise reduction (*10)
Noise reduction (*100)
Groningen SARA, NL Long term archive
Jülich
6 Key Science Projects 5 centrally coordinated 1 is individual user access.
1.5 P-Bytes first year on tape About 20% on disk. Restage unknown.