View
2
Download
0
Category
Preview:
Citation preview
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
1
DataGrid Workshop
INFN Testbed status report L. Gaido
Oxford July, 2-5 2001
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
2
Dedicated resources(available now)
Q Quantum Grid: 3-4 PCs in 15 sites: Bari, Bologna, Cagliari, Catania, Cnaf,
Ferrara, Lecce, Legnaro, Milano, Napoli,Padova, Parma, Pisa, Roma, Torino
Total: ~ 50 nodes
Q 1 SUN cluster (5 nodes) at CNAF
Q Some more hosts dedicated to specificactivities
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
3
Non dedicated resources
The LHC experiments have their own farms (~ 100nodes in 8 sites), dedicated to productionactivities, in some INFN sites.Since the experiments play a fundamental role inTestbed and some of the test steps will requireexperiments’ production environment theseresources will also be used for Testbed activities(but it is up to the experiments decide when,where and and how).
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
4
Further dedicated resources(available soon)
In order to have enough resources available forthe Testbed activities 6 new farms have beenfunded by InfnGrid project.So, in the next months, farms made up of - 1 disk server (~ 300 GB) - 5 dual PIII 1 GHzwill be bought and set up in:Catania, Cnaf, Milano, Padova, Pisa and Torino
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
5
How have they been used?The currently available resources have been used for:
- Globus Toolkit evaluation activities (a report has been produced)- InfnGrid toolkit production (Globus toolkit + GDMP +
support for various CAs + configuration scripts, etc.)- Implementation of the GIS INFN infrastructure (an index server, GIIS, in each INFN unit)- MDS-2 tests (presentation by A. Cavalli in the MDS-2
session, Wednesday 9:00-11:00)- GridFTP tests- “toys” for the site managers
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
6
How have they been used?- LHC experiments’ tests: - ALICE Catania, Cagliari, Torino, Lyon.... - ATLAS Milano, Roma, CERN - CMS Bologna, Padova, Pisa, Roma....
Testbed resources and experiments’ resources have beenused for these tests.
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
7
How have they been used?
Presentations about these activities have been given at theInfnGrid workshop held in Catania (April, 9-11) and in theDataGrid WP8 meeting (May, 31)
http://pcct108.ct.infn.it/grid/workshop-ct/ http://datagrid-wp8.web.cern.ch/DataGrid-WP8/meeting_main.htm
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
8
How will they be used?
The INFN Testbed resources will be primarily used forInfnGrid Testbed activities and tests.
Some of the InfnGrid Testbed activities are also DataGridactivities, some are INFN specific. Some words aboutthem in a few moments…
Some guidelines (and tools) to request and allocate theTestbed resources for test purposes have been producedfor INFN.
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
9
How will they be used?
The INFN Testbed resources can also be used forDataGrid Testbed activities.
But we need to: - define clearly which resources each DataGrid partner
reserves for DataGrid Testbed
- define how to manage the allocation and usage of thetestbed resources at the DataGrid level (Europe + someUSA sites)
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
10
InfnGrid guidelines
- The requester (Test Coordinator) submits a form describing the goals and all the relevant information
(users involved, needed resources, grid services, etc.)about the test to a technical group, the Planning Group.
- The PG evaluates the request and allocates the resources (via the resource database) to the test users,
gets the appropriate grid-mapfile from the Certificates database manager and sends it to the site managers.
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
11
InfnGrid guidelines
- The site managers have the responsibility to install thefarms, the needed software and have to provide thefirst basic support.
- At the end of the test activity the Test Coordinatorproduces a report
- The InfnGrid WP6 coordinators supervise the wholeactivity
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
12
Grid resource database and tools (by L. Fonti)
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
13
Grid Resource Database
§ A database has been created for theresources of the Grid pool.
§ The database backend is an XML file.§ The database management tool is a java
servlet.§ All database management is done on the
web, through web pages produced byservlet.
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
14
XML database backend
§ XML has been choosen as databasebackend, because its element nestingcapability is particularly suitable to reflectthe hierarchical tree-based structure ofGrid resource layout.
§ Moreover, XML offers an open, freelyexpandable database schema.
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
15
XML file structure§ <GRID>§ <ORGANIZATION name=“INFN”>§ <UNIT name=“Bari”>§ <GIIS>giis.ba.infn.it</GIIS>§ <GLOBUS>1.1.3</GLOBUS>§ <PERSON>Giorgio Maggi</PERSON>§ <CLUSTER>§ <GATEKEEPER>gris1.ba.infn.it</GATEKEEPER>§ <GRAM>fork</GRAM>§ <CPU units=“2”>§ <TYPE>Pentium III></TYPE>§ <OS>Linux RedHat 6.2</OS>§ <SPEED>500</SPEED>§ <RAM>256</RAM>§ <DISK>12</DISK>§ <NODE alloc=“WP1”>gris1.ba.infn.it</NODE>§ <NODE>gris2.ba.infn.it</NODE>§ </CPU>§ </CLUSTER>§ </UNIT>§ </ORGANIZATION>§ </GRID>
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
16
Database Access
§ Database access is granted via web withHTTPS protocol and it is reserved to usersowning a valid certificate.
§ Database and management tools are basedon a multi-organizational architecture, i.e.all the resources of DataGrid testbedcould be housed in a single database.
§ However, the choice between centralizedand distributed approach has not yet beentaken: this question is still subject ofdiscussion.
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
17
Database management tools
§ The main management tool is a javaservlet, which uses a simple, non validatingparser to access the XML file, and toconvert it to an object tree.
§ It also generates a set of web pageswhich allow users to examine, add, removeand modify unit entries.
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
18
Database management tools
§ Additional tools have been developed tosearch nodes with given characteristics,and to allocate them to a testing group.
§ All management tools have beenimplemented as java servlets, and toolaccess is restricted to users owning a validcertificate.
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
19
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
20
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
21
Search Tool
§ A tool has been developed to search thedatabase for nodes matching a given set ofrequirements.
§ In the current version, allowed searchparameters are: Organization, CPU typeand speed, memory size and disk space.
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
22
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
23
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
24
Resource Allocation Tool
§ Another tool has been developed to allowmarking a node as reserved to a givengroup for testing or working purposes.
§ The tool shows on a web page a list ofavailable resources, similar to the listproduced by the main management tool, butwith an allocation button shown near eachnode.
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
25
Resource Allocation Tool
§ The label on the button is “Allocate” if thenode is currently free, or “Release” if it iscurrently reserved.
§ Clicking the button will swap the nodestatus.
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
26
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
27
What about DataGrid?
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
28
What about DataGrid?May the INFN resource database and the tools be used forthe whole DataGrid Testbed?
May the InfnGrid guidelines be generalized for DataGrid? - a proposal (where the Planning Group is replaced by a Testbed Coordination Group, composed by 1-2 people for each national Testbed) has been submitted to the Integration Convergence Team (ICT) - comments by Federico Carminati - resent by FCA to WP8 TWG ** Need for discussion
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
29
InfnGrid Testbed activities
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
30
InfnGrid Testbed activities for 2001Software environment, packaging
- Integration of various software components - Installation kit production - Experiments’ software environment installation
some of the above activities are performed with WP4 and WP8
Grid services - Information Service MDS-2 tests (presentation in the MDS-2 session)
- Security, Certification and Authorization a draft document (“Authorization Service for Grid”) will be presented in the themed Security workshop
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
31
Services and Working Groups
The GRID basic services Working Groups are:
- Information Service Participants: A. Cavalli, A. Bassi, L. Fonti , F. Semeria, G. VitaFinzi
- Security, Certification and Authorization Participants: R. Cecchini, R. Alfieri, L. Dell'Agnello, A. Gianoli, C. Rocca, F. Spataro
- INFNGrid installation kit and integration Participants: F. Donno, S. Barale, A. Cavalli, A. Chierici, A. Italiano, D. Mura, M. Serra
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
32
InfnGrid Testbed activities for 2001
Resource access policies and allocation for the tests - guidelines for the resource allocation - resource database and tools for search/allocation - tools for the automatic creation of grid-mapfiles
Monitoring - development of a simple monitoring tool for whole InfnGrid Testbed starting from the tools developed by the italian components of the LHC experiments (since WP3 tools will not be available in the short term).
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
33
InfnGrid Testbed activities for 2001Support for Testbed activities: - by the site managers - by the grid services WGs - by the Installation kit group
Testbed resources validation
Goal: check if the testbed resources are running properlyOperational tests of: - grid software installation - batch systems - grid core services - networkIs this activity useful for DataGrid, too?Some preliminary discussions with S. Nicoud and E. Fedè, needfor a decision.
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
34
Validation Tools Proposal (A. Cavalli)
(cont.)
§ The tool could be done extending the features ofthe available MDS perl web browser.
§ It will retrieve the list of resources from theMDS.
§ Of each GRIS it will show:- Cpu model, speed, cpucount & Globus version;- Time needed to gather information;- Published cpu load attributes;- RTT (just this for now from network);- Defined scheduler.
(*)
* dynamical values: the 1st depends on the others
DataGrid Workshop Oxford, July 2-5
INFN Testbed status reportLuciano Gaido
35
Validation Tools Proposal (2)
§ Join the tool developed in Torino (C++ program +some simple scripts) with this one to test thedefined scheduler and show the results (open issue:a test user on all resources?).
§ Write some simple script to test GSI-ftp.
? Is it enough? Is there anything else that has to bechecked?
? Is such a tool useful for DataGrid Testbed?
Recommended