OurGrid ourgrid

www.eu-eela.eu

E-science grid facility for Europe and Latin America

OurGridhttp://www.ourgrid.org

E2GRIS1

Rafael Silva

Universidade Federal de Campina Grande (Brazil)

Itacuruça (Brazil), 2-15 November 2008

www.eu-eela.eu

Overview

Itacuruça (Brazil), E2GRIS1, 2.11.2008 - 15.11.2008 2

• OurGrid• Architecture• Infrastructure• Jobs• Installing an OurGrid Site

– XMPP Server– Peer– Worker– Broker

• Running Jobs• References

www.eu-eela.eu

OurGrid

• OurGrid is an open, free-to-join, cooperative grid in which labs donate their idle computational resources in exchange for accessing other labs’ idle resources when nedeed

• Applications Bag-of-Tasks (BoT)– However, a single OurGrid task may itself be a parallel tightly-

coupled application (written in MPI, for example)


www.eu-eela.eu

Architecture


www.eu-eela.eu

Infrastructure

• Broker– Client interface for job submission

• Worker– Executes the jobs submitted

• Peer– Controls the donating and consuming of the machines

• Discovery Service– Discover resources on the Grid

• Network of Favors (NoF)– Resource share incentive– Prevent free riding

Peers that only consumes resources


www.eu-eela.eu

Communication Infrastructure

• Commune– Built on top of XMPP (Extensible Messaging and Presence

Protocol). Thus, every communication between OurGrid components are intermediated by a XMPP server.

– OpenFire Server (WildFire) http://www.igniterealtime.org/downloads/index.jsp


www.eu-eela.eu

Jobs

• Job Description File (JDF)

• Each job is a set of tasks, where each task can have three parameters:

– Init File list to be transferred to the worker

– Remote Command to be executed in the worker

– Final Retrieve the result of the task


www.eu-eela.eu

Jobs (2)

• Requirements

– Actually, the OurGrid only has two types of requirements specification available:

Operating System Memory Size


requirements: (os == windows)

requirements: (mem == 1024)

requirements: (os == windows)

requirements: (mem == 1024)

www.eu-eela.eu

Jobs (3)

• Types of Storage– $PLAYPEN (temporary storage)

– $STORAGE (permanent storage)

• Other Variables– $PROC

Contain the grid machine chosen to run the task

– $JOB Job unique number

– $TASK Task unique number (within a job)


put <local_path> <destination_path>put <local_path> <destination_path>

store <local_path> <destination_path>store <local_path> <destination_path>

www.eu-eela.eu

An example of a JDF file


job :

label : SimpleJob

task :

init : put Fat.class Fat.class

remote : java -cp . Fat 3 261147332 6819792792357414911 output-$JOB.$TASK

final : get output-$JOB.$TASK output-$JOB.$TASK

task :

remote : hostname -f

job :

label : SimpleJob

task :

init : put Fat.class Fat.class

remote : java -cp . Fat 3 261147332 6819792792357414911 output-$JOB.$TASK

final : get output-$JOB.$TASK output-$JOB.$TASK

task :

remote : hostname -f

www.eu-eela.eu

Installing an OurGrid Site

• To install an OurGrid Site, it is only necessary to install the following services:

– XMPP Server– Peer– Workers– Broker

• In addition to these services, the only requirement is Java 1.5 or higher


Two ways to configure:

- text mode

- gui mode

www.eu-eela.eu

Installing the XMPP Server• Download the openfire package from:

– http://www.igniterealtime.org/downloads/index.jsp

• Extract the tarball and start it– <openfire-directory>/bin/openfire start

• Open your browser and type:– http://<your-openfire-server-ip>:9090

• And follow the instructions of the following link:– http://www.ourgrid.org/index.php?

option=com_content&view=article&id=52&Itemid=2&lang=en#Anc_2


www.eu-eela.eu

XMPP Server


www.eu-eela.eu

Peer Installation• Download the peer package from the OurGrid’s

website, uncompress it and start the graphical interface

• Configuring the peer


$ unzip peer-<version>.zip

$ cd peer-<version>

$ sh peer gui

$ unzip peer-<version>.zip

$ cd peer-<version>

$ sh peer gui

www.eu-eela.eu

Peer Installation (2)

• Adding Workers


www.eu-eela.eu


• Site Description File (SDF)


workerdefaults :

copyTo : scp $localfile $machine:$remotefile

remExec : ssh -x $machine $command

copyFrom : scp $machine:$remotefile $localfile

worker :

OS : linux

servername : xmpp.ourgrid.org

username : worker-001

worker :

OS : windows



workerdefaults :

copyTo : scp $localfile $machine:$remotefile

remExec : ssh -x $machine $command

copyFrom : scp $machine:$remotefile $localfile

worker :

OS : linux



worker :

OS : windows



www.eu-eela.eu


• Adding Brokers


www.eu-eela.eu

Worker Installation• Download the worker package from the OurGrid’s


• Then, configure the same variables again:– XMPP User Name

– XMPP User Password

– XMPP Server Name

• IMPORTANT!– Put the public key of the peer in the “Worker Configuration” tab.


$ unzip worker-<version>.zip

$ cd worker-<version>

$ sh worker gui

$ unzip worker-<version>.zip

$ cd worker-<version>

$ sh worker gui

www.eu-eela.eu

Broker Installation• Download the broker package from the OurGrid’s


• Then, configure the same variables again:– XMPP User Name

– XMPP User Password

– XMPP Server Name


$ unzip broker-<version>.zip

$ cd broker-<version>

$ sh broker gui

$ unzip broker-<version>.zip

$ cd broker-<version>

$ sh broker gui

www.eu-eela.eu

Broker Installation (2)

• Now, it is necessary specify the peers the broker will be able to submit jobs

• Grid Description File (GDF)


peer:

label: 127.0.0.1

username : peer-rafael

servername : rs

peer:

label: 127.0.0.1

username : peer-rafael

servername : rs

www.eu-eela.eu

Running Jobs

• Demonstration


www.eu-eela.eu

References

• OurGrid web site– http://www.ourgrid.org

• XMPP Community– http://www.xmpp.org

• Openfire– http://www.igniterealtime.org/projects/openfire/index.jsp


www.eu-eela.eu

Questions…


Documents

OurGrid ourgrid