Abhishek Singh Rana UC San Diego | USCMS | DISUN rana@fnal

Joint OSG and EGEE Operations Workshop RAL UK September 27 2005

Abhishek Singh Rana and Kate Keahey OSG Edge Services Framework www.opensciencegrid.org

The Open Science Grid Consortium

Abhishek Singh RanaUC San Diego | USCMS | DISUN

[email protected]

Kate KeaheyArgonne Lab | Globus [email protected]

Edge Services FrameworkESF in Open Science Grid

Joint OSG and EGEE Operations WorkshopSeptember 27-29, 2005 RAL, UK

2




Outline

• OSG-ESF Activity.• ESF Phase 1.

– Concepts and Design.

• ESF future direction.• Xen overview.• Plan for Phase 1 on OSG.

– Site admin responsibilities.– VO admin responsibilities.

• Status and Next Steps.

3




Vision

From talk by Frank Wuerthwein at OSG Consortium Meeting 2005

(simplified snapshot of a typical OSG/LCG site in 2007)

4




OSG-ESF Activity

• Started in September 2005.• Physicists, Computer Scientists & Engineers, Software Architects.• Chairs: Kate Keahey and Abhishek Singh Rana.

• Workspace Services Design: – Timothy Freeman & Kate Keahey (Globus Alliance).

• Edge Services Implementations:– USATLAS: Rob Gardner (U Chicago), Alexander Vaniachine (ANL).– USCMS: Frank Wuerthwein, Abhishek Singh Rana (UC San Diego).

• Mailing List – osg-edgeservices at opensciencegrid.org

• Web collaborative area –http://osg.ivdgl.org/twiki/bin/view/EdgeServices

5




Current OSG-ESF mailing list

[email protected] Ian [email protected] Barry [email protected] Simone [email protected] Kaushik [email protected] Laurence [email protected] Tim [email protected] Rob [email protected] Burt [email protected] Kate [email protected] Maarten [email protected] Stuart [email protected] Razvan [email protected] Ruth [email protected] Abhishek Singh [email protected] Markus [email protected] Jim [email protected] Dane [email protected] David [email protected] Preston [email protected] Alexander [email protected] Torre [email protected] Frank Wuerthwein

6




ESF - Phase 1

7




No ESF - Phase 0

SECE

Site

8




No ESF - Phase 0

Site

SECE

Static deployment

CMS ATLAS CDF

9




ESF?

SECE

Site

10




ESF - Phase 1

ESF

SE

Site

Snapshot ofES Wafers

implemented asVirtual Workspaces

CE CDFCMS ATLAS

GuestVO

11




An attempt at ESF Terminology

• Edge Services Wafer (ES Wafer)– A specific instance of a dynamically-created VM (workspace) is called

an Edge Services Wafer. – An ES Wafer can have several Edge Services running. – A VO can have multiple ES Wafers up at a Site.

• Edge Services Slot (ES Slot) – An ES Slot has hardware characteristics specified by the Site Admin.– An ES Slot can be leased by a VO to host an ES Wafer.

• Edge Service (ES) – A VO-specific service instantiated by a VO in a Wafer.

• Workspace Service (WS)– Service at a Site that allows VOs to instantiate ES Wafers in ES Slots.

12




ESF - Phase 1

CDFCMS ATLAS

GuestVO

ESF

SECE

Site

GT4 Workspace Service & VMM

Dynamically deployed ES Wafers for each VO

Wafer imagesstored in SE

Compute nodes and Storage nodes

Snapshot ofES Wafers

implemented asVirtual Workspaces

13




User jobs at Compute nodes using ES Wafers for VO Edge Services

ESF

SECE

Site

CDFCMS ATLAS

GuestVO

14




VO Admin transporting/storing ES image to a remote Site..

..Deploying ES using image stored in Site’s local repository

15




ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO Admin

16




ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO Admin

17




ESF - Phase 1

ESF

SECE

Site

Role=VO Admin

18




ESF - Phase 1

ESF

SECE

Site

Role=VO Admin

19




ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO Admin

20




ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO Admin

21




ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO Admin

22




A VO User using ES..

23




ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO User

24




ESF - Phase 1

ESF

SE

Site

CMS

Role=VO User

CE

25




ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO User

26




ESF - Phase 1

ESF

SECE

Site

CMS

Role=VO User

27




ESF - future direction

28





• Same concept. • ES Wafers to be deployed as an ESF cluster and be fully schedulable by any OSG VO.

29





ESF

Site

Brokering and Scheduling byEdge Services Framework

Dynamically deployed ES Wafers for many VOs

Cluster of ES Slots with different properties

ATLAS1CMS ATLAS2

CDF

30




31




Xen overview

Hardware

Virtual Machine Monitor (VMM)

Public Network

Private Network

32




Plan for Phase 1 on OSG

33




Plan for Phase 1 on OSG

• ATLAS & CMS procure one piece of hardware at their Sites on OSG that runs ESF (called ESF node).– Dual CPU.– 4GB RAM (No PAE support in Xen2, we will exploit only 4GB until

we migrate to Xen3).

• Site administrators install: – Xen (We will go for Xen 2.0.7).– GT4 Workspace Service.

• VO administrators use ESF to fire-up Xen VMs that instantiate VO Services – Edge Services in an ES Wafer.

• A single ESF node hosts ES Wafers for both ATLAS & CMS.

34




Site Administrator Responsibilities

• Start with FC4 and deploy:– Xen (We will go for Xen 2.0.7).– Custom kernel for domain 0 (grub required).– Custom kernel for domain U.– GT4.– GT4 Workspace Service.

• Provision:– One public IP, One private IP per VM.– Host certificates per VM.– Disk space per VM.

• Declare available ES Slots and their properties to ESF.

35




VO Administrator Responsibilities

• Prepare (freeze) ES Wafer instance as disk image.• Prepare image into file for transport.• SRMCP the image to remote Site’s SE.• Use ESF to fire-up Xen VM with VO-Services (ES Wafer)

at remote Site, from image file in remote SE, using role based authorization (OSG-RBAC).

• Advertise the running Edge Services as needed.

36




Status

• Xen3 on testbed.• GT4 Workspace Service being used.• Prototype 0 of ESF.

Next Steps

• Revert back to Xen2.• Develop VO-Service images.• Verify performance, functionality, robustness.

37




Thank You.

Documents

Abhishek Singh Rana UC San Diego | USCMS | DISUN rana@fnal