Authority, Virtual Organizations and

Diagnostics:Building and Managing Complexity

Ken Klingenstein

Director, Internet2 Middleware and Security

Topics

Background on Internet2 Middleware and International efforts

The model: enterprises, federations and virtual organizations; the unified field theory of trust

The deliverables• Shibboleth – interrealm exchange of attributes and authorizations• Signet – a privilege management system• Virtual organizations – serving collaborative communities in science

and humanities• Diagnostics – when it doesn’t work

The next year or so

MACE (Middleware Architecture Committee for Education)

Purpose - to provide advice, create experiments, foster standards, etc. on key technical issues for core middleware within higher education

Membership - Bob Morgan (UW) Chair, Tom Barton (Chicago), Scott Cantor (Ohio State), Steven Carmody (Brown), Michael Gettes (Duke), Keith Hazelton (Wisconsin), Paul Hill (MIT), Jim Jokl (Virginia), Mark Poepping (CMU), Bruce Vincent (Stanford), David Wasley (California), Von Welch (Grid)

European members - Brian Gilmore (Edinburgh), Ton Verschuren (Netherlands), Diego Lopez (Spain)

Creates working groups in major areas, including directories, interrealm access control, PKI, video, P2P, etc.

Works via conference calls, emails, occasional serendipitous in-person meetings...

Internet2 Middleware and the NSF Middleware Initiative (NMI)

Internet2 Middleware a major theme for the last five years, drawing support from 206 university members, 75+ corporate members, and government grants and interactions

Internet2 has an integrator role within NMI, the key NSF Program to develop and deploy common middleware infrastructures

NMI has two major themes • Scientific computing and data environments (ala Grids)

• Common campus and inter-institutional middleware infrastructure (ala Internet2/EDUCAUSE/SURA work)

Issues periodic NMI releases of software, services, architectures, objectclasses and best practices – R5 most current release

International efforts

Terena as an anchor for a succession of middleware discussions and initiatives

Conspicuous national efforts in Spain, Switzerland, The Netherlands, the Nordic countries and a few other European countries.

Major initiative now underway by JISC in the UK, with coordinated advancement in authorization, virtual organizations, digital rights management, and other areas.

Australian efforts rapidly advancing; the rest of the Pacific Rim lags…

The Model:Enterprises and Federation

Given the strong collaborations within the academic community, there is an urgent need to create inter-realm tools, so Build consistent campus and enterprise middleware infrastructure deployments, with outward facing objectclasses, service points, etc. and thenFederate those enterprise deployments, using the outward facing campus infrastructure, with interrealm attribute transports, trust services, etc. and thenLeverage that federation to enable a variety of applications from network authentication to instant messaging, from video to web services, and then, going forwardCreate tools and templates that support the management and collaboration of virtual organizations by building on the federated campus infrastructures.

Middleware Axioms

Work the core areas

Focus on support for collaboration

Use federated administration as the lever; have the enterprise broker most services (authentication, authorization, resource discovery, etc.) in inter-realm interactions

Develop a consistent directory infrastructure within R&E

Provide security while not degrading privacy.

Foster interrealm trust fabrics: federations and virtual organizations

Leverage campus expertise and build rough consensus

Support for heterogeneity and open standards

Influence the marketplace; develop where necessary

A Map of Campus Middleware Land

Federated administration

O

TO

T

T T

A CMCM A

VOVO

T

Campus 1Campus 2

Federation

Unified field theory of Trust

Bridged, global hierarchies of identification-oriented, often government based trust – laws, identity tokens, etc.

• Passports, drivers licenses • Future is typically PKI oriented

Federated enterprise-based; leverages one’s security domain; often role-based

• Enterprise does authentication and attributes• Federations of enterprises exchange assertions (identity and attributes

Peer to peer trust; ad hoc, small locus personal trust• A large part of our non-networked lives• New technology approaches to bring this into the electronic world.• Distinguishing P2P apps arch from P2P trust

Virtual organizations cross-stitch across one of the above

The Deliverables

Shibboleth – a secure, privacy-preserving transport for attributes between realms and within federations

Signet – a meta-authority system that leverages enterprise roles to drive sophisticated authorization options

Virtual organizations – combining enterprise services with stand-alone services to provide consistency and transparency to the VO participants

Diagnostics – coupling existent and yet-to-be-defined exception handling across a multi-layered (application, middleware, security, network) distributed environment

Shibboleth Architecture

Milestones

Project formation - Feb 2000 Stone Soup; process began late summer 2000 with bi-weekly calls to develop scenario, requirements and architecture.

Linkages to SAML established Dec 2000

Architecture and protocol completion - Aug 2001

Design - Oct 2001

Coding began - Nov 2001

Alpha-1 release – April 24, 2002

OpenSAML release – July 15, 2002

v1.0 April 2003; v1.1 July 2003; v1.2 May 2004

v2.0 likely end of the major evolution

Shibboleth Status

Open source, privacy preserving federating software Being very widely deployed in US and international universities Target - works with Apache(1.3 and 2.0) and IIS targets; Java

origins for a variety of Unix platforms. V1.3 likely to include portal support, identity linking, non web

services (plumbing to GSSAPI,P2P, IM, video) etc. Work underway on intuitive graphical interfaces for the powerful

underlying Attribute Authority and resource protection Likely to coexist well with Liberty Alliance and may work within the

WS framework from Microsoft. Growing development activities in several countries, providing

resource manager tools, digital rights management, listprocs, etc. http://shibboleth.internet2.edu/

Adoption

Over 50 + universities using it for access to OCLC, JSTOR, Elsevier, WebAccess, Napster, etc.

Common status is “moving into production”

The hard part is not installing Shibboleth but running “plumbing” to it: directories, attributes, authentication

Deployments in Europe, the UK, South America and Australia

Needs federations to scale; being adopted by, or catalyzing, national R&E federations in several countries

Signet: Stanford Authority System

Signet Deliverables

The deliverables consist of A recipe, with accompanying case studies, of how to take a role-based organization and develop apprpriate groups, policies, attributes etc to operate an authority serviceTemplates and tools for registries and group managementa Web interface and program APIs to provide distributed management (to the departments, to external programs) of access rights and privileges, and delivery of authority information through the infrastructure as directory data and authority events.

Home

Grant Authority Wizard

Virtual Organizations

Geographically distributed, enterprise distributed community that shares real resources as an organization.

Examples include team science (NEESGrid, HEP, BIRN, NEON), digital content managers (library cataloguers, curators, etc), life-long learning consortia, etc.

On a continuum from interrealm groups (no real resource management, few defined roles) to real organizations (primary identity/authentication providers)

Want to leverage enterprise middleware and external trust fabrics

Virtual Organizations

Some things seem consistent across almost all VO’s• The need to manage and delegate VO authorizations• Unique naming, and managed resource discovery• A set of collaboration tools, including a list manager, calendar,

shared web content management, etc that are seamlessly integrated into users’ everyday environment

• A need to factor in, and leverage, local domain requirements and capabilities

Some things are specific to each VO• The members and the resources being managed• Requirements for advanced services, such as Grids and instrument

management

Virtual organizations

Need a model to support a wide variety of use cases• Native v.o. infrastructure capabilities, differences in enterprise

readiness, etc.• Variations in collaboration modalities• Requirements of v.o.’s for authz, range of disciplines, etc

JISC in the UK has lead; solicitation is on the streets (see (http://www.jisc.ac.uk/c01_04.html); builds on NSF NMI

Tool set likely to include seamless listproc, web sharing, shared calendaring, real-time video, privilege management system, etc.

Leveraging V.O.s Today

VO

Target Resource

User

Enterprise

Federation

Leveraged V.O.s Tomorrow

VO

Target Resource

User

Enterprise

Federation

Collaborative Tools Authority Systemetc

Middleware DiagnosticsProblem Statement

• The number and complexity of distributed application initiatives and products has exploded within the last 5 years

• Each must create its own framework for providing diagnostic tools and performance metrics

• Distributed applications have become increasingly dependent not only on the system and network infrastructure that they are built upon, but also each other

• Middleware diagnostics need to integrate with network performance diagnostics and security diagnostics

Goals

• Create an event collection and dissemination infrastructure that uses existing system, network and application data (Unix/WIN logs, SNMP, Netflow©, etc.)

• Establish a standardized event record that normalizes all system, network and application events into a common data format

• Build a rich tool platform to collect, distribute, access, filter, aggregate, tag, trace, probe, anonymize, query, archive, report, notify, perform forensic and performance analysis

Cisco NetFlow Events

RMON Events

Event Record Standard

• Normalization of each diagnostic data feed type (SHIB, HTTP, Syslog, RMON, etc.) into a common event record

• The tagging of specific events to help downstream correlation processes

DB Access Log

SHIB log

HTTP Access log

GRID Application Log

NormalizationAnd EventTagging

NETFLOW:TIME:SRC:DST:…RMON:HOST:TIME:DSTPORT..DB:TIME:HOST:REQ:ASTRONSHIB:TIME:HOST:UID…HTTP:TIME:HOST:URL…GRIDAPP:TIME:HOST:UID:…

Variable Star Catalog DBApplication

Diagnostic Data Pipelining

Data flows can be constructed to provide the desired function and policy within a enterprise or federation

Filter

C-4

Network Events

ArchiveDBAnonimizationTagging AggregationNormalization

C-3

C-1

P-1C-2

P-2

P-3

P-4

P-5

C-* Collection Module HostP-* Processing Module Host

Host or Security Events

Event Record

Event Descriptor Meta Field

Event Descriptor

• Version Number• Observation Description Pointer• ID – unique event identifier• Time - start/stop• IP Address(es) – source/(destination)• Source Class – application, network, system, compound, bulk, management• Event Name Tag – Native language ID, user defined• Status – normal, informational, warning, measurement, critical, error, etc. • Major Source Name – filename, Netflow, Syslogd, SNMP, shell program, etc.• Minor Source Name – logging process name (named), SNMP variable name, etc.• Raw Data Encoding Mechanism – Binary, ASN1, ASCII, XML, etc.• Raw Event Data Description Pointer

Raw Event Data

The next year or so

An integrated marketplace for identity management services, packaged with work, home and personal forms

Federations and international peering of trust

More integration between Grids and enterprises

Virtual organization services• A mix of enterprise, community and outsourced options

Adaptation of Signet-type privilege management• New business models for content and service providers

Diagnostic hell• Things will get much worse before they get better