Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
www.kit.edu
04.0
8
KIT – the cooperation of Forschungszentrum Karlsruhe GmbH und Universität Karlsruhe (TH)
Cloud Computing mit mathematischen Anwendungen
Vorlesung SoSe 2009
Dr. Marcel KunzeKarlsruhe Institute of Technology (KIT) Steinbuch Centre for Computing (SCC)
Cloud Computing Teil 8 | SoSe 2009 | Dr. M.Kunze2
Agenda Cloud Computing1. Einleitung
Was ist Cloud Computing?2. Grundlagen
Virtualisierung, Web Services,…3. Cloud Architekturen
Infrastruktur, Plattform, Anwendung4. Cloud Services
Amazon Web Services, Google App Engine5. Aufbau einer Cloud
OpenCirrus Projekt, Eucalyptus, Hadoop6. Cloud Algorithmen
MapReduce, Optimierungsverfahren, …
Praktische Übungen und AnwendungenVorlesung im Web: http://www.mathematik.uni-karlsruhe.de/mitglieder/l ehre/cloud2009s/
2
Cloud Computing Teil 8 | SoSe 2009 | Dr. M.Kunze3
Commercial Cloud Offerings (Small Excerpt)
Problem: Commercial offerings are proprietary and u sually not open for cloud systems research and development
Cloud Computing Teil 8 | SoSe 2009 | Dr. M.Kunze4
Cloud Systems Research
Simple, transparent, controllable cloud computing infr astructureWhat types of interfaces are appropriate for clouds ?How should cloud networks be constructed/managed?How are security concerns addressed in “the cloud”?How are various workloads most efficiently transfer red?What types of applications can run in clouds?What types of service level agreements are appropri ate/possible?
Research requirementsPerform experiments also on a low system levelFlexible cloud computing frameworkCompare different methodologies and implementations
Cloud Computing Teil 8 | SoSe 2009 | Dr. M.Kunze5
Cloud Computing
� Cloud computing R&D: OpenCirrus TM project
OpenCirrusTM
Cloud Computing Teil 8 | SoSe 2009 | Dr. M.Kunze6
Cloud Computing Teil 8 | SoSe 2009 | Dr. M.Kunze7
OpenCirrus™ Cloud Computing Research Testbedhttp://opencirrus.org
An open, internet-scale global testbed for cloud computing research
Data center management & cloud servicesSystems level researchApplication level research
Structure: a loose federationSponsors: HP Labs, Intel Research, Yahoo!Partners: UIUC, Singapore IDA, KIT, NSFMembers: System and application development
Great opportunity for cloud R&D
Cloud Computing Teil 8 | SoSe 2009 | Dr. M.Kunze8
Where are the OpenCirrus™ sites? Six sites initially:
Sites distributed world-wide: HP Research, Yahoo!, UIUC, Intel Research Pittsburgh, KIT, Singapore IDA1000-4000 processor cores per siteNew CMU site coming in 2009
HPYahoo(sf)
UIUC
Intel(pgh)
KIT(de)
IDA(sg)
CMU (coming in ‘09)
Cloud Computing Teil 8 | SoSe 2009 | Dr. M.Kunze9
HC3: OpenCirrus Installation at KIT (July)
2656 Nehalem Cores / 332 Servers / 10 TB MemoryInfiniband QDR Interconnect192 TB Lustre Filesystem
MSA2000fc
Infniband QDR Interconnect
Cloud Computing Teil 8 | SoSe 2009 | Dr. M.Kunze10
OpenCirrus Global Services
Federation of sites needs common global services
Global SignOnSingle authentication at all sitesBased on distribution of ssh keys
Global MonitoringBased on Ganglia ( http://ganglia.info/ )
Scalable distributed monitoring systemSupports federations of clusters
OpenCirrus service at http://12.161.200.222/ganglia
Global DataTBD
Cloud Computing Teil 8 | SoSe 2009 | Dr. M.Kunze11
Cloud Architecture
Source: S.Tai
Cloud Computing Teil 8 | SoSe 2009 | Dr. M.Kunze12
OpenCirrus TM Blueprint
IT infrastructure layer (Physical Resource Sets)
Cloud infrastructure services
Cloud application services
Virtual Resource Sets
Eucalyptus
Cloud Computing Teil 8 | SoSe 2009 | Dr. M.Kunze13
Physical Resource Sets (PRS)
PRS service goalsProvide mini-datacenters to researchersIsolate experiments from each otherStable base for other research
PRS service approachAllocate sets of physical co-located nodes, isolate d inside VLANs.Leverage existing software (e.g. Utah Emulab, HP Op sWare)Start simple, add features as we goBase to implement virtual resource sets
Hardware as a Service (HaaS)
Cloud Computing Teil 8 | SoSe 2009 | Dr. M.Kunze14
Virtual Resource Sets (VRS)
Basic idea: Abstract from physical resource by introduct ion of avirtualization layerConcept applies to all IT aspects: CPU, storage, net works and applications, …Main advantages
Implement IT services exactly fitting customer‘s va rying needDeploy IT services on demandAutomated resource managementEasily guarantee service levelsLive migration of servicesReduce both: CapEx and OpEx
Infrastructure as a Service (IaaS)Implement Compute and Storage servicesDe-facto standard: Amazon Web Services interface
Cloud Computing Teil 8 | SoSe 2009 | Dr. M.Kunze15
Eucalyptus: A potential VRS Layerhttp://eucalyptus.cs.ucsb.edu
EUCALYPTUS - Elastic Utility Computing Architecture fo r Linking Your Programs To Useful Systems.
Open-Source software infrastructure for implementin g cloud computing on clusters from UC Santa Barbara.Impelements Infrastructure as a Service (IaaS) – gives the user theability to run and control entire virtual machine instanc es (Xen) deployed across a variety of physical resources.
Interface compatible with Amazon EC2 and S3.Potential to interact with the same tools, known to work withAmazon EC2 and S3.Linux image hosting ala Amazon Web Services.
Eucalyptus is an important step for archiving an open and established cloud computing infrastructure standard.
Spin-off Eucalyptus Systems with venture capital
Cloud Computing Teil 8 | SoSe 2009 | Dr. M.Kunze16
Eucalyptus: A potential VRS Layerhttp://eucalyptus.cs.ucsb.edu
Source: R.Wolski
Schedules the distribution
of virtual machines to the
NC. Collects (free) resource
information.
Collects resource
information from the
CC. Operates like a
meta-scheduler in the
cloud.
Runs on every node in
the cloud. Xen-
Hypervisor running.
Provides Information
about free resources to
the CC.
Cloud Computing Teil 8 | SoSe 2009 | Dr. M.Kunze17
R&D Cloud Installation at SCC/KIT
Eucalyptus R&D Cloud I2x IBM Blade LS20
Dual Core Opteron (2,4GHz)4GB RAM
2x IBM Blade HS21Dual Core Xeon (2,33GHz)16GB RAM
Eucalyptus R&D Cloud II5x HP Blade ProLiant BL2x220cEach Blade: 2 Server Nodes
2x Intel Quad-Core Xeon (2,33GHz) 16GB RAM
Cloud Computing Teil 8 | SoSe 2009 | Dr. M.Kunze18
Comparing Storage Performance between S3 and Eucalyptus
Sequential OutputPer-Character: file is written usind putc()
Block: file is written using write()
Rewrite: read() and write()
Sequential InputPer-Character: file is read usind getc()
Blockwise: file is written using read()
WOW!
Source: C.Baun
Cloud Computing Teil 8 | SoSe 2009 | Dr. M.Kunze19
Realistic values…
The RAM of the Eucalyptus Node Controller was reduced to overcome memory caching.The storage performance of Eucalyptus depends on the avai lable storage sub-system.
Source: C.Baun
Cloud Computing Teil 8 | SoSe 2009 | Dr. M.Kunze20
Performance of Random Seeks and File Creation
The RAM of the Eucalyptus Node Controller was reduced to overcome memory caching.The storage performance of Eucalyptus depends on the avai lable storage sub-system.
Source: C.Baun
Cloud Computing Teil 8 | SoSe 2009 | Dr. M.Kunze21
How is OpenCirrus™ different from other testbeds?
OpenCirrus TM supports both system- and app-level research
n/a at Google/IBM and EC2/S3OpenCirrus TM researchers will have complete access to the underlying hardware and software platform. OpenCirrus TM allows Intel platform features that support cloud computing (e.g. DCMI, NM) to be exposed, and exploited .
Virtualmachines
Hadoop
Map-Reduce apps
Google/IBMcluster
Virtual or physical machines
Cluster mgmt software
Open Cirrus cluster
Hadoop
Cloud apps and services
Map-Reduceapps
Cannot be modified by users
Can be modified by users
Can be modified by users
Cloud Computing Teil 8 | SoSe 2009 | Dr. M.Kunze22
Summary
OpenCirrus TM offers interesting R&D opportunitiesCloud systems developmentCloud application developmentAccepting research proposals
OpenCirrus TM workshop at HP Palo Alto on June 8/9https://opencirrus.org/content/agenda-open-cirrus-summit -palo-alto-june-8-9-2009
Cloud Computing Teil 8 | SoSe 2009 | Dr. M.Kunze23
Karlsruhe Institute of Technology
Thank you for your attention.Steinbuch Centre for Computing (SCC)