2013年11月7日星期四

Shuo Yang Principal Architect of Cloud Computing, US R&D Center

Compass – A Streamlined OpenStack Deployment System

1

Outline of This Talk

DRY, Truly Open Deployment

Scope of Problem for Compass

Compass Explained

1

2

3

2

A General System to Deploy Distributed Systems, Extensibility as a Primary Design Goal

Not Limited to OpenStack, but Streamlined Our OpenStack Deployment Like a Charm

100% Python, 5000 Line of Python Code

Successfully Deployed Several Dogfood Clusters

Compass Wiki Page: https://wiki.openstack.org/wiki/Compass

Compass at a Glimpse

Think Big, Start Small

To Be Open Sourced – Apache 2.0 Soon

3

Open Cloud OS (OpenStack) Open Deployment (Compass)

OpenStack

OpenStack Control Channel

Cinder Quantum Nova

NIC CPU Disk CPU Server

(2285/1285)

Disk Storage (N8000/N900)

Switch Switch (5020)

LILO/GRUB/LiveCD for OpenStack

HW/SW Configuration Deployment Channel

Auto Deploy

Data Center as a Computer

CPU CPU

Disk Disk

Switch

Live CD /

GRUB

NIC

/dev eth0, lo /proc

Linux

4

No.1 as Storage Revenue Growth

No. 2 as `x86 Server Revenue Growth

Needless to Say, Networking Gears..

Global Excellent Telecom Cloud Solution Provider of year 2012

Why We Are Doing This?

Full HW Portfolio in Data Center

OpenStack Makes the Above a Full Cloud Solution

5

Pioneer effort, a Ruby web app, Chef based

configuration management

A great web apps, Puppet based

configuration management

OpenStack Deployment System Overview

Crowbar

Fuel

Attractive concept to OpenStack folks:

deploy OpenStack from OpenStack

A great tool for simple OpenStack setup:

developers use it

TripleO (“under the cloud” mode)

DevStack

6

Zero Touch Installation

Rack and Stack

Blueprint

Robot

Compass

AI

Streamlining the Deployment of a System

7

Real Demo

Video Clips Recording Our First Deployment

8

Outline of This Talk

DRY, Truly Open Deployment

Scope of Problem for Compass

Compass Explained

1

2

3

9

Switch 1

Server 1

Host OS

Switch 2 Switch k

Server 2

Host OS

Server n

Host OS

IPMI

SNMP

Cobbler / Razor / …

Chef / Puppet / …

A Life of Deployment – How to Streamline All Controls

Plus a lot of scripts: Boilerplate Target system

description

10

A Restful API service instead of a web app

Functional modules with plug-in architecture (e.g., 200 Loc to support HP Switch)

Carefully designed boundary to mature tools like Cobbler, Chef and etc.

Work WITH the above tools, not to replace any of them.

Compass Philosophies – Programmability and Extensibility

Programmability – To Be a Programmable Framework

Extensibility – NOT To Be A Conglomerate

11

Huawei UI

DB

Cobbler plugin Razor plugin Chef plugin Ansible plugin

Package Deployment Hardware Discover

Huawei Plugin microkernel

OS Provisioning

Message bus (RabbitMQ)

Restful API Server

3rd Party ISV UI Rest Client

Compass -- Open Source

Compass Internals

12

Machine – physical compute resource

Switch – management network for machines

Cluster – pool of machines managed together

Host – machine in a cluster

Adapter – plug-in module to discover/deploy/configure hosts

Compass – A Programmable Framework

Mapping Fundamental Concepts during Software Deployment into Restful Resources

13 13

GET /machines Lists all discovered machines

GET /machines/id Lists a machine’s hardware details and status

POST /machines/id/action Modify the hardware configuration for a machine

switch=ID address=string cluster=ID status=string

Resource Example: Machines

Query parameters for filtering

marker=markerID limit=int changes-size=dateTime details=ID,switch,address,cluster,devices,status,modified

Actions

modify(hwConfig)

14

Time

(1) Find all available machines: /machines?switch=0e8756dc

(2) Select machines for my system to be deployed on to: {1, 2, 3, 4}

(3) Select an adapter, which describes the result system: /?adapter=OpenStack (4) Find components for result systems {“controller”, “compute”}

(5) Tell Compass a plan {1 -> “controller”; [2,3,4] -> “compute”} (or specify ‘?auto=true’)

(6) Poll the deployment progress

A Life of OpenStack Deployment -- as Restful Calls

15

Currently we use Chef as deployment management Engine

Puppet or Ansible devopers, we want to work with you

Networking gear based bare metal discovery (topolology awareness)

Currently we are working on RAID Controller for Huawei

OCP community, what’s your thought

NOT to Be Conglomerate – Plug-in Architecture

Plug-in architecture for configuration management system

Plug-in architecture for Hardware vendors IPMI / SNMP

16

Chef-Server

Compass

Cobbler-server

Policy Databag (e.g., exclusive; default) OpenStack Cookbook

snippets kickstart files

NOT to Be Conglomerate – Roles in the System

App writer

Lib writer

Restful Reqs

17

Outline of This Talk

DRY, Truly Open Deployment

Scope of Problem for Compass

Compass Explained

1

2

3

18

Distributed System

Hardware

HP

OS

Ubuntu

CentOS

Windows Server?

OpenStack Hadoop Ceph?

OCP?

ESXi ?

Huawei

Extensible in Use Cases

19

OS Provisioning Tools

Cobbler

Razor

Chef Puppet Ansible Salt Configuration Management Tools

Extensible in Tool Chain

20

To Summarize

General Purpose Deployment System (Ultimate Goal)

Deploys OpenStack Cloud Infrastructure Smoothly (Dogfoods Done)

Restful API: Allow Third-party UI Integration

Contact us: shuo.yang@huawei.com

Santa Clara R&D Center, US

Open Sourced under Apache 2.0

Open to Collaboration

Copyright©2013 Huawei Technologies Co., Ltd. All Rights Reserved.

The information in this document may contain predictive statements including, without limitation, statements regarding the future financial and operating results, future product

portfolio, new technology, etc. There are a number of factors that could cause actual results and developments to differ materially from those expressed or implied in the predictive

statements. Therefore, such information is provided for reference purpose only and constitutes neither an offer nor an acceptance. Huawei may change the information at any time

without notice.

22

Compass – Web Enabled Deployment

Compass

Server Farm

Administrators

Internet

Administrators

Intranet

Simple Web actions

zzz…

(40mins)

From baremetal to OpenStack DC

23

Flask for Web Server Layer

Celery for Task Management

Relational DB as Transaction Records

Compass Internal: Current Implementation

RabbitMQ as Message Broker

Chef as Package Management Executor

Cobbler as Host System Management

24

Authentication

JSON or XML request/response data formats

Paginated collections

Follow OpenStack API conventions

Compass API Design Convention

Allow multi-tenancy as an option (tenantID not required)

Use POST to create a new resource and return its ID

Use PUT to replace, PATCH/POST for a partial update

Changes-since query parameter

Version in URI or header

Common representation for faults