Build your own clouds with Chef & MCollective

Jonathan Weiss

Who am I?

Working for Peritor in Berlin, Germany

Written, maintain, or involved in

  Webistrano

  Capistrano

  SimplyStored

  Happening

  The great fire of London

http://github.com/jweiss

@jweiss

2

Scalarium

Amazon EC2 Cluster Management

  Auto-Config

  Self-Healing

  Auto-Scaling

  One-click-deployment

www.scalarium.com

3

Cloud Computing

4

Cloud Computing

5

6

What you get

Ubuntu Linux

IBM

Oracle

Red Hat

CentOS

Open Solaris Suse

Windows

Infrastructure as a Service

What you really want

Load balancer

App server

Database

Fully configured stack of servers, dependencies, firewalls, … +

your own application deployed

Automation is needed for

• Configuration

• Healing

• Scaling

• Deployment

flickr.com/photos/torek/3788181603

Automation Solutions

Reasons to build your own

  In-house capacity

 Governance: Security and regulations

  Flexibility (or lack of…)

11

12

Ingredients

Components

Host configuration

Command & Control

Communication

Command & Control

14

Responsibilities:

  Communication with IaaS API

  User Interface (Web, CLI, API): Query, deploy, start, stop

  Repository of projects &configuration

  Controls which hosts do what&when

Command & Control

Implementation:

  Fog & other API implemenations

  Rails

  CouchDB

  Async wokers & agents

15

Responsibilities:

  Communication with IaaS API

  User Interface (Web, CLI, API): Query, deploy, start, stop

  Repository of projects &configuration

  Controls which hosts do what&when

Communication

16

Responsibilities:

  Distribute commands and responses in a scalable & fault tolerant way

Communication

Implementation:

  MCollective / ActiveMQ

17

Responsibilities:

  Distribute commands and responses in a scalable & fault tolerant way

Host Configuration

18

Responsibilities:

  Actually executes commands

  Responds to cluster changes

  Ensures local host is configured according to commands: Install software, start/stop services, ...

Host Configuration

Implementation:

  Chef

  MCollective Agents

19

Responsibilities:

  Actually executes commands

  Responds to cluster changes

  Ensures local host is configured according to commands: Install software, start/stop services, ...

20

Host Configuration Bootstrapping new servers

Chef

Open Source Provisioning Tool

  Configures your servers

  Cookbooks & recipes

  Ruby DSL

21

Chef

22

Chef Server

23

Chef Solo

24

Chef-Solo vs. Chef-Server

Chef-Server   Complex

  No concept of different environments, e.g. staging vs. production

  Didn‘t want to run one Chef server per project (and env)

  No lifecycle events, e.g. deployment vs. bootstrap

  Pull vs. push

Chef-Solo

  MCollective agents to push cookbooks and Chef runs

25

Cookbooks

26

Cookbooks

apache2

memcached

mysql

rails

Memcached Cookbook

27

Cookbooks

memcachd

attributes

recipes

templates

Memcached Cookbook

28

memcached

attributes

default.rb

recipes

service.rb

install.rb

templates

default

memcached.conf.erb

Attributes

29

cookbooks/memcached/attributes/default.rb

Set default values for variables to use in recipes & templates

Recipes

30

cookbooks/memcached/recipes/service.rb

Actual instructions to execute

Recipes

31

cookbooks/memcached/recipes/install.rb

Templates

32

cookbooks/memcached/templates/default/memcached.conf.erb

Chef Solo Run

33

Configuration JSON: /etc/chef/solo.json

Run Chef Solo

Available Resources

34

Cookbook File

Cron

Deploy

Directory

Erlang Call

Execute

File

Git

Group

HTTP Request

Ifconfig

Link

Log

Mdadm

Mount

Package

Remote Directory

Remote File

Route

Ruby Block

SCM

Script

Service

Subversion

Template

User

Deployment with Chef

Deploy resource   Compatible with Capistrano layout

  Supports Capistrano hooks via deploy/hook_name.rb

  Handles source checkout & callbacks

Call custom script

  Arbitrary options, e.g:

Download WAR file

Compile Source

…

  Automatically wrapped in Chef error handling & notifications

35

Deploy Resource

36

37

Communication Relaying messages to servers

Marionette Collective

„a framework to build server orchestration

or parallel job execution systems“

38

MCollective Architecture

39

RPC Calls

Client

Service: chef-run Service: update-cookbooks

Service: chef-run Service: update-cookbooks

Agents

Message Routing

Discover ���agents

RPC Calls

Client

Service: chef-run Service: update-cookbooks

Service: chef-run Service: update-cookbooks

Agents

Message Routing

Call ���agents

MCollective Agent

42

Calling an Agent

43

Calling an Agent

44

Filters & Facts

Limit responding agents by filters of facts/classes:

Classes

  Arbitary used defined settings role

  Can be set by userdata

Facts

  Information about local machine

  Plugins for Chef/Ohai, Puppet/Facter

  Examples: Linux version, installed packages, ....

45

Filters on the Command Line

46

Filters in Code

47

MCollective & Chef

Call Chef action on remote agents/machines

48

MCollective & Chef

Agent running Chef

49

Security

Clients   Stomp / ActiveMQ user

  AES+RSA plugins: encrypt & sign every message

  SSL plugin: sign every message

  TLS for connection encryption

Middleware

  Topic permissions and subclusters

Agents

  All client security measures available

  RPC authorization and auditing

50

Scalability & Redundancy

ActiveMQ offers different means

  Network of brokers

  Broker clusters

  Master/Slave

51

52

Command & Control Controlling the cloud

Command Software

General capabilities   Stores cluster model

  Interacts with IaaS API (EC2, VMWare & friends)

  Handles recovery & presence

  Integrates communication solution

  Generates host configuration / Chef events

Custom requirements

  User management & authorization

  Business rules & processes

  Deployment recipes & scripts

53

Server Roles

54

Command Software

55

Experiences

Chef   Flexible and powerful

  Easy to learn first steps - easy to write spaghetti recipes

  Some annoyances: pseudo idempotent and 2 phases

  “Write once, test everywhere”

MCollective

  Simple and robust

  Missing presences and events (Nanite has them but has different problems)

  Integrated with Puppet in the future?

56

© Peritor GmbH - Alle Rechte vorbehalten

Peritor GmbH Blücherstr. 22, Hof III Aufgang 6 10961 Berlin

Tel.: +49 (0)30 69 20 09 84 0 Fax: +49 (0)30 69 20 09 84 9

Internet: www.peritor.com E-Mail: info@peritor.com

Q&A

Idempotent

Bug or feature?

58

59

Will never happen, even if /data/logs/redhat will be created!

Two Phases of Chef

1) Compile

Load all attributes&recipes and build list of actions/dependencies

Compute attribute tree

2) Run

Actually execute the resources

60

Random Ruby Code runs here

Two Phases of Chef

 Use only_if and not_if

 Tell Chef to execute your ressource during compile phase

61