10 Deployments a day - A brief on extreme release protocols

Preview:

DESCRIPTION

A reflection on how we migrated from an era of production deployments once a week to more than 10 deployments a day. The story of transforming a mere sys-admin to super sophisticated DevOps team, armed with multiple tools, scripts and plugins for achieving automations, accuracy and invincible agility. As we continue on our endeavor for further improvisation, I believe its worth sharing the experience with community. Tentative topics outline: 1.Arming teams for their transformation to DevOps from mere sys-admins. 2.Killing manual deployments (manual deployment->capistrano->webistrano) 3.Automation (for daily backups,monitoring applications and servers deploying dependencies) 4.Saving up time on setting up new instances(using golden image). 5.Using LDAP to maintain servers with public key(avoiding individual logins to every server makes it less messy) 6.Using Puppet for instantiating multiple servers at once. 7.Commissioning our own Cloud Infrastructure.

Citation preview

10 Deployments A Day

A brief on extreme release protocol Bhains ki Pooch… :P

weboniselab.com

An outsourced product development company building apps that people love to use!

•  A Weboniser and Rubyist •  A Blogger(vparihar01.github.com) •  A Geek •  DevOps Guy •  An Aspiring Tech Innovator •  Tweet with @vparihar

With the new born startup and freshly out of college grad’s, everything we did was an experiment. Deployments and relevant techniques were completely alien with no experience on how to deal with our first application going live.

Our Stone-Age deployment techniques: •  Created instances manually. •  Installed dependencies one-by-one. •  Pulled from GIT into each instance.

•  Doing every task manually on the server for each deployment - Not again...

•  Database migrations and rolling back is such a headache.

•  We need versions for our deployments!

This is embarrassing… 1 deployment in a week or two.

•  Started versioning viz.(test, stage, production).

•  Brought in Capistrano •  Implemented plugins to notify us of

errors and exceptions.

Wanted to deploy on test.. ..instead deployed on production

•  Rolling back application code successfully but database rollbacks were still a headache.

•  Realized that we need to control deployment access.

•  So, the introduction of SysAdmin. •  Installations through bash scripts.

2011-2012 One Step Ahead

One click deployment with Webistrano instead of Capistrano.

•  What dependencies ? •  No machines available ? •  What database ? •  Security ? •  High Availability ? •  Scalability ? •  My computer can’t install this!

“Put this Code Live, here’s a tarball” NOW!

SysAdmin DOES NOT work!

2 deployments in a week!! Hurray :-)

BUT IT’S NOT ENOUGH

2012 onwards... Bringing in Sophistication

Why SysAdmin didn’t work for us?

Introducing DevOps.. Tada!!

A growing company witnessed proportional infra growth!!!

- 15 live server instances - 45 applications - 7 deployments a week

Biggest Challenge, DevOps faced

Achieving Continuous Deployment {WEBISTRANO}

Keeping our infrastructure… up and running

What We Did…

Monitoring with

Performance with Cacti

The duo in action!

Monitoring with Performance with Cacti

DevOps To The Rescue!

For Centralized Accesses

Combine SSH Public-Key and LDAP on Ubuntu

For Security

Combine UFW and Snort

Automation:The Golden Image

More Automation:Bringing in Puppet

We were spending around 1-1.5 lac/month for our Test and Stage Servers

Why not our own Private Cloud ??

So, we went ahead and bought our own servers and migrated all test and stage instances to our own in-house private cloud.

10 and more deployments in a day

Managing:

•  70 Servers and still increasing... •  More than 120 Live apps.

A unified dashboard for managing multiple cloud services ...HIPPO We intend to opensource the app soon to cater to all our Devops friends…

Thank you

weboniselab.com