Presented By: Date:

Cost Efficient, Auto Scaling Continuous Integration with Amazon AWS Cloud

Mateusz Zając @ Software Engineer SMART

2015-10-18

Introduction

Mateusz Zajac | Software Engineer | Kainos | DD: +48 58 333 71 42 | Skype: mateuszza@kainos.com

Introduction

Few Words About Continuous Integration

Few Words About Continuous Integration

Few Words About Continuous Integration

https://en.wikipedia.org/wiki/Comparison_of_continuous_integration_software

Few Words About Continuous Integration

https://en.wikipedia.org/wiki/Comparison_of_continuous_integration_software

Motivation

Motivation

Motivation

Motivation

Motivation

Auto Scaling Concept

Table of Content How To

Creating Amazon AWS instance

Installing TeamCity

Basic TeamCity Configuration

Creating Agents

Scaling Agents

Conclusions

Creating Amazon AWS instance

Amazon offers 750h of free t2.micro instace so you can prototype yourself! I encourage you!

Once you setup your key pair for login and Security groups you are ready to setup the instance with Team City.Pre requisites:Unzip, java jdk7

Installing TeamCity

$ sudo apt-get install openjdk-7-jdk

Login to the newly create AWS instance

Download TC from https://www.jetbrains.com/teamcity/download/

$ tar xvzf TeamCity-9.1.3.tar.gz$ ./TeamCity/bin/runAll.sh start

Go to the <serverip/domain>:8111 and configure your administrator account

Job done

Basic TeamCity Configuration

Creating Agents

Once you setup your key pair for login and Security groups you are ready to setup the instance with Team City.Pre requisites:Unzip, java jdk7

Download the zip file to the agent. Unzip it

$ unzip buildAgent.zip$ chmod +x ./bin/agent.sh start

Creating AgentsConfigure the build.properties for the agent

$ cp ./conf/buildAgent.dist.properties ./conf/buildAgent.properties$ vim ./conf/buildAgent.properties

Setup the serverURL with your TC Web App addressAdd agent to boot

$ vim /etc/rc.local

Add a line:<path to where you unpacked zip>/bin/agent.sh start

Scaling Agents

Scaling Agents

Scaling AgentsFill up the queue with dummy project runs

Conclusions

-50%

Further down the road

On Demand vs. Spot Instances

FIGHT!

Further down the road

m3.xlarge 4 13 15 2 x 40 SSD $0.266 per Hour

vCPU ECU Memory (GiB)

Instance Storage

(GB)

Linux/UNIX

Usage

Further down the road

Further down the road

Vs. $0.266

Conclusions

-50%

Conclusions

-84%

Spot InstancesAmazon EC2 Spot InstancesAmazon EC2 Spot instances are spare EC2 instances that you can bid on to run your cloud computing applications. Since Spot instances are often available at a lower price, you can significantly reduce the cost of running your applications, grow your application’s compute capacity and throughput for the same budget, and enable new types of cloud computing applications.BenefitsGet the Best Value for Amazon EC2 CapacitySpot instances run when your bid price exceeds the Spot price, and provide the reliability, security, performance, control, and elasticity of Amazon EC2, at low market-driven prices.

Reduce Operating CostsReduce your operating costs by up to 50-90% with Spot, compared to On-Demand instances. To see the current prices, visit the Spot Pricing page.

Improve Application ThroughputYou can run and scale applications such as stateless web services, image rendering, big data analytics and massively parallel computations on Spot instances. Since Spot instances typically costs 50-90% less, you can increase your compute capacity by 2-10x within the same budget.

Motivation

Motivation

Q&A & Thank you