AWS와 함께하는 DevOps이야기 :: 박선용 :: AWS Summit Seoul 2016

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

박선용솔루션즈 아키텍트

2016/05/17

AWS와 함께하는 DevOps기술이야기

오늘의 내용

1. AWS DevOps 관련서비스들2. DevOps 개념3. 아마존이야기4. Code 시리즈 (Pipeline, Deploy) & Lambda5. Demo

1. AWS DevOps 관련서비스들

AWS서비스종류

Elastic Beanstalk

AWS CloudFormation

AWS CodeDeploy AWS OpsWorks

Infrastructuredeployment

Codemanagement

Application deployment

Runtime/container

AWS CodeCommit AWS CodePipeline

AWS OpsWorks Elastic Beanstalk

Elastic Beanstalk

Elastic Beanstalk

Amazon EC2 EC2 Container Service Amazon Lambda

4

AWS서비스종류

Elastic Beanstalk

AWS CloudFormation

AWS CodeDeploy AWS OpsWorks

Infrastructuredeployment

Codemanagement

Application deployment

Runtime/container

AWS CodeCommit AWS CodePipeline

AWS OpsWorks Elastic Beanstalk

Elastic Beanstalk

Elastic Beanstalk

Amazon EC2 EC2 Container Service Amazon Lambda

5

2. DevOps 개념

신속한배포를위해서어떠한도구가필요한가?

개발주도적세계에서배포는다음과같음많은도구를고려해야함:• 소프트웨어개발,릴리즈프로세스흐름을관리할도구• 코드흠결및잠재적이슈를테스트하고검토할수있는적합한도구

• 어플리케이션을배포할도구

개발주기는점점빨라지고있음

게임소프트웨어개발및배포는쉬워지면서더욱가속화됨:• 모바일게임의생명주기는아주짧음• 고객의이벤트및요구에따라잦은패치및업데이트가필요

• 수백만명에게게임을전달하는것이아주쉬운환경

• 개발속도를높이는것은중간방해물과의싸움

과거배포모델

게임배포방식이극적으로변함

새로운배포방식

시작하기앞서,소프트웨어릴리즈프로세스에대한이해가필요

https://www.flickr.com/photos/jurvetson/5201796697/

• Integration tests with other systems

• Load testing• UI tests• Penetration

testing

릴리즈프로세스의 4개주요단계

소스 빌드 테스트 프로덕션

• Check-in source code such as .java files.

• Peer review new code

• Compile code• Unit tests• Style checkers • Code metrics• Create

container images

• Deployment to production environments

릴리즈프로세스단계

소스 빌드 테스트 프로덕션

지속통합 (Continuous integration)

지속전달 (Continuous delivery)

지속배포 (Continuous deployment)

Release Processes levels

Source Build Test Production

Continuous integration

Continuous deployment

오늘살펴볼단계

지속전달 (Continuous delivery)

지속전달의잇점

개발자생산성향상 버그를 특정하고찾아서 쉽게해결

업데이트를 보다빠르게 전달

지속적인 소프트웨어릴리즈 프로세스

3. Amazon 이야기

아마존의개발을돌이켜보면..

https://secure.flickr.com/photos/pixelthing/15806918992/

2001

아마존의개발방식변환: 2001-2009

2009

monolithic application + teams

microservices + 2 pizza teams

이팀과개발문화하에서모든점이개선되었다.과거보다

릴리즈는훨씬빨라지고좋아졌다.하지만우리는여전히더개선할수있다고

느꼈다.

2009년,우리는어떤비효율성이여전히존재하는지연구를수행했다.

단지기다리고있음

WaitWrite Code WaitBuild

Code WaitDeploy to Test

Deploy to

Prod




Deploy to

Prod

Mins Days Mins Days Mins Days Mins




Deploy to

Prod

Weeks





Deploy to

Prod

Weeks


우리는소프트웨어릴리즈프로세스를자동화하는툴을만들었다

https://secure.flickr.com/photos/lindseygee/5894617854/

자동화된액션(action)과이행(transitions); 체크인부터프로덕션까지

개발이익:• 더빠름

• 더안전함

• 단순화 & 표준화• 처리의시각화

Pipelines

지금까지매우우수하게지속적으로작동:

2014년:• Amazon의수천개서비스팀• Microservices를개발• 지속전달을실제로적용• 많은환경 (스테이징,베타,프로덕션)

50 million deploys

지금까지매우우수하게지속적으로작동 :

매년 Amazon에서는 우리의소프트웨어개발자에게서베이를수행. 2014년 결과에서는보다행복한개발자와통계적으로관련이있는하나의개발툴/서비스가발견되었다.그것은바로

pipelines 서비스!

continuous delivery == happier developers!

4. Code 시리즈 ( Pipeline, Deploy) & Lambda

빠르고신뢰성있는어플리케이션업데이트를위한지속전달서비스

게임소프트웨어릴리즈프로세스의모델화및시각화

코드변경이있는경우빌드,테스트와 배포를매번수행가능

3rd파티툴과AWS서비스와 통합

AWS CodePipeline

AWS CodePipeline장점

Improved quality

Rapid delivery Get started fast

Configurable workflow Easy to integrate

Source

SourceGitHub

Build

JenkinsOnEC2Jenkins

Deploy

JavaAppElastic Beanstalk

PipelineStage

Action

Transition

CodePipelineMyApplication

Source

SourceGitHub

Build

JenkinsOnEC2Jenkins

Deploy


NotifyDevelopersLambda


Parallel actions

Source

SourceGitHub

Build

JenkinsOnEC2Jenkins

Deploy


NotifyDevelopersLambda

TestAPIRunscope


Sequential actions

8. Retrieve build artifact

EC2 instance

CodePipeline

Source

SourceGitHub

Build

JenkinsOnEC2Jenkins

Deploy


Source Artifact

S3

Build Artifact

S3

5. Get source artifact

1. Get Changes

6. Store build artifact

3. Poll for Job

4. Acknowledge Job

7. Put Success

9. Deploy build artifact

Elastic BeanstalkWeb container

Java App

MyApplication

매우다양한파트너가존재,계속증가Source Build Test Deploy

AWS 서비스통합

소스 로직호출 배포

AWS Elastic BeanstalkAmazon S3

AWS CodeDeploy

AWS Lambda

게임어플리케이션개발릴리즈파이프라인을생성

https://www.flickr.com/photos/seattlemunicipalarchives/12504672623/

게임어플리케이션빌드 &테스트

https://secure.flickr.com/photos/spenceyc/7481166880

코드빌딩

코드 “빌딩”은전형적으로컴파일이필요한언어에대해수행 :• .NET : C#, F#, VB.net, etc.• Java 와관련유사언어 : Java, Scala, JRuby• Go• iOS : Swift, Objective-C

“빌딩”이란표현은 Docker 컨테이너이미지를만들때도사용한다

EC2

빌딩단계가필요없는경우

많은다른언어들은빌딩단계가필요없음.주로이들은인터프리터언어로개발된경우에해당:• PHP• Ruby• Python• Node.js

이경우단순배포만!

EC2

코드테스팅

테스팅은과학과예술두측면을모두가짐

코드테스팅의목표:• 원하는기능에대한구현확인• 프로그램의문법에러찾기• 코드패턴과포맷에대한표준화• 원치않은어플리케이션사용법과로직상의흠결로인한버그감소

• 어플리케이션을좀더보안성높게만듬

테스팅의출발점

1. 일반적인사소한문법오류

• 브라켓,코마 등을빠뜨리고 있지않은가?2. 전체소프트웨어의가장윗단계부분인유저가사용하는화면에서부터시작

• 각 게임화면이 정확하게보여지고 있는가?• 화면별로 렌더링은잘 수행되고있는가?

3. 더깊숙한 UX 및서비스체크• 상호 싱크는맞는가?

4. 기본유저함수의유닛테스트로부터더깊숙한서버로직까지테스트

• 제품화에서의 버그를찾기 위해보다 체계화된방법에시간을 더투자

단계별테스트리소스비율:

UI

Service

Unit 70%

20%

10%

게임서버어플리케이션배포

https://secure.flickr.com/photos/simononly/15386966677

코드배포를인스턴스상관없이자동화

업데이트하는과정의복잡함을쉽게처리

어플리케이션배포중다운타임을최소화

어떤언어,어떤 O/S라도, Amazon EC2 혹은온프리미스서버에도배포가능

3rd 파티툴과AWS 서비스와통합

AWS CodeDeploy

appspec.yml Exampleversion: 0.0os: linuxfiles:

- source: /destination: /var/www/html

permissions:- object: /var/www/htmlpattern: “*.html”owner: rootgroup: rootmode: 755

hooks:ApplicationStop:- location: scripts/deregister_from_elb.sh

BeforeInstall:- location: scripts/install_dependencies.sh

ApplicationStart:- location: scripts/start_httpd.sh

ValidateService:- location: scripts/test_site.sh- location: scripts/register_with_elb.sh

appspec.yml Exampleversion: 0.0os: linuxfiles:

- source: /destination: /var/www/html

permissions:- object: /var/www/htmlpattern: “*.html”owner: rootgroup: rootmode: 755

hooks:ApplicationStop:- location: scripts/deregister_from_elb.sh

BeforeInstall:- location: scripts/install_dependencies.sh

ApplicationStart:- location: scripts/start_httpd.sh

ValidateService:- location: scripts/test_site.sh- location: scripts/register_with_elb.sh

• Remove/add instance to ELB• Install dependency packages• Start Apache• Confirm successful deploy• More!

• Send application files to one directory and configuration files to another

• Set specific permissions on specific directories & files

v2 v2 v2 v2 v2 v2

one at a time

half at a time

all at once

v2 v2 v2 v1 v1 v1

v2 v1 v1 v1 v1 v1 Agent Agent

Dev Deployment group

ORProd Deployment group

Agent

AgentAgent

Agent Agent

Agent

배포속도와그룹을선택

https://www.flickr.com/photos/spacex/16510243060/

프로덕션으로런칭

프로덕션으로런칭어플리케이션을빌드하고테스트하고나서다행스럽게몇단계의전-프로덕션단계를거치고난이후에는,실제배포를진행

다음을고려:• 고객에대한영향

• 인프라에대한영향

• 비지니스에대한영향

어떻게이과정을추적하고배포간에상호동작을하게할것인가?

커스텀액션을이용한 CodePipeline확장

티켓업데이트 리소스프로비저닝

업데이트대쉬보드 고객에게알림 보안스캔

모바일테스팅

Source

SourceGitHub

Build

JenkinsOnEC2Jenkins

Deploy

ActionCustom Action


MyApplication

CodePipeline

Deploy

JavaAppElastic Beanstalk Job Worker 3. Perform Job

1. Poll for Job

2. Acknowledge Job

4. Put Success

Amazon EC2

커스텀액션과관련해서, 잡워커(Job Worker)가존재해서이것이

AWS CodePipeline과다른어플리케이션혹은서비스와의커뮤니케이션을주도할수

있다

Source

SourceGitHub

Build

JenkinsOnEC2Jenkins

Deploy

MyActionAWS Lambda


2. Perform Job1. Invoke Lambda function

3. PutJobSuccessResult

MyApplication

CodePipeline AWS Lambda기반액션으로, AWS CodePipeline이 Lambda와의통합을

주도하고, Lambda를통해서다른어플리케이션과서비스를연결한다

AWSLambda

3. PutJobSuccessResult w/ Continuation Token4. Invoke Lambda function w/ Continuation Token5. PutJobSuccessResult #3 & #4 repeat until no continuation

token is sent, signaling the action has been completed (#5).

Lambda에서의 SNS 호출구현

var sns = new AWS.SNS();

var datetime = new Date().toISOString().replace(/T/, ' ').replace(/\..+/, '')

var params = {

Message: datetime + ' : DevOps deploy is completed', /* required */

Subject: 'Deploy',

TopicArn: 'arn:aws:sns:us-east-1:550xxxxxxxxxx:devops-deploy-topic'

};

sns.publish(params, function(err, data) {

if (err) console.log(err, err.stack); // an error occurred

else console.log(data); // successful response

});

CodePipeline을위한 Lambda에서 SNS 호출구현var codepipeline = new AWS.CodePipeline();

var jobId = event["CodePipeline.job"].id;

var url = event["CodePipeline.job"].data.actionConfiguration.configuration.UserParameters;

var putJobSuccess = function(message) {

var params = {

jobId: jobId

};

codepipeline.putJobSuccessResult(params, function(err, data) {

if(err) context.fail(err);

else context.succeed(message);

});

};

sns.publish(params, function(err, data) {

if (err) putJobFailure(ex);

else putJobSuccess("Tests passed.");

});

각방법은어떤특성이있는가?Lambda Custom Action

짧은시간동안동작하는태스크를아주쉽게구현

어떤형태의워크로드라도수행가능

장기간동작해야하는태스크의경우보다구현작업이필요

콘솔에서각각의콘트롤간의링크가디스플레이됨

Node.js, Python, Java 지원 어떤언어든지원

AWS상에서동작 on-premises에서도동작가능

프로비전이나매니지할서비스가존재하지않음

컴퓨팅리소스필요

5. Demo

Technology

AWS와 함께하는 DevOps이야기 :: 박선용 :: AWS Summit Seoul 2016