Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
⼜又拍云 CI/CD 实践
莫红波-⼜又拍云
⾃自我介绍
2014 年年加⼊入⼜又拍云
图⽚片处理理业务
⾳音视频处理理业务
私有云项⽬目
边缘计算项⽬目
⾃自动化测试部署相关
主要内容
• GitLab WorkFlow
• ⼀一些术语
• 如何使⽤用 GitLab CI
• GitLab Runner 内部实现
• ⼀一些实践经验
GitLab WorkFlow
⼀一些术语
• Job/Build:最⼩小的构建任务单元• Stage:阶段,可以通过定义不不同的阶段来决定 Job 执⾏行行的先后顺序• Pipeline:流⽔水线,可以直观的看到不不同 Job 以及 Stage 间的关系• GitLabRunner:实际处理理 GitLab 构建任务的 Worker• GitLabMultiRunner:多个 GitlabRunner 的管理理者,⽀支持动态修改 runner 配置等• Executor:执⾏行行器器类型,每个 runner 都需要指定⼀一个,决定使⽤用哪个执⾏行行器器处理理构建任务
⼀一些术语
• Job/Build:最⼩小的构建任务单元• Stage:阶段,可以通过定义不不同的阶段来决定 Job 执⾏行行的先后顺序• Pipeline:流⽔水线,可以直观的看到不不同 Job 以及 Stage 间的关系• GitLabRunner:实际处理理 GitLab 构建任务的 Worker• GitLabMultiRunner:多个 GitlabRunner 的管理理者,⽀支持动态修改 runner 配置等• Executor:执⾏行行器器类型,每个 runner 都需要指定⼀一个,决定使⽤用哪个执⾏行行器器处理理构建任务
GitLab CI 使⽤用
GitLab CI 使⽤用 - runner 注册
GitLab CI 使⽤用 - runner 注册
GitLab CI 使⽤用 - runner 注册
GitLab CI 使⽤用 - 增加 .gitlab-ci.ymlstages: - test - buildtest: stage: test script: - echo skip tags: - opentalk
build: stage: build script: - export PATH=$PATH:/usr/local/go/bin - make docker VER=$CI_BUILD_TAG tags: - on-line-docker-builder only: - tags@consumers/jigsaw
GitLab CI 使⽤用 - 增加 .gitlab-ci.ymlstages: - test - buildtest: stage: test script: - echo skip tags: - opentalk
build: stage: build script: - export PATH=$PATH:/usr/local/go/bin - make docker VER=$CI_BUILD_TAG tags: - on-line-docker-builder only: - tags@consumers/jigsaw
GitLab CI 使⽤用 - 增加 .gitlab-ci.ymlstages: - test - buildtest: stage: test script: - echo skip tags: - opentalk
build: stage: build script: - export PATH=$PATH:/usr/local/go/bin - make docker VER=$CI_BUILD_TAG tags: - on-line-docker-builder only: - tags@consumers/jigsaw
GitLab CI 介绍
GitLab CI 介绍
115.231.11.11:80
GitLabMultiRunner
10.0.5.112
…
GitLabMultiRunner
10.0.5.113
…
GitLabMultiRunner
10.0.5.114
…
GitLab CI 内部消息机制 - runner 注册
POST /api/v4/runners HTTP/1.1Host: gitlab.widget-inc.comContent-Type: application/json
{ "info": { "name": "gitlab-ci-multi-runner", "version": "9.5.0", "revision": "413da38", "platform": "linux", "architecture": "amd64", "features": { "variables": false, "image": false, "services": false, "features": false, "cache": false } }, "token": "LhDVkUM9kysPbasGuEEd", "description": "runner for OpenTalk", "tag_list": "opentalk", "run_untagged": false, "locked": true}
请求体内容 返回体内容
HTTP/1.1 201 CreatedContent-Type: application/json
{"id":136,"token":"5d36b61b35fbbfa432d8bce605c046"}
GitLab CI 内部消息机制 - runner 注册
POST /api/v4/runners HTTP/1.1Host: gitlab.widget-inc.comContent-Type: application/json
{ "info": { "name": "gitlab-ci-multi-runner", "version": "9.5.0", "revision": "413da38", "platform": "linux", "architecture": "amd64", "features": { "variables": false, "image": false, "services": false, "features": false, "cache": false } }, "token": "LhDVkUM9kysPbasGuEEd", "description": "runner for OpenTalk", "tag_list": "opentalk", "run_untagged": false, "locked": true}
请求体内容 返回体内容
HTTP/1.1 201 CreatedContent-Type: application/json
{"id":136,"token":"5d36b61b35fbbfa432d8bce605c046"}
GitLab CI 内部消息机制 - 请求 Job
POST /api/v4/jobs/request HTTP/1.1Host: gitlab.widget-inc.comContent-Type: application/json
{ "info": { "name": "gitlab-ci-multi-runner", "version": "9.5.0", "revision": "413da38", "platform": "linux", "architecture": "amd64", "executor": "docker", "features": { "variables": true, "image": true, "services": true, "features": false, "cache": false } }, "token": "5d36b61b35fbbfa432d8bce605c046", "last_update": "b03cb53e63daeb53c7f7451c10c5cd9e"}
HTTP/1.1 204 No ContentX-Gitlab-Last-Update: b03cb53e63daeb53c7f7451c10c5cd9e
情况1: 没有需要处理理的 Job请求体内容
GitLab CI 内部消息机制 - 请求 Job
POST /api/v4/jobs/request HTTP/1.1Host: gitlab.widget-inc.comContent-Type: application/json
{ "info": { "name": "gitlab-ci-multi-runner", "version": "9.5.0", "revision": "413da38", "platform": "linux", "architecture": "amd64", "executor": "docker", "features": { "variables": true, "image": true, "services": true, "features": false, "cache": false } }, "token": "5d36b61b35fbbfa432d8bce605c046", "last_update": "b03cb53e63daeb53c7f7451c10c5cd9e"}
情况2: 有需要处理理的 Job请求体内容
HTTP/1.1 201 CreatedContent-Type: application/json
{ "id": 43848, "token": "8XdyE6KdnyUkDdZZQbxs", "allow_git_fetch": true, "job_info": { "name": "test", "stage": "test", "project_id": 2220, "project_name": "jigsaw" }, "git_info": { "repo_url": "http://gitlab-ci-token:[email protected]/consumers/jigsaw.git", "ref": "master", "sha": "bead0bb1a1d717bdd8eeaf1fe75905fdf4658781", "before_sha": "bead0bb1a1d717bdd8eeaf1fe75905fdf4658781", "ref_type": "branch" },
GitLab CI 内部消息机制 - 请求 Job
POST /api/v4/jobs/request HTTP/1.1Host: gitlab.widget-inc.comContent-Type: application/json
{ "info": { "name": "gitlab-ci-multi-runner", "version": "9.5.0", "revision": "413da38", "platform": "linux", "architecture": "amd64", "executor": "docker", "features": { "variables": true, "image": true, "services": true, "features": false, "cache": false } }, "token": "5d36b61b35fbbfa432d8bce605c046", "last_update": "b03cb53e63daeb53c7f7451c10c5cd9e"}
情况2: 有需要处理理的 Job 请求体内容
"runner_info": { "timeout": 3600 }, "variables": [{ "key": "CI_JOB_ID", "value": "43848", "public": true }, { "key": "CI_JOB_NAME", "value": "test", "public": true }], "steps": [{ "name": "script", "script": ["echo skip"], "timeout": 3600, "when": "on_success", "allow_failure": false }], "image": null, "services": [], "artifacts": [null], "cache": [null], "credentials": [], "dependencies": []}
GitLab CI 内部消息机制 - 确认 Job
PUT /api/v4/jobs/43848 HTTP/1.1Host: gitlab.widget-inc.comContent-Type: application/json
{ "info": { "name": "gitlab-ci-multi-runner", "version": "9.5.0", "revision": "413da38", "platform": "linux", "architecture": "amd64", "executor": "docker", "features": { "variables": true, "image": true, "services": true, "features": false, "cache": false } }, "token": "8XdyE6KdnyUkDdZZQbxs", "state": "running"}
返回体内容请求体内容
HTTP/1.1 200 OKContent-Type: application/json
null
GitLab CI 内部消息机制 - 信息返回
PATCH /api/v4/jobs/43848/trace HTTP/1.1Host: gitlab.widget-inc.comContent-Range: 0-354Content-Type: text/plainJob-Token: 8XdyE6KdnyUkDdZZQbxs
0KRunning with gitlab-ci-multi-runner 9.5.0 (413da38) on runner for OpenTalk (5d36b61b)Using Docker executor with image repo.upyun.com:5043/opentalk:latest ...ERROR: Preparation failed: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?Will be retried in 3s ...
返回体内容
请求体内容
HTTP/1.1 202 AcceptedContent-Type: application/jsonJob-Status: running
"0-354"
GitLab CI 内部消息机制 - 结果返回
PUT /api/v4/jobs/43848 HTTP/1.1Host: gitlab.widget-inc.comContent-Type: application/json
{ "info": { "name": "gitlab-ci-multi-runner", "version": "9.5.0", "revision": "413da38", "platform": "linux", "architecture": "amd64", "executor": "docker", "features": { "variables": true, "image": true, "services": true, "features": false, "cache": false } }, "token": "8XdyE6KdnyUkDdZZQbxs", "state": "failed", "trace": "\u001b[0KRunning with gitlab-ci-multi-runner 9.5.0 (413da38)\n on runner for OpenTalk ..."}
返回体内容请求体内容
HTTP/1.1 200 OKContent-Type: application/json
true
GitLab CI 内部消息机制 - 总结
POST /api/v4/runners
201 Created
POST /api/v4/jobs/request
204 No Content
Runner
Job
GitLab CI 内部消息机制 - 总结
POST /api/v4/jobs/request
201 Created
PUT /api/v4/jobs/43848
200 OK
PATCH /api/v4/jobs/43848/trace
202 Accepted
PUT /api/v4/jobs/43848
200 OK
Job
GitLab Executor"job_info": {},"git_info": {},"runner_info": { "timeout": 3600 }, "variables": [{ "key": "CI_JOB_ID", "value": "43848", "public": true }, { "key": "CI_JOB_NAME", "value": "test", "public": true }], "steps": [{ "name": "script", "script": ["echo skip"], "timeout": 3600, "when": "on_success", "allow_failure": false }], "image": null, "services": [], "artifacts": [null], "cache": [null], "credentials": [], "dependencies": []}
GitLab Executor"job_info": {},"git_info": {},"runner_info": { "timeout": 3600 }, "variables": [{ "key": "CI_JOB_ID", "value": "43848", "public": true }, { "key": "CI_JOB_NAME", "value": "test", "public": true }], "steps": [{ "name": "script", "script": ["echo skip"], "timeout": 3600, "when": "on_success", "allow_failure": false }], "image": null, "services": [], "artifacts": [null], "cache": [null], "credentials": [], "dependencies": []}
Runner 是如何处理理 Job 的? 内部处理理逻辑是怎样的?
GitLab Executor
GitLab Executor
GitLab Executor - BuildStagePrepare
Prepare
GitLab Executor - BuildStageGetSource
Prepare
GetSource
"git_info": { "repo_url": "http://gitlab-ci-token:[email protected]/consumers/jigsaw.git", "ref": "master", "sha": "bead0bb1a1d717bdd8eeaf1fe75905fdf4658781", "before_sha": "bead0bb1a1d717bdd8eeaf1fe75905fdf4658781", "ref_type": "branch" },
GitLab Executor - BuildStageRestoreCache
Prepare
GetSource
RestoreCache
"cache": [null],
GitLab Executor - BuildStageDownloadArtifacts
Prepare
GetSource
RestoreCache
DownloadArtifacts
"dependencies": { "id": 44231, "name": "compile-alpine", "token": "rpYZNZBbWekCdpJw4RFw", "artifacts_file": { "filename": "artifacts.zip", "size": 1810651 }}
GitLab Executor - BuildStageDownloadArtifacts
Prepare
GetSource
RestoreCache
DownloadArtifacts
"dependencies": { "id": 44231, "name": "compile-alpine", "token": "rpYZNZBbWekCdpJw4RFw", "artifacts_file": { "filename": "artifacts.zip", "size": 1810651 }}
GET /api/v4/jobs/44231/artifacts HTTP/1.1Host: gitlab.widget-inc.comJob-Token: rpYZNZBbWekCdpJw4RFwAccept-Encoding: gzip
GitLab Executor - BuildStageUserScripts
Prepare
GetSource
RestoreCache
DownloadArtifacts
ExecUserScripts
"steps": [{ "name": "script", "script": ["echo skip"], "timeout": 3600, "when": "on_success", "allow_failure": false }],
Prepare
GetSource
RestoreCache
DownloadArtifacts
ExecUserScripts
ExecAfterScripts
GitLab Executor - BuildStageAfterScripts
"steps": [{ "name": “after_script", "script": ["echo skip"], "timeout": 3600, "when": "on_success", "allow_failure": false }],
GitLab Executor - BuildStageArchiveCache
Prepare
GetSource
RestoreCache
DownloadArtifacts
ExecUserScripts
ExecAfterScripts
ArchiveCache
GitLab Executor - BuildStageUploadArtifacts
Prepare
GetSource
RestoreCache
DownloadArtifacts
ExecUserScripts
ExecAfterScripts
ArchiveCache
UploadArtifacts
"artifacts": [null],
GitLab Executor
Shell SSH Docker
Docker-SSH DockerMachine DockerMachine-SSH
VirtualBox Parallels Kubernetes
GitLab Executor
Shell SSH Docker
Docker-SSH DockerMachine DockerMachine-SSH
VirtualBox Parallels Kubernetes
•秒级别启动容器器
•轻量量
•资源隔离性好
•资源回收⽅方便便
Docker Executor
test: image: ubuntu:17.04 services: - mysql - redis script: - make && make start && make test
image: 本次测试环境使⽤用的 docker 镜像
services: 本次测试依赖的服务
script: 测试脚本
Docker Executor - Prepare
Services
Redis Mysql
Volumes
/build/proj /cache
Docker Executor - Prepare
Services
Redis Mysql
Volumes
/build/proj /cache
Services
Redis Mysql
Volumes
/build/proj /cache
Predefine
VOLU
ME-FR
OM
Docker Executor - GetSource
Docker Executor - ExecScripts
Services
Redis Mysql
Build
Volumes
/build/proj /cache
VOLUME-FROM LINK
Docker Executor vs Docker SSH
$ docker run --name build --link=service-mysql:mysql —link=service-redis:redis —volumes-from=cache —volumes-from=source repo.upyun.com/shared /bin/bash < build_script.sh
docker
http://repo.upyun.com/shared
Docker Executor vs Docker SSH
$ docker run --name build --link=service-mysql:mysql —link=service-redis:redis —volumes-from=cache —volumes-from=source repo.upyun.com/shared /bin/bash < build_script.sh
$ docker run --name build --link=service-mysql:mysql —link=service-redis:redis —volumes-from=cache —volumes-from=source repo.upyun.com/shared-sshd
$ ssh root@container 'bash -s' < build_script.sh
docker
docker-ssh
http://repo.upyun.com/sharedhttp://repo.upyun.com/shared
GitLab Artifacts
MakeNamiImage
GitLab Artifacts - 优化前
MakeNamiImageCompileNamiBinary
GitLab Artifacts - 优化后
GitLab Artifacts - 优化后
stages: - compile - test - release
compile: stage: compile image: gcc:4.1.5 script: - make deps - make && make install && make tar tags: - compile artifacts: name: "binary" paths: - nami.tar.gz expire_in: "1 hour"
test: stage: test script: - make start - time make test tags: - offline-test
release: stage: release script: - make docker tags: - builder only: - tags@upyun-core/nami
GitLab Artifacts - 下载
GitLab Artifacts vs Cache
Cache 是为了了解决同个项⽬目同个 runner,前⼀一次 Job 与后⼀一次 Job 之间的缓存问题,所有数据都是缓存在 runner 所在的宿主机上
Artifacts 是为了了解决同个项⽬目不不同 runner不不同阶段之间的数据流通。所有数据都会保存在 GitLab 服务器器上,由于上传下载会影响测试效率
C/C++ 编译优化
CDN 项⽬目依赖 Nginx,存在公司定制逻辑,每次测试都需要重新编译 Nginx。
C/C++ 编译优化 - ccache
ccache( “compiler cache” 的缩写)是⼀一个编译器器驱动器器。第⼀一次编译时 ccache 会缓存GCC 的 “-E” 输出、编译选项以及 .o ⽂文件到 $HOME/.ccache。第⼆二次编译时尽量量利利⽤用缓存,必要时更更新缓存。
C/C++ 编译优化 - ccache
ccache( “compiler cache” 的缩写)是⼀一个编译器器驱动器器。第⼀一次编译时 ccache 会缓存GCC 的 “-E” 输出、编译选项以及 .o ⽂文件到 $HOME/.ccache。第⼆二次编译时尽量量利利⽤用缓存,必要时更更新缓存。
测试跑完之后容器器⽴立刻就被回收了了,如何将编译缓存保留留到下⼀一次测试呢?
C/C++ 编译优化 - ccache
ccache( “compiler cache” 的缩写)是⼀一个编译器器驱动器器。第⼀一次编译时 ccache 会缓存GCC 的 “-E” 输出、编译选项以及 .o ⽂文件到 $HOME/.ccache。第⼆二次编译时尽量量利利⽤用缓存,必要时更更新缓存。
测试跑完之后容器器⽴立刻就被回收了了,如何将编译缓存保留留到下⼀一次测试呢?
$ ccache —set-config=cache_dir=/cache/.ccache$ ccache -F 0 && ccache -M 0
解决 Localhost 问题
Services
Redis Mysql
Build
LINK
解决 Localhost 问题
Services
Redis Mysql
Build
LINK
配置中更更多使⽤用 Localhost,⽅方便便本地测试
解决 Localhost 问题 - services 配置
Services
Redis Mysql
Build
LINK
解决 Localhost 问题 - services 配置
Services
Redis Mysql
Build
LINK
修改代码,使⽤用别
名访问
解决 Localhost 问题 - 端⼝口转发
before_script: - echo "tcppm 6379 redis 6379" > /tmp/3proxy.cfg - echo "tcppm 3306 mysql 3306" >> /tmp/3proxy.cfg - 3proxy /tmp/3proxy.cfg &
THANKS!
感谢您的聆听