61
Jenkins User Conference Boston # jenkinsconf Jenkins State of Union Kohsuke Kawaguchi Creator of Jenkins / CTO, CloudBees @kohsukekawa / [email protected] #jenkinsconf

Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

  • Upload
    others

  • View
    3

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

JenkinsState of Union

Kohsuke Kawaguchi

Creator of Jenkins / CTO, CloudBees

@kohsukekawa / [email protected]

#jenkinsconf

Page 2: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Page 3: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Thank You To Our Sponsors

Platinum Gold

Silver

Page 4: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Installation

Last 12 months

Page 5: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Slaves

Last 12 months

Page 6: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Jobs

Last 12 months

Page 7: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Dockercon keynote

Page 8: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Jenkins is everywhere

Page 9: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Page 10: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Page 11: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Copyright Google

Page 12: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Dockercon keynote

Page 13: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Copyright HBO

Page 14: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Even on a TV show

Page 15: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Even on a TV show

Page 16: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

http://en.wikipedia.org/wiki/File:Grand-Bazaar_Shop.jpg

Page 17: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

DotCi

Page 18: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

DotCi Highlights

• GitHub specific opinionated flavor of Jenkins

• 1 repository = 1 build

• Configuration via source file: .ci.yaml

• Docker

• MongoDB backend for storage

Page 19: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

User Interface Refresh

Doony

Page 20: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Page 21: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

User Interface Refresh

• Short-term goals

– Series of CSS improvements

–<table> → <div>

–Bigger text, bigger controls, bigger buttons

• Pay respect to compatibility

Page 22: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

More UI/UX improvements

• Dynamic content update

– Left-over from FOSDEM UI Discussion

• A lot under discussion

–Create jobs from repos (like GitHub)

–Plugin pack

–…

Page 23: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Page 24: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

End-to-end tracking with

Page 25: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

server

Chef/Puppet

sourcerepository

Page 26: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Tracking for Continuous Delivery

• Track what was deployed when/where?

• Use that to automate even more

– Smoke test

– Show the status in the build job

• Extensible

– Integration to specific tool can be written as plugins

Page 27: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Changes needed in Ops is minimal

• Chef

– Custom report handler

• Puppet

– Just POST a standard report to Jenkins

• Request for feedback

– How should the data come back to Jenkins?

Page 28: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Jenkins and Docker

• DotCI

• Docker plugin for Jenkins

–Runs one-off slaves in Docker container

• Docker build publish plugin

–Build & push docker images

Page 29: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Page 30: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Page 31: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Page 32: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Page 33: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Page 34: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

with.node(“linux”) {

steps.git ...

sh “mvn install”

parallel(

{ sh “./regression-test.sh” },

{ sh “./performance-test.sh” }

)

}

Page 35: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Build #153Build #153

Page 36: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Page 37: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Page 38: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Page 39: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Page 40: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Page 41: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Page 42: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Page 43: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Workflow

• github.com/jenkinsci/workflow-plugin

• Bunch of new extension points

• Expect some value-adds in Jenkins Enterprise by CloudBees

• More about this later today

Page 44: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Page 45: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Train Model for LTS Releases

Main release line

LTS release line

Pick good main release

1.554.21.554.1 1.554.3

New LTS release every four weeks

Page 46: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

LTS Release Model Changes

• You can plan ahead for upgrades

• Participate in LTS qualification process

Page 47: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Acceptance Test & Harness

• Old project, renewed

• Lots of activity

–50+ commits/week by 22 people

–365 tests

–50-ish plugins

Page 48: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Acceptance Test & Harness

Page 49: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Class of problems we hope to catch

• Browser-specific issues

• Container-specific issues

• OS-specific issues

• Packaging-specific issues

• Problematic plugin combinations

• Plugin with newer versions of cores

Page 50: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

This is for you, too

I want to make sure a new version of Jenkins works in

my environment with my set of plugins.

Page 51: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Reusability

• Make your tests our tests

• Amass a bigger pool of tests

Page 52: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Separately reusable pieces

Jenkins Controller

Page ObjectsJENKINS

HOME seeder

Machine Provisioner

Dockerfixtures

Cucumber integration

Geb + Spock VagrantJUT Process

pool

Page 53: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

More work ahead

• More regular executions

–More target platforms

–More browsers

• Test stability improvements

• Expand to non-functional tests

Page 54: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

X1K

Page 55: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

More efficient master/slave communication

• NIO in remoting

–use a few threads to manage 100s of slaves

• JNLP today, coming to CLI soon

• Maven2 jobs are a lot faster

Page 56: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Speaking of JNLP slaves…

Page 57: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Speaking of JNLP slaves…

Page 58: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

http://www.flickr.com/photos/acmace/4410650624/

Page 59: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

“Eat our own dogfood” approach

• Continuous delivery pipeline for our own infra

• Shout out to Puppet Labs for their help

Page 60: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Infra Summary

• It’s all out there

– github.com/jenkins-infra/jenkins-infra

• Come join the infra team

–Work on cutting-edge stuff & brag about it elsewhere

Page 61: Jenkins State of Union20Boston%20Ke… · Geb + Spock Vagrant JUT Process pool. Jenkins User Conference Boston #jenkinsconf More work ahead •More regular executions –More target

Jenkins User Conference Boston #jenkinsconf

Conclusion

• So many exciting things going on in the project

• Let’s build this together