Bdsys icsm v3.5

An Empirical Study of Build System

Migrations in PracticeCase Studies on KDE and the Linux Kernel

1

R. Suvorov, B. Adams, M. Nagappan, A. E. Hassan, Y. Zou

Analogy: Car Assembly

2

+4x +assembly

+

Build System Transforms Source Code & Resources into Executables

3

+bdSys

+

bdSys is at the Heart of the Development Process!

4

bdSys+

Developers

QA Personnel Testers

bdSys Technology is “different”

5

bdSys Technology is “different”

5

Our record so far is a project we inherited with an Ant script weighing in at 10,000 lines of XML. Needless to say, this project required an entire team devoted to keeping the build working - a complete waste of resources.[Jez Humble & David Farley]

KDE 4 is leaving the aging "autotool" build chain behind. Some developers,

not only in KDE, like to nickname

the autotools as "auto-hell"

because of its difficult to comprehend architecture.[Alexander Neundorf]

I've been examining the existing kernel configuration system, and concluded that the best favor we could do everybody involved with it is to take it out behind the barn and shoot it through the head. [Eric Raymond]

... yet a bdSys Must Keep on Evolving

6

MLO

C

..., in fact a bdSys Changes Relatively More than Source Code

71

(c) McIntosh et al.: “An Empirical Study of

Build Maintenance Effort”

..., in fact a bdSys Changes Relatively More than Source Code

71

ArgoUML Hibernate Eclipse GCC Git Linux Mozilla PLPlot PostgreSQL0

0.51

1.52

2.53

3.54

4.55

# changes / source file# changes / build file

(c) McIntosh et al.: “An Empirical Study of

Build Maintenance Effort”

..., in fact a bdSys Changes Relatively More than Source Code

71

ArgoUML Hibernate Eclipse GCC Git Linux Mozilla PLPlot PostgreSQL0

0.51

1.52

2.53

3.54

4.55

# changes / source file# changes / build file

(c) McIntosh et al.: “An Empirical Study of

Build Maintenance Effort”

..., in fact a bdSys Changes Relatively More than Source Code

71

ArgoUML Hibernate Eclipse GCC Git Linux Mozilla PLPlot PostgreSQL0

0.51

1.52

2.53

3.54

4.55

# changes / source file# changes / build file

(c) McIntosh et al.: “An Empirical Study of

Build Maintenance Effort”

In Other Words ...

8

In Other Words ...

8

Software Projects need HELP Maintaining their Build Systems

The “easy” way out: Build System Migrations

9

The “easy” way out: Build System Migrations

9

10

11

12

12

Build system migrations are not that easy! But why?

Major Contributions

13

Common Migration

Methodology

Major Migration Challenges

Major Challenge: Requirements

Gathering

Study Setup

• Linux & KDE:

• Two established projects: 21 & 16 years old

• Large: 25 & 4.3 MLOC

• 1 failed & 1 successful migration each

• Data sources:

• git repos: 300 K & 100 K commits

• 4 mailing lists, over 1.5 million messages

14

Quantitative Study of Developer Commits

• git repositories

• Native git tools + bash scripts

• Statistics on source & bdSys code:

• Evolution

• Churn

• Distinguish between “normal” periods & migrations

15

Qualitative Analysis of Developer Communication

• Personal emails to migration participants

• Mailing lists

• Mass-downloaded and analyzed

• Statistics on number of messages:

• By groups (roles)

• By period of time

• Distinguish between “normal” periods & migrations

16

Major Contributions

17

Common Migration

Methodology

Major Migration Challenges

Major Challenge: Requirements

Gathering

18

Common Migration

Methodology

The Spiral Model of bdSys Migration

19

1. Planning 2. Risk Analysis

3. Development4. Evaluation

Why? Migration is Incremental!

• Linux kernel

• 33K+ source code files

• 2,300+ subdirectories

• 5 major subsystems

• KDE

• 11K+ source code files

• 22 packages

20

21

Common Migration

Methodology

21

Common Migration

Methodology

spiral model!

21

Common Migration

Methodology

spiral model!

migration happens

incrementally

22

Common Migration

Methodology

Major Migration Challenges

spiral model!

migration happens

incrementally

Communication AmongKey Participants

23

Communication AmongKey Participants

23

bdSys Managermajor decisions

Communication AmongKey Participants

23

bdSys Managermajor decisions

bdSys Championleading bdSys dev’t

Communication AmongKey Participants

23

bdSys Managermajor decisions

bdSys Championleading bdSys dev’t

bdSys ExpertbdSys contributor

Communication AmongKey Participants

23

bdSys Managermajor decisions

bdSys Championleading bdSys dev’t

bdSys ExpertbdSys contributor

Core Developersource contributor

Major Challenges

• Communication issues

• Build experts form a sub-community often reluctant to communicate effectively

• Performance vs. complexity

• Improving performance often means introducing hacks

• Effective evaluation

• When is migration "successfully finished"? How to measure?

• Requirements gathering (see remainder)

24

25

Common Migration

Methodology

Major Migration Challenges

spiral model!

migration happens

incrementally

25

Common Migration

Methodology

Major Migration Challenges

requirementsspiral model!

migration happens

incrementally

25

Common Migration

Methodology

Major Migration Challenges

requirements

communicationspiral model!

migration happens

incrementally

25

Common Migration

Methodology

Major Migration Challenges

requirements

communication

performance

spiral model!

migration happens

incrementally

25

Common Migration

Methodology

Major Migration Challenges

requirements

communication

performance

evaluation

spiral model!

migration happens

incrementally

26

Common Migration

Methodology

Major Migration Challenges

Major Challenge: Requirements

Gathering

requirements

communication

performance

evaluation

spiral model!

migration happens

incrementally

Example bdSys Requirements

• Platform independence

• Two new major platforms: Mac OS X & Windows

• High-level configuration

• Single-pass methodology

• Away with multiple-steps build!

• Simple and platform-independent syntax

27

Requirements Gathering Process1. Stakeholder identification - who?

• Who needs to be involved?

2. Elicitation

• What requirements are there?

3. Analysis

• Get all requirement data. Resolve conflicts, refine and organize.

4. Specification & Maintenance

• Document & order resulting requirements.

28

How SCons failed• Stakeholder identification

• Only 150 of 800 developers attended aKademy

• Elicitation

• Too little, too late: “trawled” for requirements

• Analysis

• Not enough attention paid to this phase!

• Specification & Maintenance

• Not done at all!

29

30

“the main SCons tree has intolerable problems, and there are so many layers [...] that it barely

resembles what a normal SCons build system looks like. Don't get me wrong; I use SCons at work and I

really like it, but if it isn't capable of handling KDE then why are we trying to make it do so? Is it really

worth keeping a KDE specific build system (which is what we are moving towards)?

[...] Perhaps it’s time to cut our losses and run.” [Jaison Lee]

As a Result:

Rise and Fall of SCons

31

% m

ax (

LOC

)

How succeeded

• Stakeholder identification

• Learned from SCons. Involved experts early.

• Elicitation

• Reuse some of SCons’ data.Active participation by experts!

• Analysis

• Identified conflicting requirements early.

• Specification & Maintenance

• An initiative by the build champion.32

33

Common Migration

Methodology

Major Migration Challenges

Major Challenge: Requirements

Gathering

spiral model!requirements

communication

performance

evaluation

migration happens

incrementally

33

Common Migration

Methodology

Major Migration Challenges

Major Challenge: Requirements

Gathering

spiral model!requirements

communication

performance

evaluation

migration happens

incrementally

involve all stakeholders

33

Common Migration

Methodology

Major Migration Challenges

Major Challenge: Requirements

Gathering

spiral model!requirements

communication

performance

evaluation

migration happens

incrementally

involve all stakeholders

identify &

resolve conflicts

33

Common Migration

Methodology

Major Migration Challenges

Major Challenge: Requirements

Gathering

spiral model!requirements

communication

performance

evaluation

migration happens

incrementally

involve all stakeholders

identify &

resolve conflicts

efficient elicitation

34

34

Common Migration

Methodology

spiral model!

migration happens

incrementally

34

Major Migration Challenges

requirements

communication

performance

evaluation

Common Migration

Methodology

spiral model!

migration happens

incrementally

34

Major Migration Challenges

requirements

communication

performance

evaluation

Common Migration

Methodology

spiral model!

migration happens

incrementally

Major Challenge: Requirements

Gathering

involve all stakeholders

identify &

resolve conflicts

efficient elicitation