Rapid Continuous Software Engineering - Meeting the challenges of modern software development

Rapid Con*nuous So.ware Engineering

-‐ Mee*ng the challenges of modern so.ware development

Ivica Crnkovic Mälardalen University à Chalmers & Gothenburg University, Sweden

[email protected] hHp://ivica-‐crnkovic.net

RCoSE, June 2, 2014, Hyderabad

Con*nuous SE – some history •  1990…2000 – get it big* •  2000…2005 – get it manageable •  2005 -‐ 2010 -‐ get it simple •  2010 – get it wide •  Now: get it sustainable (survivable)

* -‐ func*onality, business, in use

2014-‐06-‐02 2

1990…2000 – big shots – so.ware releases So.ware Major.Minor/Revision –  (SW 4.5/11)

User func*onality

*me

1.0 1.1

1.2

2.0

2014-‐06-‐02 3

Cash flow

User func*onality

*me

1.0

1.1

1.2

2.0

2014-‐06-‐02 4

Refinement

User func*onality

*me

1.0

1.1

1.2

2.0

2014-‐06-‐02 5

Ge_ng complex…. maintenance, variants…

User func*onality

*me

1.0

1.1

1.2

2.0

2014-‐06-‐02 6

1.0/1

1.2 1.2/2 1.2/1

1.1/1 1.1/2 1.1/3

2.0-‐variant A

Ge_ng complex…. New roles

User func*onality

*me

1.0

1.1

1.2

2.0

2014-‐06-‐02 7

1.0/1

1.2 1.2/1 1.2/2

1.1/1 1.1/2 1.1/3

Developers

maintainers

Get it manageable..

2014-‐06-‐02 8

!"#$%&'()*+(,-./0%

123%

121%

124%

423%

12351%

124%12454%12451%

12151% 12154% 12156%

42378,$.,(/%9%

Developers Maintainers Architects SCM Manager Quality team

CMM(I) Process Improvements Quality Assurance

So.ware Product Lines

Component-‐based SE Aspects SCM tools MDD

Get is simple!

2014-‐06-‐02 9

User func*onality

Focus on: Code Changes Short *me planning Customers Team-‐working Refactoring Pragma*c (ad hoc) reuse

Things are ge_ng wide

2014-‐06-‐02 10

But: Distributed development Outsourcing, offshoring crowdsourcing, Crowd-‐tes*ng, Eco-‐systems, Services & Cloud Compu*ng COMPETITION HARDER THAN EVER

Challenge

2014-‐06-‐02 11

How to enable rapid changes yet insure improvements on long and short terms? How to be compeGGve and sustainable? How to be rapid and conGnuous?

What do we want to achieve?

2014-‐06-‐02 12

How to achieve this?

2014-‐06-‐02

a)  Let the system improve/adjust itself b) Make it easy for the users to adapt the system c)  Let others do the improvements (for you) d) React rapidly on requirements & users’

behavior

e)  Ensure correctness and quality f)  Ensure innovaGon g)  Ensure (increased) outcome

13

SE challenges* •  Dealing with con.nuously and rapidly changing systems at run-‐.me: –  How to achieve that the system itself maintains the architectural integrity and ensures its quality?

•  Dealing with con.nuously and rapidly changing systems at development-‐.me: –  How to facilitate developers to effec*vely predict consequences of poten*ally applied changes?

•  Suppor.ng and synchronizing development-‐ and run-‐.me: –  How to enable maintaining alignment of all lifecycle ar*facts in a con*nuously evolving system?

2014-‐06-‐02 14 * Jan Bosch, Michel Chaudron, Ivica Crnkovic, Patrizio Pelliccione, MaHhias Tichy: Controlled Con*nuous So.ware Engineering, Research proposal SW Research Council

Research objec*ves 1.  So=ware architecture for con.nuous experimenta.on

–  experimental evalua*on of different SA solu*ons in parallel to the exis*ng “baselined” SA.

2.  Managing changes of non-‐func.onal proper.es (NFP) –  Rapid analysis of (possible) changes of NFP caused by changes in SA

3.  Methods for synchronizing architecture/designs and source code –  co-‐evolu*on of both source code and SA.

4.  Suppor.ng controlled and reliable adapta.ons –  support unplanned adapta*ons in a controlled and reliable way, i.e.,

by ensuring the preserva*on of system invariants.

2014-‐06-‐02 15

Example 1: dependable systems •  Dependability Ability of a

system to deliver service that can jus*fiably be trusted (Ability of a system to avoid failures that are more frequent or more severe than is acceptable to user(s)

•  Robustness is the persistence of a system’s characteris*c behavior under perturba*ons or condi*ons of uncertainty.

2014-‐06-‐03 DFG 2014 16

Dependability

Attributes

Threats

Means

Availability Reliability Safety Confidentiality Integrity Maintainability

Fault Prevention Fault Tolerance Fault Removal Fault Forecasting

Faults Errors Failures

Dependable systems

•  From Sta*c and Robust to Adaptable and resilient systems

2014-‐06-‐02 17

Resilience vs. dependability vs. robustness.

2014-‐06-‐02 DFG 2014 18

dependable (robust&resistent) systems

System states

•  Highly controlled •  Operates in a narrow band •  Predefined states (“modes”) •  Top-‐down design •  Challenge: predict all states

caused by the environment

•  A broad spectrum of possible equilibrium state •  Not necessary all states are predicted •  Adap*ve and evolving systems •  impact of the system on the environment •  Challenge:

•  Adapta*on •  Op*mal performance in different states

“Resilient systems” Environment

Extended development process

2014-‐06-‐02 DFG 2014 19

1JOSEPH FIKSEL Designing Resilient, Sustainable Systems , Environ. Sci. Technol. 2003, 37, 5330-‐5339

Example 2: Crowdsourcing

2014-‐06-‐02 20

Crowdsourcing the cosmos: Astronomers welcome all to idenGfy star clusters in Andromeda galaxy www.andromedaproject.org.

Crowdsourcing assump*on

2014-‐06-‐02 21

OEM Product company

In-house development

Off-shore development

centre COTS Out-

sourcing Crowd-

sourcing

Amount of accessible expertise

Crowdsourcing -‐ challenges •  Provide aHrac*ve calls that will bring the wanted results

•  Provide open architectures that enable –  a constant flow of innova*on –  efficient integra*on of components from both OEM and supplier perspec*ves.

•  Efficient organiza*ons and processes –  support balancing of openness and IPR management.

•  Efficient assessment of quality aHributes for components and systems.

2014-‐06-‐02 22

Integra*on Engine

Component-‐tool

System design tool

Synthesis tool

Refactoring tool

Repository

Integrator

Supplier

External Repository view

CALL

Component development

Conformance checking

Acceptance test

Component Submission

Conclusion: Rapid con*nuous SE

•  From Agile to Agile with systema*c SE •  Enabling openness and con*nuous change of so.ware (at design and run-‐*me)

•  Enabling adaptability, evolu*on •  Keep quality •  Increase innova*on capabili*es

2014-‐06-‐03 24

Papers @ ICSE 2014 •  So.ware Engineering at the Speed of Light: How Developers Stay

Current using TwiHer, Leif Singer, Fernando Figueira Filho, and Margaret-‐Anne Storey

•  Mining Behavior Models from User-‐Intensive Web Applica*ons, Carlo Ghezzi, Mauro Pezzè, Michele Sama, and Giordano Tamburrelli

•  Automated Design of Self-‐Adap*ve So.ware with Control-‐Theore*cal Formal Guarantees, Antonio Filieri, Henry Hoffmann, and Mar.na Maggio

•  Trading Robustness for Maintainability: An Empirical Study of

Evolving C# Programs, Nélio Cacho et al.

2014-‐06-‐02 25

FOSE, Session, Papers @ ICSE 2014 •  So.ware Evolu*on and Maintenance, Václav Rajlich

•  Session: Automated Bug Detec*on and Repair

•  Session: Processes and Agile Development –  Automated So.ware Integra*on Flows in Industry: A Mul*ple-‐Case Study, Daniel Ståhl and Jan Bosch

–  Evidence-‐Based Decision Making in Lean So.ware Project Management, Brian Fitzgerald, Mariusz Musiał, and Klaas-‐Jan Stol

2014-‐06-‐02 26

2014-‐06-‐03 27