HIS 2015: Neil White - Advances in Practical Techniques for Critical Development Software

1

ALTRAN, GLOBAL LEADER IN INNOVATION

Advances in Practical Techniques for Critical Software Development

5 t h N o v e m b e r 2 0 1 5

Agenda

• Who am I? Who are Altran?

• Our approach to building software

• Methods and tool support for building software

• Strengthening the weakest link – advances in test approaches

• Conclusion

Agenda





• Conclusion

Who am I?

4

Neil White

Head of Engineering Altran UK

5

Bath, UK Paris, France Annecy, France

Munich, Germany

Hamburg, Germany

Shanghai, China

Bangalore, India

Sophia-Antipolis, France

Toulouse, France

Milan, Italy

Turin, Italy

Atlanta, USA

Barcelona, Spain

Madrid, Spain

Lille, France

Pisa, Italy

Bologna, Italy

Rome, Italy

Naples, Italy

Who are Altran?

CMMI L3, ISO 9001, ISO 27001, EN 9100, ISO 13485

SYSTEMS ENGINEERING

SOFTWARE ENGINEERING

ELECTRONICS

SAFETY

SECURITY

CONNECTIVITY

UK FOOTPRINT

6

Bristol

Bristol London

Warwick Cambridge*

Reading

* Cambridge Consultants, a member of the Altran Group

Bath

€112.3M Revenues (2014)

Offices in 12 locations

850+ FTEs

Derby

Slough

Penrith

Manchester

Glasgow

Agenda





• Conclusion

The cost of errors Correctness by Construction

Source: CMM Data from Jones, Caspers: Software Assessments, Benchmarks and Best Practices. Reading, MA: Addison-Wesley, 2002 Source: C By C data from Correctness by Construction: A manifesto for High-Integrity software, Croxford and Chapman 2005

Source:Leffingwell http://www.rational.com/media/whitepapers/roi1.pdf

Source: IEEE Software. Correctness by Construction: Developing a Commercial Secure System, Hall and Chapman, Jan 2002

Principles

Avoid introducing defects

Introducing defects is easy – removing them is hard, and expensive

Generate evidence as you go

Evidence needed for certification is produced naturally as a by-product of the process

Remove defects early

Defects removed early when changes are cheap

Correctness by Construction

Testing is a demonstration of correctness

Not the point where we start debugging. Prediction over observation.

Better can be cheaper

Safety is given. How you get there determines the cost.

Zero tolerance of defects

We cannot claim zero defects but we can have a zero tolerance attitude to them.

Strategy

• Use precise or formal notations for each step

• Design the software to simplify verification and validation

• Small steps verified at every stage

• Use strong, tool-supported methods to verify each step

• Say things only once

• Do the hard / risky things first

Correctness by Construction

Agenda





• Conclusion

Bu i ld ing B locks

13

Requirements Specification Design Implement Analyse & Prove Test

Bu i ld ing B locks

14


REVEAL

Bu i ld ing B locks

15


REVEAL

Z, CSP, UML, SCADE, Matlab/Simulink

Bu i ld ing B locks

16


REVEAL


INFORMED

Bu i ld ing B locks

17


REVEAL


INFORMED

SPARK, MISRA C, SCADE, QGen

Bu i ld ing B locks

18


REVEAL


INFORMED


SPARK, CodePeer

Bu i ld ing B locks

19


REVEAL


INFORMED


SPARK, CodePeer

??

Bu i ld ing B locks

20


REVEAL


INFORMED


SPARK, CodePeer

ConTestor

Agenda





• Conclusion

Traditional Dynamic Test Approach

Requirements

Verification Conditions

System Under Test

Comparator

Test Scripts

Inputs

Expected outputs

Actual outputs

Code Coverage

VC Coverage

Results 22

Dynamic Test Approach with an Oracle

Requirements


System Under Test

Comparator

Test Scripts

Inputs

Expected outputs

Actual outputs

Code Coverage

Test Oracle

Inputs

Expected outputs

VC Coverage

Results 23

Test Oracle Test Oracle

The ConTestor Approach

24

Requirements


System Under Test

Comparator

Test Descriptions

Inputs

Actual outputs

Code Coverage

Inputs

Expected outputs

VC Coverage

VC Checker

Results

VC Coverage

Test Scripts

Agenda





• Conclusion

Conc lus ion

› Automating the running of Test Scripts has been standard practice for years.

› Automating the production of Test Scripts for Safety Critical software is now possible

› Reduces time

› Reduces cost

› Reduces the opportunity for human error

› Improves depth of testing with brute force

› Reduces maintenance costs

› Why ever write Test Scripts again?

26

27

Business

HIS 2015: Neil White - Advances in Practical Techniques for Critical Development Software