Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
Workshop TDD4ES
16/11/2010
EP 1
iwt-090191 TDD4ES
Workshop
Building blocks for
Test-Driven Development of Embedded Software
16/11/2010
dr. ing. Jeroen Boydens
ing. Piet Cordemans
ing. Sille Van Landschoot
ing. Wim Catteeuw
@ep_research http://ep.khbo.be
16/11/2010 TDD4ES
EP research group
2
http://ep.khbo.be
Founded: 2006 – 2007
dr. ing. Jeroen Boydens + founder + professor + project leader
ing. Sille Van Landschoot + TDD4ES + lab assistant
ing. Piet Cordemans + TDD4ES + PhD student + lab assistant
ing. Wim Catteeuw + APS4ES
ing. Olivier Rosseel + academic staff + ICTO coach
Workshop TDD4ES
16/11/2010
EP 2
16/11/2010 TDD4ES 3
EP research group
Threefold mission:
Education Research Social services
• Software courses Prof. Bachelor EO/ICT Ac. Bachelor EO/ICT Master ICT
• With K.U.Leuven Concurrent software Software testing Transactional behavior
• Tetra/PWO/theses Embedded development
methodologies Gaming frameworks
• Seminars Software testing State-of-the-art
technologies
• LED eg. Medical
Reminder System
16/11/2010 TDD4ES 4
IWT - Tetra 090191: TDD4ES
Test on target Test on host Remote testing
Case studies
3 strategies
Literature study
Manual
• State of the art • Best practices
CMS
http://ep.khbo.be/TDD4ES
Workshop TDD4ES
16/11/2010
EP 3
16/11/2010 TDD4ES
User committee TDD4ES IWT/090191
5
www.edna.eu www.tesco.be
www.qstar.be
www.marelec.com
www.unitron.be
www.summa.be
www.sirris.be
www.televic.com
www.newtec.eu
www.vandewiele.com
www.dspvalley.com www.kdg.be
www.kuleuven.be www.fmtc.be
www.iwt.be
16/11/2010 TDD4ES
Content
Session 1 (Tuesday 16/11/2010)
1. Introduction
2. TDD in practice
3. TDD for embedded software
Session 2 (Thursday 18/11/2010)
1. Test on target
2. Test on host
3. Remote testing 6
Workshop TDD4ES
16/11/2010
EP 4
16/11/2010 TDD4ES
Content: session 1
1. Introduction
1. Testing embedded software
2. Test-Driven Development
3. Unit test framework
4. TDD strategy
5. TDD pitfalls
2. TDD in practice
1. Do it yourself
2. Temperature library
3. UnitTest++
4. Exercise
7
16/11/2010 TDD4ES
Content: session 1 cont’d
3. TDD for embedded software
1. Embedded constraints
2. TDD migration strategies
3. Test on target in practice
4. Do it yourself
5. EmbUnit
6. Exercise
7. Testable design
8
Workshop TDD4ES
16/11/2010
EP 5
16/11/2010 TDD4ES
1 Introduction
9
1. Testing embedded software
2. Test-Driven Development
3. Unit test framework
4. TDD strategy
5. TDD pitfalls
16/11/2010 TDD4ES
1.1 Testing embedded software
10
Quality assurance in embedded software development is mostly limited to debugging and final testing, only
focusing on the current issue…
Embedded software testing gap
Importance versus state of testing
Lifecycle
Co
st
of
Re
pa
ir
Specification Programming ProductionDesign
1000
0
100
10
© B.W. Boehm, 1981
Software Engineering Economics
Workshop TDD4ES
16/11/2010
EP 6
16/11/2010 TDD4ES
Lifecycle
Co
st
of
Re
pa
ir
Specification Programming ProductionDesign
1000
0
100
10
© B.W. Boehm, 1981
Software Engineering Economics
1. Find bugs early
2. Measurable quality
3. Increasing confidence
4. Detecting regression
5. Encapsulating third party code
11
Timely detection of bugs by automated testing
Running test suite frequently during development Incrementally expanding the test suite Support from unit testing framework
1.1 Testing embedded software
16/11/2010 TDD4ES
1.2 Test-Driven Development
TDD cycle
1. Write failing test (red)
• New behavior
• Minimal skeleton to get through compilation
2. Write code to pass test (green)
• Minimal implementation
3. Refactor
• No new behavior
• Clean code
• Keep tests passing
12
Workshop TDD4ES
16/11/2010
EP 7
16/11/2010 TDD4ES 13
1.2 Test-Driven Development
Advantages
1. Code is tested while it’s written
2. Fast feedback cycle
3. Extensive & safe refactoring
4. Focus on current functionality
5. Tests become living documentation
Disadvantages
1. 2x code
increase development time
16/11/2010 TDD4ES
1.3 Unit test framework
Why?
• Organization
• Execution
• Reuse
• Reporting
Many exist for C/C++ (>40) minUnit, CppUnitLite, UnitTest++, GoogleTest, CxxTest,
EmbUnit, …
Or: write your own (TDD-style)
14
Workshop TDD4ES
16/11/2010
EP 8
16/11/2010 TDD4ES
1.3 Unit test framework
xUnit model
15
16/11/2010 TDD4ES
1.4 TDD strategy
Black box tests vs. White box tests
a.k.a. glass box tests
16
• Internal perspective • Implementation • Focus on developer • 20% coverage • Test Afterwards
• External perspective • Specification • Focus on end-user • 80% coverage • Test First
Workshop TDD4ES
16/11/2010
EP 9
16/11/2010 TDD4ES
Write test first
• Inspiration
• Specifications
• Use cases
• Notes
• …
• Procedure
1. Define interface
2. Select values for test
« By writing a test before implementing the item under test, attention is focussed on the item’s interface and observable behavior » Beck (2003)
17
1.4 TDD strategy
16/11/2010 TDD4ES
1.5 TDD pitfalls
18
• Broken window syndrome
– One broken test, which remains unresolved, leads quickly to many broken tests.
• Test cancer [Fowler]
– Removing broken tests is a bad habit.
Maintaining tests is necessary
Workshop TDD4ES
16/11/2010
EP 10
16/11/2010 TDD4ES
2 TDD in practice
19
1. Do it yourself
2. Temperature library
3. UnitTest++
4. Exercise
16/11/2010 TDD4ES
2.1 Do it yourself
Software
• Virtual machine
Admin rights
Slow
• IDE: Eclipse CDT
Open source
Customizable
Popular
20
Workshop TDD4ES
16/11/2010
EP 11
16/11/2010 TDD4ES
2.1 Do it yourself
Procedure
21
1. Explanation
2. Demonstration
3. Exercise similar to demo
4. Solution
16/11/2010 TDD4ES
2.2 Temperature library
01_TDDonHost_TempLibrary
Requirements • Stores internally a temperature value in Kelvin
• Offers an interface to get and set values in Kelvin & Celsius
Goals • TDD without hardware involvement
• Getting acquainted with TDD
22
Workshop TDD4ES
16/11/2010
EP 12
16/11/2010 TDD4ES
2.3 UnitTest++
23
Hello unit test framework: UnitTest++
RunTests.cpp
#include "UnitTestppHeaders\UnitTest++.h"
TEST(alwaysFail)
{
CHECK(false);
}
int main()
{
return UnitTest::RunAllTests();
}
16/11/2010 TDD4ES
CHECK(true); //succeeds
CHECK_EQUAL(1, 2); //fails
CHECK_EQUAL("EP", "ep"); //fails
CHECK_CLOSE(3.14, 3.1415, 0.01); //succeeds
24
• Grouping
• Setup & Teardown
• Assertions
2.3 UnitTest++
SUITE(ThermometerTests)
{ TEST(TestName){}
TEST(AnotherTestName){}
}
struct TestFixture
{ TestFixture(){/*setup*/}
~TestFixture(){/*teardown*/}
};
TEST_FIXTURE(TestFixture, TestName) {}
Workshop TDD4ES
16/11/2010
EP 13
16/11/2010 TDD4ES
2.4 Exercise
25
Some inspiration…
Kelvin = Celsius + 273,15
KelvinTest
CelsiusTest
Celsius2KelvinTest
Kelvin2CelsiusTest
Celsius2Kelvin2CelsiusTest
…
16/11/2010 TDD4ES
3 TDD for embedded software
26
1. Embedded constraints
2. TDD migration strategies
3. Test on target in practice
4. Do it yourself
5. EmbUnit
6. Exercise
7. Testable design
Workshop TDD4ES
16/11/2010
EP 14
16/11/2010 TDD4ES
3.1 Embedded constraints
27
TDD 1. Tests + Program +
Unit test framework
Embedded 1. Limited memory
footprint
2. Automated testing
3. Fast programming cycle
2. Hardware dependencies
3. Slow programming cycle
16/11/2010 TDD4ES
3.1 Embedded constraints
28
Embedded TDD challenges
1. Limit the memory footprint needed for TDD on
target
2. Tests for
1. Hardware independent code 2. Hardware aware code 3. Hardware specific code
3. Maintain a fast programming cycle
Workshop TDD4ES
16/11/2010
EP 15
16/11/2010 TDD4ES
3.2 Migration strategies
29
Test on target
Test on host
Remote testing
1. Limited number of tests
2. Minimalistic framework
3. Program code
1. Unlimited number of tests
2. Full option framework
3. Program code with hardware virtualization
1. Unlimited number of tests
2. Full option framework with remote testing functionality
3. Program code with remote testing functionality
16/11/2010 TDD4ES
3.3 Test on target in practice
30
Tests
Workshop TDD4ES
16/11/2010
EP 16
16/11/2010 TDD4ES
3.4 Do it yourself
• EasyARM development board
– Mikroelektronika
– Temperature sensor (DS1820)
– 2x16 LCD
– RS232
– Drivers library (Keil compiler)
• Two versions (1.01 & 1.51)
– Hardware changes (switch on/off, …)
– Two different programming tools
31
Hardware
16/11/2010 TDD4ES
3.4 Do it yourself
• 2 versions
– 2.01
– 2.51
• Load hex file
• Write to flash of target
32
Workshop TDD4ES
16/11/2010
EP 17
16/11/2010 TDD4ES
3.5 EmbUnit
33
• Written in C
• No library includes needed
– But flexible inclusion of stdio possible
• Small memory footprint
– 8KB on it’s own
– 18KB with stdio library
– 34KB with UART driver on LPC2148
• Adaptable & extensible
• Verbose
• Not scalable
– (+/- 2 KB per test)
• Limited functionality
16/11/2010 TDD4ES
3.5 EmbUnit
34
Hello unit test framework: EmbUnit
main.cpp
#include “EmbUnit_lib/embUnit.h”
#include “EmbUnitTests.h”
int main()
{ extern TestRef FirstTest(void);
TestRunner_start();
TestRunner_runTest(FirstTest());
TestRunner_end();
return 0;
}
Workshop TDD4ES
16/11/2010
EP 18
16/11/2010 TDD4ES
3.5 EmbUnit
35
Hello unit test framework: EmbUnit EmbUnitTests.cpp
#include “EmbUnit_lib/embUnit.h”
static void setUp(void){}
static void tearDown(void){}
static void HelloTest(void){TEST_ASSERT(true)}
TestRef FirstTest(void)
{ EMB_UNIT_TESTFIXTURES(fixtures)
{
new_TestFixture(“HelloTest”, HelloTest)
};
EMB_UNIT_TESTCALLER(FirstTestset,
“FirstTestset”,setUp,tearDown,fixtures);
return (TestRef) &FirstTestset; }
16/11/2010 TDD4ES 36
• Assertions
3.5 EmbUnit
• No floating point support
• Advanced output possible – XML outputter library
• Scripts available to deal with verbosity
TEST_ASSERT(true); //succeeds
TEST_ASSERT_MESSAGE(false, “message”) //fails
TEST_ASSERT_EQUAL_INT(1,2) //fails
TEST_ASSERT_EQUAL_STRING(“EP”, “ep”) //fails
TEST_ASSERT_NULL(pointer) //NULL = succeeds
Workshop TDD4ES
16/11/2010
EP 19
16/11/2010 TDD4ES
3.6 Exercise
37
02_TDDonTarget_TempLibrary Requirements
• Port and extend the temperature library to the EasyARM development board
Goals
• TDD without hardware involvement on target • Getting acquainted with TDD on target
16/11/2010 TDD4ES
3.6 Exercise
38
Some inspiration…
Kelvin = Celsius + 273,15
Celsius = (Fahrenheit - 32) * 5/9
KelvinTest
CelsiusTest
FahrenheitTest
Celsius2KelvinTest
Kelvin2CelsiusTest
Fahrenheit2KelvinTest
Kelvin2FahrenheitTest
Fahrenheit2CelsiusTest …
Workshop TDD4ES
16/11/2010
EP 20
16/11/2010 TDD4ES
3.7 Testable design
39
3-Tier design
Hardware aware
Hardware specific
Hardware independent
• Reuse
• Change implementation
• Data hiding
• Testable design
16/11/2010 TDD4ES
3.7 Testable design
40
Example problem:
Generic definition of pin & port number of a hardware component, without duplication all over the code.
Typical C solution:
/* hardware.h */
#define MEANINGFUL_NAME extern const int pinNumber;
/* hardware.c */
const int pinNumber = 1 << N;
/* main.c */
init_hardware(MEANINGFUL_NAME);
Workshop TDD4ES
16/11/2010
EP 21
16/11/2010 TDD4ES
3.7 Testable design
41
(simplified adapter pattern)
TempSensor - port - pin # getPin() # getPort()
DS1820
production
TempSensorAdapter + getPin() + getPort()
testing
unable to call from test
C++ solution
notation UML 2.0
16/11/2010 TDD4ES
Next session
Session 2 (Thursday 18/11/2010)
1. Test on target (continued)
2. Test on host
3. Remote testing
42
«Program testing can be a very effective way to show the presence of bugs, but is hopelessly inadequate for showing their absence. » Dijkstra (1972)
Workshop TDD4ES
16/11/2010
EP 22
16/11/2010 TDD4ES
Literature
• K. Beck. Test-Driven Development: By Example. Addison-Wesley, 2003.
• K. Beck and C. Andres. Extreme Programming Explained: Embrace Change (2nd ed.). Addison-Wesley, 2004.
• B.W. Boehm. Software Engineering Economics (Prentice-Hall Advances in Computing Science & Technology Series). Prentice Hall PTR, October 1981.
• J. Boydens, P. Cordemans, and E. Steegmans. Test-Driven Development of Embedded Software. In European Conference on the Use of Modern Information and Communication Technologies, 2010.
• E. Dijkstra. The humble programmer. Commun. ACM, 15(10):859866, 1972.
• M. Feathers. Working Effectively with Legacy Code. Prentice Hall PTR, 2005.
• M. Fowler. UML Distilled (3rd ed.). Addison-Wesley, 2004.
• M. Fowler. Refactoring: Improving the design of existing code. Addison-Wesley, 1999.
• E. Gamma, R. Helm, R. Johnson, and J. Vlissides. Design patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley, 1995.
• J. Grenning. Progress before hardware. Agile Alliance Newsletter, 4:74_79, 2004.
43
16/11/2010 TDD4ES
Literature continued
• M. Karlesky, W. Bereza, and C. Erickson. Effective test driven development for embedded software. In IEEE 2006 Electro/Information Technology Conference, 2006.
• R. Koss, and J. Langr. Test-Driven Development in C. In C/C++ Users Journal, 2002.
• M. Müller and F. Padberg. About the return on investment of test-driven development. In International Workshop on Economics-Driven Software Engineering Research EDSER-4, 2003.
• R. Osherove. The Art of Unit Testing. Manning, 2009.
44