Continuous Database BuildAnd: Practical Approaches For DB Based Functional Testing
Stefan Rufer – Netcetera AG
Michael Wernli – Telekurs Card Solutions AG
3020
2
Speakers
> Stefan Rufer
– Studied business IT at the University of Applied Sciences in
Bern
– Senior Software Engineer and Architect at Netcetera
– Main interest: Server side applicat ion development using JEE
> Michael Wernli
– Cert if ied engineer in IT
– Senior Software Engineer at Telekurs Card Solut ions
– As Applicat ion DBA in charge of large- scale cashless payment
t ransact ion system.
3
Why are we here?
> You will see how a cont inuous database build improved our
overall build stability and helped us to focus on business
problems during integrat ion tests.
4
Motivation
> “Who the #@!* dropped the salary column in the build database?!”
– Dude, you didn‘t run the script to add it in the f irst place…
> Did the build just turn red…
– because someone forgot to clean up the build database –
AGAIN?
> svn revert drop-that-table.sql
– Hmm, the table is st ill missing in the DB…?
> ……
5
Motivation
6
AGENDA
> Continuous Build For The Database
> Re- Runnable Functional JUnit Tests
> Eclipse/ Maven Setup For Functional JUnit Tests
> Outlook / References
7
> Continuous Build For The Database
> Re- Runnable Functional JUnit Tests
> Eclipse/ Maven Setup For Functional JUnit Tests
> Outlook / References
8
The Problem
> 100..200 SQL scripts (DDL+ DML) per major release
> Scripts have to be runnable f irst - to- last at all t imes
> 20 developer/ build/ integrat ion databases to keep in sync
Poor Database Administrator!
9
The Idea
> Install database scripts as soon as they are commited to the
subversion repository.
> Run SQL script against reference database. If installat ion
terminates without error, populate to all other databases.
> Store in the database what script was installed in what subversion
(SVN) revision.
10
Basic Setup of a Continuous Database Build
11
Demo Part I
12
Different behaviour of database sources and Java sources during build
13
Demo Part II
14
We need re- runnable (idempotent) database source scripts
15
Delta and Source Scripts
> Most dif f icult is data definit ion
or data modif icat ion scripts.
– delta-scripts: „run once“
– Add idempotency manually
> Packages, t riggers, views can
be reinstalled easily.
– source-scripts: „run
however“
– Idempotent by design
16
Branching for database source scripts is a bit different
17
Demo Part III
18
Quality Checks
> Automated quality checks for
– Primary key constraint names
– Unique key constraint names
– Foreign key constraint names
– Check constraint names
– Index names for all of the above
– Sequence names
> Hooks provided to start quality
checks
19
> Continuous Build For The Database
> Re- Runnable Functional JUnit Tests
> Eclipse/ Maven Setup For Functional JUnit Tests
> Outlook / References
20
JUnit Tests Modifying Database
21
How This All Relates
> JUnit tests that do insert / update/ delete must not leave their
t races in the build database. Auto rollback JUnit class
> Good experience by providing personal database to each
developer.
– But: How to keep it up to date? Cont inuous Database Build.
> Developers want to switch between a personal and a build
database. Eclipse/ Maven Setup
22
The Idea
> AutoRollbackTestCase (extends JUnits class TestCase)
> Datasource and JDBC connect ion managed internally
> Rollback called during tearDown method of test no t races!
23
Undo, Undo!
class ExampleDBTestCase extends AutoRollbackTestCase {
public void testDbInsert() throws SQLException { Statement stmt = getDataSource().getConnection().createStatement();
stmt.executeUpdate("INSERT INTO testtable values ('X')");
ResultSet rs = stmt.executeQuery("SELECT * FROM testtable"); assertTrue("expect one row", rs.next()); assertEquals("X", rs.getString("testrow")); assertFalse("expect only one row", rs.next()); }
}
24
Little Burden – Big Deal
> JDBC insert / update/ delete statements are a crucial part of
enterprise applicat ions. They need to be tested!
> Avoid the burden of cleaning up the database (someone will
forget anyway…)
> Unit tests that roll back at the end offer a simple way to run SQL:
– Verify correct SQL syntax
– Verify database state after update
> Spring offers the possibilit ies of AutoRollbackTestCase (and much
more), check out spring-test.jar /
org.springframework.test
25
> Continuous Build For The Database
> Re- Runnable Functional JUnit Tests
> Eclipse/ Maven Setup For Functional JUnit Tests
> Outlook / References
26
Properties Overloading
> ConfiguredTestCase reads junit .propert ies
> If junit_fritz.propert ies is available for user „fritz“, it is read as
well
Overwrites propert ies in junit .propert ies
> How does this look like?
user=buildpassword=buildurl=jdbc:derby:builddb
example=buildsrv:80
user=integrationpassword=integration
example=localhost:8080
reuse URL
junit.properties junit_fritz.properties
27
KISS – But How?
> Scope: Mult imodule build setup using Maven 2 and Eclipse
> Eclipse and Maven classpath handling dif ferent
> Eclipse: Mult iple projects referencing each other results in a
„global“ classpath.
Where to place database connect ion informat ion for unit tests?
> Maven: Each module needs junit .propert ies in
src/test/resources if we want to avoid test resources in the
main source t ree.
Where to place database connect ion informat ion for unit tests?
28
Eclipse Setup For Functional JUnit Tests
> User adds his one and only junit_USERNAME.propert ies f ile to
„toplevel“ project
> Eclipse uses this for all JUnit runs as it is always on the classpath
User specif ic database connect ion set t ings in Eclipse.
29
Maven Setup For Functional JUnit Tests
> junit .propert ies f ile in each submodule
> junit .propert ies contains variables:
> Values def ined in Maven POM…
…and injected during Maven lifecycle process-test-resources
db.user=${db.user}db.password=${db.password}db.schema=${db.schema}db.url=${db.url}
<db.user>build</db.user> <db.password>build</db.password><db.schema>build</db.schema><db.url>dpas10</db.url>
junit.properties
pom.xml
30
Test- properties: Define once, use everywhere
31
> Continuous Build For The Database
> Re- Runnable Functional JUnit Tests
> Eclipse/ Maven Setup For Functional JUnit Tests
> Outlook / References
32
Outlook
> Continuous Database Script Suite current ly tailored to Oracle
systems
– Consider generic implementat ion or at least ident ify and
isolate specialized points
> Test data management is the next big topic on our radar. As we
have to select sensible stuff f rom 1TB of data we need some
clever ideas…
> Integrate Maven/ Eclipse setup with a tool like MavenIDE.
33
Links / References
> Full source for continuous database build system
http:/ / www.stefanrufer.ch/ dbbuild
> Spring Test inghttp:/ / stat ic.springframework.org/ spring/ docs/ 2.5.x / reference/ test ing.html
34
The End
35
Two things to remember
Store in the database how the database was
built.
Write JUnit tests that roll back.
Stefan Rufer [email protected]
Netcetera AG www.netcetera.ch
Michael Wernli [email protected]
Telekurs Card Solutions www.telekurs.com
Recommended
