April 9, 2023

Evolutionary database design

DevClub.eu,30.06.2009

Andrei Solntsev

Agenda

begin» background» my experience» tools

end;

begin

процесс разработки программного обеспечения

Плановый («водопад»)На раннем этапе • выявить требования, • согласовать, • спроектировать, • согласоватьи затем приступить к кодированию

минимизация изменений за счет обширной предварительной работы.

Эволюционный• Допускать измененияна всех стадиях• Но управлять изменениями

принципиальная позиция – максимально упростить возможность изменений

@author

1999: Refactoring: Improving

the Design of Existing Codehttp://www.amazon.com/exec/obidos/ASIN/0201485672

2000: Continuous Integrationhttp://www.martinfowler.com/articles/continuousIntegration.html

2003: Evolutionary Database Designhttp://www.martinfowler.com/articles/evodb.html

Martin Fowler

@compare

Рефакторинг базы данных сложнее, чем рефакторинг кода

• Сохранение информационной семантики• Миграция данных

• большое количество связей• период поддержки устаревшего кода

• В отличие от обычного ПО, невозможно просто удалить старый код и заменить новым.

<img src=“book.pdf”/>

@experience

My experience PRIA HireRight HireRight 2.0 ?

PRIA

ProductionProduction

Users

Developers

Testers

SalespeopleProduct Manager

Single DB environment

Deployment is not a problem

Multi-DB environment

Production

ProductionUsers

Salespeople

Developer #1

Developer #2

Tester #1

Tester #2

Demo

Deployment becomes a problem

HireRight: deployment

ALTER TABLE man ADD COLUMN length

Create index man_idx

CREATE INDEX women_id_idx

DROP TABLE raha

CREATE TABLE money ( eek NUMBER);

ALTER TABLE money ADD CONS

CVS DB

Installationscripts

Problems:• scripts reinstallation• DB recreation• DB synchronization

Database Change Management

DB Deploy - http://dbdeploy.com/

LiquiBase - http://www.liquibase.org/

My findings:

MySQL Workbench – Diff & Synchronizationhttp://dev.mysql.com/tech-resources/articles/workbench_database_management.html

Oracle Enterprise Manager (change management pack)

An automated processAn automated process is needed to make the process of is needed to make the process of upgrading out-of-date databases simpleupgrading out-of-date databases simple

We need a Database Change ManagementDatabase Change Management tool.

RedGate SQL compare - http://www.red-gate.com/products/SQL_Compare/index.htm

Ruby migrations - http://www.oracle.com/technology/pub/articles/kern-rails-migrations.html

Apache DDL Utils

<database name="testdb"> <table name="author"> <column name="author_id" type="INTEGER"

primaryKey="true" required="true"/> <column name="name" type="VARCHAR"

size="50" required="true"/> <column name="organisation" type="VARCHAR"

size="50" required="false"/> </table>

<index name="book_isbn"> <index-column name="isbn"/> </index></database>

Based on XML, automatically generates SQL scripts to update DB.

http://db.apache.org/ddlutils/

Incremental Schema Changes

All database schemas can be thought of as a compilation of incremental changes made over time to accommodate new functionality

As updates are applied to a database, the changes will be recorded in a table similar to the following:

Change Date

1_Create_Customer_Table.sql 4-15-07

2_Add_e-mail_address_column.sql 4-17-07

3_Add_fax_number_column.sql 4-18-07

4_Add_transaction_table.sql 4-21-07

5_Add_transaction_status_column.sql 4-24-07

6_Add_customer-transaction_view.sql 4-27-07

DBDeploy

1. Go to directory with SQL files:1. “1 create_customer_table.sql”2. “2 add_customer_id_constraint.sql”3. …

2. Run “ant”

Output:

[dbdeploy] dbdeploy 3.0-SNAPSHOT[dbdeploy] Reading change scripts from directory

/tmp/dbdeploy/dbdeploy-3.0-SNAPSHOT/example...[dbdeploy] Changes currently applied to database:[dbdeploy]   (none)[dbdeploy] Scripts available:[dbdeploy]   1, 2[dbdeploy] To be applied:[dbdeploy]   1, 2[dbdeploy] Generating undo scripts...     [sql] Executing resource:

/tmp/dbdeploy/dbdeploy-3.0-SNAPSHOT/example/output.sql     [sql] 7 of 7 SQL statements executed successfully

ThoughWorks.com

DBDeploy options

http://dbdeploy.com/documentation/getting-started/rules-for-using-dbdeploy/

• Naming convention for delta scripts:NUMBER COMMENT.SQL

• Re-execution – NOT POSSIBLE

• Undo section – marked by comments:

CREATE TABLE FOO ( FOO_ID INTEGER NOT NULL, FOO_VALUE VARCHAR(30) );

ALTER TABLE FOO ADD CONSTRAINT PK_FOO PRIMARY KEY (FOO_ID);--//@UNDO DROP TABLE FOO;

LiquiBase

The main XML describing all delta scripts:

<databaseChangeLog> <changeSet id="1" author="bob">

<createTable tableName="department"> <column name="id" type="int">

<constraints primaryKey="true" nullable="false"/> </column> <column name="name" type="varchar(50)">

<constraints nullable="false"/> </column> <column name="active" type="boolean" defaultValue="1"/></createTable>

</changeSet> </databaseChangeLog>

LiquiBase.org

LiquiBase

Run:

liquibase --driver=com.mysql.jdbc.Driver \ --classpath=/path/to/classes \ --changeLogFile=com/example/db.changelog.xml \ --url="jdbc:mysql://localhost/example" \ --username=user \ --password=asdf \ migrate

Also possible to run:• Ant script• Maven plugin• Built-in java code• Spring• Grails• Servlet Listener

LiquiBase options

• Custom order of scripts (user-defined)

• Re-execution – POSSIBLE (with attributes)

LiquiBase options

And many other options:• DBMS• SQL checksum• context• long transaction• pre-conditions

Outcome

DB Deploy

LiquiBase

My findings:

MySQL Workbench – Diff & Synchronization

Oracle Enterprise Manager

RedGate SQL compare

Ruby migrations

1. commercial2. RDBMS-specific3. but probably good

- Seems to be cool, need to try- Written on Ruby

- Written by authoritative people- Simple- Convection over configuration

- More configuration than DBDeploy- More options than DBDeploy- Works with any RDBMS- Eclipse plugin

Written on

Java

end;

Links:http://www.refactoring.com/ - Code refactoring (1999)http://databaserefactoring.com/ - DB refactoring (2003)

Books:Рефакторинг баз данных: эволюционное проектирование. http://www.williamspublishing.com/Books/978-5-8459-1157-5.html