Searching for Liveness Property Violations in Concurrent Systems with ACO

1 / 34GECCO 2008, Atlanta, USA, July 12-16, 2008

Searching for Liveness

Property Violations in Concurrent Systems with ACO

Francisco Chicano

and Enrique Alba

Introduction Background Algorithmic Proposal Experiments

Conclusions & Future Work


• Concurrent software is difficult to test ...

• ... and it is in the heart of a lot of

critical systems

• Techniques for proving the correctness

of concurrent software are required•Model checking

→ fully automatic

• In the past the work using metaheuristics

focused on safety properties• In this work we focus on liveness

properties

MotivationMotivation




Intersection Büchi

automaton

• Objective: Prove that model

M satisfies the property :

• HSF-SPIN: the property f is an LTL formula

s0

s4s7

s6

s2

s1

s8s9

s5

s3

s0

s1s2

s3

s4

s5

Model M

s1

s0 s2

qp∧!q

!p∨q

LTL formula ¬

f(never claim)

∩ =

Explicit State Model CheckingExplicit State MC State Explosion Heuristic MC

Safety & Liveness

SCCs




Intersection Büchi

automaton




s0

s4s7

s6

s2

s1

s8s9

s5

s3

s0

s1s2

s3

s4

s5

Model M

s1

s0 s2

qp∧!q

!p∨q

LTL formula ¬

f(never claim)

∩ =


Safety & Liveness

SCCs




Intersection Büchi

automaton




s0

s4s7

s6

s2

s1

s8s9

s5

s3

s0

s1s2

s3

s4

s5

Model M

s1

s0 s2

qp∧!q

!p∨q

LTL formula ¬

f(never claim)

∩

Using Nested-DFS

=


Safety & Liveness

SCCs




s0

s4s7

s6

s2

s1

s8s9

s5

s3

• Number of states very large

even for small models

• Example: Dining philosophers with n

philosophers

→ 3n

states20 philosophers → 1039 GB

for storing the states

•

Solutions: collapse compression, minimized automaton representation, bitstate hashing, partial order reduction, symmetry reduction

• Large models cannot be verified but errors can be found

Memory

State Explosion Problem



Explicit State MC

State Explosion

Heuristic MC

Safety & Liveness

SCCs


• The search for errors can be directed by using heuristic information

• Different kinds of heuristic functions have been proposed in the past:

• Formula-based

heuristics

• Structural

heuristics

s0

s4

s7

s6

s2

s1

s8s9

s5

s32

0

3

5

1

2

40

76

Heuristic value



Heuristic Model Checking

• Deadlock-detection

heuristics

• State-dependent

heuristics

Explicit State MC

State Explosion Heuristic MC

Safety & Liveness

SCCs


Safety property

• Counterexample ≡

path to accepting state

• Graph exploration algorithms can be used: DFS

and BFS

Liveness

property

• Counterexample

≡

path to accepting cycle

• It is not possible to apply DFS or BFS

s0

s4s7

s6

s2

s1

s8s9

s5

s3

Safety and Liveness

Properties



s0

s4s7

s6

s2

s1

s8s9

s5

s3

Explicit State MC


Safety & Liveness

SCCs


Strongly Connected Components



a b

ce

df g

Explicit State MC


Safety & Liveness

SCCs





Explicit State MC


Safety & Liveness

SCCs





Explicit State MC


Safety & Liveness

SCCs





Explicit State MC


Safety & Liveness

SCCs

a b

ce

df gF-SCC

N-SCC

P-SCC


OPTIMIZATION/SEARCH TECHNIQUESOPTIMIZATION/SEARCH TECHNIQUES

EXACTEXACT APPROXIMATEDAPPROXIMATED

Ad Hoc

HeuristicsAd Hoc

Heuristics METAHEURISTICSMETAHEURISTICS

Optimization/Search

Techniques

• Newton• Gradient

Based on CalculusBased on Calculus

• Depth

First

Search• Branch

and

Bound

EnumerativesEnumeratives

• SA• VNS• TS

Trayectory-basedTrayectory-based

• EA• ACO• PSO

Population-basedPopulation-based



Metaheuristics

ACO

ACOhg

ACOhg-live


•

Ant Colony Optimization (ACO) metaheuristic

is inspired by the foraging behaviour of real ants

• ACO Pseudo-code

ACO: Introduction



Metaheuristics

ACO ACOhg

ACOhg-live


• The ant selects its next node stochastically

• The probability of selecting one node depends on the pheromone trail and theheuristic value (optional) of the edge/node

• The ant stops when a complete solution is built

i

j

l

m

k

Ni

τij

ηij

kTrail

Heuristic

ACO: Construction Phase



Metaheuristics

ACO ACOhg

ACOhg-live


• Pheromone update

During the construction phase

After the construction phase

• Trail limits (particular of MMAS)

Pheromones are kept in the interval [τmin, τmax]

ACO: Pheromone Update



with

Metaheuristics

ACO ACOhg

ACOhg-live

with


The length of the ant paths is limited by λant λant Objective node

What if…?

Starting nodes for path construction change

After σs

steps

Second stage Third stage

Initial node

ACOhg: Huge Graphs Exploration



Metaheuristics

ACO

ACOhg

ACOhg-live


• The search is an alternation of two phases

First phase: search for accepting states

Second phase: search for cycles from the accepting states

ACOhg-live



Metaheuristics

ACO

ACOhg

ACOhg-live

ACOhg-live

Pseudocode

First

phase





ACOhg-live



Metaheuristics

ACO

ACOhg

ACOhg-live

ACOhg-live

Pseudocode

Second

phase





ACOhg-live



Metaheuristics

ACO

ACOhg

ACOhg-live

ACOhg-live

Pseudocode

Tabu

list





ACOhg-live



Metaheuristics

ACO

ACOhg

ACOhg-live

ACOhg-live

Pseudocode • Improvement using SCCs

First phase: accepting states in N-SCC are ignored

Both phases: cycle detected in F-SCC is a violation

Tabu

list


•We used 11 Promela

models

for the experiments

• Parameters for ACOhg-live

• Formula-based

and finite state machine

heuristics• ACOhg-live implemented in HSF-SPIN• 100

independent executions and statistical validation

Model LoC Scalable Processes LTL formula (liveness)alter 64 no 2 □(p → ◊q) ^ □(r → ◊s) giopj 740 yes j+6 □(p → ◊q)phij 57 yes j+1 □(p → ◊q)elevj 191 yes j+3 □(p → ◊q)sgc 1001 no 20 ◊p

Promela

Models



Models & parameters Results Discussion

Parameter msteps colsize λant σs ξ a ρ α β

1st phase100

10 204

0.75 0.2 1.0 2.0

2nd phase 20 4 0.5


•We used 11 Promela

models

for the experiments

• Parameters for ACOhg-live

• Formula-based

and finite state machine

heuristics• ACOhg-live implemented in HSF-SPIN• 100

independent executions and statistical validation

Model LoC Scalable Processes LTL formula (liveness)alter 64 no 2 □(p → ◊q) ^ □(r → ◊s) giopj 740 yes j+6 □(p → ◊q)phij 57 yes j+1 □(p → ◊q)elevj 191 yes j+3 □(p → ◊q)sgc 1001 no 20 ◊p

Promela

Models



Models & parameters Results Discussion

j=10,15,20

j=20,30,40

Parameter msteps colsize λant σs ξ a ρ α β

1st phase100

10 204

0.75 0.2 1.0 2.0

2nd phase 20 4 0.5

j=10,15,20


Results I: Influence of the SCC Improvement



Models & parameters

Results

Discussion

SDSD





Models & parameters

Results

Discussion

SDSD

SDSD

SD SD SDSD





Models & parameters

Results

Discussion

SDSD

SDSD

SD SD SDSD

SDSD

SD





Models & parameters

Results

Discussion

SDSD

SDSD

SD SD SDSD

SDSD

SD

SD

SD

575192

SD

SD SD

SD

SD

SD


Results II: ACOhg-live+

vs. NDFS and INDFS



Models & parameters

Results

Discussion

Models ACOhg-live+ NDFS INDFSaltergiop10giop15giop20phi20phi30phi40elev10elev15elev20sgc



vs. NDFS and INDFS



Models & parameters

Results

Discussion


10001

10001

1069

1059

9999

SD



vs. NDFS and INDFS



Models & parameters

Results

Discussion


10001

10001

1069

1059

9999

SD

392192 388096 15360

SD



vs. NDFS and INDFS



Models & parameters

Results

Discussion


10001

10001

1069

1059

9999

SD

392192 388096 15360

SD

1567015320

SD


How to use ACOhg-live



Models & parameters

Results Discussion

Model

ACOhg-live

(I)NDFS

Large

model

ora short couterexample

is

needed

Small

model

andany

counterexampleis

needed

fast

•

ACOhg-live should be used in the first/middle stages

of the software development, when software errors are expected

•

ACOhg-live can also be used in other phases of the software development for testing concurrent software


•

ACOhg-live is the first algorithm

based on metaheuristics

(to the best of our knowledge) applied to the search for liveness

errors in concurrent models

•

The improvement based on the SCCs

of the never claim outperforms

the efficacy of ACOhg-live

•

ACOhg-live is able to outperform (Improved) Nested-DFS

in efficacy and efficiency in the search for liveness

errors

Conclusions

Future Work• Analysis of parameterization for reducing the parameters

• Include ACOhg-live into JavaPathFinder

for finding liveness

errors in Java programs

•

Combine ACOhg-live with techniques for reducing the memory

required for the search such as partial order reduction

(work in progress)






Thanks for your attention !!!

Searching for Liveness

Property Violations in Concurrent Systems with ACO

Technology

Searching for Liveness Property Violations in Concurrent Systems with ACO