Embed Size (px)
DESCRIPTION
Strongest and Weakest Interpolants If I and I′ are both interpolants for (F,G), then so are I ∧ I′ and I ∨ I′ Let F ∧ G be unsatisfiable. The strongest interpolant for (F, G), denoted SI (F, G), is the unique interpolant for (F, G) that implies any other interpolant. The weakest interpolant for (F,G), denoted WI(F,G), is the unique interpolant that is implied by any other interpolant SI (F, G) implies WI (F, G) December 3,
Citation preview
mm
Lazy Annotation for Program Testing and Verification
(Supplementary Materials)
Speaker: Chen-Hsuan Adonis LinAdvisor: Jie-Hong Roland Jiang
December 3, 2010
1
mm
OutlineHow to compute Interpolants of
program sequenceConcolic Approach (without learning)
Dart: Directed Automated Random Testing
December 3, 2010
2
mm
Strongest and Weakest InterpolantsIf I and I′ are both interpolants for (F,G), then
so are I∧I′ and I∨I′Let F ∧ G be unsatisfiable. The strongest
interpolant for (F, G), denoted SI (F, G), is the unique interpolant for (F, G) that implies any other interpolant. The weakest interpolant for (F,G), denoted WI(F,G), is the unique interpolant that is implied by any other interpolant
SI (F, G) implies WI (F, G)
December 3, 2010
3
mm
Interpolants of SequencesWe want to handle program paths, therefore a
generalization of interpolant is needed.Given a sequence of formulas Γ = A1,A2,…,An, we
say that Ā 0, Ā 1,…, Ā n is in an interpolant for Γ when:Ā 0 = TRUE and Ā n = FALSE,For all 1≤i≤n, Ā i-1∧Ai implies Ā i, andFor all 1≤i≤n, Ā i is in L(A1,…,Ai)∩L(Ai+1,…,An)
If Γ is quantifier-free we can derive a quantifier-free interpolant for Γ (from the refutation of Γ )
December 3, 2010
4
mm
Interpolants for Sequences (con’t)
An intuition:
So this is a structured refutation of A1, …, Ak
(Ā i ∧ Ai+1) implies Ā i+1
December 3, 2010
5
A1 A2 A3 Ak...
Ā1 Ā2 Ā3 Āk-1...True False
mm
Iterative Computation of Interpolants
Given a formula F = F1 ∧ . . . ∧ Fn, determine whether F is unsatisfiable, and if so, find interpolants for the pairs (F ..i, F i+1..), i∈{1,...,n}, where F..i := F1∧...∧Fi and Fi+1.. :=Fi+1∧...∧Fn
Each formula Fi models a program instruction
A formula F = F1 ∧ . . . ∧ Fn models a trace through a program
In order to check if the trace is feasible or spurious, one can check if F is satisfiable or unsatisfiable
December 3, 2010
6
mm
Iterative Computation of Interpolants (con’t)
Definition (Tracking Property) Let F1 ∧ . . . ∧ Fn be unsatisfiable, and let Ki be interpolants
for (F ..i, F i+1..). We say that the family {Ki} satisfies the tracking property if ( Ki ∧ Fi+1 |= Ki+1)
Proposition: Let F1 ∧ F2 ∧ . . . ∧ Fn be unsatisfiable. Let {Ii} and {Ji} be families of predicates defined according to the following procedures: I0 := true, Ii+1 := any interpolant for (Ii ∧Fi+1, Fi+2..),
where i=0, ..., n−1 Jn := false, Ji−1 := any interpolant for (F ..i−1, ¬(Fi → Ji)),
where i=n, ..., 1{Ii} and {Ji} are interpolants for (F ..i,F i+1..) and satisfy
the tracking property
December 3, 2010
7
mm
Iterative Computation of Interpolants (con’t)
Ｉ nterpolants satisfying the tracking property “explain” the infeasibility of a trace by providing Hoare annotations
Evaluate strongest interpolants (Ii) {true} X := true {X} Y := X {X ∧ Y } assume(¬Y ∧ Z)
{false}
Evaluate weakest interpolants (Ji) {true} X := true {X ∨ ¬Z} Y := X {Y ∨ ¬Z} assume(¬Y ∧ Z)
{false}
By definition, Ii |= Ji; Ex: (X∧Y) |= (Y∨¬Z)
December 3, 2010
8
mm
Iterative Computation of Interpolants (con’t)
Evaluate strongest interpolants (Ii) {true} X := true {X} Y := X {X ∧ Y } assume(¬Y ∧ Z) {false
Evaluate weakest interpolants (Ji) {true} X := true {X ∨ ¬Z} Y := X {Y ∨ ¬Z} assume(¬Y ∧ Z) {false}
Intuitively, the strongest interpolants at node n records all facts that are established by the path leading up to n Ex: the strongest interpolant at node 2 is {X ∧ Y}
Intuitively, the weakest interpolant at n represents the disjunction of all conditions that make the trace infeasible if they hold at n Ex: the weakest interpolant at node 2 is {Y ∨¬Z}
December 3, 2010
9
mm
OutlineHow to compute Interpolants of
program sequenceConcolic Approach (without learning)
Dart: Directed Automated Random Testing
December 3, 2010
10
mm
Motivation of software testingToday, QA is mostly testing
“50% of my company employees are testers, and the rest spends 50% of their time testing!”
-- Bill Gates 1995
December 3, 2010
11
mm
Concolic ApproachCombine concrete and symbolic execution for
unit testing (Concrete + Symbolic = Concolic)DART: Directed Automated Random Testing
Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation
Authors Patrice Godefroid (Bell Labs)Nils Klarlund (Bell Labs)Koushik Sen (CS, UIUC)
December 3, 2010
12
mm
Example (C code)int double(int x) {
return 2 * x;
}
void test_me(int x, int y) {
int z = double(x);
if (z==y) {
if (y == x+10)
abort(); /* error */
}
}
(1) Interface extraction:• parameters of toplevel function• external variables• return values of external functions
main(){
int tmp1 = randomInt();
int tmp2 = randomInt();
test_me(tmp1,tmp2);
}
(2) Generation of test driver for random testing:
Problem: probability of reaching abort() is extremely low!December 3, 2010
13
mm
DART: Directed Searchmain(){
int t1 = randomInt();
int t2 = randomInt();
test_me(t1,t2);
}
int double(int x) {return 2 * x; }
void test_me(int x, int y) {
int z = double(x);
if (z==y) {
if (y == x+10)
abort(); /* error */
}
}
Concrete Execution
Symbolic Execution
Path Constraint
x = 36, y = 99x = 36, y = 99create create symbolicsymbolicvariables x, y variables x, y
December 3, 2010
14
mm
DART: Directed Searchmain(){
int t1 = randomInt();
int t2 = randomInt();
test_me(t1,t2);
}
int double(int x) {return 2 * x; }
void test_me(int x, int y) {
int z = double(x);
if (z==y) {
if (y == x+10)
abort(); /* error */
}
}
Concrete Execution
Symbolic Execution
Path Constraint
create symboliccreate symbolicvariables x, y variables x, y
x = 36, y = 99,x = 36, y = 99,z = 72z = 72
z = 2 * xz = 2 * x
December 3, 2010
15
mm
DART: Directed Searchmain(){
int t1 = randomInt();
int t2 = randomInt();
test_me(t1,t2);
}
int double(int x) {return 2 * x; }
void test_me(int x, int y) {
int z = double(x);
if (z==y) {
if (y == x+10)
abort(); /* error */
}
}
Concrete Execution
Symbolic Execution
Path Constraint
create symboliccreate symbolicvariables x, y variables x, y
x = 36, y = 99,x = 36, y = 99,z = 72z = 72
z = 2 * xz = 2 * x
2 * x != y2 * x != y
Solve: 2 * x == ySolve: 2 * x == y
Solution: x = 1, y = 2Solution: x = 1, y = 2
December 3, 2010
16
mm
DART: Directed Searchmain(){
int t1 = randomInt();
int t2 = randomInt();
test_me(t1,t2);
}
int double(int x) {return 2 * x; }
void test_me(int x, int y) {
int z = double(x);
if (z==y) {
if (y == x+10)
abort(); /* error */
}
}
Concrete Execution
Symbolic Execution
Path Constraint
x = 1, y = 2x = 1, y = 2create symboliccreate symbolicvariables x, y variables x, y
December 3, 2010
17
mm
DART: Directed Searchmain(){
int t1 = randomInt();
int t2 = randomInt();
test_me(t1,t2);
}
int double(int x) {return 2 * x; }
void test_me(int x, int y) {
int z = double(x);
if (z==y) {
if (y == x+10)
abort(); /* error */
}
}
Concrete Execution
Symbolic Execution
Path Constraint
create symboliccreate symbolicvariables x, y variables x, y
x = 1, y = 2, z = 2x = 1, y = 2, z = 2 z = 2 * xz = 2 * x
December 3, 2010
18
mm
DART: Directed Searchmain(){
int t1 = randomInt();
int t2 = randomInt();
test_me(t1,t2);
}
int double(int x) {return 2 * x; }
void test_me(int x, int y) {
int z = double(x);
if (z==y) {
if (y == x+10)
abort(); /* error */
}
}
Concrete Execution
Symbolic Execution
Path Constraint
create symboliccreate symbolicvariables x, y variables x, y
x = 1, y = 2, z = 2x = 1, y = 2, z = 2 z = 2 * xz = 2 * x 2 * x == y2 * x == y
December 3, 2010
19
mm
DART: Directed Searchmain(){
int t1 = randomInt();
int t2 = randomInt();
test_me(t1,t2);
}
int double(int x) {return 2 * x; }
void test_me(int x, int y) {
int z = double(x);
if (z==y) {
if (y == x+10)
abort(); /* error */
}
}
Concrete Execution
Symbolic Execution
Path Constraint
create symboliccreate symbolicvariables x, y variables x, y
2 * x == y2 * x == y
x = 1, y = 2, z = 2x = 1, y = 2, z = 2 z = 2 * xz = 2 * x
y != x + 10y != x + 10
Solve: (2 * x == y) Solve: (2 * x == y) Æ Æ (y == x +10)(y == x +10)
Solution: x = 10, y = 20Solution: x = 10, y = 20
December 3, 2010
20
mm
DART: Directed Searchmain(){
int t1 = randomInt();
int t2 = randomInt();
test_me(t1,t2);
}
int double(int x) {return 2 * x; }
void test_me(int x, int y) {
int z = double(x);
if (z==y) {
if (y == x+10)
abort(); /* error */
}
}
Concrete Execution
Symbolic Execution
Path Constraint
x = 10, y = 20x = 10, y = 20create symboliccreate symbolicvariables x, y variables x, y
December 3, 2010
21
mm
DART: Directed Searchmain(){
int t1 = randomInt();
int t2 = randomInt();
test_me(t1,t2);
}
int double(int x) {return 2 * x; }
void test_me(int x, int y) {
int z = double(x);
if (z==y) {
if (y == x+10)
abort(); /* error */
}
}
Concrete Execution
Symbolic Execution
Path Constraint
create symboliccreate symbolicvariables x, y variables x, y
x = 10, y = 20, z = 20x = 10, y = 20, z = 20 z = 2 * xz = 2 * x
December 3, 2010
22
mm
DART: Directed Searchmain(){
int t1 = randomInt();
int t2 = randomInt();
test_me(t1,t2);
}
int double(int x) {return 2 * x; }
void test_me(int x, int y) {
int z = double(x);
if (z==y) {
if (y == x+10)
abort(); /* error */
}
}
Concrete Execution
Symbolic Execution
Path Constraint
create symboliccreate symbolicvariables x, y variables x, y
x = 10, y = 20, z = 20x = 10, y = 20, z = 20 z = 2 * xz = 2 * x 2 * x == y2 * x == y
December 3, 2010
23
mm
DART: Directed Searchmain(){
int t1 = randomInt();
int t2 = randomInt();
test_me(t1,t2);
}
int double(int x) {return 2 * x; }
void test_me(int x, int y) {
int z = double(x);
if (z==y) {
if (y == x+10)
abort(); /* error */
}
}
Concrete Execution
Symbolic Execution
Path Constraint
create symboliccreate symbolicvariables x, y variables x, y
2 * x == y2 * x == y
y == x +10y == x +10z = 2 * xz = 2 * xx = 10, y = 20, z = 20x = 10, y = 20, z = 20
Program Error
December 3, 2010
24
mm
Concolic Testing: A Middle Approach
+ Complex programs
+ Efficient
- Less coverage
+ No false positive
- Simple programs
- Not efficient
+ High coverage
- False positive
Random Testing
Symbolic Testing
Concolic Testing
+ Complex programs+/- Somewhat efficient+ High coverage+ No false positive
December 3, 2010
25
mm
Limitations: A Comparative View
Concolic: Broad, shallow
Random: Narrow, deepDecember 3, 2010
26
mm
Hybrid Concolic Testing Interleave Random Testing and Concolic Testing to increase
coverage
Deep, broad, hybrid Search
December 3, 2010
27
mm
Thanks for your attention
December 3, 2010
28
