A Logic Your Typechecker Can Count On:

Unordered Tree Types in Practice

Nate Foster (Penn)Benjamin C. Pierce (Penn)

Alan Schmitt (INRIA Rhône-Alpes)

PLAN-X ’07

µX . {}|(hd[T]+tl[X])

φ(x0, .., x4),24 hd[T ] , hd[¬T ] ,tl[X ] , tl[¬X ] ,{hd, tl}[True]

35

µX . {}|(hd[T]+tl[X])

φ(x0, .., x4),hd[T ] , hd[¬T ] ,tl[X ] , tl[¬X ] ,{hd, tl}[True]

Types in

Sync

A B

A’ B’

O

T

Harmony

A generic synchronization frameworkI Architecture takes two replicas + original ⇒ updated

replicas.

I Data model is “deterministic” trees: unordered,edge-labeled trees.

Types in

Sync

A B

A’ B’

O

T

Harmony: Typed Synchronization [DBPL ’05]

Behavior of synchronizer guided by type.I If inputs well-typed, so are outputs.

I Required operations: membership of trees in type [alsosets of names].

Types in

Sync

A B

A’ B’

O

T

Harmony: Lenses [POPL ’05]

Pre-/post-process replicas using bi-directional programs.I Facilitates heterogeneous synchronization.

I Types in conditionals, run-time asserts, static checkers.

I Required operations: membership, inclusion, equivalence,emptiness, [projection, injection, etc.].

Deterministic Tree Types

Syntax

T ::= {} | n[T] |T+T |T|T | ~T |X| !\{n1, .., nk}[T] | *\{n1, .., nk}[T]

Deterministic Tree Types

Syntax

T ::= {} | n[T] |T+T |T|T | ~T |X| !\{n1, .., nk}[T] | *\{n1, .., nk}[T]

Semantics

Singleton denoting the unique tree with no children:

∈ {}

Deterministic Tree Types

Syntax

T ::= {} | n[T] |T+T |T|T | ~T |X| !\{n1, .., nk}[T] | *\{n1, .., nk}[T]

Semantics

Atoms: trees with single child n and subtree in T :

If

t

∈ T , then n

t

∈ n[T]

Deterministic Tree Types

Syntax

T ::= {} | n[T] |T+T |T|T | ~T |X| !\{n1, .., nk}[T] | *\{n1, .., nk}[T]

Semantics

Commutative concatenation operator:

If

t

∈ T and

t’

∈ T ′, thent t’

∈ T+T ′

Deterministic Tree Types

Syntax

T ::= {} | n[T] |T+T |T|T | ~T |X| !\{n1, .., nk}[T] | *\{n1, .., nk}[T]

Semantics

Boolean operations and recursion:

X1 = T1...

Xn = Tn

Deterministic Tree Types

Syntax

T ::= {} | n[T] |T+T |T|T | ~T |X| !\{n1, .., nk}[T] | *\{n1, .., nk}[T]

Semantics

If m 6∈ {n1, .., nk} and

t

∈ T , then m

t

∈!\{n1, .., nk}[T]

Deterministic Tree Types

Syntax

T ::= {} | n[T] |T+T |T|T | ~T |X| !\{n1, .., nk}[T] | *\{n1, .., nk}[T]

Semantics

If m1, ..,mk 6∈ {n1, .., nk} and

t1

..

tk

∈ T , thenm1 mk

..

..t1 tk

∈ *\{n1, .., nk}[T]

Deterministic Tree Types

Syntax

T ::= {} | n[T] |T+T |T|T | ~T |X| !\{n1, .., nk}[T] | *\{n1, .., nk}[T]

Example: hd[True]+tl[True]

hd tl

Deterministic Tree Types

Syntax

T ::= {} | n[T] |T+T |T|T | ~T |X| !\{n1, .., nk}[T] | *\{n1, .., nk}[T]

Example: {}|(hd[True]+tl[True])

or

hd tl

Deterministic Tree Types

Syntax

T ::= {} | n[T] |T+T |T|T | ~T |X| !\{n1, .., nk}[T] | *\{n1, .., nk}[T]

Example: X = {}|(hd[True]+tl[X])

or

hdtl

hdtl

hdtl

Deterministic Tree Types

Syntax

T ::= {} | n[T] |T+T |T|T | ~T |X| !\{n1, .., nk}[T] | *\{n1, .., nk}[T]

Example: ![True]+![True]

Deterministic Tree Types

Syntax

T ::= {} | n[T] |T+T |T|T | ~T |X| !\{n1, .., nk}[T] | *\{n1, .., nk}[T]

Example: ~(![True]+![True])

or or or ...

Can eliminate negations, and use direct algorithms, buttypes get large...

Sheaves Formulas

Formulas

S =φ(x0, .., xk),[r0[S0] , .., rk [Sk ]]

where φ is a Presburger formulaand ri a set of names.

[Dal Zilio, Lugiez, Meyssonnier, POPL ’04]

Sheaves Formulas

Formulas

S =φ(x0, .., xk),[r0[S0] , .., rk [Sk ]]

where φ is a Presburger formulaand ri a set of names.

φ(x0, x1),[b[True], {a, c}[True]] 0 0

ab

c

Sheaves Formulas

Formulas

S =φ(x0, .., xk),[r0[S0] , .., rk [Sk ]]

where φ is a Presburger formulaand ri a set of names.

φ(x0, x1),[b[True], {a, c}[True]] 0 1

ab

c

Sheaves Formulas

Formulas

S =φ(x0, .., xk),[r0[S0] , .., rk [Sk ]]

where φ is a Presburger formulaand ri a set of names.

φ(x0, x1),[b[True], {a, c}[True]] 1 1

ab

c

Sheaves Formulas

Formulas

S =φ(x0, .., xk),[r0[S0] , .., rk [Sk ]]

where φ is a Presburger formulaand ri a set of names.

φ(x0, x1),[b[True], {a, c}[True]] 1 2

ab

c

Sheaves Formulas

Formulas

S =φ(x0, .., xk),[r0[S0] , .., rk [Sk ]]

where φ is a Presburger formulaand ri a set of names.

φ(x0, x1),[b[True], {a, c}[True]] 1 2

?|= φ(1, 2)

Sheaves Formulas

Formulas

S =φ(x0, .., xk),[r0[S0] , .., rk [Sk ]]

where φ is a Presburger formulaand ri a set of names.

φ(x0, x1, x2),[b[True], {a, c}[True], {a, b, c}[True]

]

For coherence: ri [Si ] must partition set of atoms.Note: does not ensure determinism.

Examples as Sheaves Formulas

X = ({}|hd[True]+tl[X])

X =

(x0 =x1 =x2 =x3 =0) ∨(x0 =x1 =1 ∧ x2 =x3 =0),[hd[True] , tl[X ] , tl[¬X ] , {hd, tl}[True]

]

~(![True]+![True])

x0 6= 2,[{}[True]

]

Examples as Sheaves Formulas

X = ({}|hd[True]+tl[X])

X =

(x0 =x1 =x2 =x3 =0) ∨(x0 =x1 =1 ∧ x2 =x3 =0),[hd[True] , tl[X ] , tl[¬X ] , {hd, tl}[True]

]

~(![True]+![True])

x0 6= 2,[{}[True]

]

Challenges and Strategies

Blowup in naive compilation from types to formulas.I Syntactic optimizations avoid blowup in common cases.

Backtracking in top-down, non-deterministic traversal.I Incremental algorithm avoids useless paths.

Presburger arithmetic requires double-exponential time.I Compile Presburger formulas to MONA representation.I Hash-consing allocation + aggressive memoization.

Contributions

I Strategies and algorithms;

I Implementation in Harmony;

I Experimental results.

Challenges and Strategies

Blowup in naive compilation from types to formulas.I Syntactic optimizations avoid blowup in common cases.

Backtracking in top-down, non-deterministic traversal.I Incremental algorithm avoids useless paths.

Presburger arithmetic requires double-exponential time.I Compile Presburger formulas to MONA representation.I Hash-consing allocation + aggressive memoization.

Contributions

I Strategies and algorithms;

I Implementation in Harmony;

I Experimental results.

Incremental Algorithm

φ(x0, .., xk),[r0[S0] , ..rk [Sk ]]

0 0 .. 0

n1 n2 nk−1 nk..

..

Incremental Algorithm

φ(x0, .., xk),[r0[S0] , ..rk [Sk ]]

(φ)

n1 n2 nk−1 nk..

..

Incremental Algorithm

φ(x0, .., xk),[r0[S0] , ..rk [Sk ]]

(φ ∧ ψdom)

n1 n2 nk−1 nk..

..

Incremental Algorithm

φ(x0, .., xk),[r0[S0] , ..rk [Sk ]]

(φ ∧ ψdom ∧ ψ1)

n1 n2 nk−1 nk..

..

Incremental Algorithm

φ(x0, .., xk),[r0[S0] , ..rk [Sk ]]

(φ ∧ ψdom ∧ ψ1 ∧ ψ2)

n1 n2 nk−1 nk..

..

Incremental Algorithm

φ(x0, .., xk),[r0[S0] , ..rk [Sk ]]

(φ ∧ ψdom ∧ ψ1 ∧ .. ∧ ψk−1)

n1 n2 nk−1 nk..

..

Incremental Algorithm

φ(x0, .., xk),[r0[S0] , ..rk [Sk ]]

(φ ∧ ψdom ∧ ψ1 ∧ .. ∧ ψk)

n1 n2 nk−1 nk..

..

Hash-Consing and Memoization

Thousands of formulas and trees, but many repeats.

Suggests hash-consed allocation:

I Sheaves formulas;

I Presburger formulas;

I Trees.

Memoization of intermediate results:

I MONA representations of Presburger formulas;

I Satisfiability of Presburger formulas;

I Membership results;

I Partially-evaluated member functions.

Experiments

Programs:

I Structured text parser;

I Address book validator;

I iCalendar lens.

Experimental setup: structures populated with snippets ofJoyce’s Ulysses; 1.4GHz Intel Pentium III, 2GB RAM, SuSELinux OS kernel 2.6.16; execution times collected from POSIXfunctions.

Experiments: Address Book Validator

0

Tim

e(se

conds)

50

100

150

Input Size (# lines)

1000 2000 3000 4000 5000 6000 7000 8000 9000 10000

basebase-memo

States Formulas Sat Trees312 107711 99.8% 25744 99.9% 107711 42.1%

Experiments: Address Book Validator

0

Tim

e(se

conds)

50

100

150

Input Size (# lines)

1000 2000 3000 4000 5000 6000 7000 8000 9000 10000

basebase-memoincr-all-off

incr

States Formulas Sat Trees312 107711 99.8% 25744 99.9% 107711 42.1%

Experiments: Address Book Validator

0

Tim

e(se

conds)

50

100

150

Input Size (# lines)

1000 2000 3000 4000 5000 6000 7000 8000 9000 10000

basebase-memoincr-all-offincr-phi-off

incr-member-offincr

States Formulas Sat Trees312 107711 99.8% 25744 99.9% 107711 42.1%

Experiments: Structured Text Parser

0

Tim

e(se

conds)

50

100

150

200

Input Size (# lines)

1000 2000 3000 4000 5000 6000 7000 8000 9000 10000

basebase-memoincr-all-offincr-phi-off

incr-member-offincr

States Formulas Sat Trees105 12580 99.1% 222 92.8% 3507706 81.4%

Experiments: iCalendar Lens

0

Tim

e(se

conds)

50

100

150

200

250

Input Size (# lines)

100 200 300 400 500 600 700 800 900 1000

basebase-memoincr-all-offincr-phi-off

incr-member-offincr

States Formulas Sat Trees361 116939 97.4% 17600 87.8% 407652 76.5%

Related Work

Types and Automata:

I TQL [Cardelli and Ghelli, ESOP ’01]

I “A Logic You Can Count On”[Dal Zilio, Lugiez, Meyssonnier, POPL ’04]

I “Counting In Trees For Free”[Seidl, Schwentick, Muscholl, Habermehl, ICALP ’04]

I Survey and Foundations:[Boneva and Talbot, RTA ’05, LICS ’05]

Implementations:

I “Static Checkers for Tree Structrures and Heaps”[Hague ’04]

I “Boolean Operations and Inclusion Test for AttributeElement Constraints” [Hosoya and Murata, ICALP ’03]

Conclusions and Future Work

Summary

I Strategies and algorithms;

I Implemented in Harmony;

I Reasonable performance.

Tune algorithm, hash-consing, memoization parameters.

Determinize sheaves formulas.

Implement Presburger arithmetic directly, optimized for addingconstraints incrementally; also restricted fragments.

Extend to new structures and types: multitrees, ordered trees,also horizontal recursion, adjoint operators, etc.

Acknowledgements

Haruo Hosoya, Christian Kirkegaard, Stéphane Lescuyer,Thang Nguyen, Val Tannen, Penn PLClub and DB Group.

http://www.seas.upenn.edu/∼harmony/

http://www.seas.upenn.edu/~harmony/