Advances in Abstract Categorial Grammars

Language Theory and Linguistic Modeling

Lecture 1

This course

• Formal properties of second-order ACGs- Characterization of generative power- Datalog representation- Earley recognizer

• Linguistic modeling- ACG perspective on syntax-semantics

interface

The theory of general ACGs is very difficult, but the special case of second-order ACGs is almost completely understood.

Second-order ACGs

π1

π2

π3 π3

L (π2)

L (π3) L (π3)

L (π1)lexicon

L

β

treelinear λ-term

stringtree

logical formetc.

object language

abstract language

local set of derivation trees

∈ ∈

This is what a second-order ACG officially looks like.Second-order ACGs are “context-free” grammars on linear lambda-terms.Use notation that makes ACGs closer to other “context-free” formalisms.

S⎛⎝⎜

⎞⎠⎟

:− B(X ),B(Y ),S(Z ). X

Y Z

“Context-free” grammar formalisms

S

B B S

S → B

B S

top-down view bottom-up view

derivation tree “yield”

Context-free grammars

S → aBBS

S(aXYZ ) :− B(X ),B(Y ),S(Z ).

top-down view

bottom-up view

S

B B C

S

a B CB

Multiple context-free grammars

am bn cm dn

yield = tuple of strings

derivation tree

S

P Q

S(x1y

1x

2y

2) :− P(x

1, x

2),Q(y

1, y

2).

P(ε ,ε ).P(ax

1,cx

2) :− P(x

1, x

2).

Q(ε ,ε ).Q(by

1,dy

2) :−Q(y

1, y

2).

dnbncmam

bottom-up

This is an example of a 2-MCFG.An m-MCFG allows nonterminals to take up to m arguments.

• • • Each xi,j occurs at most once in t1…tr

m-multiple context-free grammars

G = (N,Σ,ρ,P ,S)

ρ(S) =1

B(t

1,…, t

r) :− B

1(x

1,1,…, x

1,r1),…,B

n(x

n,1,…, x

n,rn).

Seki et al. 1991

t1…tr ∈(Σ∪ X )∗

ρ(B) = r,ρ(Bi ) = ri

L(G ) = { w ∈Σ∗∣P S(w ) }

ρ: N→[1,m]

Top-down vs. bottom-upType 0

Type 1 = CSG

Type 2 = CFG

Type 3

EFS

l.b. EFS ≡ CSG

simple LMG ≡ P

PMCFG

MCFG

simple EFS ≡ CFG

Smullyan 1961

Arikawa et al. 1989

Arikawa 1970

Groenink 1997

Seki et al. 1991, Groenink 1997

Seki et al. 1991, Groenink 1997

rewriting systems logic programs on strings

Thue Post Chomsky

Actual world

Smullyan

Not much happensfor a long time

Smullyan Chomsky

Better possible worldFormal grammar

flourishes

1956

1961

Second-order ACGs as context-free grammars on λ-terms

• Σ = (A,C,τ) is a higher-order signature- A is a set of atomic types- C is a set of constants- τ: C→T(A)

• σ: N→T(A)

G = (N,Σ,σ ,P ,S)

α ,β ∈T (A) ⇒α → β ∈T (A)

complexity = max(ord(σ(B)))

Second-order ACGs as context-free grammars on λ-terms

• Rules in P are of the form

•

G = (N,Σ,σ ,P ,S)

B(M) : −B1(X1),…,Bn(Xn ).

where X1 : σ (B1),…, Xn : σ (Bn ) |−Σ M : σ (B).

L(G ) = { |M|β∣P S(M) }

β-normal form

Typed linear λ-calculus

MNP means (MN)Pλx1…xn.M means λx1.(…(λxn.M)…)

α1→…→αn→ β means α1→(…→(αn→ β)…)

Second-order ACGS

P

R

P λx .gx λx1x2 .fx1(gx2 )

λx1.q(λx2 .fx1(gx2 ))

q(λx1.q(λx2 .fx1(gx2 )))

S(q(λx .Xx )) :− P(X ).P(λx

1.q(λx

2.Yx

1x

2) :− R(Y ).

P(λx2.q(λx

1.Yx

1x

2)) :− R(Y ).

P(λx .gx ).R(λx

1x

2.fx

1(Xx

2)) :− P(X ).

τ (q) = (o → o)→ oτ (f ) = o → o → oτ (g) = o → o

σ (S) = oσ (P ) = o → oσ (R) = o → o → o

complexity 2

“Context-free” grammars on …

• RTG: regular tree grammar• LCFTG: linear context-free tree grammar• MCFG: multiple context-free grammar• MRTG: multiple regular tree grammar• MCTAG: multi-component tree-adjoining grammar• MLCFTG: multiple linear context-free tree

grammar

strings trees unary tree contextsC[x1]

n-ary tree contexts C[x1,…,xn]

single CFG RTG TAG (monadic LCFTG)

LCFTG

multiple MCFG MRTG MCTAG MLCFTG

S ⎛⎝

⎞⎠ :− P(Y ).

Linear Context-Free Tree Grammar

Y

e e

f

b b

a

Y

b b

a

P⎛

⎝⎜⎜

⎞

⎠⎟⎟

:− P(Y ). P ⎛⎝

⎞⎠ .

f

b b

a

b b

a

f

b b

a

b b

a

e e

f

S

P

P

Pf

Ranked trees

Δ = Δ( r )

r∈

f ∈Δ(n ) ,T

1,…,T

n∈TΔ ⇒ (fT1…Tn ) ∈TΔ

f (T1,…,Tn )

ranked alphabet

Tree signature

ΣΔtree = ({o},Δ,τΔ )

f ∈Δ(n ) ⇒τΔ (f ) = o

n → o

• Trees over Δ are β-normal closed λ-terms of type o

• n-ary tree contexts over Δ are β-normal closed λ-terms of type on→o

S ⎛⎝

⎞⎠ :− P(Y ).

LCFTG as second-order ACG

Y

e e

f

b b

a

Y

b b

a

P⎛

⎝⎜⎜

⎞

⎠⎟⎟

:− P(Y ). P ⎛⎝

⎞⎠ .

f

b b

a

b b

a

f

b b

a

b b

a

e e

f

S

P

P

P f

S ⎛⎝

⎞⎠ :− P(Y ).

LCFTG as second-order ACG

Y

e e

f

b b

a

Y

b b

a

P⎛

⎝⎜⎜

⎞

⎠⎟⎟

:− P(Y ). P ⎛⎝

⎞⎠ .

f

b b

a

b b

a

f

b b

a

b b

a

e e

f

S

P

P

P

x1 x2

λx1x2.

λx1x2.

x1 x2

f

x1 x2

λx1x2.

x1 x2

λx1x2.

x1 x2

λx1x2.

complexity 2 σ (P ) = o → o → o

Context-free grammars usually do not use lambda.Cleaner notation with lambda.

TR(ACG(2,2) ) ⊇ LCFTL

treegenerating

power

second-orderACGs of

complexity 2

• Converse can be shown as well.

TR(ACG(2,2) ) = LCFTL

treegenerating

power

second-orderACGs of

complexity 2

P ,⎛⎝⎜

⎞⎠⎟

:− P(X1, X

2).

S ⎛⎝

⎞⎠ :− P(X1, X2 ).

Multiple regular tree grammar

f

X1 X2 X2

g

P(e,e).

S

P

P

P

c dX1

g

a b

e ee

g

c de

g

a b

g

c d

g

a b

e

g

c de

g

a b

g

c d

g

a b

e

g

c de

g

a b

f

Encoding tuples

M1, M2: linear λ-terms of type α1, α2

〈M1, M2〉= λw.wM1M2linear

type (α1→α2→ β)→ β for any β

(λx1x2.C[x1, x2]) ↠β C[M1, M2]〈M1, M2〉Usage:C[ , ] must be of type β

S X λx

1x

2.( )⎛⎝

⎞⎠ :− P(X ).

P ,⎛⎝⎜

⎞⎠⎟

:− P(X1, X

2).

S ⎛⎝

⎞⎠ :− P(X1, X2 ).

f

X1 X2

X2

g

P(λw .wee).

c dX1

g

a b

P X λx

1x

2.λw .w ( )( )( )⎛

⎝⎜⎞

⎠⎟:− P(X ).

f

x1 x2

x2

g

P(e,e).

c dx1

g

a b

MRTG ACG

context of type o

context of type (o→o→o)→o

✕

Encoding tuples

M1, M2: linear λ-terms of type α1, α2

〈M1, M2〉= λw.wM1M2linear

type (α1→α2→ o)→ o

(λx1x2.C[x1, x2]y1…ym) ↠βη C[M1, M2]〈M1, M2〉Usage:

where C[ , ] is of type β1→…→ βm→oλy1…ym.

S X λx

1x

2.( )⎛⎝

⎞⎠ :− P(X ).

P ,⎛⎝⎜

⎞⎠⎟

:− P(X1, X

2).

S ⎛⎝

⎞⎠ :− P(X1, X2 ).

f

X1 X2

X2

g

c dX1

g

a b

P X λx

1x

2.λw .w ( )( )( )⎛

⎝⎜⎞

⎠⎟:− P(X ).

f

x1 x2

x2

g

c dx1

g

a b

MRTG ACG

context of type o

context of type (o→o→o)→o

P λw .X λx

1x

2.w ( )( )( )⎛

⎝⎜⎞

⎠⎟:− P(X ).

x2

g

c dx1

g

a b✓

✕

complexity 3 σ (P ) = (o → o → o)→ o

TR(ACG(2,3) ) ⊇MRTL

treegenerating

power

second-orderACGs of

complexity 3

• Inclusion is proper because LCFTL ⊈ MRTL

P ,⎛

⎝⎜⎜

⎞

⎠⎟⎟

:− P(Y1,Y

2).

S ⎛⎝

⎞⎠ :− P(Y1,Y2 ).

Multiple Linear Context-Free Tree Grammar

Y1

e e

f

b b

a

Y1

b b

a

P ,⎛⎝

⎞⎠ .

f

b b

a

b b

a

f

b b

a

b b

a

e e

f

S

P

P

Pf f

f

b b

a f

b b

a

b b

a

f

b b

a

b b

a

e e

f

fY2

b b

a

Y2

e e

f

TR(ACG(2,4) ) ⊇MLCFTL

treegenerating

power

second-orderACGs of

complexity 4

TR(ACG(2,3) ) ⊇MRTL∪LCFTL

TR(ACG(2,2) ) = LCFTL

TR(ACG(2,1) ) = RTL

TR(ACG(2,4) ) ⊇MLCFTL

Encoding tree grammars

• View strings as unary tree contexts over a monadic alphabet

Encoding strings

/aaba/ = λz. a

a

b

a

z

String signature

τ(a) = o→ o

ΣstringΥ = ({o},Υ, τ)

• Concatenation is expressed by B = λxyz.x(yz)

Yield

f λx1…x

nz.x

1(…(x

nz)…) if f ∈Δ(n ) for n ≥1

c λx .cx if c ∈Δ(0)

o o → o

yield : Σtree∆ → Σstring∆(0)

G G′

tree generatingcomplexity n

string generatingcomplexity n+1

L( ′G ) = yL(G )

STR(ACG(2,4) ) ⊇MCFL = yMRTL

STR(ACG(2,3) ) = yLCFTL

STR(ACG(2,2) ) = CFL = yRTL

Encoding string grammars

?

• This implies (for n ≥ 5)

Salvati’s Theorem

STR(ACG(2,n ) ) ⊆ MCFL

STR(ACG(2,n ) ) = STR(ACG(2,4) ) = MCFL

Extraction of a tuple of strings from a λ-term

|−

ΣΥstring M :α

(w1,…,wm ) P[z1,…, zm ]tuple of strings pure λ-term

z1 : o → o,…, zm : o → o |− P[z1,…, zm ] :α

P[/w1/,…, /wm /] ↠β M

2m ≤ |α |

Given α, there are only finitely many possibilities for P[z1,...zm].Information carried by M is just (w1,...,wm) plus some bounded amount of information.

z2

z3

z1

aba a

bbbb

b

aba

abbb

bb

z2

z3

z1

z1 : o → o,…, zm : o → o |− P[z1,…, zm ] :α

only finitely many

B(M)

(B,P[z1,…, zm ])(w1,…,wn )

ACG

MCFG

• The same technique applies to the tree case.

Create a new nonterminal with B and P[z1,...,zm].You get a tuple of tree contexts in the tree case.

STR(ACG(2,4) ) = MCFL

STR(ACG(2,3) ) = yLCFTL

STR(ACG(2,2) ) = CFL

STR(ACG(2,n ) ) (n ≥ 5)

=

TR(ACG(2,3) ) ⊇MRTL∪LCFTL

TR(ACG(2,2) ) = LCFTL

TR(ACG(2,1) ) = RTL

TR(ACG(2,4) ) = MLCFTL

TR(ACG(2,n ) ) (n ≥ 5)

=

Second-order ACG hierarchy