1

Chinese WordSketch

Engine

Online, corpus-based summaries of word usage

2

Designers of the original WordSketch Engine for English

• Adam Kilgarriff, Lexical Computing, UK

• David Tugwell, Tech University Budapest

• Pavel Rychly, Brno University

3

Facing the problem: lexical choice

• “You shall know a word by the company it keeps” (Firth, 1957)

• The meaning of face depends on the collocation ( 詞語搭配 ) – 學漢語的外國人要面對詞語選擇的問題– 許多種動物正在面臨絕種的問題

• Similarly with save– Save money– Save life

4

Look in a dictionary? A corpus?

• Some modern English dictionaries give some collocation ( 詞語搭配 ) information– Chinese dictionaries give very limited help

• Since the 1980s, corpus KWIC (KeyWord In Context) concordances have been available

5

Pre-computer corpus!

Oxford English Dictionary:• 20 million index cards

6

KWIC Concordance1 arity, which will be used to take a party of under-privileged children to D 2 from outside. You are invited to a party and after a couple of drinks you d 3 tion, we believe politicians of all parties will listen to our views. &equo 4 ould be reaching agreement with all parties concerned, as to which events, 5 lack people. I have certainly been party to one or two discussions amongst 6 . These should be discussed by both parties before entering into the relatio 7 presents They had hosted a cocktail party at Kensington palace, for example 8 akes. By midnight the end-of-course party is in full swing, but most cadet 9 e should be a right for the injured party to terminate the contract. A mana 10 by the Safran Peoples ' Liberation Party. This presents the powerful neigh 11 s. Ahead I could see the rest of my party plodding towards the final slope t 12 cial ethic. The two main political parties - the Tories and the Liberals - 13 ritish successes in Perth The small party of British players competing in th 14 to help control. One member of the party went to summon the rescue team and 15 rket society fashion magazine. The party was held at his flat which was a l 16 security and secrecy than any Tory Party Conference : it seems that bootleg

7

1 political association 4 person in an agreement/dispute 2 social event 5 to be party to something...3 group of people

1 arity, which will be used to take a party of under-privileged children to D2 from outside. You are invited to a party and after a couple of drinks you d3 tion, we believe politicians of all parties will listen to our views. &equo4 ould be reaching agreement with all parties concerned, as to which events,5 lack people. I have certainly been party to one or two discussions amongst6 . These should be discussed by both parties before entering into the relatio7 presents They had hosted a cocktail party at Kensington palace, for example8 akes. By midnight the end-of-course party is in full swing, but most cadet9 e should be a right for the injured party to terminate the contract. A mana10 by the Safran Peoples ' Liberation Party. This presents the powerful neigh11 s. Ahead I could see the rest of my party plodding towards the final slope t12 cial ethic. The two main political parties - the Tories and the Liberals -13 ritish successes in Perth The small party of British players competing in th14 to help control. One member of the party went to summon the rescue team and15 rket society fashion magazine. The party was held at his flat which was a l16 security and secrecy than any Tory Party Conference : it seems that bootleg

The coloured pens method

8

Limitation of KWIC analysis

• As corpora get bigger: too much data– 50 lines for a word: read all– 500 lines: could read all, takes a long time– 5000 lines: no

• Instead, create a statistical summary of word usage– Show most salient 最有顯著性 collocates

(Mutual Information)

9

Mutual Information

• Church and Hanks 1989• MI: How much more often does a

word pair occur, than one might expect by chance:

)(

)|(log);(

xP

yxPyxI

10

Collocation listing For right collocates of save (>5 hits)

word f(x+y) f(y) word f(x+y) f(y)

forests 6 170 life 36 4875

$1.2 6 180 dollars 8 1668

lives 37 1697 costs 7 1719

enormous 6 301 thousands 6 1481

annually 7 447 face 9 2590

jobs 20 2001 estimated 6 2387

money 64 6776 your 7 3141

11

Limitations of collocation listing

• Some items are not genuine collocates– yours appears only because it is adjacent to save

• The collocates can belong to any part of speech– It would better if they were classified into POS– and the role they play in the sentence

• Thus,– for arrest in “The police were quick to arrest a number of suspects on

the spot”• We would like to see

– Keyword: arrest– Subject: police– Object: suspect(s)– Modifier: on the spot

• We would not be especially interested in to, a and number– These non-collocates happen to be close to the keyword

12

Wordsketch

• Attempts to meet these requirements

• A corpus-derived one-page summary of a word’s grammatical and collocational behaviour

• Implemented for English and Czech

• Chinese and Irish implementations in progress

13

Grammatical relations

• Salience is calculated for a keyword and its collocates in particular grammatical configurations – Police and arrest as subject and verb– Suspect and arrest as object and verb– Bank and post office in a coordinative

relationship (and/or) – And others: altogether 27, for English

14

The corpus: Chinese Gigaword

• A Linguistic Data Consortium corpus– Very large: over 1 billion characters– Compiled by David Graff & Ke Chen in 2003– Minimally tagged

• 286 newswire stories, half from each of:– CNA Taiwan (740 million traditional characters)– Xinhua PRC (380 million simplified characters)

• Corpus was segmented and tagged using Academia Sinica tools

15

http://corpora.fi.muni.cz/chinese/

• 逮捕• 教• 學習• 銀行• 捉

16

17

18

19

20

21

22

Grammar: examples of current constraints

• Object relation1:trans_verb particle? any_noun? "DE"? any_noun? 2:common_noun

• And/or relation1:any_noun listcomma 2:any_noun & 1.tag = 2.tag1:any_noun conj 2:any_noun & 1.tag = 2.tag

23

Functions• KWIC concordance

– Sorting, filtering etc

• Word sketch• Automatic thesaurus• Sketch difference

– discriminate near-synonyms

24

25

Further work

• Annotate Gigaword corpus with AS semantic tagset

• Improve grammatical relations, especially sentence objects, to account for– topicalization ( 啤酒 , 葡萄酒 , 他都愛喝 )– 把 fronting ( 請把啤酒喝完 )

• Create “Dr Eye” style interface, to show common collocations online, in a text.