Introduction to Machine Translation

Machine TranslationAdam Lopez, Matt Post, & Yuan Cao

Machine TranslationAdam Lopez, Matt Post, & Yuan Cao

虽然北风呼啸 , 但天空依然十分清澈。


However , the sky remained clear under the strong north wind .

The Tower of Babel

Pieter Brueghel the Elder (1563)

Georges Artsrouni’s “mechanical brain”, patented 1933 (France)

ENIAC (1946)

When I look at an article in Russian, I say: “This

is really written in English, but it has been coded in some strange symbols. I will now proceed to decode.”

Warren Weaver (1949)

Popular view of MT in 2003


Popular view of NLP:



and NLP:

Because we want to provide everyone with access to all the world's information, including information

written in every language, one of the exciting projects at Google Research is machine translation... Now you can see the results for yourself. We recently launched

an online version of our system for Arabic-English and English-Arabic. Try it out!

Statistical Machine Translation Live

4/28/2006 03:40:00 PMFranz Och

Translation Exercise(Knight, 1997)

iat lat pippat eneat hilat oloat at-yurp .

ok-voon ororok sprok .

at-voon bichat dat . ok-drubel ok-voon anok plok sprok .

at-drubel at-voon pippat rrat dat .

erok sprok izok hihok ghirok .

totat dat arrat vat hilat . ok-voon anok drok brok jok .

at-voon krat pippat sat lat . wiwok farok izok stok .

totat jjat quat cat . lalok sprok izok jok stok .

wat dat krat quat cat .

lalok farok ororok lalok sprok izok enemok .

wat jjat bichat wat dat vat eneat . lalok brok anok plok nok .

iat lat pippat rrat nnat . wiwok nok izok kantok ok-yurp .

totat nnat quat oloat at-yurp . lalok mok nok yorok ghirok clok .

wat nnat gat mat bat hilat . lalok nok crrrok hihok yorok zanzanok .

wat nnat arrat mat zanzanat . lalok rarok nok izok hihok mok .

wat nnat forat arrat vat gat .

















pippat : anokquat : izokrrat : ploksat : drok

totat : eroktotat : wiwok

vat : izokwat : lalok

zanzanat : zanzanok??? : crrrok

arrat : hihokat-drubel : ok-drubel

at-voon : ok-voonat-yurp : ok-yurp

bat : clokbichat : ororok

cat : stokdat : sprok

eneat : enemokforat : rarok


gat : mokhilat : ghirok

iat : lalokjjat : farokkrat : joklat : brok

mat : yoroknnat : nok

oloat : kantok














oloat : kantok














oloat : kantok

lalok brok anok enemok ghirok kantok ok-yurp .


































lalok anok brok enemok kantok ghirok ok-yurp .

















lalok brok anok kantok ghirok enemok ok-yurp .

















lalok brok anok kantok ghirok enemok ok-yurp .

5040 possible orderings!



ok-drubel anok ghirok farok . wiwok rarok nok zerok ghirok enemok .

ok-drubel ziplok stok vok erok enemok kantok ok-yurp zinok jok yorok clok .lalok clok izok vok ok-drubel .

ok-voon ororok sprok . ok-drubel ok-voon anok plok sprok .

erok sprok izok hihok ghirok . ok-voon anok drok brok jok .

wiwok farok izok stok .lalok sprok izok jok stok .lalok brok anok plok nok .

lalok farok ororok lalok sprok izok enemok .wiwok nok izok kantok ok-yurp .lalok mok nok yorok ghirok clok .

lalok nok crrrok hihok yorok zanzanok .lalok rarok nok izok hihok mok .










































































lalok brok anok ghirok enemok kantok ok-yurp .







































































totat nnat forat arrat mat bat .














oloat : kantok














oloat : kantok














oloat : kantok

erok|wiwok nok rarok hihok yorok clok .





























erok|wiwok rarok nok hihok yorok clok .




































wiwok rarok nok crrrok hihok yorok clok .



































Is this exercise realistic?

•In real natural language...

•...most words are ambiguous.

•...and may have complex morphology.

•...and non-standard orthography.

•...sentences are longer.

•...output word order may depend on input order. (e.g. “Mary loves John” vs. “John loves Mary”).

Is this exercise realistic?

•What about pronouns?

•What about inflectional morphology?

•What about structural ambiguity?

•What about domain knowledge?

•What about negation?

















the clients and the associates are enemies .

los clientes y los asociados son enemigos .the company has three groups .

la empresa tiene tres grupos .its groups are in Europe .

sus grupos estan en Europa .the modern groups sell strong pharmaceuticals .

los grupos modernos venden medicinas fuertes .the groups do not sell zanzanine .

los grupos no venden zanzanina .the small groups are not modern .

los grupos pequenos no son modernos .

Garcia and associates .

Garcia y asociados .Carlos Garcia has three associates .

Carlos Garcia tiene tres asociados .his associates are not strong .

sus asociados no son fuertes .Garcia has a company also .

Garcia tambien tiene una empresa .its clients are angry .

sus clientes estan enfadados .the associates are also angry .

los asociados tambien estan enfadados .

la empresa tiene enemigos fuertes en Europa .

the clients and the associates are enemies .

los clientes y los asociados son enemigos .the company has three groups .

la empresa tiene tres grupos .its groups are in Europe .

sus grupos estan en Europa .the modern groups sell strong pharmaceuticals .

los grupos modernos venden medicinas fuertes .the groups do not sell zanzanine .

los grupos no venden zanzanina .the small groups are not modern .

los grupos pequenos no son modernos .

Garcia and associates .

Garcia y asociados .Carlos Garcia has three associates .

Carlos Garcia tiene tres asociados .his associates are not strong .

sus asociados no son fuertes .Garcia has a company also .

Garcia tambien tiene una empresa .its clients are angry .

sus clientes estan enfadados .the associates are also angry .

los asociados tambien estan enfadados .

la empresa tiene enemigos fuertes en Europa .

the company has strong enemies in Europe .





Although north wind howls , but sky still very clear .





































Despite the strong northerly winds , the sky remains very clear .




Despite the strong northerly winds , the sky remains very clear .The sky was still crystal clear , though the north wind was howling .

Although a north wind was howling , the sky remained clear and blue .




Despite the strong northerly winds , the sky remains very clear .The sky was still crystal clear , though the north wind was howling .

Although a north wind was howling , the sky remained clear and blue .

Questions

•How can we formalize knowledge acquisition from text as an algorithm?

•How can we formalize decoding as an algorithm?

•How well does it work?

Statistical Machine Translation

Develop a statistical model of translation that can be learned from data and used to predict the correct

English translation of new Chinese sentences.


Develop a statistical model of translation that can be learned from data and used to predict the correct

English translation of new Chinese sentences.


linguistics


linguistics

algorithms


linguistics

algorithms

machine learning


linguistics

algorithms

machine learning

formal language

theory


linguistics

algorithms

machine learning

formal language

theory info

rmat

ion

theo

ry


linguistics

algorithms

machine learning

formal language

theory info

rmat

ion

theo

ry


Course Outline

Course Outline

•February: basic tools of modeling translation.

Course Outline


•Things we understand pretty well.

Course Outline



•March: formal tools and syntax.

Course Outline




Course Outline




•Things we’re trying to understand.

Course Outline





•April: advanced topics.

Course Outline





•April: advanced topics.

•Things we don’t understand very well (Research!)

Book

Book

Grading•Five homework assignments (12 points each)

•Homework 1 is posted!

•Assignments are open-ended. You must do more than implement an algorithm that we specify.

•Presentation: Language in 10 minutes (10 points)

•Final project (30 points)

•Writing component to homeworks and project.

http://mt-class.org/jhu

Please sign up on piazza for discussion.



Technology

Introduction to Machine Translation