PowerPoint Presentationbitec.snubi.org/AlphaGo(I).pdf · 2016-07-25 · 2016: An Intelligence Odyssey The Mystery of Go (WIRED, 2014.12.05)• “The Mystery of Go”, 1965, New Scientist

2016: An Intelligence Odyssey

2016 An Intelligence

Odyssey

March, 2016

Ju Han Kim, M.D., Ph.D.



March, 2016 入試說明會


A Space Odyssey

March, 2016



The Dawn of Man

• 25s

• 1m

• 3m 10s

• 5m 30s

https://www.youtube.com/watch?v=ypEaGQb6dJk

https://www.youtube.com/watch?v=bbUGT--UfJE


Discovery One


EVA Pod

Ex Machina

α


An Intelligence Odyssey

March, 2016



Deep Thought에서 DeepMind까지


Deep Thought, Deep Blue, and DeepMind

• Average branching factors for

Chess (35) and Go (250)

• Combinatorial explosion!

• Brut force search, infeasible!

• Pruning algorithm can cut down

the branching factor (i.e., alpha-

beta pruning and minmax algorithm, a

branch and bound algorithm)


Time complexity: Θ(bd) b= branching factor, d=depth

형세판단: Value networks, Policy networks

Daylen Yang, University of

California at Berkeley, TNG

Technology Consulting,

https://www.youtube.com/wat

ch?v=pUyURF1Tqvg

A ridiculously big number!!

Search Space Reduction

https://www.youtube.com/watch?v=7iCV509dXUE


“알파고 승리는 희대의 사기극”

Time complexity: Θ(bd) b= branching factor, d=depth A ridiculously big number!!


The Mystery of Go (WIRED, 2014.12.05)

• “The Mystery of Go”, 1965, New Scientist

• Impossibly high branching factor (250, cf. 35 for Chess) and

state space makes MinMax extremely costly.

• Alfred Zorist’s first Go program in 1968

• In Search of the Mental Leap: Many people peak out at a certain level of amateur and never get any stronger. Bruce Wilcox won James Kerwin in 1979 with divide and conquer mainly with expert modeling. Mark Boon (Goliath), David Fotland (Many Faces of Go), Chen Zhixing (Handtalk and Goemate) were all excellent players.

• The Monte Carlo Bet: (Coulom’s Crazy stone ) MC reduces search space with statistical sampling. Crazy stone MCTS won its first tournament in 2006, commercialized in 2011, winning Yoshio in 2013. “After the match, I ask Coulom when a machine will win without a handicap, “I think maye 10 years, but I don’t like to make predictions.” Feng-Hsing Hsu (Deep Thought) also favored alpha-beta search over MC.

• When AI Is Not AI “Even more surpprising was that no programmers

think of their creations as ‘intelligent’”

• IBM Watson becomes the gatekeeper of a new era.



ChipTest (1985), Deep Thought (by Feng-hsiung Hsu, a Taiwanese American Scientist at Carnegie Mellon University), and Deep Blue (IBM)

Deep Fritz (Germani), Deep Junior (Israeli)

Deep Blue (IBM) won its first game against a world champion, Garry Kasparov, on February 10, 1996, but defeated by 4-2. Deep Blue was then heavily upgraded, and defeated Kasparov again in May 1997, becoming the first computer system to defeat a reigning world champion in a match under standard chess tournament.

Blue Gene is an IBM supercomputers that can reach operating speeds in the PFLOPS (petaFLOPS) range.

Watson is a QA computer system capable of answering questions posed in natural language,[2] developed in IBM's DeepQA project, specifically developed to answer questions on the quiz show Jeopardy! Watson won Jeopardy in 2011 and awared $1M.

DeepMind & AlphaGo is claimed to be not pre-programmed. Technically it uses deep learning on a convolutional neural network, with a novel form of Q-learning, a form of model-free reinforcement learning.



Deep Thought does not know the Ultimate Question to Life, the Universe and Everything, but offers to design an even more powerful computer, (planet) Earth, to calculate it. After 10M years of calculation, the Earth is destroyed by Vogons* 5 min. before the computation is complete.

1971

1979.10.12

Deep Thought is a super computer created by some hyper-intelligent, pan-galactic beings (whose three-dimensional protrusions into

our universe are white mice). Deep Thought is the size of a small city. Deep Thought takes 7.5M years to compute the Answer to “The Ultimate Question of Life, the Universe, and

Everything“, which turns out to be 42,

quite definitely after very thorough checking.

*Vogons are a fictional alien race from the planet Vogsphere who are responsible for the

destruction of the Earth, in order to facilitate an intergalactic highway construction project.


A Towel and A Screwdriver

Bring your towel.

May 25th.


Riddles, Tricks and Intelligence

March, 2016


The first riddle of the sphinx

What goes on four legs in the morning, on two legs

at noon, and on three legs in the evening?

Oedipus solved the riddle, and the Sphinx destroyed herself. 17

The second riddle of the sphinx

"There are two sisters: one gives birth to

the other and she, in turn, gives birth to

the first. Who are the two sisters?" 18


A Riddle of My Son, 양말을 꿀꺽 삼켜버린 수학


Bachet’s game or Nim

• The 21 game The first player says "1" and each player in turn increases the number by 1, 2, or 3, but may not exceed 21; the player forced to say "21" loses.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

노오오오오~오~력!

• The 100 game Two players start from 0 and alternatively add a number from 1 to 10 to the sum. The player who reaches 100 wins. The winning strategy is to reach a number in which the digits are subsequent (e.g. 01, 12, 23, 34,...) and control the game by jumping through all the numbers of this sequence. Once reached 89, the opponent has lost (he can only tell numbers from 90 to 99, and the next answer can in any case be 100).


Nimatron, an Electric Brain?

• At the 1940 New York World's Fair, Westinghouse displayed a machine, the Nimatron, that played Nim. It was also one of the first ever electronic computerized games.

The Nimrod, built in Britain by Ferranti for the 1951 Festival of Britain, was an early computer custom-built to play a computer game, one of the first games developed in the early history of video games.


Can Machine Think?

March, 2016



Monte, Nim, Chess, Go, and Human Insight

1060


The Imitation Game

Alan Turing’s Machine Intelligence (since 1941)

Computing Machinery and Intelligence (1950) "I propose to

consider the question, 'Can machines think?'"

Because "thinking" is difficult to define, Turing chooses to replace

the question, "Are there imaginable digital computers which

would do well (what we (as thinking entities) can do) in the

Imitation Game?", one that can actually be answered.

But it never happens that it arranges its speech in various ways, in order to reply

appropriately to everything that may be said in its presence, as even the lowest type

of man can do. (René Descartes in 1637, in his Discourse on the Method)


Monte, Nim, Chess, Go, Elisa, and Human

• Eliza, the psychotherapist, is the grandma of Siri.

• Turing test for Elisa in Blade Runner

• Eugene Goostman

• Siri by Apple

29s

https://www.youtube.com/watch?v=Q6CycvRbHfM

Eliza AI Therapy Program.mp4


The Chinese Room (Strong AI hypothesis)

• John Searle, a philosopher, proposed a thought experiment to challenge

the claim that it is possible for a computer running a program to have a

"mind" and "consciousness" in the same sense that people do,

simply by virtue of running the right program.

"The appropriately programmed computer with the right inputs and outputs would thereby have a

mind in exactly the same sense human beings have minds."

Blade Runner (1-10) Movie CLIP - She's a Replicant (1982) HD.mp4


Strong vs. Weak AI

• Strong AI: Performing any intellectual task

that a human being can.

• The primary goal of AI

• SF writers and futurists

• Artificial General Intelligence

• Consciousness, sentience, sapience, self-

awareness

• Weak AI: applied or narrow AI

• Do not necessarily have a mind, mental states,

or consciousness.


A Strong AI, w/ Emergent Properties


A Strong AI, having emotion/motivation?

•알파고 승리 배경

http://www.todayhumor.co.kr/board/view.php?table=bestofbest&no=235211&s_no=235211&page=1


알파고 승리 배경 A Strong AI, having emotion/motivation?

, .

Feel

http://www.todayhumor.co.kr/board/view.php?table=bestofbest&no=235211&s_no=235211&page=1


A Strong AI, showing delinquent behavior?


Reverse Turing Test


Stupidity or ‘Natural’ Intelligence


Artificial Stupidity vs. Human Stupidity


An Odyssey for Intelligence or Search?

March, 2016



Time complexity: Θ(bd) b= branching factor, d=depth

형세판단: Value networks, Policy networks

Daylen Yang, University of

California at Berkeley, TNG

Technology Consulting,

https://www.youtube.com/wat

ch?v=pUyURF1Tqvg

A ridiculously big number!!


S

A B C

G

F E D

3

4

2

5

4

3

5

4

4

Graph and Tree, Path and Search


Tree Search

S

A D

B D A E

C E

D F

G

E

B F

C G

B

C E

F

G

B

A C

F

G

S

A B C

G

F E D

3

4 2

5

4

3

5

4

4


S

A B C

G

F E D

3

4 2

5

4

3

5

4

4

S : Start G : Goal N(x) : list of 1st degree neighbors Path : [S, D, E, F, G] Paths : [[S, A, B, C], [S, D, E, F, G],….]

[[S]] [[S,A], [S,D]] [[S,D], [S,A,B], [S,A,D]] [[S,A,B], [S,A,D], [S,D,A], [S,D,E]] [[S,A,D], [S,D,A], [S,D,E], [S,A,B,C], [S,A,B,E]] [[S,D,A], [S,D,E], [S,A,B,C], [S,A,B,E], [S,A,D,E]] [[S,D,E], [S,A,B,C], [S,A,B,E], [S,A,D,E], [S,D,A,B]] [[S,A,B,C], [S,A,B,E], [S,A,D,E], [S,D,A,B], [S,D,E,B], [S,D,E,F]] [[S,A,B,E], [S,A,D,E], [S,D,A,B], [S,D,E,B], [S,D,E,F], [S,A,B,C]] [[S,A,D,E], [S,D,A,B], [S,D,E,B], [S,D,E,F], [S,A,B,C], [S,A,B,E,D], [S,A,B,E,F]] [[S,D,A,B], [S,D,E,B], [S,D,E,F], [S,A,B,C], [S,A,B,E,D], [S,A,B,E,F], [S,A,D,E,B], [S,A,D,E,F]] [[S,D,E,B], [S,D,E,F], [S,A,B,C], [S,A,B,E,D], [S,A,B,E,F], [S,A,D,E,B], [S,A,D,E,F], [S,D,A,B,C], [S,D,A,B,E]] [[S,D,E,F], [S,A,B,C], [S,A,B,E,D], [S,A,B,E,F], [S,A,D,E,B], [S,A,D,E,F], [S,D,A,B,C], [S,D,A,B,E], [S,D,E,B,A], [S,D,E,B,C]] [[S,D,E,F], [S,A,B,C], [S,A,B,E,D], [S,A,B,E,F], [S,A,D,E,B], [S,A,D,E,F], [S,D,A,B,C], [S,D,A,B,E], [S,D,E,B,A], [S,D,E,B,C]] [[S,A,B,C], [S,A,B,E,D], [S,A,B,E,F], [S,A,D,E,B], [S,A,D,E,F], [S,D,A,B,C], [S,D,A,B,E], [S,D,E,B,A], [S,D,E,B,C], [S,D,E,F,G]] [[S,A,B,E,D], [S,A,B,E,F], [S,A,D,E,B], [S,A,D,E,F], [S,D,A,B,C], [S,D,A,B,E], [S,D,E,B,A], [S,D,E,B,C], [S,D,E,F,G], [S,A,B,C]] [[S,A,B,E,F], [S,A,D,E,B], [S,A,D,E,F], [S,D,A,B,C], [S,D,A,B,E], [S,D,E,B,A], [S,D,E,B,C], [S,D,E,F,G], [S,A,B,C], [S,A,B,E,D]] [[S,A,D,E,B], [S,A,D,E,F], [S,D,A,B,C], [S,D,A,B,E], [S,D,E,B,A], [S,D,E,B,C], [S,D,E,F,G], [S,A,B,C], [S,A,B,E,D], [S,A,B,E,F,G]] [[S,A,D,E,F], [S,D,A,B,C], [S,D,A,B,E], [S,D,E,B,A], [S,D,E,B,C], [S,D,E,F,G], [S,A,B,C], [S,A,B,E,D], [S,A,B,E,F,G], [S,A,D,E,B,C]] [[S,D,A,B,C], [S,D,A,B,E], [S,D,E,B,A], [S,D,E,B,C], [S,D,E,F,G], [S,A,B,C], [S,A,B,E,D], [S,A,B,E,F,G], [S,A,D,E,B,C], [S,A,D,E,F,G]] [[S,D,A,B,E], [S,D,E,B,A], [S,D,E,B,C], [S,D,E,F,G], [S,A,B,C], [S,A,B,E,D], [S,A,B,E,F,G], [S,A,D,E,B,C], [S,A,D,E,F,G], [S,D,A,B,C]]

S

A D

B D A E

C E

D F

G

E

B F

C G

B

C E

F

G

B

A C

F

G



S

A B C

G

F E D

3

4 2

5

4

3

5

4

4

S : Start G : Goal N(x) : list of 1st degree neighbors Path : [S, D, E, F, G] Paths : [[S, A, B, C], [S, D, E, F, G],….]

[[S]] [[S,A], [S,D]] [[S,A,D], [S,A,B], [S,D]] [[S,A,D,E], [S,A,B], [S,D]] [[S,A,D,E,B], [S,A,D,E,F], [S,A,B], [S,D]] [[S,A,D,E,B,C], [S,A,D,E,F], [S,A,B], [S,D]] [[S,A,D,E,F], [S,A,B], [S,D], [S,A,D,E,BC]] [[S,A,D,E,F,G],[S,A,B], [S,D], [S,A,D,E,BC]] [[S,A,B], [S,D], [S,A,D,E,BC], [S,A,D,E,F,G]] [[S,A,B,C], [S,A,B,E], [S,D], [S,A,D,E,BC], [S,A,D,E,F,G]] [[S,A,B,E], [S,D], [S,A,D,E,BC], [S,A,D,E,F,G], [S,A,B,C]] [[S,A,B,E,D], [S,A,B,E,F], [S,D], [S,A,D,E,BC], [S,A,D,E,F,G], [S,A,B,C]] [[S,A,B,E,F], [S,D], [S,A,D,E,BC], [S,A,D,E,F,G], [S,A,B,C], [S,A,B,E,D]] [[S,A,B,E,F,G], [S,D], [S,A,D,E,BC], [S,A,D,E,F,G], [S,A,B,C], [S,A,B,E,D]] [[S,D], [S,A,D,E,BC], [S,A,D,E,F,G], [S,A,B,C], [S,A,B,E,D], [S,A,B,E,F,G],]

S

A D

B D A E

C E

D F

G

E

B F

C G

B

C E

F

G

B

A C

F

G



g = ((0,1),(0,4),(1,4),(1,2),(2,3),(4,5),(2,5),(5,6),(6,7)) def nextNodes(graph, paths, goal): path = paths[0] for each g[i] if g[i] has path[-1] and g[i] not in path: paths = paths + [path + g[i]] if goal == g[i]: BINGO!! return paths def dfs(graph, paths, goal): if ?nextNodes for all: exit() else: return dfs(graph, nextNodes(graph, paths, goal), goal)

S

A D

B D A E

C E

D F

G

E

B F

C G

B

C E

F

G

B

A C

F

G

S

A B C

G

F E D

3

4 2

5

4

3

5

4

4

H

Depth First Search, Breath First Search


Create a Web-bot!

ALTRUISM-

.| .||

PL---ASMA ALTRUI

.|..--

PLAS


Graph and tree search algorithms

•α — β •A* •B* •Beam •Bellman–Ford •Best-first •Bidirectional •Borůvka •Branch & bound •BFS •British Museum •D* •DFS •Depth-limited •Dijkstra •Edmonds •Floyd-Warshall •Hill climbing •Iterative deepening •Kruskal •Johnson •Lexicographic BFS •Prim •Uniform-cost Listings •Graph algorithms •Search algorithms •List of graph algorithms Related topics •Dynamic programming •Graph traversal •Tree traversal •Search games

Most AI problems are search problems!

• (Deep) Neural Nets • SVMs • Genetic Algorithms • Graph Algorithms

• Bayse Net • Semi-supervised MLs • Unsupervised MLs • Reinforcement Learning

…

http://en.wikipedia.org/wiki/Graph_traversal

http://en.wikipedia.org/wiki/Tree_traversal



http://en.wikipedia.org/wiki/Alpha-beta_pruning




http://en.wikipedia.org/wiki/A*_search_algorithm

http://en.wikipedia.org/wiki/B*

http://en.wikipedia.org/wiki/Bellman%E2%80%93Ford_algorithm



http://en.wikipedia.org/wiki/Best-first_search



http://en.wikipedia.org/wiki/Bidirectional_search

http://en.wikipedia.org/wiki/Bor%C5%AFvka's_algorithm



http://en.wikipedia.org/wiki/Branch_and_bound

http://en.wikipedia.org/wiki/Breadth-first_search

http://en.wikipedia.org/wiki/British_Museum_algorithm

http://en.wikipedia.org/wiki/D*

http://en.wikipedia.org/wiki/Depth-first_search

http://en.wikipedia.org/wiki/Depth-limited_search



http://en.wikipedia.org/wiki/Dijkstra's_algorithm

http://en.wikipedia.org/wiki/Edmonds'_algorithm

http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm



http://en.wikipedia.org/wiki/Hill_climbing

http://en.wikipedia.org/wiki/Iterative_deepening_depth-first_search

http://en.wikipedia.org/wiki/Kruskal's_algorithm

http://en.wikipedia.org/wiki/Johnson's_algorithm

http://en.wikipedia.org/wiki/Lexicographic_breadth-first_search

http://en.wikipedia.org/wiki/Prim's_algorithm

http://en.wikipedia.org/wiki/Uniform-cost_search



http://en.wikipedia.org/wiki/Category:Graph_algorithms

http://en.wikipedia.org/wiki/Category:Search_algorithms

http://en.wikipedia.org/wiki/List_of_algorithms

http://en.wikipedia.org/wiki/Dynamic_programming

http://en.wikipedia.org/wiki/Graph_traversal


http://en.wikipedia.org/wiki/Search_game


Deep Learning

• The modern reincarnation of Artificial Neural Networks from the 80s and 90s.

• A multi-layer feed forward NNs, w/ an input layer and multiple layers for non-linear transformations.

• Hinton and Salakhutdinov (2000’s): many-layered feedforward NN can effectively pre-train one layer

at a time, treating each layer as an unsupervised restricted Boltzmann machine, then fine-tune using

supervised backpropagation.

• Good in Speech (signal) and Image Recognition: Deep Learning outperforms the traditional GMM-

HMM large-scale speech recognition around 2010.

• A set of algorithms attempting to model higher level abstractions in data by using multi-layered

architectures.

• General algorithms that learn from observations in big-compute, big-data era.

• Critic: Black box, mostly empirical, rather than theoretical.


• Learn action policy: s a to

maximize rewards

• Value function: expected future rewards

• Temporal difference error

• Exploration vs. Exploitation

Reinforcement Learning and Co-evolution

r

a

State s

https://www.youtube.com/watch?v=aTpJJR1WBuc


Reinforcement Learning and Co-evolution

• Reinforcement learning is an area of ML inspired by behaviorism (ako

Skinnerian), concerned with how software agents ought to take actions in an

environment so as to maximize some notion of cumulative reward.

• Claimed to differ from standard supervised learning in that correct input

/output pairs are never presented, nor sub-optimal actions explicitly corrected.

• Approximate Dynamic Programming

• Markov Decision Process: a discrete time stochastic control process.

• Exploration (of uncharted territory) vs. Exploitation (of current knowledge)

trade-off

• Q-learning is a model-free RL technique for an optimal action-selection policy

for any given (finite) MDP.

• Co-evolution strategy by self-playing


The Go Space

March, 2016



Go Problem Space

• Max. branching factor: 19 * 19

• Max. Depth: 150

• Ko rule (覇), 삼패빅, 공배

• Finite vs. infinite

• Computability, Provability, Definability

♥ α


Go space

• Problem space

Local optima

global optima

perturbation

Hill climbing Optimization Brut force?

Curse of dimensionality, the long narrow valley problem


AlphaGo vs. Sedol Lee


`

DeathGo, 제국의 구조

• 바둑의 문제공간 크기는 생각했던 것보다 작다.

• 특히, 바둑이 진행될수록 문제공간은 급격히 좁혀진다.

• ‘패’는 생각보다 쉽게 피하거나 무시할 수 있다.

• 그렇다면 바둑은 그저 좀 큰 체스에 불과하다.

• Balance A* w/ Deep Learning ‘near’ brut-force search.

Win. prob.

“딱 이길 만큼만 둔다!”

Space size


“At the Moment of Truth” Strategy!

1. Go space as latent ko rule 승부수

실수 없이 마무리

2. 흉내바둑 승부수 low-tech. brut force attack

Winning strategy, AlphaGo

“딱 이길 만큼만 둔다!”

Space size

Win. prob.


AlphaGo vs. Lee Sedol, Match 4

• . 78 . . .

• : , . . 78 .

. .


DeathGo, 제국의 구조

• Troopers, Darth Vader, Dark Lord, Death Star

• When Luke Skywalker solved the riddle, the Death Star destroyed herself.

Space size Win. prob.


Death Go, the Dark Lord

1. 흉내바둑 승부수 low-tech. brut force attack



The Last Hero


The Hero’s Odyssey


The Last Hero



Taking Count of Crosses

March, 2016


計家


Take Count the Crosses

•패자는?

•승자는?

•대책은?


Don’t Panic Bring Your Towel

March, 2016



Too Deep Thought…

https://www.youtube.com/watch?v=wCWeLY_aEPs


I think the problem,

to be quite honest with you

is that you’ve never known

what the question was.

Too Deep Thought…


• Formal System

• Computability, Definability, Provability

• Hilbert's problem No. 2 vs. Gödel's Incompleteness Theorems

The Question Was..

“ .”

“( ) (Formal System) ”

( ).

“( )

.

,

.”

“ ”, “

.”

“

.”

David Hilbert Kurt Gödel

Gerhard Karl Erich Gentzen


AI BI CI

• AI: Artificial Intelligence

• BI: Biomedical Informatics

• CI: Clinical Intelligence

• DI: Dasein Intelligence


Always Know Where Your Towel Is

Return Of the Jedi opened

on May 25, 1983.

Bring Your Towel, May 25


AlphaGo Strikes Back

March, 2016



When AlphaGo Strikes Back…


The Problem Begins

• Problem space


From the Dawn of Computer…


고용의 미래 소비

생산

Luddite, Chartist, Suffragettes

Hitchhikers, Hackers, Heroes


의료의 미래

Dr.


When the Empire Strikes Back…

Space size


Hitchhikers, Hackers, and Heroes


The Troopers

우리가 모르고 있는 것 때문에 문제가 일어나는

것이 아니다. 문제는 우리가 진실이라고 믿고

있는 것이 틀렸을 때 발생한다. -- 마크 트웨인


An Eternal Odyssey

March, 2016



An Eternal Odyssey

♥ α

2001- A Space Odyssey - The Dawn of Man.mp4


The Final Odyssey…


After the Odyssey…

♥ α



March, 2016 入試說明會


What if the space is not the one that we’ve known?


What if the space is not the one that we’ve known?


Q&A

CU on May 25th

Documents

PowerPoint Presentationbitec.snubi.org/AlphaGo(I).pdf · 2016-07-25 · 2016: An Intelligence Odyssey The Mystery of Go (WIRED, 2014.12.05)• “The Mystery of Go”, 1965, New Scientist