Knowledge acquisition for ada-tive game AI

Marc Ponsen et al.

Science of Computer programming

vol. 67, pp. 59-75, 2007

장수형

Outline

• Introduction• Related work• Adaptive Script of Wargus• Experiment• Result• Alternative method

2 /24

Introduction

• Game– Become increasingly realistic– Graphical presentation – Capabilities of characters ‘living’

• Game AI– Game developers

• Encompass techniques such as pathfinding, animation, collision physics

– Academic researchers• Intelligent behavior

– Inferior quality• Benefit from academic research into commercial games

3 /24

Introduction

• Adaptive game AI– Behavior of computer-controlled opponents– Potentially increase the quality of game AI– Incorporate a sufficient amount of correct prior domain

knowledge• Dynamic scripting

– Offline reinforcement learning technique– Dynamic scripting in a real-time strategy game called

Wargus– Ambitious performance task– The quality of the knowledge base is essential

4 /24

Introduction

• Knowledge base– Manually encode

• Take a long time• Sub-optimal due to analysis • Not generate satisfying result

– Semi-automatically• Increase the performance• Machine learning• Added to knowledge bases• Evolution algorithm

– Automatically• Evolutionary algorithm• Automatically transfers the domain knowledge

5 /24

Related work

• Few studies exist on learning to win complex strategy games

• Focusing on simpler tasks– Relational Markov decision process model to some lim-

ited Wargus scenarios(Guestrin et al.)– Case-bases plan recognition approach for assisting War-

gus player(Cheng and Thawonmas)• Manual knowledge acquisition

– Typical RTS games(Age of Empires and Command & Cun-quer)

• Semi-automatic knowledge acquisition– Pattern recognition technique(Street et al.)

• Automatic knowledge acquisition– Neural network for Backgammon, GO, Chess(Kirby)

6 /24

RTS games

• Usually focus on military combat• Control armies and defeat all opposing forces that are situated in a virtual battlefiled(often called a map) in real-time

• Collecting and managing resources• Determines all decision for a computer opponent over the course of the whole game

– Form of scripts which are list of game action that are ex-ecuted sequentially

– Constricting buildings, researching new technologies, and combat

7 /24

Wargus

• Clone of the popular RTS game Warcraft II

• Open source• Stratagus engine• Strategy

– Small Balanced Land Attack– Large Balanced Land Attack– Soldier’s Rush– Knight’s Rush

8 /24

Complexity of Wargus

• No single tactic dominates all others– The rock-paper-scissors principle

• Large action space– The set of possible actions that can be executed at a par-

ticular moment• In Wargus…

– A : number of assignments workers can perform– P : average number of workplace– T : number of troops– D : Average number of directions that a unit can moves– S : number of choices for a troop’s stance– B : number of buildings– R : average number of choices for research objectives at

a building– C : average number of choice of units to create at a build-

ing9 /24

Complexity of Wargus

10 /24

Complexity of Wargus

• Decision complex of each state– – Higher than the average number of possible moves in

many board game such as chess(30)

11 /24

Dynamic Scripting for Wargus

• Game AI for complex games is mostly defines in scripts

– Contain weaknesses, which human players can exploit– Dynamic script

• Introduced by Spronck et al.• Ability to adapt to a human player’s behavior• The probability that a tactics is selected for a script is an increasing

function of its associated weight value

– Requirements• The game AI can be scripted• Domain knowledge on the characteristics of a successful script can be

collected• Evaluation function can be designed to assess the success of the func-

tion’s execution

12 /24

Dynamic Scripting for Wargus

• Divide the game into a small number of distinct game states

• Each state corresponds to a unique knowledge base

13 /24

Weight adaptation in Wargus

• F : The overall fitness• Fi : the stats fitness(state i)• Sd : the score for the dynamic player• So : the score for the player’s opponent

14 /24

Weight adaptation in Wargus

• Sx : the score of the dynamic player state x• Mx : the military points for player x• Bx : building points for player x

15 /24

EA(Fitness Function)

• Md : Military points for the dynamic player• Mo : Military points for the dynamic player’s opponent• b : break-even point• Ct : game cycle• Cmax : maximum game cycle(the longest time a game is allowed to continue)

16 /24

EA(Encoding)

• Construct, research, economy, combat genes..

17 /24

Performance evaluation

• Dynamic scripting under three condition– Manually acquired– Semi-automatically acquired– Automatically acquired

• The other is controlled by a static script• Four strategy

– SBLA, LBLA, SR, KR• Randomization turning point

– Number of the first game in which the dynamic player statistically outperforms the static player

– A low RTP value indicates good efficiency

18 /24

Result

19 /24

Conclusions

• Three alternative for acquiring high-quality domain knowledge used by adaptive game AI

– Manual, semi-automatic, automatic• Discussed dynamic scripting• Domain knowledge is crucial factor to the perfor-mance of dynamic scripting

• The automatic knowledge acquisition approach takes best performance

20 /24

Alternative method

• Alternative method of script handling– Bayesian Network

• Case study : StarCraft• ‘Adaptive Reasoning Mechanism with Uncertain Knowledge for Improving Performance of Artificial In-telligence in StarCraft

21 /24

상성파악

• 전략과 유닛의 상성 파악

22 /24

베이지안 네트워크 설계

• 불확실한 지식정보– 상대방 진영으로의 정찰 시도– 지어진 건물들의 구성– 생산한 유닛의 구성– 건물과 유닛의 개수– 위의 정보들을 얻어낸 시각

• 거짓정보는 아니지만 완벽한 정보도 아니다– 숨겨진 유닛 , 숨겨진 건물 , 지어지다가 취소된 건물

23 /24

스크립트 선택

• 정보 추론 후 가장 효과적인 대응 스크립트 선택

24 /24

결과

• 실험결과

25 /26

E.N.D