Daniele Loiacono

Laboratorio di Intelligenza Artificiale e RoboticaA.A. 2008-2009

2

Daniele Loiacono

Outline

Machine LearningUnsupervised LearningSupervised LearningReinforcement Learning

Genetic AlgorithmsGenetics-Based Machine LearningApplications

Computational Intelligence and GamesMachine Learning for Embedded Systems Design

What isMachine Learning?

4

Daniele Loiacono

What is Machine Learning?

“The field of machine learning is concerned with the question of how to construct computer programs that automatically improve with experience.” Tom Mitchell (1997)

A program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.

A well-defined learning task is defined by P, T, and E.

5

Daniele Loiacono

Example: checkers

Task T: playing checkers

Artificial IntelligenceDesign and implement a computer-based system that exhibit intelligent action

Machine LearningWrite a program that can learn how to playIt can learn from examples of previous games, by playing against another opponent, by playing against itself

UnsupervisedLearning

7

Daniele Loiacono

SupervisedLearning

9

Daniele Loiacono

What is an apple?

10

Daniele Loiacono

What is an apple?

ReinforcementLearning

12

Daniele Loiacono

RL framework

Agent

Environment

...delay…...delay…

actionstate

reward

13

Daniele Loiacono

Optimal value function

The problem becomes learning how to maximize the amountof reward collected (payoff)Usually RL algorithms try to learn the action-value functionQ(s,a) and derive from it the optimal policy

Genetic Algorithms

15

Daniele Loiacono

The population

16

Daniele Loiacono

Ranking by fitness

17

Daniele Loiacono

Mate Selection

Selection probability is proportional to fitness

18

Daniele Loiacono

Crossover

Exploit goodness of

parents

19

Daniele Loiacono

Mutation

Exploreunknown

20

Daniele Loiacono

Best solution

21

Daniele Loiacono

The GA recipe

Genetic Algorithms +

Machine Learning=

Genetic Based Machine Learning

23

Daniele Loiacono

Examples of GBML

NeuroevolutionClassifiers structure and parameters optimization with GAsFeature selection with GAsRule-based Evolutionary SystemsGA-based approach for training classifiers…

24

Daniele Loiacono

GoalImplementation of a GBML approach, or extension of an available implementation, or comparison of different approaches Experimental analysisTechnical report

SkillsBasic Knowledge of EC and MLGood analysis capabilitiesGood programming skills (C++)

Students: max 2

GBML projects

Computational Intelligence

and Games

26

Daniele Loiacono

Why games?

Modern computer games are ideal benchmarks for computational intelligence techniques

InexpensiveChallenging

Computational intelligence could be useful for modern games

Speedup AI designImprovement of AIAdaptive AICustomization of game content

Unreal TournamentNeverwinter Nights

Urban CombatCivilization II

27

Daniele Loiacono

Issues in CIG

CIG is an attractive field but involves several technical issues:

InterfaceExperiments designSimulation speedCustomizationDocumentation

Typical benchmarks are specifically designed for doing researchModern computer games are very complex and with different goals

28

Daniele Loiacono

Projects available on TORCS

TORCS is an open source car racing simulatorWell suited for CIG research

Software API availableExamples availableCompetitions

Available projectsAutomated generation of tracks Visual input for TORCSC++ Learning APILearn your bot!

29

Daniele Loiacono

SmarTrack

GoalGeneration, evaluation and classification of tracks for TORCS

SkillsGood programming skills (C++)Good analysis capabilitiesBasic knowledge of EC and/or ML

Students: max 3Reference

http://cig.dei.polimi.ithttp://torcs.sourceforge.net/

30

Daniele Loiacono

EyeBot

GoalDevelop a TORCS controller based on visual inputs

SkillsSome knowledge/experience with computer vision is preferredGood programming skills (C++)

Students: max 2Reference

http://cig.dei.polimi.ithttp://torcs.sourceforge.net/

31

Daniele Loiacono

C++ Learning API

GoalDeveloping a learning framework for TORCS (partially as porting of an existing JAVA API)

SkillsBasic knowledge of EC and/or MLGood programming skills (C++)

Students: max 2Reference

http://cig.dei.polimi.ithttp://torcs.sourceforge.net/

32

Daniele Loiacono

Learn your bot!

GoalApply any evolutionary computation or machine learning techniques to develop a controller for TORCSProvide the software, the controller and a presentation

SkillsKnowledge of EC and/or MLGood analysis capabilitiesBasic programming skills (C++)

Students: max 1Reference

http://cig.dei.polimi.ithttp://torcs.sourceforge.net/

33

Daniele Loiacono

Racing Games for CIG research

GoalAnalysis of racing games for research purposesComparison of different gamesPrototype and tutorial for a specific game

SkillsGood programming skillGood analysis capabilities

Referencehttp://www.rfactor.net/http://vdrift.net/