Computer Science and Mathematical Basics

Chap. 3

발표자 : 김정집

Introduction

Fundamental notions of computer science and mathematics necessary for understanding the GP approach

leading question What are the mathematical and information-

processing contexts of GP? What are the tools from these contexts that GP has

to work with

3.1 The Importance of Randomness in Evolutionary

Learning Evolution in Nature vs. Evolution in Computer

s in nature, mutation is basically “free”

The Costs of Variation in nature, sexual reproduction is not “free”

GP as a General Search Process “non-deterministic” algorithm depend on randomness

3.2 Mathematical Basics

Randomness and Probability random events play such a prominent role in GP

3.2.1 Combinatorics and the Search Space

Permutation N different elements constituting the set E can be o

rdered in N! different permutations Combination

Variation

3.2.2 Random numbers

Quasi-random number generator Linear Congruential Method

Randomness test

X2 Test randomness test

if X2 is near to k, then the random number generator is good

3.2.3 Probability

Elementary Events random experiments - flip a coin events - “heads” or “tails”

Relative Frequency

Probability

Random Variables and Probability Distributions probability distribution p(x) of random variable x

Expectation Value and Variance moment quantity

Expectation value

Variance

Bernoulli Process and the Binomial Distribution

Probability Density Functions

Normal Distribution

Multiplicative Variation and the Log-Normal Distribution

Three distributions

3.3 Computer Science Background and

Terminology 3.3.1 The Turing Machine, Turing

Completeness, and Universal Computation

Turing Completeness a programming language allows to write a program

that emulates the behavior of a certain arbitrary TM Structure and Function of a TM

Universal TM and Universal Computation A Universal TM U can emulate any TM T U is said to be able to perform universal computation

3.3.2 The Halting Problem

Halting Theorem there is no problem that can determine the terminat

ion properties of all programs time bounded excution of GP

3.3.3 Complexity

Complexity measure # of nodes, # of bits needed to express a program in

linear form, or # of instructions Kolmogorov Complexity and Generalization

Kolmogorov Complexity “complexity of a computable object”

the shortest program that produces the object upon execution

if two programs model the same data, the shorter one can be argued to have a higher probability of being general

Different complexity measures

3.4 Computer Hardware

Von Neumann machine a computer where the program resides in the same

storage as the data used by that program

3.4.1 Von Neumann Machines

The Processor

RISC/CISC RISC(SPARC or PowerPC)

extensive use of registers

CISC(Pentium)

Schematic view of CPU

3.4.2 Evolvable Hardware

FPGAs EHW

When HW has failures, there is no need to discard the entire HW; instead one simply reprogram the chip

3.5 Computer Science

Elementary representation of software machine language, assembly language higher language data structures

3.5.1 Machine Language and Assembler Language

Machine Language A sequence of integers impractical to use numbers for instructions not natural to remember

Assembly very simple grammar

3.5.2 Higher Languages

Imperative Language BASIC, C, FORTRAN, Pascal, SIMULA program statements explicitly order (Latin imperare) the com

puter how to perform a certain task

Functional, Applicative Language LISP, LOGO, ML, BETA a program represents a function that maps input data and inter

nal data into output data using a function on its arguments is called application, so a fu

nctional language is also called applicative

Predictive Language PROLOG programming means describing to the computer

what is wanted as result Objective-Oriented Languages

SMALLTALK-80, C++, JAVA the principle behind these languages is modeling a

system by objects

Language classes

3.5.3 Data Structures

Aggregation cartesian product of structures

Generalization unites structures

Recursion Graph, Tree, List Power Set Function Space Selector

3.5.4 Manual versus Genetic Programming

From Bits to Memo Code From Assembler to High-Level Languages From High-Level Languages to Algebraic Spe

cification A Programmer’s Heuristics

“cut and paste” strategy cut and paste <-> crossover generation of new segments <-> mutation debugging and testing <-> selection unused code <-> introns

The main difference GP can afford to evolve a population of programs

simultaneously a programmer only work in this way if

the environments changed only slightly between applications or

the programming language was hard to handle

hard for GP system to generate code without any idea of what a given argument or function

could mean to the output