Lecture 7

Engineering Engineering Engineering Engineering Systems Systems Systems Systems Design and Analysis Design and Analysis Design and Analysis Design and Analysis (ENG (ENG (ENG (ENG 504504504504))))

/ /

Email: [email protected] / [email protected]

Webpage: http://www.staff.zu.edu.eg/amhm/

Lecture Lecture Lecture Lecture 7777

Dr. A. Helmi Eng. Sys. Design and Analysis (ENG 504) Zagazig Univ-Eng. Faculty 2013/2014Eng. Sys. Design and Analysis (ENG 504)

1In the previous lecturesIn the previous lecturesIn the previous lecturesIn the previous lectures

Expert systems

Propositional logic

Predicate logic

Fuzzy logic


2AgendaAgendaAgendaAgenda

Expert systems

Artificial Neural Networks

Real & ArtificialReal & ArtificialReal & ArtificialReal & ArtificialReal & ArtificialReal & ArtificialReal & ArtificialReal & ArtificialNeuronsNeuronsNeuronsNeuronsNeuronsNeuronsNeuronsNeurons


3Real NeuronsReal NeuronsReal NeuronsReal NeuronsReal NeuronsReal NeuronsReal NeuronsReal Neurons

Human brain contains a bout 1011 neurons.

Neurons can receive, process and transmit electrochemical signals over neural

pathways that make up the brains communication system.

Structure of a neuron:

Cellular body: called soma

Dendrites: branches conduct information into a cell

Axon: branches conduct information (reaction) out of the cell

Action potential: is an activation of a neuron.

A spike: an emitted signal from a neuron and characterized by frequency,

duration and amplitude.



A Neuron has many inputs but only one output.

Synapse is the connection between neurons.

The brain consists of different types of neurons with different functionality.


5Artificial NeuronsArtificial NeuronsArtificial NeuronsArtificial NeuronsArtificial NeuronsArtificial NeuronsArtificial NeuronsArtificial Neurons

Inputs

Outputw2

w1

w3

wn

wn-1.

..

x1

x2

x3

xn-1

xn

y)(;1

zHyxwzn

iii ==

=

The McCullogh-Pitts model

Binary inputs.

Binary output.

Fixed activation threshold.


6

Parameters:

1. Input connections with weights.

2. Input function: the aggregate net input signal: z=f(x,w)

3. Activation signal: calculates the activation level: a = s(z)

4. Output function: calculates the output signal: y = H(a)

Artificial NeuronsArtificial NeuronsArtificial NeuronsArtificial NeuronsArtificial NeuronsArtificial NeuronsArtificial NeuronsArtificial Neurons


7

Input and output values can be:

Binary: {0,1}

Bivalent: {-1,1}

Continuous: [0,1]

Discrete numbers in a defined interval.



8

Most used activation functions, a=s(z):


uea

+=

11


9Structure of Neural NetworksStructure of Neural NetworksStructure of Neural NetworksStructure of Neural NetworksStructure of Neural NetworksStructure of Neural NetworksStructure of Neural NetworksStructure of Neural Networks

Inputs Output

Input layer

Hidden layer

Output layer

This is the feed-forward style.

There is also the feed-back style.


10Neural NetworksNeural NetworksNeural NetworksNeural NetworksNeural NetworksNeural NetworksNeural NetworksNeural Networks

An artificial neural network is a computational model defined by four parameters:

1. Type of neurons.

2. Connectionist architecture.

3. Learning algorithm.

4. Recall algorithm.

Notice that:

The weight matrix represents the "knowledge", the long-term memory of

the system.

The activation of the neurons represents the current state,

the short-term memory.

Learning of NeuralLearning of NeuralLearning of NeuralLearning of NeuralLearning of NeuralLearning of NeuralLearning of NeuralLearning of NeuralNetworksNetworksNetworksNetworksNetworksNetworksNetworksNetworks


11

The brain's ability to analyze complex problems and to react adequately to

unfamiliar situations is due to its heuristic faculty of taking decisions on the

basis of previously stored knowledge and its ability to adapt to new situations.

The human brain has the ability to learn and to generalize (recall).

The information we accumulate as a result of our learning is stored in the

synapses in the form of concentrated chemical substances.

Real NeuronsReal NeuronsReal NeuronsReal NeuronsReal NeuronsReal NeuronsReal NeuronsReal Neurons



Learning

is achieved in the brain through the process of chemical change

in synaptic connections.

Recall

activates a collection of neurons in time.


13Artificial Neural Networks Artificial Neural Networks Artificial Neural Networks Artificial Neural Networks Artificial Neural Networks Artificial Neural Networks Artificial Neural Networks Artificial Neural Networks

A neural network has to be trained: we apply a set of input vectors,

then we change the weights until the desired output obtained.

Three categories of learning algorithms:

1. Supervised: Training is performed until the neural network "learns" to

associate each input vector x to its corresponding and desired output

vector y. (with a teacher)

2. Unsupervised: Only input vectors are supplied, then the neural network

learns some internal features of the whole set of all the input vectors.

3. Reinforcement: is a combination of the above two techniques.

is called also reward-penalty learning or learning with a critic (without

teacher).

Artificial Neural NetworksArtificial Neural NetworksArtificial Neural NetworksArtificial Neural NetworksArtificial Neural NetworksArtificial Neural NetworksArtificial Neural NetworksArtificial Neural Networks

as a problemas a problemas a problemas a problemas a problemas a problemas a problemas a problem--------solving solving solving solving solving solving solving solving paradigmparadigmparadigmparadigmparadigmparadigmparadigmparadigm


14Solving problems paradigmSolving problems paradigmSolving problems paradigmSolving problems paradigmSolving problems paradigmSolving problems paradigmSolving problems paradigmSolving problems paradigm

Tasks to be solved by artificial neural networks:

Controlling the movements of a robot based on self-perception and other

information (e.g., visual information).

Predicting where a moving object goes, when a robot wants to catch it.

Deciding the category of potential food items

(e.g., edible or non-edible) in an artificial world;

Image/speech recognition.

Weather forecasting.


15Solving problems paradigmSolving problems paradigmSolving problems paradigmSolving problems paradigmSolving problems paradigmSolving problems paradigmSolving problems paradigmSolving problems paradigm

The problem-solving process, when using neural networks, comprises two phases:


16Connectionist Expert SystemsConnectionist Expert SystemsConnectionist Expert SystemsConnectionist Expert SystemsConnectionist Expert SystemsConnectionist Expert SystemsConnectionist Expert SystemsConnectionist Expert Systems

System ExampleSystem ExampleSystem ExampleSystem ExampleSystem ExampleSystem ExampleSystem ExampleSystem Example


17Optical Character Optical Character Optical Character Optical Character Optical Character Optical Character Optical Character Optical Character Recognition (OCR)Recognition (OCR)Recognition (OCR)Recognition (OCR)Recognition (OCR)Recognition (OCR)Recognition (OCR)Recognition (OCR)

A sample set of handwritten characters:


18OCROCROCROCROCROCROCROCR

Two ANNs schemes for OCR.

Results (around 1996) for isolated

characters:

98% for digits.

96% for uppercase characters.

87% for lowercase characters.

ThanksThanks

Documents

Lecture 7