Basics on Decision Making

Motivation Phases Implementation

Artificial IntelligenceGraph theory

G. Guérard

Department of Nouvelles EnergiesEcole Supérieure d’Ingénieurs Léonard de Vinci

Lecture 0

GG | A.I. 1/26


OutlineOutlineOutlineOutlineOutlineOutlineOutlineOutlineOutlineOutlineOutlineOutlineOutlineOutlineOutlineOutlineOutline

1 Motivation

2 PhasesInformation gathering phaseDesign phaseChoice phase

3 ImplementationDefinitionImplementationParadigmAlgorithm’s efficiency

GG | A.I. 2/26


What is decision making ?What is decision making ?What is decision making ?What is decision making ?What is decision making ?What is decision making ?What is decision making ?What is decision making ?What is decision making ?What is decision making ?What is decision making ?What is decision making ?What is decision making ?What is decision making ?What is decision making ?What is decision making ?What is decision making ?

MotivationDecision making is a process of choosing among two or morealternate courses of action for the purpose of attaining a goal (orgoals).

How to choose between production’s strategies ?How to find a shortest path in a traffic ?Etc.

GG | A.I. 3/26


Decision Making PhasesDecision Making PhasesDecision Making PhasesDecision Making PhasesDecision Making PhasesDecision Making PhasesDecision Making PhasesDecision Making PhasesDecision Making PhasesDecision Making PhasesDecision Making PhasesDecision Making PhasesDecision Making PhasesDecision Making PhasesDecision Making PhasesDecision Making PhasesDecision Making Phases

Decision making involves three major phases followed by thevalidation phase:

Information gathering,Design,Choice,Implementation,Validation.

To examine and to identify the problem (variables, functions, val-ues, etc.).

GG | A.I. 4/26





To construct a model that represents the system.

GG | A.I. 4/26





To select a solution to the model.

GG | A.I. 4/26





To implemente an informatic software to solve any instance of themodel.

GG | A.I. 4/26





To validate the software if it shows reasonable computing time andgood solution.

GG | A.I. 4/26


Information gathering phaseInformation gathering phaseInformation gathering phaseInformation gathering phaseInformation gathering phaseInformation gathering phaseInformation gathering phaseInformation gathering phaseInformation gathering phaseInformation gathering phaseInformation gathering phaseInformation gathering phaseInformation gathering phaseInformation gathering phaseInformation gathering phaseInformation gathering phaseInformation gathering phase

Information gathering includes several activities aimed at identifyingproblem situations or opportunities :

1 Problem identification2 Problem classification3 Problem decomposition

Identification of organized goals and objectives related to an issueof concern.

GG | A.I. 5/26





To determine whether a problem exists, identify its symptoms, de-termine its magnitude, and explicitly define it.

GG | A.I. 5/26





Some issues may arise during data collection and estimation likelack of data, inoccurate or inprecise data, bad estimation, informationoverload, only simulated data, etc.

GG | A.I. 5/26





Problem classification is the placement of a problem in a definablecategory. This leads to a standard solution approach.

GG | A.I. 5/26





Many complex problems can be divided into sub-problems.Some unstructured problems may have some highly structured sub-problems.

GG | A.I. 5/26


Design phaseDesign phaseDesign phaseDesign phaseDesign phaseDesign phaseDesign phaseDesign phaseDesign phaseDesign phaseDesign phaseDesign phaseDesign phaseDesign phaseDesign phaseDesign phaseDesign phase

The design phase involves finding and analyzing possible coursesof action :

1 To understand the problem2 A model is constructed, tested, and validated

Modelling involves abstracting the problem to quantitative and/orqualitative forms.

GG | A.I. 6/26





For a mathematical model, the variables are identified and the rela-tionships among them are established.

GG | A.I. 6/26





All models are made up of three basic components: decision vari-ables, uncontrollable variables, and result variables. Mathamticalrelationships link these components together.

GG | A.I. 6/26





In a non-quantitative model, the relationships are symbolic or quali-tative.

GG | A.I. 6/26



All models are made up of three basic components :

1 Result variables2 Decision variables3 Uncontrollable variables

Result variables are outputs. The reflect the level of effectivenessof the system. These are dependent variables.

GG | A.I. 7/26





They describe alternative courses of action.

GG | A.I. 7/26





They are factors that affect the result variables but are not undercontrol of the decision maker. Either these factors are fixed or theycan vary.

GG | A.I. 7/26



Structure of quantitative modelsThe components are linked together by mathematicalexpressions.

Principle of ChoiceA principle of choice is a criterion that describes the acceptabilityof a solution approach

GG | A.I. 8/26



Normative modelsThe chosen solution is demonstrably the best of all possiblealternatives. To find it, one should examine all alternatives andprove that the one selected is indeed the best one. The process isbasically Optimization.

Descriptive modelsThey investigate alternate courses of action under differentconfigurations of inputs and processes. All the alternatives are notchecked, only a given set are.

GG | A.I. 9/26


Choice phaseChoice phaseChoice phaseChoice phaseChoice phaseChoice phaseChoice phaseChoice phaseChoice phaseChoice phaseChoice phaseChoice phaseChoice phaseChoice phaseChoice phaseChoice phaseChoice phase

The choice phase includes:Recommendation of a solutionEvaluation of a solution

When the problem is solved, a solution is chosen based on themodel . This one is recommended over the set of solutions.

GG | A.I. 10/26




Evaluation is possible if the result variables give a quantitative so-lution. A preference relation over the set of solution give an order:f (a) � f (b) means that f (b) is better or equal to f (a) where a, bare variables and f (.) is a mathematical function based on resultvariables.

GG | A.I. 10/26




b is the best solution iff: f (x)� f (b) for any x in the set of solution.

GG | A.I. 10/26


AlgorithmsAlgorithmsAlgorithmsAlgorithmsAlgorithmsAlgorithmsAlgorithmsAlgorithmsAlgorithmsAlgorithmsAlgorithmsAlgorithmsAlgorithmsAlgorithmsAlgorithmsAlgorithmsAlgorithms

An algorithm is a self-contained step-by-step set of operations to beperformed.

An algorithm is an effective method that can be expressed withina finite amount of space and time and in a well-defined formallanguage for calculating a function. Starting from an initial stateand initial input (perhaps empty), the instructions describe acomputation that, when executed, proceeds through a finite numberof well-defined successive states, eventually producing output andterminating at a final ending state. The transition from one stateto the next is not necessarily deterministic; some algorithms, knownas randomized algorithms, incorporate random input.

GG | A.I. 11/26


FlowchartFlowchartFlowchartFlowchartFlowchartFlowchartFlowchartFlowchartFlowchartFlowchartFlowchartFlowchartFlowchartFlowchartFlowchartFlowchartFlowchart

GG | A.I. 12/26


PseudocodePseudocodePseudocodePseudocodePseudocodePseudocodePseudocodePseudocodePseudocodePseudocodePseudocodePseudocodePseudocodePseudocodePseudocodePseudocodePseudocode

GG | A.I. 13/26


Classification by implementationClassification by implementationClassification by implementationClassification by implementationClassification by implementationClassification by implementationClassification by implementationClassification by implementationClassification by implementationClassification by implementationClassification by implementationClassification by implementationClassification by implementationClassification by implementationClassification by implementationClassification by implementationClassification by implementation

1 Classic, iterative implementation2 Recursion3 Logical4 Deterministic or non-deterministic5 Exact or approximate6 Serial, parallel or distributed

Iterative algorithms use repetitive constructs like loops and some-times additional data structures like stacks to solve the given prob-lems.

GG | A.I. 14/26




A recursive algorithm is one that makes reference to itself repeat-edly until a termination condition matches.

GG | A.I. 14/26




A logical algorithm is composed by logic component and controlcomponent.

GG | A.I. 14/26




Deterministic algorithms solve the problem with exact decision atevery step of the algorithm whereas non-deterministic algorithmssolve problems via guessing . A deterministic algorithm is an algo-rithm which, given a particular input, will always produce the sameoutput.

GG | A.I. 14/26




While many algorithms reach an exact solution, approximation algo-rithms seek an approximation that is close to the true solution.When it is not possible to find a best solution in human time, onecan seek for an approximate solution by less complex algorithms.

GG | A.I. 14/26




Those implementations depend of computer architectures, i.e. num-ber of processors that can work on a problem at the same time, andhow to decompose the whole process into independent processes.

GG | A.I. 14/26


RecursionRecursionRecursionRecursionRecursionRecursionRecursionRecursionRecursionRecursionRecursionRecursionRecursionRecursionRecursionRecursionRecursion

A recursive algorithm is one that makes reference to itselfrepeatedly until a termination condition matches.

GG | A.I. 15/26


LogicalLogicalLogicalLogicalLogicalLogicalLogicalLogicalLogicalLogicalLogicalLogicalLogicalLogicalLogicalLogicalLogical

A logical algorithm is composed by logic component andcontrol component.

The logic component expresses the axioms that may be used in thecomputation and the control component determines the way inwhich deduction is applied to the axioms. This is the basis for thelogic programming paradigm. In pure logic programming languagesthe control component is fixed and algorithms are specified bysupplying only the logic component.

GG | A.I. 16/26


Logical: prologLogical: prologLogical: prologLogical: prologLogical: prologLogical: prologLogical: prologLogical: prologLogical: prologLogical: prologLogical: prologLogical: prologLogical: prologLogical: prologLogical: prologLogical: prologLogical: prolog

Basics: man(adam). man(peter). man(paul). woman(marry).woman(eve).parent(adam,peter). parent(eve,peter). parent(adam,paul).parent(marry,paul).

Controls: father(F,C):-man(F),parent(F,C).mother(M,C):-woman(M),parent(M,C). is_father(F):-father(F,_).is_mother(M):-mother(M,_).

Question: ?-is_father(adam).

GG | A.I. 17/26


Classification by paradigmClassification by paradigmClassification by paradigmClassification by paradigmClassification by paradigmClassification by paradigmClassification by paradigmClassification by paradigmClassification by paradigmClassification by paradigmClassification by paradigmClassification by paradigmClassification by paradigmClassification by paradigmClassification by paradigmClassification by paradigmClassification by paradigm

1 Brute-force2 Divide and conquer3 Dynamic programming4 Search5 Randomized6 Reduction

Naive algorithms try every possible solution to see which is best.

GG | A.I. 18/26




A divide and conquer algorithm repeatedly reduces an instance ofa problem to one or more smaller instances of the same problem untilthe instances are small enough to solve easily.

GG | A.I. 18/26




A problem that can be solved optimally by breaking it into sub-problems and then recursively finding the optimal solutions to thesub-problems is said to have optimal substructure.

GG | A.I. 18/26




Many problems can be modeled as problems on graphs. A graphexploration algorithm specifies rules for moving around a graph andis useful for such problems.

GG | A.I. 18/26




Such algorithms make some choices randomly . They can be veryuseful in finding approximate solutions for problems where findingexact solutions can be impractical.

GG | A.I. 18/26




This technique involves solving a difficult problem by transformingit into a better known problem for which we have optimal algo-rithms. The goal is to find a reducing algorithm whose complexity isnot dominated by the resulting reduced algorithm’s.

GG | A.I. 18/26


Divide and conquerDivide and conquerDivide and conquerDivide and conquerDivide and conquerDivide and conquerDivide and conquerDivide and conquerDivide and conquerDivide and conquerDivide and conquerDivide and conquerDivide and conquerDivide and conquerDivide and conquerDivide and conquerDivide and conquer

GG | A.I. 19/26


Dynamic programmingDynamic programmingDynamic programmingDynamic programmingDynamic programmingDynamic programmingDynamic programmingDynamic programmingDynamic programmingDynamic programmingDynamic programmingDynamic programmingDynamic programmingDynamic programmingDynamic programmingDynamic programmingDynamic programming

The longest common subsequence (LCS) problem is the problem offinding the longest subsequence common to all sequences in a setof sequences.LCS (Xi ,Yj) =

if i = 0 or j = 0LCS (Xi−1,Yj−1)_ xi if xi = yj

longest(LCS (Xi ,Yj−1) ,LCS (Xi−1,Yj)) if xi 6= yj

GG | A.I. 20/26


ReductionReductionReductionReductionReductionReductionReductionReductionReductionReductionReductionReductionReductionReductionReductionReductionReduction

We tranform a bipartite matching problem into a flow problem :

GG | A.I. 21/26


BasicsBasicsBasicsBasicsBasicsBasicsBasicsBasicsBasicsBasicsBasicsBasicsBasicsBasicsBasicsBasicsBasics

TerminationTo determine whether the evaluation of a given program willdefinitely terminate

CorrectnessAn input give a solution of the problem.

CompletenessFor a feasible domain, the algorithm give a solution.

GG | A.I. 22/26


ExampleExampleExampleExampleExampleExampleExampleExampleExampleExampleExampleExampleExampleExampleExampleExampleExample

Termination: in all condition, the algorithm returns a value.Correctness: for an input n, the algorithm returns a factorial.Completeness: n is an integer.

GG | A.I. 23/26


ComplexityComplexityComplexityComplexityComplexityComplexityComplexityComplexityComplexityComplexityComplexityComplexityComplexityComplexityComplexityComplexityComplexity

The best, worst and average case complexity refer to three different ways ofmeasuring the time complexity (or any other complexity measure) of different inputsof the same size. Since some inputs of size n may be faster to solve than others, wedefine the following complexities:

Best-case complexityThis is the complexity of solving the problem for the best input of size n.

Worst-case complexityThis is the complexity of solving the problem for the worst input of size n.

Average-case complexityThis is the complexity of solving the problem on an average. This complexity is onlydefined with respect to a probability distribution over the inputs. For instance, if allinputs of the same size are assumed to be equally likely to appear, the average casecomplexity can be defined with respect to the uniform distribution over all inputs ofsize n.

GG | A.I. 24/26


Complexity (time)Complexity (time)Complexity (time)Complexity (time)Complexity (time)Complexity (time)Complexity (time)Complexity (time)Complexity (time)Complexity (time)Complexity (time)Complexity (time)Complexity (time)Complexity (time)Complexity (time)Complexity (time)Complexity (time)

Notation Examples

O(1) Constant: condition, elementary computingO(log(n)) Logarithm: tree, binary search

O(n) Linear: for, while, etc.O(n ∗ log(n)) Quasilinear: quicksort, mergesort, tree search

O(n²) Quadratic: multiplying two n-digit numbers, sort, matrixO(np) Polynomial: grammar, matching, reductionO(2n) Exponential: brute-force, search treeO(pn) Exponential: dynamic programming, logical brute-forceO(n!) Factorial: naive combinatory algorithm

GG | A.I. 25/26


ExampleExampleExampleExampleExampleExampleExampleExampleExampleExampleExampleExampleExampleExampleExampleExampleExample

Bubble sort: sort an array of n numbers by switching twofollowing indexes.

The best time complexity is when the algorithm do one loop:O(n).The worst time complexity is when the algorithm do n loop:O(n2).In average, the algorithm is quadratric : O(n2).Data are stored in an array, data are reached directly by areference in memory. Data complexity is: O(1).

GG | A.I. 26/26

Science

Basics on Decision Making