Fundamentals of the Analysis of Algorithm Efficiency · Fundamentals of the Analysis of Algorithm Efficiency DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ

Fundamentals of the Analysis of Algorithm Efficiency

DR. J IRABHORN CHAIWONGSAI

ดร.จิราพร ไชยวงศสายD E PA R T M E N T O F C O M P U T E R E N G I N E E R I N G

S C H O O L O F I N F O R M AT I O N A N D C O M M U N I C AT I O N T E C H N O L O GY

U N I V E R S I T Y O F P H AYA O

Correctness

Time efficiency

o How fast an algorithm in question runs

Space efficiency

o Extra space the algorithm requires

Speed can be achieve much more spectacular progress than space

Focus on time efficiency

Analysis of algorithms

2AJ. JIRABHORN CHAIWONGSAI

Measuring an Input’s Size◦ All algorithms run longer on larger inputs

◦ Ex. take longer to sort larger arrays

Unit for measuring running time◦ Time efficiency is analyzed by determining the number of

repetitions of the basic operation as a function of input size

◦ basic operation is the most important operation of the algorithm

◦ Contribute the total running time

◦ Compute the number of times that the basic operation is executed

◦ The most consuming operation in the algorithm

Analysis framework


Ex.1 Sorting algorithm

Work by comparing elements (keys) of a list being sorted with each other

Basic operation = ?

Input size = ?


Input size and basic operation examples

ProblemProblem Input sizeInput size Basic operationBasic operation

Searching for key in a list of n items

Number of list’s items, i.e. n

Key comparison

Multiplication of two matrices

Matrix dimensions or total number of elements

Multiplication of two numbers

Checking primality of a given integer n

n’size = number of digits (in binary representation)

Division

Typical graph problem #vertices and/or edgesVisiting a vertex or traversing an edge


Basic operation: the operation that contributes most towards the running time of the algorithm

T(n) ≈ copC(n)

Unit for measuring running time

running time

execution timefor basic operation

Number of times basic operation is executed

input size

Let cop be the execution time of an algorithm’s basic operation

C(n) be the number of times this operation needs to be executed for this algorithm


Why do we use “≈” ?

◦ cop is not always easy to assess

◦ How much faster would this algorithm run on a machine that is ten times faster than the one we have?

◦ Assume that a addition operation requires a cycle to execute

◦ We need 99 addition operations

◦ Compare the clock rate of a CPU between running at 1.0 GHz and 2.0 GHz

Unit for measuring running time (cont.)


What about C(n)?

Homework1Assume that

How much longer will the algorithm run if we double its input size?

Unit for measuring running time (cont.)


Order of Growth


Order of Growth (cont.)time


Order of Growth (cont.)


Best-case, Average-case, Worst-caseFor some algorithms efficiency depends on form of input:

Worst case: Cworst(n) = maximum over inputs of size n

Best case: Cbest(n) = minimum over inputs of size n

Average case: Cavg(n) = “average” over inputs of size n

◦ Number of times the basic operation will be executed on typical input

◦ NOT the average of worst and best case

◦ Expected number of basic operations considered as a random variable under some assumption about the probability distribution of all possible inputs


Worst case:

Best case:

Average case:

Example2: Sequential search


Cworst(n) = the algorithm makes the largest number of comparisons among all possible inputs of size n

For any instance of size n, the running time will not exceed Cworst(n)

Cworst(n) = ?

Example2: Sequential search (cont.)


Cbest(n) = the algorithm runs the fastest among all possible inputs of size n

The best-case efficiency is not nearly as important as the worst-case

Cbest(n) = ?



To analyze the average-case efficiency, we must make some assumption about possible inputs of size n





Cavg(n) = ?

(a) The probability of successful search =

(b) The probability of the first match occurring in the

-th position of the list is the same for every

In case of successful search, the probability of the first match occurring in the -th position of the list is

for every

In case of unsuccessful search, the number of the comparisons is with the probability


If = 1 (the search must be successful),

If (the search must be unsuccessful),


Best-case, average-case, worst-case


Average-case efficiency is considerably more difficult than the worst-case and best-case efficiencies

The average-case efficiency can’t be obtained by taking the average of the worst-case and the best-case efficiencies

Best-case, average-case, worst-case


Documents

Fundamentals of the Analysis of Algorithm Efficiency · Fundamentals of the Analysis of Algorithm Efficiency DR. JIRABHORNCHAIWONGSAI ดร.จิราพร ไชยวงศ