Lp_solve example, LP tactics, “Network reliability”, Min cost network flow, multi- commodity max flow, more on combinatorial optimization... W. D. Grover

Lp_solve example, LP tactics, “Network Lp_solve example, LP tactics, “Network reliability”, Min cost network flow, multi-reliability”, Min cost network flow, multi-

commodity max flow, more on commodity max flow, more on combinatorial optimization...combinatorial optimization...

W. D. Grover

TRLabs & University of Alberta © Wayne D. Grover 2002, 2003

E E 681 - Module 6

E E 681 - Module 6 © Wayne D. Grover 2002, 2003 2

s

jjj sCMinimize

1

.,...,2,1 Si i

P

p

pi wf

i

1

jiSji .,...,2,1),(

iP

p

pi

pjij fs

1,

S.t.

How would this be expressed as an lp_solve model... ?

Min sum of spare capacity

Restoration flows for each failure = failed working capacity

Restoration flows must be feasible under spare capacity

Revisit mesh SCP problem


Sample network for example of mesh “SCP”

A B

E

ZG

C

F

7 4

8

107 2

412

6 9

•SCP = spare capacity placement

•here, spans all have equal length or cost

•span working capacity quantities are shown

•Problem is to place spare capacity for 100% restorability via span restoration.


Sample network for example of mesh “SCP”

A B

E

ZG

C

F

7 4

8

107 2

412

6 9

• as a first step the “eligible restoration routes” for each failure are identified.

• For example, three distinct eligible routes for restoration of span A-B under 4 hops might be as shown…(examples only)

f1 f2 f3

Illustration explaining the first “restorability” constraint to follow...

Flow variables defined for A-B as a failure span


/* Objective function */min: Sab+Sbe+Seg+Sag+Saf+Sgf+Sbc+Scz+Sez+Sfz;

/* The sum of all the flows on the different routes for restoration *//* of span i must be greater or equal to wi */

f1 + f2 + f3 >= 7; /* span A-B */ (see prior slide for this one example)f4 + f5 + f6 + f7 >= 8; /* span B-E */f8 + f9 + f10 + f11 >= 12; /* span E-G */f12 + f13 + f14 >= 5; /* span A-G */f15 + f16 + f17 + f18 >= 10; /* span A-F */f19 + f20 >= 6; /* span G-F */f21 + f22 >= 4; /* span B-C */f23 + f24 >= 6; /* span C-Z */f25 + f26 + f27 + f28 >= 4; /* span E-Z */f29 + f30 + f31 + f32 >= 9; /* span F-Z */

One line per span failureThe details come from the route

identified for restoration,and the associated

flow variable for defined for each route

Lp_solve example for mesh SCP


/* The spare capacity on each span must be >= to the sum of the flows/* on routes that cross this span for every failure scenario */

/* Span A-B */Sab - f4 - f5 - f6 >= 0; /* failure of B-E */Sab - f8 - f9 >= 0; /* failure of E-G */Sab - f14 >= 0; /* failure of A-G */Sab - f17 - f18 >= 0; /* failure of A-F*/Sab - f22 >= 0; /* failure of B-C */Sab - f24 >= 0; /* failure of C-Z */Sab - f26 >= 0; /* failure of E-Z */Sab - f31 - f32 >= 0; /* failure of F-Z */

/* Span B-E */Sbe - f1 - f2 >= 0; /* failure of A-B */Sbe - f8 - f9 >= 0; /* failure of E-G */Sbe - f14 >= 0; /* failure of A-G */

....etc... (produces S sets of (S-1) additional constraints)

Lp_solve example for mesh SCP

The exact definition of eligiblerestoration routes and corresponding

flow variables (not shown) is needed to

produce this level of detail.

This shows an example of the logical form that is taken


Illustration: explaining the first spare-capacity generating constraint

A B

E

ZG

C

F

7 4

8

107 2

412

6 9

• as a first step the “eligible restoration routes” for each failure are identified.

• For example, three distinct eligible routes for restoration of span A-B under 4 hops might be as shown…(examples only)

/* Span A-B */Sab - f4 - f5 - f6 >= 0; /* failure of B-E */

…….

f1

f2

f3

F1, f2, f3 do not cross span A-B when B-E failure is considered

f4

f5 f6

f4, f5, f6 do cross span A-B when B-E failure is considered


• Mutual exclusion:- e.g. Each working demand flow may have only one route- use a “radio button” constraint:

- where is a decision variable to use route i or not.

• Peak minimizing: - e.g. Minimize the worst case “oversubscription” on all spans j

over all failures, i

- cannot write min { max {function}} in an LP / IP- can write :

min Z .... Z - zij >= 0 for every i,j

1

1n

ii

{1,0}i

Tips on formulating LP / IP’s


• Sometimes our notion of optimality depends on more than one criterion (with different units or measures)

- can form a two term objective function- “blended” by some weighting factor

e.g.:

perhaps: (Z1 ~ capacity Z2~ restoration paths length) (Z1 ~ cost Z2~ uncertainty)

• Solve family of problem instances with varying “alpha”

- result is the “Pareto optimal” solution set - shows the shape of the “trade-off” of one goal versus the other - discrete (as shown) if IP- continuous Pareto curve if LP

AB

C

Pareto solution point

Sub-optimal solution point

D

Bi-objective LP / IP’s

min { 1 (1 ) 2}Z Z


• Sometimes capacity can only be placed in discrete modules, with associated cost e.g. OC-48, OC-192

- but demand flows are still integer or even real valued - change objective function from...

Min (capacity)

to ....

Min (sum of all modules used * module cost)

s.t. Sum of (modules placed * module capacity) >= capacity needed

(see book p. 249)

Handling Modular Capacity


• A specialized field of graph theory that attempts to solve the computational problem posed by the question:

“If all each edge of a graph is in a working state with probability (1-p) and failed state with probability p, what is the probability that a route exists between “:

- nodes s,t -existence of a spanning tree- all node pairs, - etc.

0

12

3

4

56

7

89

10

111 2

131 4

0

12

3

4

56

7

89

10

111 2

131 4

0

12

3

4

56

7

89

10

111 2

131 4

0

12

3

4

56

7

89

10

111 2

131 4

(a) (b) (c) (d)

Network reliability = How likely is it that at least one route exists between nodes? (228 link state combinations to consider here)

Example;p ~ 0.32

yes noyesyes

“Network Reliability” (book p. 163)


• “Reliability polynomial”:

• 0

( ,{ , }, ) ( ,{ , }) (1 )m

i

m iiR G s t p N G s t p pi

Where:

G = graph

{s,t} = a specific node pair

p = edge failure probability

Ni() = number of edge state combinations containing exactly i routes {s,t}

m = number of edges in graph

The real problem: how toenumerate or estimate these

numbers of states that containa path between {s,t} when i edges are

failed

Prob.of exactly i failed and (m-i) working edges

“Network Reliability”


• two-terminal (s,t} reliability is a bit like a path availability assessment

• However...

- we also need to consider specific structures (e,g rings), not

just the network topology graph.

- we need to consider capacity and exact restoration mechanism

- we don’t need to consider high link failure probabilities nor high

numbers of simultaneous failures

- we can accept the numerical approximation that

“unavailibilities add for elements in series”

(i.e., Ni reduces to ~ number of elements

in normal path plus restoration considerations)

• Graph reduction, factoring and decomposition methods (p. 33) can however

be applied directly to problems of availability analysis.

“Network Reliability”(relevance to survivable transport network design )


• so far with the LP method we have looked at finding maximum flow, subject to constraining capacities.

• A related problem is to support a specified flow, between source and

target nodes at min cost for the edges used.

• Two - terminal MCNF :

minij E

x ci ij j

;

sj E jt E

x d x ds s st tj jt

S.t.

0 { { , }}ij jij i j i

x x i E s t

0 x C ij Eijij Where Cij = edge capacitiesOptionally also :

Min-cost Network flow (MCNF)

dst


If all the inputs are integer, ...

the solution will be integer even if you solveit as a Linear Program not as an Integer Program.

Maximum flow

and

Min cost network flow

are problems know to have “special structure”

also called “network structure” .....

The mathematical property is called “unimodularity”and it is an important computational advantage whenever a problem

has this special mathematical structure.

Intuitively, the solution is “trapped” onto discrete integer solution values...no operations in the formulation can fractionate the flow variables

“Unimodularity”


More formally, unimodularity requires that the matrix A formed by the constraint system of an IP formulation:

Ax < chas the properties *:

(i) every ai,j is {-1,0,1}.

(ii) each column contains at most 2 non-zero coefficients.

(iii) there exists a partition (M1,M2) of the set M rows such that the sum of coefficients in columns of M1 is equal and opposite to the corresponding

sum of coefficients M2

* Possible project : investigate this criterion further, write program to detect the “unimodular part” of any given problem

“Unimodularity”


• Multisource-sink Min cost network flow - same objective- every node has source or sink b_i quantity of “commodity” i - this canonical “transportation problem” is not quite the same as the communications version…. - It describes net flows from several factories to several distribution sites.

• Multi-commodity max flow (MCMF)- numerous source-sink pairs (each a “commodity”) are simultaneously trying to maximize flow through common set of finite edge capacities ----> Closely related to later “path restorable mesh” problem.

(a) pure max sum version(b) maximize smallest proportion of a requirement

Related network flow formulations(Chapter 4)


Classical min-cost “transportation problem”

A B

E

ZG

C

F

7 4

8

107 2

412

6 9

factories

stores

• All one commodity

Routing costs

Question is: what assignment of flows toedges supplies all stores from available factories at minimum cost?


Communications version: min-cost multi-commodity flow

A B

E

ZG

C

F

7 4

8

107 2

412

6 9

City Z (source ~factory)

stores

Routing costs

Question is: what assignment of flows toeach edge for each commodity meets all requirements at minimum routing cost?

• Each city pair is a “commodity”

• only one factory and one store for each commodity

City A (sink ~store)

A-Z commodity

C-G commodity


Communications version: min-cost multi-commodity flow

Two cases of MCMF:

A. no limits on capacities of edges….

B. finite capacities on edges.

---> shortest path (least cost) routing for each commodity is optimal (trivial problem)

---> shortest path (least cost) routing for each commodity is not optimal

• becomes “NP- hard” problem

• the difficulty is the “mutual capacity” issue

• will be at the heart of our later study of the path-restoration problem


Problems that are in:

P

NP

NP-hard

NP-complete

have the property that:

-a polynomial time algorithm exists, or can be developed, forexact solution of the problem. example: The shortest paththrough a graph (by Dijkstra’s algorithm) is O(n2)

- if a guess at the solution is provided, you can at least test itsvalidity in polynomial time. example: Is there a tour of thisgraph that is less that 20 miles?

- it is not even possible to test the validity of a postulated solu-tion in polynomial time. example: Is this specific tour of agraph the minimal tour?

-as for NP-hard but with the additional generality of scope thatevery other problem in NP is reducible to it in polynomial time.example: (Satifisability) Given a set of boolean logic expres-sions (clauses) on n binary variables, does any set of binaryvalues exist that simultaneously satisfies all clauses ?

Quick guide to computational complexity


• Integer programming (IP) is NP-hard.. and exhibits this behaviour in practise

• Linear programming (LP) (by simplex algorithm) is theoretically NP-hard but polynomial time in most cases.

• Some newer LP algorithms are provably polynomial time.

• Some Integer Programming problems are “uni-modular” - solve as an LP without losing integrality

• Other IP s or combinatorial optimization problems in general require heuristic (or meta-heuristic) solutions.

Why theoretical complexity matters


(1) Simulated Annealing (SA) (2) Genetic Algorithms (3) Tabu Search

• A “meta-heuristic” is an approximate, but highly general optimization technique that can be used as an alternative to solving LP / IP formulations.

• It is a “meta- heuristic” because the basic search / improvement tactic can be explainedin a way that is not specific to any one problem.

• Advantages may be:

- handling larger size problems

- representing problem detail not amenable to an LP / IP

- non-linear constraints or objectives

• Results not provably optimal but often this is OK in practice.

• All need quantifiable / calculable objective function

•Central issue each is trying to deal with is effecting a suitably broad search of the solution space without getting trapped in local optima

Other approaches to combinatorial optimization

“Meta-heuristics” (book: p. 259-267.)


Basic ideas / concept:

• Inspired by mathematical analogy to strain energy minimization in the slow cooling (annealing) of metals.

• Changes to the current design are made randomly.

• Improvements are kept, steps backward are normally rejected, except...

• A step “backward” (i.e., a degrading of the objective function) may be kept with a random probability.

•The probability of keeping such a change slowly decreases.

( )

E

kT tacceptp e

Where:

E = the magnitude of degradation

t = time (elapsed simulation time)

T(t) = the “cooling schedule”

k = Boltzmanns constant

Meta-heuristics: Simulated Annealing (SA)


Define a cooling function or schedule T(t) and set: t = 0, temperature T(0) := Tmax ;

while T > Tmin doFor n = 1 to Nc dopropose a change in the solution;evaluate the change in the objective function DE ; if DE > 0 , accept the moveotherwise, if rand() < Paccept[T(t), DE], accept the move

otherwise reject the move; End for

t := t+1 ( or often: T := T )

Also: keep note of global best ever found at any stage, not just current solution



• An additional resource on Simulated Annealing is at the web site:

Simulated Annealing.htm

on the course web-site (related reading section for this lecture).

From “Overview of Modern Heuristic Methods” at http://www.cs.cf.ac.uk/User/S.U.Thiel/ra/int_rep.html

(where meta-heuristics are being studied for radio frequency assignment problems).




• Inspired by concepts of natural selection and evolutionary improvement of an evolving population.

• Concepts / terms:

- a population, an individual, a generation

- genome or chromosome, encoding scheme or “schemata”

- selection, fitness function

- cross-over (‘breeding’)

- mutation

• invented / developed by John Holland

Meta-heuristics: Genetic Algorithms (GA)


• A key to use of GA is finding a suitable schemata

- must encode all relevant aspects of the design

- should permit easy evaluation of fitness function

- must lend itself to a suitable “cross-over” operator

• Crossover example:

-uniform random cross-over point

-cross-over may be followed by “chromosome repair” • Mutation:

- a small random probability of (~ 0.1% typ.) of flipping

a chromosome value (reason ? - search diversification)

1 2 3 4 5 F G H1 2 3 4 5 6 7 8

A B C D E F G H+

Crossover Point

A B C D E 6 7 8

Meta-heuristics: GA Algorithms


t := 0;

initialize a (usually random) population of individuals, P (t=0);

evaluate fitness of all individuals F{ P (t=0) };

repeat

t := t + 1;

P(t) := selectparents {F {P (t-1)}}; (selects sub-population for offspring production

- “roulette wheel” is a common

approach)

crossover on P(t); (recombine the "genes" of selected parents)

mutate P(t); (perturb the mated population stochastically)

evaluate fitness of all new individuals F{ P(t) };

select the survivors P(t) := survive (F{P(t)});

until {test for termination criterion (time, fitness, etc.)}.

Meta-heuristics: Basic GA Algorithm


• Additional resource on Genetic Algorithms:

Overview_of_GAs.pdf (white paper by Beasley et. al)

on course web-site (related reading section).

• GA s also covered in “Overview of Modern Heuristic

Methods” at http://www.cs.cf.ac.uk/User/S.U.Thiel/ra/int_rep.html

Meta-heuristics: GA Algorithms



• uses ideas of conventional “hill climbing” / gradient- type search but with addition of temporary prohibition of moves that lead back to recently discovered local optimum.

Steepest-descent

Tabu Search

Cos

t

Tabu search is forced to accept “uphill moves”

after reaching a local optimumbecause the return move is tabufor some time following discovery

of the local optimum.

Meta-heuristics: Tabu Search

Main proponent and book: Tabu Search by Fred Glover and M. Laguna


• Key to a good TS algorithm is usually quite problem-specific policies and memory structures for deciding what a “move” is and how long moves are “tabu” after leading to a local minimum.

• “Aspiration criterion” are exceptions to the tabu tenure rules for moves that happen to lead directly to large solution improvements.

• Key concepts / terms:

- a “move”

- a “neighbourhood”

- tabu moves, tabu tenure rules

- recency and frequency lists

- aspiration criteria

- search diversification

Meta-heuristics: Tabu Search


i := 0; generate initial solution, so;

repeat

generate neighbour set, N(si), and objective function values for each neighbour

identify taboo neighbor set T(si), (from current list of tabu moves);

identify aspirant subset of taboo neighbors, A(si);

move to best neighbour amongst {N(si) - T(si) + A(si)};

i := i+1 ;

update recency or frequency memories and aspiration criteria (count down tenure

of any currently tabu moves)

Until {test for termination criterion (time, objective value, etc.)}.

Meta-heuristics: Basic TS algorithm


• part of PhD studies by D. Morley (U of A 2001). •See " Tabu Search Optimization of Optical Ring Transport Networks," Globecom 2001, San Antonio, Nov. 25-29, 2001, vol.4, pp. 2160-2164.

• Two types of “move”:

- add a ring (from candidate set)- delete a ring

• Short-term tabu memory:

- if ring r1 is added, we discourage it from being dropped for a specified number of iterations, called the tabu add tenure. - if ring r2 is dropped, we discourage it from being added back to the solution for the tabu drop tenure.

• Search diversification strategy: a entirely new starting point design is generated when:

- a prior solution is revisited

- % improvement in best solution over last n iterations is too low

- absolute improvement in best solution over last n iterations is too low

A recent application of Tabu Search Application to Ring network design optimization


Start

Save solution

Exit

Alldemandserved?

Abort?

Restart?Generate new

startingsolution

Drop ring

Alldemandserved?

Add ring

Alldemandserved?

Save solution

Save solution

Yes

No

No

No

No

Yes

Yes

No

Yes

Feasibility checks:Is a routing through rings possible that serves all demand?

Progress checks- could generate diversification re-start here

Basic “move” strategy- Drop ring each iteration

- Only add ring if feasibility lost

Enter with a RingBuilderTM starting design

Tabu policy is in here- Deciding which ring add / drop

moves are not allowed (discouraged) at present

Anatomy of a tabu search

algorithm

A recent application of Tabu Search(2)


• For more on Tabu Search see: Tabu Search.htm from the

“Overview of Modern Heuristic Methods” at http://www.cs.cf.ac.uk/User/S.U.Thiel/ra/int_rep.html.

0

20

40

60

80

100

120

140

0 20 40 60 80 100

iteration

tota

l des

ign

cost

Current SolutionBest SolutionGap ExceededPrev. Solution

Understanding Tabu search : typical TS trajectory(sample result from D. Morley)

• notice how each “restart” is followed first by a climb in cost (probably due to initial bad drop moves, requiring adds for feasibility) then by settling into a new local cost-minima

Search diversification re-starts

Tabu Search

Documents

Lp_solve example, LP tactics, “Network reliability”, Min cost network flow, multi- commodity max flow, more on combinatorial optimization... W. D. Grover