Upload
buicong
View
258
Download
1
Embed Size (px)
Citation preview
Mixed Integer Nonlinear Programming
IMA New Directions Short Course on Mathematical OptimizationJeff Linderoth and Jim Luedtke
Department of Industrial and Systems EngineeringUniversity of Wisconsin-Madison
August 12, 2016
Introduction
Mixed-Integer Nonlinear Optimization
Mixed Integer Nonlinear Program: (MINLP)
zminlp = minx
η
subject to gi (x) ≤ 0, i = 1, . . . ,m
f (x) ≤ ηx ∈ X , xI ∈ Z|I |
f : Rn → R, g : Rn → Rm smooth, sometimes convex functions.X ∈ Rn bounded, polyhedral set, e.g. X = {x : l ≤ AT x ≤ u}I ⊆ {1, . . . , n} subset of integer variablesWe may assume that f is a linear function.
NP-Super-Hard
Combines challenges of handling nonlinearitieswith combinatorial explosion of integer variables
Introduction
The MINLP Family
The great watershed inoptimization isn’t betweenlinearity and nonlinearity,but convexity andnonconvexity.
- R. Tyrrell Rockafellar
If f and g are convex functions,then we have a convex MINLP
If f and g are not convex, thenwe have a nonconvex MINLP
Optimization 101
Know the convexity properties of your instance!
Algorithms for two different classes of problems are different
We can solve significantly larger convex MINLP instances thatnonconvex MINLP instances
Algorithms designed for convex-MINLP are often used as heuristicsfor nonconvex MINLPs
Introduction
Specializations
Functional Form Problem Type
gi (x) = ‖Aix + bi‖2 − pTi x + qi MISOCP
gi (x) = polynomial MIPP
gi (x) = xTQix + pTi + qi MIQP
gi (x) = aTi x − bi MILP
MISOCP: Mixed Integer Second Order Cone ProgramAre convex-MINLP
MIPP: Mixed Integer Polynomial ProgramMIQP: Mixed Integer Quadratic Program
May be convex or nonconvexConvex MIQP is a special case of MISOCPIf f is convex quadratic and c is an affine mapping, then there arespecialized algorithms for convex-MIQP
MILP: Mixed Integer Linear Program
Introduction
Aside – MISOCP
The feasible regions of surprisingly many MINLP problems can beexpressed as MISOCP:
Hyperbolic constraints: Product of (non-negative) variables ≥ anorm2.
wTw ≤ xy , x ≥ 0, y ≥ 0 ⇔∥∥∥∥[ 2w
x − y
]∥∥∥∥ ≤ x + y
Convex Quadratic: ci (x) = xTQix + pTi x + qi ≤ t
Qi is positive semidefinite, so it has Cholesky factorization Qi = LTL
y = Lx ⇒ g(x) = yT y + pTi x + qi ≤ t
{(x , t) ∈ Rn+1 | xTQx + pTi x + qi ≤ t} =
Proj(x ,t){(x , y , t, p) ∈ R2n+2 | y = Lx , p + qT x + r ≤ t, p ≥ ‖y‖2}
Introduction
MINLP Tree
MINLP
NonconvexMINLP
ConvexMINLP
NonconvexNLP
ConvexNLP
PolynomialOpt.
MI-Polynomial
Opt.MISOCP SOCP
NonconvexQP
NonconvexMIQP
ConvexMIQP
Convex QP
MILP LP
Introduction
How Big Is It?
The Leyffer-Linderoth-Luedtke (LLL) Measure of Complexity
You have a problem of class X that has Y decision variables? What is thelargest value of Y for which one of Sven, Jim, and Jeff would be willingto bet $50 that a “state-of-the-art” solver could solve the problem?
Convex NonconvexProblem Class (X ) # Var (Y ) Problem Class (X ) # Var (Y )
MINLP 500 MINLP 100NLP 5× 104 NLP 100
MISOCP 1000 MIPP 150SOCP 105 PP 150MIQP 1000 MIQP 300
QP 5× 105 QP 300MILP 2× 104
LP 5× 107
Introduction Software Tools and Online Resources
Solvers for MINLP
Convex Nonconvexα-ECP α-BBBONMIN ANTIGONEDICOPT BARONFilMINT COCONUTGUROBI† COUENNEKNITRO CPLEX‡MILANO LGOMINLPBB LindoGlobalMINOPT SCIPMINOTAURSBBXpress††MIQP ‡MIQP/convex MIQP
Open-source and commercial solvers
Introduction Software Tools and Online Resources
Software for MINLP
Convex MINLP
ALPHA-ECP, Bonmin, DICOPT, FilMINT, KNITRO, MINLP-BB,SBB
(Convex) MIQP
CPLEX, GUROBI, MOSEK, XPRESS
(Nonconvex) MIQP
GLOMIQO
General Global Optimization
BARON, Couenne, LINDOGlobal, SCIP
Try ’em on NEOS
http://www.neos-server.org/neos/solvers/index.html
Introduction Basic Concepts
Simple Example
(x1 − 2)2 + (x2 + 1)2 ≤ 36
3x1 − x2 ≤ 6
−2.5(x1 − 1)2 − x2 ≤ −4
0 ≤ x1, x2 ≤ 5
x2 ∈ Z
x1
x2
Doubly NP-Hard!
Feasible region is not convex for two reasons—Integrality of x2 andnonconvexity of constraint
Introduction Basic Concepts
Q: What to Do?!
Relax!
Remove some constraints.Solution of relaxed problem provide a lower bound on optimal solutionvalue
Introduction Basic Concepts
Natural Relaxation
“Natural” relaxation is toremove constraints xi ∈ Z ∀i ∈ I
But this still leaves a nonconvexNLP relaxation
x1
x2
Introduction Basic Concepts
Relax More
We need a mechanism forrelaxing the nonconvexconstraints.
Secant underestimator forconcave functions
“Easy” to get polyhedralouterapproximation ofnonconvex constraints
Ideally, we would like to get theconvex hull of the feasible points
x1
x2
Introduction Basic Concepts
Linear Objective Is Important Here!
min(x1 − 1/2)2 + (x2 − 1/2)2
s.t. x1 ∈ {0, 1}, x2 ∈ {0, 1}
η ≥ (x1 − 1/2)2 + (x2 − 1/2)2
x1
x2
(x1, x2)
η
Without linear objective, optimal solution may be interior to theconvex hull ⇒ convexifying may do you no good!
Introduction Basic Concepts
Algorithm Ingredients
1 Relaxation
Used to compute a lower bound onthe optimum valueObtained by enlarging feasible set; e.g.ignore constraintsShould be “tractable” and “tight”
2 Constraint Enforcement
Exclude solutions from relaxations not feasible to MINLPRefine or tighten of relaxation—Branching and Cutting
3 Upper Bounds
Obtained from any feasible point; e.g. solve NLP for fixed xI .(NLP(xI )).Other heuristic techniques have been developed
Modeling and Applications
Models are Important!
Writing “good” models is extremelyimportant in MILP
Probably more important for MINLP
More Complexity
Tradeoff between physical accuracy and mathematical tractability
Convex reformulations? Linearize expressions?
MINLP Modeling Preference
We want strong formulations—Relaxations that can be relaxed tosomething that closely approximates the convex hull
We prefer linear over convex over nonconvex formulations
Modeling and Applications
Some MINLP Applications
1 Supply Chain—Convex MINLPDiscrete: Fixed charges for opening facilitiesNonlinear: Nonlinear transportation costs
2 Gas/Water Network Design—Nonconvex MINLPDiscrete: Pipe connections/sizesNonlinear: Pressure Loss
3 Pooling/Petrochemical—Nonconvex MINLPDiscrete: Which process to use?Nonlinear: Product Blending (among others)
Luedtke Theorem
97.5% of all practical MINLPs have integer variables for one of twopurposes...
1 Binary variables to make a “multiple choice” selection2 Binary indicator variables that turn on/off continuous variables
and/or constraints.
Modeling and Applications Supply Chain
Uncapacitated Facility Location
Problem introduced by Gunluk, Lee,and Weismantel (’07) and classes ofstrong cutting planes derived
M: Facility
N: Customer
xij : percentage of customer j ∈ N demand met by facility i ∈ M
zi = 1⇔ facility i ∈ M is built
Fixed cost for opening facility i ∈ M
Quadratic cost for meeting demand j ∈ N from facility i ∈ M
Modeling and Applications Supply Chain
Quadratic Uncapacitated Facility Location Problem
A very simple MIQP
z∗def= min
∑i∈M
cizi +∑i∈M
∑j∈N
qijx2ij
subject to
xij ≤ zi ∀i ∈ M,∀j ∈ N∑i∈M
xij = 1 ∀j ∈ N
xij ≥ 0 ∀i ∈ M, ∀j ∈ N
zi ∈ {0, 1} ∀i ∈ M
Modeling and Applications Supply Chain
Base Case of Induction to “Luedtke Theorem”
Note that binary variables are used as indicators: zi = 0⇒ xij = 0
If z = 1, then we need to model the epigraph of x2ij
Intager Programmerz ’r’ Smrt
Mixed Integer Linear Programmerscarefully study simple problemstructures to come up with“good” formulations for problems
Good formulations closelyapproximate convex hull offeasible solutions
Important to understand thestructure of a special MINLP withindicator variables
Modeling and Applications Supply Chain
A Very Simple Structure
Rdef={
(x , y , z) ∈ R2 × B | y ≥ x2, 0 ≤ x ≤ uz}
z = 0⇒ x = 0, y ≥ 0
z = 1⇒ x ≤ u, y ≥ x2
x
y
z = 1
z
y ≥ x2
Deep Insights
conv(R) ≡ line connecting (0, 0, 0) to y = x2 in the z = 1 plane
Modeling and Applications Supply Chain
Characterization of Convex Hull
Deep Theorem #1
R ={
(x , y , z) ∈ R2 × B | y ≥ x2, 0 ≤ x ≤ uz}
conv(R) ={
(x , y , z) ∈ R3 | yz ≥ x2, 0 ≤ x ≤ uz , 0 ≤ z ≤ 1, y ≥ 0}
x2 ≤ yz , y , z ≥ 0 ≡
Second Order Cone Programming
There are effective, robust algorithms for optimizing over conv(R)
Modeling and Applications Supply Chain
In General—Giving You Some Perspective
For a convex function f : Rn → R, the perspective functionP : Rn+1 → R of f is
P(x , z)def=
{0 if z = 0zf (x/z) if z > 0
The epigraph of P(x , z) is a cone pointed at the origin whose lowershape is f (x)
Exploiting Your Perspective
If zi is an indicator that the (nonlinear, convex) inequalityf (x) ≤ 0 must hold, (otherwise x = 0), replace the inequality withits perspective version:
zi f (x/zi ) ≤ 0
The resulting (convex) inequality is a much tighter relaxation ofthe feasible region.
Modeling and Applications Supply Chain
Perspectivizing Quadratic Uncapacitated Facility Location
z∗def= min
∑i∈M
cizi +∑i∈M
∑j∈N
qijx2ijyij
xij ≤ zi ∀i ∈ M, ∀j ∈ N∑i∈M
xij = 1 ∀j ∈ N
xij ≥ 0 ∀i ∈ M,∀j ∈ N
zi ∈ {0, 1} ∀i ∈ M
x2ij − ziyij ≤ 0 ∀i ∈ M,∀j ∈ N
Steps
1 Make ObjectiveLinear
2 ApplyPerspective
3 Make GiantProfit
Modeling and Applications Supply Chain
Strength of Relaxations
zR : Value of NLP relaxation
zGLW : Value of NLP relaxation after GLW cuts
zP : Value of perspective relaxation
z∗: Optimal solution value
|M| |N| zR zGLW zP z∗
10 30 140.6 326.4 346.5 348.715 50 141.3 312.2 380.0 384.120 65 122.5 248.7 288.9 289.325 80 121.3 260.1 314.8 315.830 100 128.0 327.0 391.7 393.2
Cheers!
Modeling and Applications Supply Chain
Impact of SOCP
m = 30, n = 100
Convex MINLP, Original: 16697 CPU seconds, 45901 nodes
Convex MINLP, w/GLW Ineq.: 21206 CPU seconds, 29277 nodes
MISOCP, Perspective, 23 CPU seconds, 44 B&B nodes
Larger Instances
m n T N
30 200 141.9 6340 100 76.4 5440 200 101.3 4550 100 61.6 4950 200 140.4 47
“Poifect”
Modeling and Applications Water Network Design
Design of Water Distribution Networks
Goal: Design minimum costnetwork from discrete pipediameters to meet waterdemand
Sets
N nodes in network
S source nodes
A: arcs in the network
K: Pipe diameters
Modeling and Applications Water Network Design
Design of Water Distribution Networks
Variables
qij : flow pipe (i , j) ∈ Ahi : hydraulic head at node i ∈ Ndij : diameter of pipe (i , j) ∈ A, where dij ∈ {P1, . . . ,Pr}aij : area of cross section (i , j) ∈ Ayijk : Binary variable = 1 if pipe of size k ∈ K chosen for (i , j) ∈ A
Constraints
Conservation of flow∑(i ,j)∈A
qij −∑
(j ,i)∈A
qji = Di , ∀i ∈ N \ S.
Flow bounds: (linear in a, but since aij = 0.25πd2ij , nonlinear in d)
−Vmaxaij ≤ qij ≤ Vmaxaij , ∀(i , j) ∈ A.
Modeling and Applications Water Network Design
Design of Water Distribution Networks
Discrete pipe sizes dij ∈ {P1, . . . ,Pr}Use “multiple choice” integrality to remove nonlinearity aij = πd2
ij/4
This is in general a good modeling practice
1 Binary variables yijk ∈ {0, 1} for k ∈ K2 Model discrete choice as∑
k∈Kyijk = 1, and
∑k∈K
Pkyijk = dij . ∀(i , j) ∈ A
3 Model aij = πd2ij/4 (nonlinear) as
aij =∑k∈K
(π4
Pk
)yijk ∀(i , j) ∈ A
Modeling and Applications Water Network Design
Design of Water Distribution Networks
Nonsmooth, nonlinear pressure loss as a function of flow qij along arc(i , j) ∈ A and diameter dij :
hi − hj = κijd−c1ij sgn(qij)|qij |c2 ∀(i , j) ∈ A
Another trick: disaggregate flow into flow variables for each pipe size:
qij =∑k∈K
qijk
(−Vmaxaij)yijk ≤ qijk ≤ (Vmaxaij)yijk
Modeling and Applications Water Network Design
Why On Earth?
Doing this allows us to write the(nonlinear) headloss as afunction of a single variable qijk
Often in applications, thenonlinearity is low-dimensional(low ∈ {1, 2})Therefore a piecewise linearapproximation (or relaxation)may be sufficiently accurate
Luedtke’s Theorem!
Note also thatyijk = 0⇒ qijk = 0
q
f (q) = sgn(q)|q|c2
Modeling and Applications Water Network Design
Modeling Piecewise-Linear Functions
Take Your Pick
SOS2 Model [Beale and Tomlin, 1970, Beale and Forrest, 1976]
Incremental Model [Markowitz and Manne, 1957]
Multiple Choice Model [Jeroslow and Lowe, 1984]
Convex Combination Model [Dantzig, 1960, Padberg, 2000]
Disaggregated Convex Combination Model [Meyer, 1976]
Logarithmic Model [Vielma and Nemhauser, 2011]
Modeling and Applications Water Network Design
Multiple Choice Model
Want to model (∀i)
f (q) = miq + ci , q ∈ [Bi−1,Bi ]
Introduce New Variables
wi : = q if q ∈ [Bi−1,Bi ]
bi : = 1 if q ∈ [Bi−1,Bi ]
t ≈ f (q)
q
f (q) ≈ sgn(q)|q|c2
B0 B1 B2
B3 B4
q =n∑
i=1
wi ,
t =n∑
i=1
(miwi + cibi )
Bi−1bi ≤ wi ≤ Bibi ∀i ∈ 1, . . . , n
1 =n∑
i=1
bi
B0y ≤ q ≤ Bny
Modeling and Applications Water Network Design
A Simple Trick
Instead of modeling the piecewise-linear indicator in the standard way:
B0y ≤ q ≤ Bny , 1 =n∑
i=1
bi
Model it as
y =n∑
i=1
bi
Resulting formulation is provably stronger (locally-ideal)
All piecewise-linear functions turned on/off by an indicator have asimilar modeling trick [Sridhar et al., 2013]
It is the exact extension of the “perspective reformulation” to thisnonconvex case
Modeling and Applications Pooling
The Pooling Problem
The Pooling Problem
A canonical problem in the petrochemical industry
Decide how to blend (pool) feeds together to make productsmeeting “quality requirements” at minimum cost
Feeds I Pools J Products K
Generally each product has multiple attributes
Modeling and Applications Pooling
Pooling Problem
Basic variables
xij : Flow of input i to pool j
xik : Flow of input i to product k
xjk : Flow from pool j to product k
Key Parameters
λi : Concentration of attribute in feed i
`k , υk : Bounds on attribute concentration in product k
Ci ,Cj ,Ck : Capacities on feeds, pools, and outputs
Uij ,Uik ,Ukj : Flow capacities (some may be zero)
Simple (Linear) Constraints
Linear Constraints: Flow balance, bounds on flows,feed/pool/output capacity
Modeling and Applications Pooling
P-formulation
(For example) Input capacity:∑`∈L
xi` +∑j∈J
xij ≤ Ci ∀i ∈ I
Flow Balance: ∑i∈I
xi` =∑j∈J
x`j ∀` ∈ L
Pool Quality: ∑i∈I
λixi` = p`∑i∈I
xi` ∀` ∈ L
Product Quality:∑`∈L
p`x`j +∑i∈I
λixij = pj
(∑`∈L
x`j +∑i∈I
xij
)∀j ∈ J
`j ≤ pj ≤ υj ∀j ∈ J
Modeling and Applications Pooling
Modeling the Quality Constraints
Q-formulation [Ben-Tal et al., 1994]
New variables qij : proportion of flow to pool j coming from feed i :∑i∈I
qij = 1
qij satisfy the relations:
qij =xij∑t∈I xtj
⇔ xij = qij
∑t∈I
xtj
Reformulate using flow balance at pool j :∑
t∈I xtj =∑
k∈K xjk
xij = qij
∑k∈K
xjk
Concentration of attribute at pool i is then:∑
i∈I λiqij
Modeling and Applications Pooling
Modeling the Quality Constraints (2)
Concentration of attribute at pool i :∑
i∈I λiqij
Quality constraint on product k:
`k ≤∑
i∈I λixik +∑
j∈J∑
i∈I λiqijxjk∑i∈I xik +
∑j∈J xjk
≤ υk
Multiply and gather terms (e.g., for upper bound):∑i∈I
(λi − υk)xik +∑j∈J
∑i∈I
(λi − υk)qijxjk ≤ 0
Introduce variables wijk to represent bilinear terms qijxjk :
⇒ xij = qij
∑k∈K
xjk =∑k∈K
wijk
Modeling and Applications Pooling
Modeling the Quality Constraints (3)
Bringing it all together∑i∈I
qij = 1, j ∈ J
xij =∑k∈K
wijk , i ∈ I , j ∈ J∑i∈I
(λi − υk)xik +∑j∈J
∑i∈I
(λi − υk)wijk ≤ 0, k ∈ K
∑i∈I
(`k − λi )xik +∑j∈J
∑i∈I
(`k − λi )wijk ≥ 0 k ∈ K
wijk = qijxjk , i ∈ I , j ∈ J, k ∈ K
Modeling and Applications Pooling
PQ-Formulation [Tawarmalani and Sahinidis, 2002]
1 For each j ∈ J, multiply the constraint∑
i∈I qij = 1 with xjk :∑i∈I
qijxjk = xjk
2 For each j ∈ J, multiply the constraint:∑
k∈K xjk ≤ Cj with qij :∑k∈K
qijxjk ≤ Cjqij
3 Replace qijxjk with wijk :∑i∈I
wijk = xjk ,∑k∈K
wijk ≤ Cjqj
Much better relaxations!
Takeaway: Adding redundant constraints can significantly improve relax-ations of nonconvex MINLPs.
Modeling and Applications Pooling
The Pooling Problem with Binary Variables
Arcs connecting the input streams toother nodes may be used only bypaying a fixed cost.
Network Design
Ship assignment
Add binary variables zij for these arcs:
xij ≤ Uijzij
A real nonconvex, integer problem.
Modeling and Applications Pooling
Practical Advice About the Luedtke Theorem
We have seen many cases where we have continuous variable0 ≤ x ≤ U and associated binary variable z ∈ {0, 1}.We see many models that write the expression w = zx .
On The Horror!
Never multiply a binary variable by abounded continuous variable
You can linearize this:
0 ≤ w ≤ Uz
−U(1− z) ≤ x − w ≤ U(1− z)
Modeling and Applications Pooling
Other MINLP Applications
1 Portfolio Management—Convex MIQP
Discrete: Trading StrategyNonlinear: Utility
2 Reactor Core Reloading—Nonconvex MINLP
Discrete: Bundle ReorderingNonlinear: Neutron Transport
3 Power Grid Operation—Nonconvex MINLP
Discrete: Unit CommittmentNonlinear: AC Power Flow
4 Building Co-Generation—Nonconvex MINLP
Discrete: Unit CommittmentNonlinear: Ramping Constraints
5 Oil-Spill Response—Nonconvex MINLP
Discrete: Resource AllocationNonlinear: Fluid Flow & Chemistry
2. Core-Reloading
4. Building Demand
5. Oil-Spill Response
Algorithms – Convex
Algorithms for Convex MINLP
Algorithms for convex MINLP are very similar to those for MILP
NLP-based branch-and-bound
Solve NLP relaxation at each node
Cutting plane based algorithms:
Use linear inequalities to “outer approximate” continuous relaxation
Solve LP relaxations instead of NLP relaxations
Update LP relaxations by adding cuts
Benders decomposition [Geoffrion, 1972], Outer approximation[Duran and Grossmann, 1986], LP/NLP branch-and-bound[Quesada and Grossmann, 1992], Extended cutting plane[Westerlund and Pettersson, 1995], Algorithmic refinements, e.g.[Abhishek et al., 2010, Bonami et al., 2008]
Algorithms – Convex
Valid Inequalities for Convex MINLP
Many classes of inequalities for MILP have been extended/studied forconvex MINLP
Disjunctive/split cuts[Ceria and Soares, 1999, Stubbs and Mehrotra, 1999,Belotti et al., 2012, Bonami, 2011, Modaresi et al., 2014b,Kılınc et al., 2010, Kılınc-Karzan and Yildiz, 2014]
Chvatal-Gomory rounding and mixed-integer rounding for conicMINLP [Cezik and Iyengar, 2005, Drewes, 2009]
Conic mixed-integer rounding [Atamturk and Narayanan, 2010]
Intersection cuts [Andersen and Jensen, 2013, Modaresi et al., 2014a]
Minimal valid inequalities and cut generating functions[Kılınc-Karzan, 2013, Moran R. et al., 2012]
Closures [Dadush et al., 2011b, Dey and Moran R., 2012,Dadush et al., 2011a]
Algorithms – Nonconvex Constructing Relaxations of Nonconvex Constraints
General Approach to Nonconvex MINLP
minimizex
f (x) subject to g(x) ≤ 0, x ∈ X , xi ∈ Z ∀ i ∈ I
Use our old trick: convex (often polyhedral) relaxation!
Relax integrality as before: xi ∈ R ∀ i ∈ I
New: R(Θ) ⊇ Θ = {x ∈ X : g(x) ≤ 0}Ensure relaxation is tractable: e.g. R(Θ) is convex
Algorithms – Nonconvex Constructing Relaxations of Nonconvex Constraints
Factorable Functions and MINLP
Consider MINLP with nonconvex, factorable f (x) and g(x)
minimizex
f (x) subject to g(x) ≤ 0, x ∈ X , xi ∈ Z ∀ i ∈ I
Factorable Functions
g(x) is factorable if it can be expressed as a combination of functionsfrom finite set of operators O = {+,×, /, , sin, cos, exp, log, | · |} whosearguments are variables, constants, or other factorable functions.
Excludes integrals∫ xξ=x0
h(ξ)dξ and black-box functions
Represented as expression trees
Algorithms – Nonconvex Constructing Relaxations of Nonconvex Constraints
Expression Tree Example
+
log
^
31
2
2x x
x
*
Expression tree of f (x1, x2) = x1 log(x2) + x32
Modeling languages (e.g. AMPL, GAMS) have expression tree “API”
Algorithms – Nonconvex Constructing Relaxations of Nonconvex Constraints
Reformulation of Factorable MINLP
Reformulate factorable MINLP as
minimizex
xn+q
subject to xk = ϑk(x) k = n + 1, n + 2, . . . , n + qli ≤ xi ≤ ui i = 1, 2, . . . , n + qx ∈ X ,xi ∈ Z, ∀i ∈ I ,
see e.g. [Smith and Pantelides, 1997]
q new auxiliary variables, xn+1, . . . , xn+q
ϑk is operator from O{+,×, /, , sin, cos, exp, log}
Algorithms – Nonconvex Constructing Relaxations of Nonconvex Constraints
Example of Reformulation of Factorable MINLP
min x1 + x1x22
s.t. x1x2 + sin x2 ≤ 4,x1 ∈ [−4, 4] ∩ Z,x2 ∈ [0, 10] ∩ Z.
Reformulation
min x7s.t. x3 = sin x2 −4 ≤ x1 ≤ 4 0 ≤ x5 ≤ 45
x4 = x1x2 0 ≤ x2 ≤ 10 −400 ≤ x6 ≤ 400x5 = 4− x3 − x4 −1 ≤ x3 ≤ 1 −404 ≤ x7 ≤ 404x6 = x2x4 −40 ≤ x4 ≤ 40x7 = x1 + x6 x1, x2, x4, x6, x7 ∈ Z.
Factorization is not unique!Integrality inherited from function and integrality on argumentsBounds inherited from function and argument bounds
Algorithms – Nonconvex Constructing Relaxations of Nonconvex Constraints
Reformulation of Factorable MINLP
Factorable form allows systematic construction of convex relaxation:
Nonconvex sets, k = n + 1, n + 2, . . . , n + q
Θk = {x ∈ Rn+q : xk = ϑk(x), x ∈ X , l ≤ x ≤ u, xi ∈ Z, i ∈ I}
Let R(Θk) ⊃ Θk be a convex relaxation
minimize
xxn+q
subject to x ∈ R(Θk) k = n + 1, n + 2, . . . , n + qli ≤ xi ≤ ui i = 1, 2, . . . , n + qx ∈ X .
... convex relaxation ... only look at simple sets!
Algorithms – Nonconvex Constructing Relaxations of Nonconvex Constraints
Examples of Relaxations
Construct relaxation for each operator ∈ O{+,×, /, , sin, cos, exp, log}Odd-degree monomials, xk = x2p+1
i , see[Liberti and Pantelides, 2003]
Bilinear functions xk = xixj , [McCormick, 1976]Let x = (xi , xj), L = (li , lj),U = (ui , uj)get convex hull of Θk = {(x , xk) : xk = xixj , L ≤ x ≤ U}:
xk ≥ ljxi + lixj − li lj xk ≤ ljxi + uixj − ui ljxk ≥ ujxi + uixj − uiuj xk ≤ ujxi + lixj − liuj
Important!
Tightness of convex hull depends on bounds li , lj , ui , uj
Algorithms – Nonconvex Constructing Relaxations of Nonconvex Constraints
Examples of Relaxations
Polyhedral relaxation, R(Θk), of xk = x3i and xk = xixj
l
u
xi
i
i
xk
x
k
j
i
x
x
Algorithms – Nonconvex Constructing Relaxations of Nonconvex Constraints
Examples of Relaxations
Polyhedral relaxation, R(Θk), of xk = x2i with xi continuous/integer
k
i
i
x
x
uli
i
liui
kx
x
... if xi ∈ Z then add inequalities violated at x ′i 6∈ Z
Algorithms – Nonconvex Spatial Branch-and-Bound
Spatial Branch-and-Bound (BnB)
To enforce constraints violated by relaxation solution use spatial BnB
Implicit enumeration technique like integer BnB
Recursively define partitions of feasible set into two sets by changingcontinuous and integer variable bounds
Solve LP (or convex NLP) relaxations (⇒ lower bounds)... and nonconvex NLPs (⇒ upper bound if feasible)
Lower bounding problem at node with bounds (l , u), e.g. LP(l , u)minimize
xxn+q
subject to x ∈ R(Θk) k = n + 1, n + 2, . . . , n + qli ≤ xi ≤ ui i = 1, 2, . . . , n + qx ∈ X .
Algorithms – Nonconvex Spatial Branch-and-Bound
Spatial Branch-and-Bound (BnB)
If LP(l , u) infeasible, then prune node.
Otherwise, x optimal solution of LP(l , u):
If x integer feasible and feasible in NLP(l , u) (hence MINLP), thenfathom node (new incumbent)
Otherwise ... branch ...
Two possible ways to branch (integer / nonlinear):
1 x not integral: xi ≤ bxic ∨ xi ≥ dxie like integer BnB2 ∃k : xk 6= ϑk(x) nonlinear infeasible:
Choose branching variable xi from arguments of ϑk(x)Branch xi ≤ xi ∨ xi ≥ xi ... two subproblemsRefine convex relaxation in each branch ... tighter bounds
Algorithms – Nonconvex Spatial Branch-and-Bound
Branching for Spatial Branch-and-Bound
Branching example xk = ϑk(xi ) = (xi )2 violated
x
^^ji
j
i
ii ul
(x ,x )
x
(x ,x )
x
x
^^ji
j
i
ii b ul
Algorithms – Nonconvex Spatial Branch-and-Bound
Branching for Spatial Branch-and-Bound
Branching based on xi ≤ b ∨ xi ≥ b
Good performance depends on good choice of i and b
Goal: Increase both bounds LP(l−, u−) and LP(l+, u+)
Strong branching, pseudocost branching, and reliability branchinggeneralized from convex MINLP
Other techniques (e.g., violation transfer) specific to nonconvexMINLP
Finding continuous branching candidates xi :
xi not fixed in parent problem
xi is argument of violated function xk 6= ϑk(x)
Algorithms – Nonconvex Tightening Bounds and Relaxations
Tightening Bounds and Relaxations
Bound tightening to reduce range of bounds xi ∈ [li , ui ]... because tighter bounds ⇒ tighter relaxations ⇒ smaller trees
Spatial BnB + Bound tightening = Branch-and-reduce[Ryoo and Sahinidis, 1996, Belotti et al., 2009]
Conceptual bound-tightening procedure:Feasible set F = {x ∈ [l , u] : g(x) ≤ 0, x ∈ X , xI ∈ Zp}Solve 2n (global) optimization problems, given upper bound U:
l ′i = min{xi : x ∈ F , f (x) ≤ U}; u′i = max{xi : x ∈ F , f (x) ≤ U}.
... nonconvex MINLPs just as hard ⇒ use relaxations:
1 FBBT: feasibility-based bound tightening
2 OBBT: optimality-based bound tightening
Algorithms – Nonconvex Tightening Bounds and Relaxations
FBBT: Feasibility-Based Bound Tightening
FBBT broadly used:
Artificial intelligence community & constraint programming
NLP solvers [Messine, 2004]
MILP solvers [Savelsbergh, 1994]
Basic Principle of FBBT
Infer bounds on xi from tighter bounds on xj for j 6= i .
Example: xj = x3i and xi ∈ [li , ui ]
Tighten interval of xj to [lj , uj ] ∩ [l3i , u3i ]
Tightened l ′j on xj ⇒ tighter l ′i = 3√
lj for xi
FBBT is efficient and fast, but convergence can be slow
Algorithms – Nonconvex Tightening Bounds and Relaxations
OBBT: Optimality-Based Bound Tightening
Solving min /max xi s.t. x ∈ F (nonconvex MINLP) not practicalInstead, use current relaxation
F(l , u) =
x ∈ Rn+q :x ∈ R(Θ)k k = n + 1, n + 2, . . . , n + qli ≤ xi ≤ ui i = 1, 2, . . . , n + qx ∈ X
.
Now get bounds on xi for i = 1, . . . , n by solving 2n LPs:
l ′i = min{xi : x ∈ F(l , u)}u′i = max{xi : x ∈ F(l , u)}
... only apply at root node, or small number of nodes
Algorithms – Nonconvex Exploiting Nonconvex Structures
The Trouble with Factorable Relaxations
Factorable relaxations are primarily based on local structure ⇒ Potentiallyweak bounds
Example: Θ = {(x1, x2) : x21 + x2
2 ≥ 1, 0 ≤ xi ≤ 2}
Convex hull: x1 + x2 ≥ 1Best possible factorable relaxation:
x3 = x21 ⇒ (1/2)x3 ≤ x1
x4 = x22 ⇒ (1/2)x4 ≤ x2
x3 + x4 ≥ 1⇒x1 + x2 ≥ (1/2)(x3 + x4) ≥ 1/2
Remedy: Study relaxations of more global structures
Algorithms – Nonconvex Exploiting Nonconvex Structures
A Special Structure: Nonconvex Quadratic Functions
Quadratically constrained quadratic program (QCQP)minimize
xxTQ0x + cT
0 x ,
subject to xTQkx + cTk x ≤ bk , k = 1, . . . , q
Ax ≤ b,0 ≤ x ≤ u, xi ∈ Z, ∀i ∈ I ,
Qk not necessarily convex ⇒ nonconvex problem
Problem is NP-hard even if all variables are continuousMany important special cases: Bilinear programming, poolingproblem, integer linear programming, max cut,. . .
Extensively studied: E.g.,
[Poljak and Wolkowicz, 1995, Sherali and Adams, 1998, Yajima and Fujie, 1998,
Burer and Vandenbussche, 2009, Burer, 2009, Burer and Letchford, 2009,
Burer and Letchford, 2012, Vandenbussche and Nemhauser, 2005,
Saxena et al., 2008, Misener and Floudas, 2012, Buchheim and Wiegele, 2012]
Algorithms – Nonconvex Exploiting Nonconvex Structures
General Nonconvex Quadratic Functions
Equivalent reformulation of QCQP: introduce Xij for all i , j pairs
minimizex ,X
Q0 • X + cT0 x ,
subject to Qk • X + cTk x ≤ bk , k = 1, . . . , q
Ax ≤ b,0 ≤ x ≤ u, xi ∈ Z, ∀i ∈ I ,X = xxT ,
where X = [Xij ] matrix, Qk • X =∑
ij [Qk ]ijXij =∑
ij [Qk ]ijxixj .
X = xxT represents nonconvex constraint Xij = xixj... otherwise problem is linear!
Relaxing equality X = xxT gives convex (linear) relaxation
We show two approaches: RLT and SDP
Algorithms – Nonconvex Exploiting Nonconvex Structures
Reformulation-Linearization Technique (RLT)[Adams and Sherali, 1986]
Begin with standard polyhedral relaxation of Xij = xixj :
Xij ≥ 0, Xij ≥ uixj + ujxi − uiuj , Xij ≤ ujxi , Xij ≤ uixj
Strengthen using more structure
1 Exploiting binary variables: xi ∈ {0, 1} ⇒ x2i = xi
Add linear constraints Xii = xi
2 Multiply linear constraints (e.g., PQ formulation for pooling)Multiplying xi ≥ 0 and bt −
∑j atjxj ≥ 0 gives
btxi −∑j
atjxixj ≥ 0
Replace xixj with Xij yields linear inequality
btxi −∑j
atjXij ≥ 0
Algorithms – Nonconvex Exploiting Nonconvex Structures
Semidefinite Programming
Semidefinite Programming (SDP) relaxations of QCQPs
1 Relax X − xxT = 0 to X − xxT � 0
2
X − xxT � 0 ⇔[
1 xT
x X
]� 0
3 Improve by adding Xii = xi for binary xi ∈ {0, 1}4 Additional constraints by squaring & linearizing constraints
... here H � 0 means H positive semidefinite (xTHx ≥ 0, ∀x)
Both RLT and SDP provide good relaxations ... can also combine them
The Future of MINLP?
Some Open Research Question in MINLP
A very biased view of open research in MINLP
Strong relaxations for nonlinear structures
Implementation & software design issues
Open questions from previous slide
Going beyond traditional MINLP
Black-box functions or simulations ...unrelaxable variables
Partial-differential equation constraints
Bilevel problems, e.g., leader-follower games
Stochastic/robust optimization
Make impact in important applications
The Future of MINLP?
Take Home Messages
Modeling is extremely important for MINLP... strong relaxations are key
Effective techniques from MILP useful for MINLP... more work can be done!
MINLP software maturing,... but lags behind commercial MILP software
Linear better than convex better than nonconvex
Exploit structure whenever possible
The Future of MINLP?
Abhishek, K., Leyffer, S., and Linderoth, J. T. (2010).FilMINT: An outer-approximation-based solver for nonlinear mixedinteger programs.INFORMS Journal on Computing, 22:555–567.DOI:10.1287/ijoc.1090.0373.
Adams, W. and Sherali, H. (1986).A tight linearization and an algorithm for zero-one quadraticprogramming problems.Management Science, 32(10):1274–1290.
Andersen, K. and Jensen, A. N. (2013).Intersection cuts for mixed integer conic quadratic sets.pages 37–48.
Atamturk, A. and Narayanan, V. (2010).Conic mixed-integer rounding cuts.Mathematical Programming A, 122(1):1–20.
Beale, E. and Tomlin, J. (1970).
The Future of MINLP?
Special facilities in a general mathematical programming system fornon- convex problems using ordered sets of variables.In Lawrence, J., editor, Proceedings of the 5th InternationalConference on Operations Research, pages 447–454, Venice, Italy.
Beale, E. M. L. and Forrest, J. J. H. (1976).Global optimization using special ordered sets.Mathematical Programming, 10:52–69.
Belotti, P., Goez, J., Polik, I., Ralphs, T., and Terlaky, T. (2012).A conic representation of the convex hull of disjunctive sets and coniccuts for integer second order cone optimization.Technical report, n. 12T-009, Lehigh University, Department ofIndustrial and Systems Engineering.http:
//www.optimization-online.org/DB_FILE/2012/06/3494.pdf.
Belotti, P., Lee, J., Liberti, L., Margot, F., and Wachter, A. (2009).Branching and bounds tightening techniques for non-convex MINLP.Optimization Methods and Software, 24(4-5):597–634.
The Future of MINLP?
Ben-Tal, A., Eiger, G., and Gershovitz, V. (1994).Global minimization by reducing the duality gap.Mathematical Programming, 63(1):193–212.
Bonami, P. (2011).Lift-and-project cuts for mixed integer convex programs.In Gunluk, O. and Woeginger, G., editors, Integer Programming andCombinatoral Optimization, volume 6655 of Lecture Notes inComputer Science, pages 52–64. Springer, Berlin.
Bonami, P., Biegler, L., Conn, A., Cornuejols, G., Grossmann, I.,Laird, C., Lee, J., Lodi, A., Margot, F., Sawaya, N., and Wachter, A.(2008).An algorithmic framework for convex mixed integer nonlinearprograms.Discrete Optimization, 5(2):186–204.
Buchheim, C. and Wiegele, A. (2012).
The Future of MINLP?
Semidefinite relaxations for non-convex quadratic mixed-integerprogramming.Mathematical Programming, pages 1–18.10.1007/s10107-012-0534-y.
Burer, S. (2009).On the copositive representation of binary and continuous nonconvexquadratic programs.Mathematical Programming, 120:479–495.
Burer, S. and Letchford, A. (2009).On nonconvex quadratic programming with box constraints.SIAM Journal on Optimization, 20(2):1073 – 89.
Burer, S. and Letchford, A. (2012).Unbounded convex sets for non-convex mixed-integer quadraticprogramming.Mathematical Programming, pages 1–26.10.1007/s10107-012-0609-9.
The Future of MINLP?
Burer, S. and Vandenbussche, D. (2009).Globally solving box-constrained nonconvex quadratic programs withsemidefinite-based finite branch-and-bound.Computational Optimization and Applications, 43(2):181 – 195.
Ceria, S. and Soares, J. (1999).Convex programming for disjunctive optimization.Mathematical Programming, 86:595–614.
Cezik, M. T. and Iyengar, G. (2005).Cuts for mixed 0-1 conic programming.Mathematical Programming, 104:179–202.
Dadush, D., Dey, S., and Vielma, J. P. (2011a).The split closure of a strictly convex body.Operations Research Letters, 39(2):121 – 126.
Dadush, D., Dey, S. S., and Vielma, J. P. (2011b).The chvatal-gomory closure of a strictly convex body.Mathematics of Operations Research, 36(2):227 – 239.
The Future of MINLP?
Dantzig, G. B. (1960).On the significance of solving linear programming problems with someinteger variables.Econometrica, 28:30–44.
Dey, S. S. and Moran R., D. A. (2012).Some properties of convex hulls of integer points contained in generalconvex sets.Mathematical Programming, pages 1 – 20.
Drewes, S. (2009).Mixed Integer Second Order Cone Programming.PhD thesis, Technische Universitat Darmstadt.
Duran, M. A. and Grossmann, I. (1986).An outer-approximation algorithm for a class of mixed-integernonlinear programs.Mathematical Programming, 36:307–339.
Geoffrion, A. M. (1972).
The Future of MINLP?
Generalized Benders decomposition.Journal of Optimization Theory and Applications, 10(4):237–260.
Jeroslow, R. and Lowe, J. (1984).Modelling with integer variables.Mathematical Programming Studies, 22:167–84.
Kılınc, M., Linderoth, J., and Luedtke, J. (2010).Effective separation of disjunctive cuts for convex mixed integernonlinear programs.Technical Report 1681, Computer Sciences Department, University ofWisconsin-Madison.
Kılınc-Karzan, F. (2013).On minimal valid inequalities for mixed integer conic programs.
Kılınc-Karzan, F. and Yildiz, S. (2014).Two-term disjunctions on the second-order cone.
Liberti, L. and Pantelides, C. C. (2003).Convex envelopes of monomials of odd degree.
The Future of MINLP?
Journal of Global Optimization, 25(2):157–168.
Markowitz, H. M. and Manne, A. S. (1957).On the solution of discrete programming problems.Econometrica, 25(1):84–110.
McCormick, G. P. (1976).Computability of global solutions to factorable nonconvex programs:Part I — Convex underestimating problems.Mathematical Programming, 10:147–175.
Messine, F. (2004).Deterministic global optimization using interval constraint propagationtechniques.RAIRO-RO, 38(4):277–294.
Meyer, R. (1976).Mixed integer minimization models for piecewise-linear functions of asingle variable.Discrete Mathematics, 16(2):163–71.
The Future of MINLP?
Misener, R. and Floudas, C. (2012).GloMIQO: Global mixed-integer quadratic optimizer.Journal of Global Optimization, pages 1–48.
Modaresi, S., Kılınc, M., and Vielma, J. (2014a).Intersection cuts for nonlinear integer programming: convexificationtechniques for structured sets.
Modaresi, S., Kılınc, M., and Vielma, J. (2014b).Split cuts and extended formulations for mixed integer conic quadraticprogramming.
Moran R., D. A., Dey, S. S., and Vielma, J. P. (2012).A strong dual for conic mixed-integer programs.SIAM Journal on Optimization, 22(3):1136 – 1150.
Padberg, M. (2000).Approximating separable nonlinear functions via mixed zero-oneprograms.Operations Research Letters, 27(1):1–5.
The Future of MINLP?
Poljak, S. and Wolkowicz, H. (1995).Convex relaxations of (0-1) quadratic programming.Mathematics of Operations Research, 20:550–561.
Quesada, I. and Grossmann, I. E. (1992).An LP/NLP based branch-and-bound algorithm for convex MINLPoptimization problems.Computers & Chemical Engineering, 16:937–947.
Ryoo, H. S. and Sahinidis, N. V. (1996).A branch-and-reduce approach to global optimization.Journal of Global Optimization, 8:107–139.
Savelsbergh, M. W. P. (1994).Preprocessing and probing techniques for mixed integer programmingproblems.ORSA Journal on Computing, 6:445–454.
Saxena, A., Bonami, P., and Lee, J. (2008).
The Future of MINLP?
Disjunctive cuts for non-convex mixed integer quadraticallyconstrained programs.Manuscript, available on Optimization Online,(www.optimization-online.org).
Sherali, H. and Adams, W. (1998).A Reformulation-Linearization Technique for Solving Discrete andContinuous Nonconvex Problems.Kluwer, Dordrecht.
Smith, E. M. B. and Pantelides, C. C. (1997).Global optimization of nonconvex MINLPs.Computers & Chemical Engineering, 21:S791–S796.
Sridhar, S., Linderoth, J., and Luedtke, J. (2013).Locally ideal formulations for piecewise linear functions with indicatorvariables.Operations Research Letters, 41:627–632.
Stubbs, R. and Mehrotra, S. (1999).
The Future of MINLP?
A branch-and-cut method for 0-1 mixed convex programming.Mathematical Programming, 86:515–532.
Tawarmalani, M. and Sahinidis, N. V. (2002).Convexification and Global Optimization in Continuous andMixed-Integer Nonlinear Programming: Theory, Algorithms, Software,and Applications.Kluwer Academic Publishers, Boston MA.
Vandenbussche, D. and Nemhauser, G. L. (2005).A polyhedral study of nonconvex quadratic programs with boxconstraints.Mathematical Programming, 102:531–557.
Vielma, J. P. and Nemhauser, G. (2011).Modeling disjunctive constraints with a logarithmic number of binaryvariables and constraints.Mathematical Programming, 128(1):49–72.
Westerlund, T. and Pettersson, F. (1995).
The Future of MINLP?
A cutting plane method for solving convex MINLP problems.Computers & Chemical Engineering, 19:s131–s136.
Yajima, Y. and Fujie, T. (1998).Polyhedral approach for nonconvex quadratic programming problemswith box constraints.Journal of Global Optimization, 13(2):151 – 170.