View
3
Download
1
Category
Preview:
Citation preview
Spectrum SlicingSLEPc
Evaluation
Strategies for Spectrum Slicing Based onRestarted Lanczos Methods
Carmen Campos and Jose E. RomanUniversitat Politecnica de Valencia, Spain
SC2011
Spectrum SlicingSLEPc
Evaluation
Goal
Context: symmetric-definite generalized eigenvalue problem
Ax = λBx B ≥ 0
Eigenvalues are real: λ1 ≤ λ2 ≤ . . . ≤ λnVery large, sparse matrices → iterative solvers, parallel computing,small part of the spectrum
Computational interval
I In many applications: structures, electromagnetism, etc.
I All eigenvalues in a given interval [a, b] (a or b must be finite)
I Do not miss eigenvalues (could be 1000’s)
I Determine multiplicity correctly (could be as high as 400)
Spectrum SlicingSLEPc
Evaluation
Outline
1 Spectrum SlicingRelated workProposed variants
2 SLEPcOverview of SLEPcImplementation of Spectrum Slicing
3 Evaluation
Spectrum SlicingSLEPc
Evaluation
Spectral Transformation
The spectral transformation [Ericsson & Ruhe 1980] enablesLanczos methods to compute interior eigenvalues
Ax = λBx =⇒ (A− σB)−1Bx = θx
I Trivial mapping of eigenvalues: θ = (λ− σ)−1
I Eigenvectors are not modified
I Very fast convergence close to σ
Things to consider:
I Implicit inverse (A− σB)−1 via linear solves
I Direct linear solver for robustness
I Less effective for eigenvalues far away from σ
Spectrum SlicingSLEPc
Evaluation
Spectrum SlicingIndefinite (block-)triangular factorization:
A− σB = LDLT
By Sylvester’s law of inertia, we get as a byproduct the number ofeigenvalues on the left of σ
ν(A− σB) = ν(D)
Spectrum slicing
I Multi-shift approach that sweeps all the intervalI Compute eigenvalues by chunksI Use inertia to validate sub-intervals
a b
σ1 σ2 σ3
Spectrum SlicingSLEPc
Evaluation
Spectrum Slicing: Grimes et al. ApproachGrimes et al. [1994] proposed an “industrial strength” scheme
I Block Lanczos, with blocksize depending on multiplicity
I B-orthogonalization, partial and external selective reorthog.
I Create an initial trust interval, extend it until finished
I Unrestarted Lanczos, tracking eigenvalue convergence
Choice of new shift:
I Asumes there are as many eigenvalues around σi and σi+1
I Uses non-converged Ritz approximations if available
I Sometimes need to fill-in gaps
Deflation with sentinel mechanism:
I Deflation against (at least) one vector from previous shift
I Goal: orthogonality in clusters, suppress eigenvectors mostlikely to reappear
Spectrum SlicingSLEPc
Evaluation
Grimes et al.: Potential Pitfalls
Possible problems of Grimes et al. approach:
I Exploits a priori knowledge of multiplicity
I Assumes all multiplicities are of same rank
I Block size cannot be arbitrarily large, difficulties with highmultiplicities
I Irregular spectra produce bad choice of shifts, with big gaps
I Wasteful work: many repeated eigenvalues are discarded
Using an unrestarted block Lanczos has strong implications onheuristics
Spectrum SlicingSLEPc
Evaluation
New Context: Restarted Lanczos
Now we have restarted Lanczos methods
I Thick-restart Lanczos [Wu et al. 1999]
I Equivalent to implicit-restart Lanczos, or symm. Krylov-Schur
New assumptions:
I Lanczos convergence is not a problem; also multiple eigs.
I Orthogonalization is relatively cheap and scales very well
I Performance of factorization degrades with n
I Triangular solves are not scalable in parallel
Goal: spectrum slicing technique that can be robust enough forirregular spectra with high multiplicity, scalable to 100’s processors
Strategy: avoid new shifts by orthogonalizing more
Spectrum SlicingSLEPc
Evaluation
Proposed Method (1)
Main idea: At each shift σi request fixed number of eigenvalues(nev), with a limited number of restarts (maxit)
Selection of new shift σi+1:
I Cannot rely on approximate Ritz values
I Separation of eigenvalues computed at σi is not reliable
I Use average eigenvalue separation in [σi−1, σi]
Backtrack: if number of eigenvalues computed in [σi−1, σi] doesnot match inertia, create a new shift somewhere inbetween
I All eigenvectors available in [σi−1, σi] are deflated
I Guarantees orthogonality of eigenvectors of multiples/clusters
Spectrum SlicingSLEPc
Evaluation
Proposed Method (2)
Deflation
I Avoid reappearance of already computed eigenvalues
I Also allow missing multiples to ariseI Two options (flag defl):
1. At σi+1, deflate all eigenvectors available in [σi, σi+1]2. Minimal deflation, with sentinels similar to Grimes
If possible, avoid backtracking
I A new factorization to compute a few eigenvalues is wasteful
I Parameter compl: try to complete interval if missingeigenvalues less or equal than compl
Spectrum SlicingSLEPc
Evaluation
Proposed Method (3)
With backtracking
I nev=10, maxit=10, with deflation
a b
σ1 σ2σ3 σ4
Avoiding backtracking
I nev=10, maxit=10, with deflation, compl=5
a b
σ1 σ2 σ3
Spectrum SlicingSLEPc
Evaluation
SLEPc: Scalable Library for Eigenvalue Problem Computations
A general library for solving large-scale sparse eigenproblems onparallel computers
I For standard and generalized eigenproblems
I For real and complex arithmetic
I For Hermitian or non-Hermitian problems
I Also support for SVD and QEP
Ax = λx Ax = λBx Avi = σiui (λ2M+λC+K)x = 0
Developed at U. Politecnica de Valencia since 2000
http://www.grycap.upv.es/slepc
Current version: 3.1 (released Aug 2010)
Spectrum SlicingSLEPc
Evaluation
PETSc/SLEPc Numerical Components
PETSc
Vectors
Standard CUSP
Index Sets
Indices Block Stride Other
Matrices
CompressedSparse Row
BlockCSR
SymmetricBlock CSR
Dense CUSP Other
Preconditioners
AdditiveSchwarz
BlockJacobi
Jacobi ILU ICC LU Other
Krylov Subspace Methods
GMRES CG CGS Bi-CGStab TFQMR Richardson Chebychev Other
Nonlinear Systems
LineSearch
TrustRegion Other
Time Steppers
EulerBackward
Euler
PseudoTime Step Other
SLEPc
SVD Solvers
CrossProduct
CyclicMatrix
LanczosThick R.Lanczos
Quadratic
Linear-ization
Q-Arnoldi
Eigensolvers
Krylov-Schur Arnoldi Lanczos GD JD Other
Spectral Transformation
Shift Shift-and-invert Cayley Fold Preconditioner
Spectrum SlicingSLEPc
Evaluation
m-step Lanczos Method
Computes Vm and TmI M = (A− σB)−1B
I Vm is a basis of the Krylov space Km(M, v1), VTmBVm = I
I Tm = V ∗mBMVm provides Ritz approximations, (θi, Vmyi)
for j = 1, 2, . . . ,mw =Mvjt1:j,j = V ∗
j Bw
w = w − Vjt1:j,jtj+1,j = ‖w‖Bvj+1 = w/tj+1,j
end
Orthogonalization:
I Full B-orthogonalization
I Do not bother about partialreorthog.
I SLEPc uses iterated CGSbut MGS also available
Use MUMPS for (A− σB)−1 = L−TD−1L−1, get inertia info
Spectrum SlicingSLEPc
Evaluation
Symmetric Krylov-Schur
A restarting mechanism that filters out unwanted eigenvectors
1. Build Lanczos factorization of order m
2. Diagonalize projected matrix
3. Check convergence, sort
4. Truncate to a factorization of order p
5. Extend to a factorization of order m
6. If not finished, go to step 2
Vm
v m+1
Sm
b∗m+1Vp
v m+1
Sp
b∗pVp
v m+1
Sp
b∗p
For spectrum slicing, the basis expansion needs to orthogonalizealso against an arbitrary set of vectorsRestarts until nev converged eigenvalues (or subinterval complete)
Spectrum SlicingSLEPc
Evaluation
Computing Platform
IBM BladeCenter cluster with Myrinet interconnect
I 256 JS20 nodes
I Two 64-bit PowerPC 970+ @ 2.2 GHz processors
I 4 GB memory per node (1 TB total)
Tests with up to 128 MPI processes (2 per node)
Spectrum SlicingSLEPc
Evaluation
Test Case: Aircraft Fuselage
Simplified but realisticmodel: cylinder with skin,frames, and stringers
Parametric, “scalable”
First vibration mode (5.34 Hz)
Spectrum SlicingSLEPc
Evaluation
Test Case: Matrix Properties
Analysis of frequency range[0–60] Hz
1 million dof’s
I Dimension: 1,036,698
I Nonzeros: ∼29 million
I Eigenvalues in interval: 1989
2 million dof’s
I Dimension: 2,141,646
I Nonzeros: ∼59 million
I Eigenvalues in interval: 2039
Maximum multiplicity: 2 B is singular
Spectrum SlicingSLEPc
Evaluation
Evaluation: Solver Parameters1 million, 16 processors
nev maxit defl comp Shifts Rest Its Time80 10 1 - 22 36 5,109 11,07480 10 1 40 17 35 4,696 9,97880 10 0 - 33 48 7,021 14,251
120 10 1 - 14 23 4,874 10,770120 10 1 50 13 27 4,565 10,125120 10 0 - 21 29 6,417 13,052
Unrest., 300 vecs, no defl. 22 - 6,622 13,305
2 million, 32 processorsnev maxit defl comp Shifts Rest Its Time80 5 1 40 20 40 5,200 14,83380 5 0 - 30 47 6,740 18,235
120 5 1 50 12 23 4,500 13,389120 5 0 - 24 33 7,290 19,659
Unrest., 300 vecs, no defl. 24 - 7,224 18,605
Spectrum SlicingSLEPc
Evaluation
Evaluation: Parallel Performance
1 million, nev=120
p Shft Rest Its Time Num. Sym. Tri. Orth.8 12 21 4,318 16,399 3,153 594 5,270 6,218
16 13 27 4,565 10,125 1,739 595 4,277 3,36432 10 21 4,101 5,500 519 653 2,839 1,42864 12 22 4,338 4,064 375 654 2,482 580
128 12 20 4,155 3,394 273 596 2,265 245
2 million, nev=120
p Shft Rest Its Time Num. Sym. Tri. Orth.32 13 24 4,873 14,429 2,012 1,691 6,536 3,90264 12 22 4,472 9,120 922 1,697 4,886 1,263
128 12 23 4,501 7,817 707 1,692 4,709 668
Spectrum SlicingSLEPc
Evaluation
Conclusion
We have developed a robust spectrum slicing method, can copewith high multiplicities
I Based on previous work by Grimes et al.
I Main focus on restarted Lanczos methods
I Our heuristics tend to favour scalability
Evaluation
I 4 times faster than plain shift-and-invert
I Compared to Grimes et al., 30-40% gain
I Reasonable scalability up to 128 processors
Future work
I Improve scalability by splitting in subcommunicators (similarto [Zhang et al. 2007])
Spectrum SlicingSLEPc
Evaluation
Thanks!
Information on SLEPc
http://www.grycap.upv.es/slepc
Recommended