Geometry Processingで学ぶSparse Matrix

GEOMETRY PROCESSINGで学ぶ SPARSE MATRIX

2012/3/18 Tokyo.SciPy #3 齊藤 淳 Jun Saito @dukecyto

本日の概要

Laplacian Mesh Processingを通じて sparse matrix一般、scipy.sparseの理解を深める

!! = ! − !L !

Computer Graphics Animation

(c) copyright 2008, Blender Foundation / www.bigbuckbunny.org

Computer Graphics Animation

Modeling • 作る

Animation • 動かす

Rendering • 絵にする

SIGGRAPH 2011 Papers Categories

Modeling

• Understanding Shapes

• Mapping & Warping Shapes

• Capturing Geometry and Appearance

• Geometry Processing

• Discrete Differential Geometry

• Geometry Acquisition

• Surfaces • Procedural &

Interactive Modeling • Fun With Shapes

Animation

• Capturing & Modeling Humans

• Facial Animation • Call Animal Control! • Contact and

Constraints • Example-Based

Simulation • Fluid Simulation • Fast Simulation

Rendering

• Stochastic Rendering & Visibility

• Volumes & Photons • Real-Time

Rendering Hardware

• Sampling & Noise

Images

• Drawing, Painting and Stylization

• Tone Editing • By-Example Image

Synthesis •  Image Processing • Video Resizing &

Stabilization • Stereo & Disparity •  Interactive Image

Editing • Colorful

SIGGRAPH 2011 Papers Categories

Modeling

• Understanding Shapes

• Mapping & Warping Shapes

• Capturing Geometry and Appearance

• Geometry Processing

• Discrete Differential Geometry

• Geometry Acquisition

• Surfaces • Procedural &

Interactive Modeling • Fun With Shapes

Animation

• Capturing & Modeling Humans

• Facial Animation • Call Animal Control! • Contact and

Constraints • Example-Based

Simulation • Fluid Simulation • Fast Simulation

Rendering

• Stochastic Rendering & Visibility

• Volumes & Photons • Real-Time

Rendering Hardware

• Sampling & Noise

Images

• Drawing, Painting and Stylization

• Tone Editing • By-Example Image

Synthesis •  Image Processing • Video Resizing &

Stabilization • Stereo & Disparity •  Interactive Image

Editing • Colorful

Geometry Processingとは

Geometry processing, or mesh processing, is a fast-growing area of research that uses concepts from applied mathematics, computer science and engineering to design efficient algorithms for the ¨  acquisition ¨  reconstruction ¨  analysis ¨  manipulation ¨  simulation ¨  transmission of complex 3D models.

http://en.wikipedia.org/wiki/Geometry_processing

Triangle Meshによる形状の表現

Differential Coordinates

¨ 形状の局所的な特徴 ¤ 方向は法線、大きさは平均曲率の近似

( )( )

1N iid ∈

= −∑i iv

δ v v ( )1( )

dslen γγ ∈

−∫ iv

v v

γ

Continuous Laplace-Beltrami operator Discrete Laplace-Beltrami operator

[Sorkine 2005]

Discrete Laplace Operator

Applications

Mesh Parameterization

Fairing / Smoothing

Denoising

Manipulation / Editing

Compression

Shape Analysis

Physical Simulation

“Majority of contemporary geometry processing tools rely on discrete Laplace operators”

[Alexa 2011]

Laplacian Mesh Processing

Graph Laplacian (a.k.a. Uniform Laplacian, Topological Laplacian)

http://en.wikipedia.org/wiki/Laplacian_matrix

Cotan-Weighted Laplacian

緑: Graph Laplacian 赤: Cotan-Weighted Laplacian

!!! =1

4!(!!)12 cot!!" + cot!!"

!∈! !(!! − !!)

[Sorkine 2005]

Stanford 3D Scan Repository

¨  Stanford Bunny ¤ 頂点数: 35,947 ¤ 35,947 x 35,947 =

1.2 G

Stanford 3D Scan Repository

¨  Dragon ¤ 頂点数: 566,098 ¤ 566,098 x 566,098 =

298G

疎行列 Sparse Matrix

成分のほとんどがゼロであることを活用した形式で行列を記憶・演算 “Sparse matrices are widely used in scientific computation, especially in large-scale optimization, structural and circuit analysis, computational fluid dynamics, and, generally, the numerical solution of partial differential equations.” Sparse Matrices in MATLAB: Design and Implementation

疎行列 Sparse Matrix

成分のほとんどがゼロであることを活用した形式で行列を記憶・演算

Laplacian Matrixは

¨  sparse ¨  symmetric

¨  positive semi-definite

Python Sparse Matrix Packages

SciPy Sparse

PySparse

CVXOPT

Python Sparse Matrix Packages

SciPy Sparse

PySparse

CVXOPT

From OpenOpt doc...

“Unfortunately, sparse matrices still remains one of most weak features in Python usage for scientific purposes”

From OpenOpt doc...

“Unlike MATLAB, Octave, and a number of other software, there is not standard Python library for sparse matrices: someone uses scipy.sparse, someone PySparse, someone (as CVXOPT) uses its own library and/or BLAS, someone just uses 3 columns (for the number indexes and value). SciPy developers refused to author of scipy.sparse to include it into NumPy, I think it's a big mistake, probably now it would been a unified standard. Still I hope in future numpy versions difference in API for handling sparse and dense arrays will be removed. “

基本的な流れ

構築に適した形式で行列を作る

演算に適した形式に変換し演算を行う

A = lil_matrix((N,N))!A[i,j] = a!!

A = A.tocsr()!A = A.dot(D)!f = factorized(A)!x = f.solve(b)!

Graph Laplacian (a.k.a. Uniform Laplacian, Topological Laplacian)

http://en.wikipedia.org/wiki/Laplacian_matrix

Graph Laplacian in scipy.sparse

from scipy import sparse!!

N = len(points)!

L = sparse.lil_matrix((N, N))!

for vids in faces:!

L[vids, vids] = 1!D = sparse.spdiags(L.sum(axis=0), 0, N, N)!

L = L.tocsc()-D!

Graph Laplacian in scipy.sparse

from scipy import sparse!!

N = len(points)!

L = sparse.lil_matrix((N, N))!

for vids in faces:!

L[vids, vids] = 1!D = sparse.spdiags(L.sum(axis=0), 0, N, N)!

L = L.tocsc()-D!

points and faces from Maya

!import numpy as np!

import pymel.core as pm!

!

mesh = pm.PyNode(‘bunny’)!

points = np.array(mesh.getPoints())!faces = np.array(mesh.getVertices()!

! !　[1]).reshape(mesh.numFaces(),3)!

Graph Laplacian in scipy.sparse

from scipy import sparse!!

N = len(points)!

L = sparse.lil_matrix((N, N))!

for vids in faces:!

L[vids, vids] = 1!D = sparse.spdiags(L.sum(axis=0), 0, N, N)!

L = L.tocsc()-D!

Sparse Matrix Storage Formats

¨  LIst of Lists (LIL)

¨  COOrdinate lists (COO)

¨  Dictionary Of Keys (DOK)

¨  Compressed Sparse Row (CSR)

¨  Compressed Sparse Column (CSC)

¨  Block Sparse Row (BSR)

¨  DIAgonal (DIA)

構築に適した形式 演算に適した形式

sparse.lil_matrix LIst of Lists (LIL) 形式

¨  データ構造 ¤ 行毎に非ゼロ要素の

ソートされた列インデックスを保持するarray

¤ 対応する非ゼロ要素の値も同様に保持

¨  用途 ¤ 行列の構築用 ¤ 巨大な行列を構築する

際にはCOOも検討すると良い

¨  利点 ¤ 柔軟なslice ¤ 他のmatrix形式への変

換が速い ¨  欠点

¤  LIL + LIL が遅い (CSR/CSC推奨)

¤  column slicingが遅い (CSC推奨)

¤ 行列ベクトル積が遅い (CSR/CSC推奨)

sparse.csr_matrix Compressed Sparse Rows (CSR)形式

¨  CSC: 行と列が逆 ¨  データ構造

A = [1 2 3 1 2 2 1] 非ゼロ要素リスト!

(IA = [1 1 1 2 3 3 4] i行)!

IA' = [1 4 5 7] !

JA = [1 2 3 4 1 4 4] j列!

A, IA’, JAを保持

¨  利点 ¤ 高速な演算 CSR + CSR,

CSR * CSR,等. ¤  Row slicingが速い ¤ 行列ベクトル積が速い

¨  欠点 ¤ Column slicingが遅い

(CSC推奨) ¤ 他のmatrix形式への変

換が遅い (そうでもない？後述ベンチマーク参照)

http://ja.wikipedia.org/wiki/%E7%96%8E%E8%A1%8C%E5%88%97

sparse.dia_matrix DIAgonal (DIA)形式

¨  帯行列に適した形式 ¨  オフセットを指定可能 >>> data = array([[1,2,3,4]]).repeat(3,axis=0)!>>> offsets = array([0,-1,2])!>>> dia_matrix( (data,offsets), shape=(4,4)).todense()!matrix([[1, 0, 3, 0],! [1, 2, 0, 4],! [0, 2, 3, 0],! [0, 0, 3, 4]])!

Sparse Matrix Format Benchmark: Construction

Benchmark by modified bench_sparse.py in SciPy distribution

Sparse Matrix Format Benchmark: Conversion

Benchmark by modified bench_sparse.py in SciPy distribution

Sparse Matrix Format Benchmark: Matrix Vector Product

Benchmark by modified bench_sparse.py in SciPy distribution

Graph Laplacian in scipy.sparse

from scipy import sparse!!

N = len(points)!

L = sparse.lil_matrix((N, N))!

for vids in faces:!

L[vids, vids] = 1!D = sparse.spdiags(L.sum(axis=0), 0, N, N)!

L = L.tocsc()-D!

Fancy indexingの仕様の違い

>>> L = sparse.lil_matrix((N,N))!

>>> ix = [1,3,4]!

>>> L[ix,ix] = 1!

>>> L.todense()!

matrix([[ 0., 0., 0., 0., 0.],!

[ 0., 1., 0., 1., 1.],!

[ 0., 0., 0., 0., 0.],!

[ 0., 1., 0., 1., 1.],!

[ 0., 1., 0., 1., 1.]])!

>>> Ldense = np.zeros((N,N))!

>>> ix = [1,3,4]!

>>> Ldense[ix,ix] = 1!

>>> Ldense!

array([[ 0., 0., 0., 0., 0.],!

[ 0., 1., 0., 0., 0.],!

[ 0., 0., 0., 0., 0.],!

[ 0., 0., 0., 1., 0.],!

[ 0., 0., 0., 0., 1.]])!

sparse.lil_matrix numpy.array

補足: numpy.ix_()

>>> Ldense = np.zeros((N,N))!>>> ix = [1,3,4]!>>> Ldense[np.ix_(ix,ix)] = 1!>>> Ldense!array([[ 0., 0., 0., 0., 0.],! [ 0., 1., 0., 1., 1.],! [ 0., 0., 0., 0., 0.],! [ 0., 1., 0., 1., 1.],! [ 0., 1., 0., 1., 1.]])!

Graph Laplacian in scipy.sparse

from scipy import sparse!!

N = len(points)!

L = sparse.lil_matrix((N, N))!

for vids in faces:!

L[vids, vids] = 1!D = sparse.spdiags(L.sum(axis=0), 0, N, N)!

L = L.tocsc()-D!

細かい演算の仕様

!# LがCSC,CSRだとNotImplementedError!L -= D!!# OK!L = L-D!

Laplacian Mesh Fairing

A Signal Processing Approach To Fair Surface Design [Taubin 95]

!! = ! − !L !

Mesh Fairing in scipy.sparse

from scipy.sparse import linalg as spla!!A = sparse.identity(L.shape[0]) - _lambda * L!solve = spla.factorized(A.tocsc())!x = solve(points[:,0])!y = solve(points[:,1])!z = solve(points[:,2])!new_points = np.array([x, y, z]).T!!# Set result back to Maya mesh!mesh.setPoints(new_points)!

Mesh Fairing in scipy.sparse

from scipy.sparse import linalg as spla!!A = sparse.identity(L.shape[0]) - _lambda * L!solve = spla.factorized(A.tocsc())!x = solve(points[:,0])!y = solve(points[:,1])!z = solve(points[:,2])!new_points = np.array([x, y, z]).T!!# Set result back to Maya mesh!mesh.setPoints(new_points)!

Solving sparse linear system

¨  sparse.linalg.factorized(A) ¤ AのLU分解結果を保持したlinear system solver関数を

返す ¤ AはCSC形式で渡す必要がある

SparseEfficiencyWarning: splu requires CSC matrix format

Mesh Fairing in scipy.sparse

from scipy.sparse import linalg as spla!!A = sparse.identity(L.shape[0]) - _lambda * L!solve = spla.factorized(A.tocsc())!x = solve(points[:,0])!y = solve(points[:,1])!z = solve(points[:,2])!new_points = np.array([x, y, z]).T!!# Set result back to Maya mesh!mesh.setPoints(new_points)!

Solving sparse linear system

# Error! 右辺はベクトルのみ!new_points = solve(points)!!# 仕方がないので一列ずつ!x = solve(points[:,0])!y = solve(points[:,1])!z = solve(points[:,2])!new_points = np.array([x, y, z]).T!

Cotan-Weighted Laplacian

緑: Graph Laplacian 赤: Cotan-Weighted Laplacian

!!! =1

4!(!!)12 cot!!" + cot!!"

!∈! !(!! − !!)

[Sorkine 2005]

Cotan Laplacian in scipy.sparse (1/2)

def laplacian_cotan(points, faces):! EDGE_LOOP = [(0,1,2),(0,2,1),(1,2,0)]! N = len(points)! point_area = np.zeros(N)! L = sparse.lil_matrix((N, N))! for vids in faces:! point_area[vids] += area_triangle(points[vids]) / 3.0! for i in EDGE_LOOP:! v0 = vids[i[0]]! v1 = vids[i[1]]! v2 = vids[i[2]]! e1 = points[v1] - points[v2]! e2 = points[v0] - points[v2]!

Cotan Laplacian in scipy.sparse (2/2)

cosa = np.dot(e1, e2) / math.sqrt(np.dot(e1, e1) * np.dot(e2, e2))!

sina = math.sqrt(1 - cosa * cosa)!

cota = cosa / sina!

w = 0.5 * cota!

L[v0, v0] -= w!

L[v0, v1] += w!

L[v1, v1] -= w!

L[v1, v0] += w!

D = sparse.spdiags(0.25/point_area, 0, N, N)!

return D.dot(L.tocsc())!

Cotan Laplacian in scipy.sparse (2/2)

cosa = np.dot(e1, e2) / math.sqrt(np.dot(e1, e1) * np.dot(e2, e2))!

sina = math.sqrt(1 - cosa * cosa)!

cota = cosa / sina!

w = 0.5 * cota!

L[v0, v0] -= w!

L[v0, v1] += w!

L[v1, v1] -= w!

L[v1, v0] += w!

D = sparse.spdiags(0.25/point_area, 0, N, N)!

return D.dot(L.tocsc())!

行列積について

# Denseに変換されてしまう!np.dot(D, L)!!# Sparseのまま行列積!D.dot(L)!

Laplacian Matrix構築 Cythonによる高速化

¨ Python: 2.6秒

¨ Cython: 1.8秒

¨ Cython + Dense: 0.8秒！ ¤ あまり大きくないMeshならばDenseで

Laplacianを作った方が速い

ファイル入出力

¨  numpy.savetxt() ¤  Sparse Matrixでは使えない

¨  scipy.io.savemat() / loadmat() ¤  MATLAB .mat形式で保存 ¤  Sparse Matrixは強制的に

CSCで保存される

>>> io.savemat('sparsetest.mat', {'lil': L.tolil(), 'csr': L.tocsr()},

oned_as='row')

>>> M = io.loadmat(‘sparsetest.mat')

>>> M['lil'] <5x5 sparse matrix of type '<type 'numpy.float64'>'

with 9 stored elements in Compressed Sparse Column format>

>>> M['csr']

<5x5 sparse matrix of type '<type 'numpy.float64'>'

with 9 stored elements in Compressed Sparse Column format>

MATLAB vs. SciPy – Sparse Matrices

¨  sparse()で行列を作るだけ！

¨  後の演算はdenseと全く同じ

¨  内部形式はCSC

¨  内部形式(denseも含め)を意識して使う

¨  形式によって同じ演算、関数が使えないケースがある

MATLAB SciPy Sparse

まとめ

¨  Sparse matrix一般 ¤ よく使われるデータ構造を理解した

¨  scipy.sparse ¤ 使い方の基本がわかった ¤ いいところ

n やりたいことはできる。実用可能。 n  Mesh Processingが省メモリで計算可能。〜1000倍のオーダーで

速くなる。 ¤ 悪いところ

n ドキュメントされていない仕様が多過ぎ。 n  ndarrayと透過的に使えるようにしてください L

¨  Laplacian Mesh Processingはおもしろいですよ J

Future Work

¨  scipy.sparse.linalg の調査 ¤  Iterative sparse linear system solver系

¨  Mesh Processingのもうちょっと深いネタを紹介

Further Readings

[Sorkine 2005] Sorkine, Olga. “Laplacian Mesh Processing” Eurographics 2005 http://igl.ethz.ch/projects/Laplacian-mesh-processing/STAR/STAR-Laplacian-mesh-processing.pdf

¤ Mesh Laplacianの基礎と応用に関するまとめ

Further Readings

Levy, Bruno and Zhang, Hao. “Spectral Mesh Processing” ACM SIGGRAPH 2010 http://alice.loria.fr/WIKI/index.php/Graphite/SpectralMeshProcessing

¤ Laplacianの特異値分解を用いてMeshの周波数解析

Further Readings

[Alexa 2011] Alexa, Marc and Wardetzky, Max. “Discrete Laplacians on General Polygonal Meshes” ACM SIGGRAPH 2011 http://cybertron.cg.tu-berlin.de/polymesh/ (ソースコード有)

¤ Discrete Laplacianを一般的なポリゴンメッシュに対し定義