Mesh Quilting For Geomet-ric Texture Synthesis

Kun Zhou et al.In SIGGRAPH 2006

발표 이성호2009년 4월 15일

2

Abstract• Mesh quilting

– Geometric texture synthesis algorithm– 3D texture sample given in the form of a triangle

• inside a thin shell around an arbitrary surface

• Allow interactive and versatile editing and animation

• Based on stitching together 3D geometry elements

• On curved surfaces– Reduce distortion of geometry elements

• inside the 3D space of the thin shell– Low-distortion parameterization

3

Introduction• Today’s commodity video cards– Exquisite details can be purely geometrically

modeled• Modeling such complex geometric details– A tedious process

• Creating mesh-based 3D geometric tex-tures– Remains challenging

4

Mesh quilting• To synthesize geometric details by

stitching together small patches – of an input geometric texture sample

• Tools to further edit and animate – these geometric details

5

6

Related work• Modeling of Geomet-

ric Detail on Surfaces– Fur• [Kajiya and Kay 1989]

– Rendering fur with three dimensional textures.

– In Proceedings of SIG-GRAPH 89

– Volume textures• [Neyret 1998]

7

• More versatile repre-sentations– Geometric textures• [Elber 2005]

– Shell map• [Porumbescu et al.

2005]

8

• Limitations– Periodic textures

• Mesh-based creation – Of geometric textures on arbitrary

meshes– [Fleischer et al. 1995]–Mostly restricted to the dissemination • Of simple texture elements over the surface

9

Example-based Texture Synthe-sis

• Synthesis based on per-pixel non-para-metric sampling– [Turk 2001;Wei and Levoy 2001; Ying et al.

2001; Tong et al. 2002; Zelinka and Garland 2003]

• Based on the L2-norm– a relatively poor measure

• of perceptual similarity, – such algorithms are not applicable

• to a large spectrum of textures.

10

• Textures by directly copying small parts – of an input texture sample– alpha-blending [Praun et al. 2000]– quilting

• [Efros and Freeman 2001; Liang et al. 2001; Soler et al. 2002; Magda and riegman 2003; Kwatra et al. 2003; Wu and Yu 2004; Zhou et al. 2005]

• Searching for the “min-cut” seams– further enhance the smoothness across the seams

• [Efros and Freeman 2001; Kwatra et al. 2003; Zhou et al. 2005]

11

• Feature matching– [Wu and Yu 2004]– Human visual system is so sensitive

• to edges, corners and other high-level features in textures

• Parallel controllable texture synthesis on GPU– [Lefebvre and Hoppe 2005]

• Texture synthesis using Expectation Maximiza-tion optimization– [Kwatra et al. 2005]

12

Graphcut textures

13

14

15

Shell map

16

17

18

Challenges• Little work to provide tool – for 3D geometric texture synthesis.• [Bhat et al. 2004; Lagae et al. 2005]

[Lagae et al. 2005]

19

• [Bhat et al. 2004] – voxel-based approach

• [Lagae et al. 2005] – used distance fields

[Bhat et al. 2004]

20

Challenges• Irregular mesh– The input texture sample • is not a regular array of pixel values

• Geometry elements– Each being truly a small 3D object identified• As a connected component in 3D

• Quilting is performed on curved surfaces– Severe distortion in the 3D space

21

Contributions• Mesh-based geometric texture synthesis– Synthesized over the base mesh.

• Triangle meshes– Both the input geometry and output geome-

try• Integrity– Maintains the integrity of geometry elements

• In the synthesized texture• Texture editing and texture animation

– can be easily performed

22

• Aligns elements – through local deformation– And merges elements to connect texture

patches• Mesh quilting on curved surfaces– Low-distortion parameterization • of the shell space

23

Mesh Quilting Synthesis• Setup & Nomenclature

24

Algorithm Overview• Seed Finding

– Find a seed region R from which to grow the output mesh texture further out

• Geometry Matching– Find the best patch placement around region R using geometry

matching to minimize mismatch between the new and the old patch• Element Correspondences

– Find correspondences between elements in the new patch and those in the old patch

• Element Deformation– Align the corresponding elements through local deformation

• Element Merging– Expand the output texture by merging the new patch into the output

texture space

25

26

Seed Finding• Grid-based approach

– The bounding boxes of both Mout and Min are subdi-vided in finer regular grids

• These grids are only two-dimensional• Initially, the cells of Mout are tagged unprocessed• Each time we wish to grow out the current mesh

Mout, – we look for an unprocessed cell with the largest num-

ber of adjacent cells that are already processed• this will be the seed cell that we will try to process next.

27

Geometry Matching• Find how to complete the mesh tex-

ture in the seed cell– and possibly add to its surroundings too.

Using the nearby existing mesh texture available near the seed cell

– Find a portion of the original swatch Min best matching this surrounding to ex-tend Mout .

28

29

30

• Restrict the translation t to be in – grid unit

• Element deformation described in Section 2.6 – will compensate for an imperfect element

alignment• Octree data structure for the input

texture– Significant speed-up

31

Element Correspon-dences

• the overlapping region is usually larger than the small sub-patch Pout – since the input mesh texture covers Pout

completely.

32

33

Element Deformation

34

35

36

Element Merging• Every element (either from Cout or Cin)

without correspondence – directly added to Mout .

• For every established correspondence (Cout ,Cin)– If Cout is entirely within the overlapping re-

gion, Cout is ignored • and Cin is instead added to the final results

– if Cin is entirely within the overlapping region,• Cin is ignored and Cout is added to Mout .

37

In all other cases• stitch parts of Cin and Cout– to get a singly-connected, combined element– seek a cut path in each element– the graph cut algorithm

• [Boykov et al. 2001]

38

39

40

41

Mesh Quilting Over Curved Sur-faces

• Setup– Let Mbase be the base mesh that we wish

to enhance with added geometric de-tails.

–Min the geometric texture mesh • used as a swatch

– seamlessly tile the base mesh– S• the scale of the geometric details

42

From Planar to Curved• 2D grid -> base mesh– Quilting process will stop

• Only when there are no more unprocessed tri-angles

• Define a local surface patch – By starting from the chosen triangle – Growing the region

• Using breadth-first traversal – Until we reach a certain depth– Or when the total area of the patch exceeds a user-

defined threshold

43

• Position of vertices located with re-spect to the base mesh

• Location of a vertex v – over a triangle Tbase – is defined by the barycentric coordinates – of its orthogonal projection • on Tbase

– along with the orthogonal distance (i.e., height) » from the triangle to v

44

discrete conformal map-ping

• surface patch is flattened over the 2D plane – using a discrete conformal mapping• DCM [Desbrun et al. 2002]

45

• Local operations – Described for planar mesh quilting– Can be performed • Over this parameterization plane

• Position of the newly synthesized vertices–Will be reprojected • Onto the local mesh-based coordinate sys-

tem

46

in very curved regions• If the area distortion induced – by the local parameterization is too

large– Reduce the area of the surface patch• This will decrease

– the size of the output-sub-patch Pout

47

Final Mesh Embedding• Convert the vertex positions

– Stored in local coordinates for now– Into a stand-alone, common embedding

• Self-intersections can be created• Build a texture atlas for Mbase

– Convert the above local representation of vertex posi-tions to locations • in a geometry texture space

• Then, construct a shell space around Mbase– Mapping the vertices

• from the geometry texture space to the shell space– will fix the location of the vertices in 3D space

48

49

50

Shell Mapping• Porumbescu et al. [2005]– Creates large distortion in curved regions

• We alleviate this!– By optimizing a stretch metric on this

tetrahedral mesh– A natural extension of

• low-distortion parameterization – of triangle meshes

• [Sander et al. 2001]

51

52

53

Minimization Algorithm• Only update the u and v texture co-

ordinates of the vertices on the offset surface

• Optimization of the stretch metric – along a randomly chosen search direc-

tion – in the (u, v) plane – as in [Sander et al. 2001].

54

55

56

57

58

Discussions• Regions with very high curvature can be

badly handled– parametric distortion of small surface

patches may be high• Cannot always achieve perfect matching – if the swatch is untileable• even with major element deformation

– Postprocessing step is performed • to remove those visually-displeasing elements• Figure 5(b)

59

Conclusions• Mesh-based 3D texture synthesis al-

gorithm