Deep Geometric Functional Maps: RobustFeature Learning for Shape Correspondence

Nicolas DonatiLIX, Ecole Polytechnique

nicolas.donati@polytechnique.edu

Abhishek SharmaLIX, Ecole Polytechniquekein.iitian@gmail.com

Maks OvsjanikovLIX, Ecole Polytechniquemaks@lix.polytechnique.fr

Abstract

We present a novel learning-based approach for comput-ing correspondences between non-rigid 3D shapes. Unlikeprevious methods that either require extensive training dataor operate on handcrafted input descriptors and thus gen-eralize poorly across diverse datasets, our approach is bothaccurate and robust to changes in shape structure. Key toour method is a feature-extraction network that learns di-rectly from raw shape geometry, combined with a novel reg-ularized map extraction layer and loss, based on the func-tional map representation. We demonstrate through exten-sive experiments in challenging shape matching scenariosthat our method can learn from less training data than ex-isting supervised approaches and generalizes significantlybetter than current descriptor-based learning methods. Oursource code is available at: https://github.com/LIX-shape-analysis/GeomFmaps.

1. Introduction

Shape correspondence is a key problem in computer vi-sion, computer graphics and related fields with a broadrange of applications, including texture or deformationtransfer and statistical shape analysis [7], among many oth-ers. While classical correspondence methods have beenbased on handcrafted features or deformation models [49],more recent approaches have focused on learning an opti-mal model from the data either in supervised [11, 53, 25, 16]or even unsupervised settings [19, 43, 17].

Despite significant progress in recent years, however,learning-based approaches for shape correspondence typ-ically require large amounts of training data in order tolearn a model that generalizes well to diverse shape classes[53, 16]. Several existing methods address this chal-lenge by learning a derived representation, through a non-linear transformation of pre-computed feature descriptors[11, 25, 19, 43], rather than on the geometry of the shapesthemselves. Unfortunately, as we demonstrate below, this

Point Cloud Descriptors

Point-to-Point Map

Figure 1. Given a pair of shapes, our approach builds consistent de-scriptors directly from the underlying point clouds (left), and auto-matically computes an accurate pointwise correspondence (right).

reliance on a priori hand-crafted descriptors makes the re-sulting learned models both less robust and less accurateleading to a significant drop in generalization power to newshape classes or instances.

In this work, we propose an approach that combines thepower of learning directly from the 3D shapes with strongregularization based on a novel spectral correspondence ex-traction layer. Our method is inspired by recent learn-ing techniques employing the functional map representa-tion [25, 43]; however, we extend them to learn the featuresfrom 3D geometry rather than from some pre-computed de-

1

arX

iv:2

003.

1428

6v1

[st

at.M

L]

31

Mar

202

0

scriptors. Furthermore, we introduce a regularizer into thefunctional map computation layer that greatly improves thespeed and robustness of training. Finally, we demonstratehow the spectral loss based on the functional map repre-sentation in the reduced basis significantly reduces over-fitting, while still leading to accurate correspondences cou-pled with recent post-processing techniques. As a result,our overall pipeline is both more robust and has greater gen-eralization power than existing methods, while still beingable to learn from limited training data.

2. Related WorkComputing point-to-point maps between two 3D discrete

surfaces is a very well-studied area of computer vision. Be-low, we review those closest to our method, or with the bestknown results to serve as baselines, and refer to recent sur-veys [49, 6, 44] for an in-depth discussion.

Our method is built upon the functional map representa-tion, which was originally introduced in [32] as a tool fornon-rigid shape matching, and then extended in follow-upworks [33]. The key property of this representation is be-ing able to express maps as small matrices, encoded in areduced basis, which greatly simplifies the associated opti-mization problems.

The original work used only a basic set of constraintson functional maps, which have been extended significantlyin, e.g., [24, 2, 20, 14, 9, 41, 31, 21, 40] among many otherworks. These approaches both extend the generality andimprove the robustness of the functional map estimationpipeline, by using regularizers, robust penalties and pow-erful post-processing of the computed maps.

A key challenge in all of functional map estimation tech-niques, however, is the strong reliance on given input de-scriptors used for computing the maps. Several approacheshave suggested to use robust norms [24, 23], improvedpointwise map recovery [42, 15] or more principled reg-ularizers [39] which can help alleviate noise in the inputdescriptors to a certain extent but do not resolve strong in-consistencies in challenging cases.

More recent techniques have advocated learning optimaldescriptors for functional map estimation directly from thedata [11, 25]. These methods compute a transformationof given input descriptors so that the estimated functionalmaps are close to ground truth maps given during training.This idea was very recently extended to the unsupervisedsetting [19, 43] where the supervised loss was replaced withstructural penalties on the computed maps.

Despite significant progress, however, in all of thesecases, the descriptors are optimized through a transforma-tion of hand-crafted input features, such as SHOT [48],Heat [45] or Wave kernel signatures [5]. This has two severeconsequences: first, any information not present in the inputfeatures will be absent from the optimized descriptors, and

second, such approaches generalize poorly across datasetsas the input features can change significantly. This is par-ticularly true of the commonly-used SHOT descriptors [48],which are sensitive to the triangle mesh structure and, as weshow below, can vary drastically across different datasets.

A number of other techniques have also been proposedfor shape correspondence learning without using the func-tional map representation. These include approaches thatexploit novel convolutional layers on triangle meshes [27,8, 30, 36] and more general methods that use learning fromdepth-maps [53] or in some feature space [46, 10] amongmany others. Remarkably, relatively few methods aim tolearn directly from the raw 3D shape geometry for shapecorrespondence, with the notable exceptions of [16, 12]. Inlarge part this is due to the complexity of the correspon-dence problem, where unlike, e.g., shape segmentation, thenumber of labels can be unbounded. As a result, exist-ing techniques address this either by learning from precom-puted features, or relying on template-based matching andlarge training sets [16, 12], that might even require manualcuration. Although PointNet [37] and its variants [38, 4, 47]achieve impressive results from raw point clouds for classi-fication tasks, they are not yet competitive for shape corre-spondence task.

Contribution

In this paper we show that feature learning for shapematching can be done directly from the raw 3D geometryeven in the presence of relatively little training data, andwithout relying on a template or an a priori parametric (e.g.,human body) model. Our main contribution is a end-to-end learnable pipeline that computes features from the 3Dshapes and uses them for accurate dense point-to-point cor-respondence. We achieve this by introducing a novel mapextraction layer using the functional map representation in areduced basis, which provides a very strong regularization.Finally, we demonstrate that recent refinement techniquesadapted to small functional maps [29], combined with ourefficient learning pipeline jointly result in accurate densemaps at the fraction of the cost of existing methods.

3. Shape Matching and Functional MapsOne of the building blocks in our pipeline work is based

on the functional map framework and representation. Forcompleteness, we briefly review the basic notions for esti-mating functional maps, and refer the interested reader to arecent course [33] for a more in-depth discussion.

Basic Pipeline Given a pair of 3D shapes, M,N repre-sented in a discrete setting as triangle meshes, and contain-ing respectivelym and n vertices, this pipeline aims at com-puting a map between them.

It consists in four main steps. First, the first few eigen-

functions of the discrete Laplace-Beltrami operator arecomputed on each shape, namely kM and kN functions re-spectively.

Second, a set of descriptor functions on each shape thatare expected to be approximately preserved by the unknownmap. For instance, a descriptor function can correspond to aparticular dimension of the Heat or Wave Kernel Signatures[45, 5] computed at each point. Their coefficients are storedin the respective basis as columns in matrices A,B.

Third, the optimal functional map C is then computedby solving the following optimization problem:

Copt = arg minC

Edesc(C)

+ αEreg(C), (1)

where the first term aims at preserving the descriptors:Edesc

(C)

=∥∥CA − B

∥∥2, whereas the second term reg-ularizes the map by promoting the correctness of its overallstructural properties. It is common to use Frbenius norm tocompute the distance between these matrices. This Eq. (1)leads to a simple least-squares problem with kM × kN un-knowns, independent of the number of points on the shapes.

As a last step, the estimated functional map C, whichmaps across the spectral domains and converted to a point-to-point map. As a post processing step, called refinement,a number of advanced techniques are available [42, 15, 40,29]. Most of them iteratively take the map from spectral tospatial domain, until it reaches a local optimum.

3.1 . Deep Functional Maps

Despite its simplicity and efficiency, being a sequen-tial framework, the functional map estimation pipeline de-scribed above is fundamentally error prone, due to the initialchoice of descriptor functions. To alleviate this dependence,several approaches have been proposed to learn an optimaltransformation of initial descriptors from data [11, 25, 43].These works aim at transforming a given set of descriptorsso that the optimal computed map satisfies some desired cri-teria during training. This transformation can be learnedwith a supervised loss, as in [11, 25], as well as with anunsupervised loss as in the more recent works of [19, 43].

More specifically, the FMNet approach proposed in [25]assumes to have as input, a set of shape pairs for whichground truth point-wise maps are known, and aims to solvethe following problem:

minT

∑(S1,S2)∈Train

lF (Soft(Copt), GT(S1,S2)), where (2)

Copt = arg minC

‖CAT (D1) −AT (D2)‖. (3)

Here, adopting the notation from [43] T is a non-lineartransformation, in the form of a neural network, to be ap-plied to some input descriptor functions D, Train is theset of training pairs for which ground truth correspondence

GT(S1,S2) is known, lF is the soft error loss, which penal-izes the deviation of the computed functional map Copt, af-ter converting it to a soft map Soft(Copt) from the groundtruth correspondence, and AT (D1) denotes the transformeddescriptors D1 written in the basis of shape 1. Thus, theFMNet framework [25] learns a transformation T of de-scriptors T (D1), T (D2) based on a supervised loss thatminimizes the discrepancy between the resulting soft mapand the known ground truth correspondence.

A related recent approach, SURFMNet [43] follows asimilar strategy but replaces lF with an unsupervised lossthat enforces the desired structural properties on the result-ing map, such as its bijectivity, orthonormality and commu-tativity with the Laplacian.

3D-CODED In contrast to the the methods described abovethat primarily operate in the spectral domain, there are alsosome approaches that never leave the spatial domain. Withthe recent works on point clouds neural networks, pioneeredby PointNet [37], and significantly extended by [4, 47],to name a few, it is now possible to learn 3D features di-rectly from point clouds. 3D-CODED [16, 12] is basedon this approach, as it is a method built on a variationalauto-encoder with a PointNet architecture for the encoder.Their method relies on a template that is supposed to bedeformable in a non-rigid but isometric way to any of theshape of the datasets. It is a supervised method, and re-quires the knowledge of all ground-truth correspondencesbetween any shape of the dataset and the deformable tem-plate. 3D-CODED is trained on 230K shapes, introduced inSURREAL [50], and generated with SMPL [26].

Motivation The two main classes of existing approacheshave their associated benefits and drawbacks. On the onehand, spectral methods are able to use small matrices in-stead of all the points of the shape, and operate on intrinsicproperties of the 3D surfaces, making them resilient to achange in pose, and allowing them to train on really smalldatasets. However, due to their use of input descriptors (typ-ically SHOT [48]), they tend to overfit to the connectivity ofthe training set, which can lead to catastrophic results evenin apparently simple cases. On the other hand, 3D-CODEDshows extreme efficiency when trained on enough data, re-gardless of the connectivity, but with a small dataset, it isprone to overfitting and fails to generalize the training posesto predict the different poses of the test set.

Our method is a mix of the two approaches, and, as weshow below, can obtain accurate results with little trainingdata leading to state-of-the-art accuracy on a challengingrecent benchmark of human shapes in different poses andwith different connectivity [28].

4. Method

4.1 . Overview

In this paper, we introduce a novel approach to learn-ing descriptors on shapes in order to get correspondencesthrough the functional map framework. Our method is com-posed of two main parts, labeled as Feat and FMReg in Fig-ure 2. The first aims at optimizing point cloud convolu-tional filters [4, 47] to extract features from the raw geome-try of the shapes. These filters are learned using a Siamesenetwork on the source and a target shapes by using sharedlearnable parameters Θ, in a similar way as in [25]. How-ever, unlike that approach and follow-up works [19, 43] welearn the features directly from the geometry of the shapesrather than computing a transformation of some pre-definedexisting descriptors. These learned descriptors are projectedin the spectral bases of the shapes and fed to the secondblock of the method, which uses them in a novel regularizedfunctional map estimation layer. Finally, we use a spec-tral loss, based on the difference between the computed andthe ground truth functional maps. This makes our approachvery efficient as it operates purely in the spectral domain,avoiding expensive geodesic distance matrix computationsas in [25, 19] and moreover allows us to handle functionalor soft ground truth input maps without requiring the train-ing shapes to have the same number of points or fixed meshconnectivity.

We stress again that the two components: learning fea-tures directly from the shapes and using the functional maprepresentation both play a crucial role in our setup. The for-mer allows us to learn robust and informative features inde-pendently from the mesh structure, while the latter allowsus to strongly regularize correspondence learning, resultingin a method that generalizes even in the presence of a rela-tively small training set.

4.2 . Architecture

The novelty of our architecture lies in its hybrid charac-ter. The first part, which we will refer to as the feature ex-tractor in the following, aims at computing point-wise fea-tures on the input shapes. It corresponds to the Feat blockin Figure 2, and takes as input only the point clouds makingit robust towards changes in connectivity.

The purpose of the second part is to recover robust func-tional maps using these learned features. This block is builtaccording to the pipeline of [32], first taking the features tothe spectral domain over the two shapes (which correspondsto the dot products blocks after the Feat blocks in Figure2), and then computing the map by minimizing an energy.However, since our method is based on a neural network,this operation should be differentiable with respect to thefeatures over the shapes for the back-propagation algorithmto work. We extend the previously proposed functional map

Feat

Feat

FMreg

Figure 2. Overview of our approach: given a pair of shapes, weoptimize for a point cloud convolutional model to get point-wisefeatures for each shape, that we convert to a functional map usingour FMReg block. The loss that we put forward penalizes mapsaccording to their distance to the ground-truth map between thetwo shapes.

layers [25] to also incorporate a differentiable regularizer,which results in the very robust map extraction, representedas FMReg in Figure 2.

4.3 . The feature extractor

The goal of this block is to learn functional characteri-zations of point clouds that will later be used to computespectral descriptors and then functional maps. To this end,this network must be applied with the same weights to thesource and target shapes, as represented in Figure 2, andmust result in informative descriptors, extracted from thepoint clouds of the two shapes.

For this part, we chose the state of the art point cloudlearning method KPConv [47], by extending the segmenta-tion network proposed in that work. Our feature extractoris thus a Siamese version of the segmentation network de-scribed in KPConv, which we review for completeness inthe supplementary materials.

4.4 . The regularized FMap layer

This block provides a novel fully differentiable way tocompute a robust functional map from potentially low di-mensional spectral descriptors.

The main goal is, as in Section 3, to recover the ground-truth bijection between M and N , on which we have thecomputed raw-data features F and G.

For this we first express the computed feature func-tions in the respective spectral basis, which we denote byΦM and ΦN . This leads to the spectral descriptors A =(ΦM)†F and B = (ΦM)†G, with Φ† the Moore pseudo-inverse of Φ. We stress again that this step is where weshift focus from the spatial to the spectral domain, and cor-responds to the dot product blocks in Figure 2.

In the pipeline first introduced in [32] and then widelyused in the follow-up works [34], the standard strategy is tocompute the functional map C that optimizes the followingenergy:

minC

∥∥CA−B∥∥2 + λ

∥∥C∆M −∆NC∥∥2, (4)

where λ is a scalar regularization parameter.Remark that the optimization problem in Eq. (4.4 ) is

quadratic in terms of C and can be solved e.g. via standardconvex optimization techniques. However, in the learningcontext, we need to differentiate the solution with respectto the spectral features A,B, which is challenging when Cis computed via an iterative solver. Alternatively, the prob-lem in Eq. can be written directly in terms of a large leastsquares system, by vectorizing the matrix C as was sug-gested in [32]. However, for a k × k functional map, thisleads to a system of size k2×k2 which becomes prohibitiveeven for moderate values of k. To avoid these issues, pre-vious learning-based approaches based on functional maps[25, 19, 43] have only optimized for C using the first part ofthe energy in Eq. (4.4 ):

∥∥CA−B∥∥2. This results in a sim-

ple linear system for which the derivatives can be computedin closed form. This has two major limitations, however:first the linear system is only invertible if there are at least klinearly independent feature functions. This condition caneasily be violated in practice, especially in the early stagesof learning, potentially resulting in a fatal error. Further-more, the lack of regularization makes the solved-for func-tional map very sensitive to inconsistencies in the computeddescriptors, which leads to an overall loss of robustness.

In our work we address this problem by using the full en-ergy in Eq. (4.4 ) in a fully differentiable way. In particular,we use the fact that the operators ∆M and ∆N are diagonalwhen expressed in their own eigen-basis.

Indeed we remark that the gradient of the energy in Eq.(4.4 ) vanishes whenever CAAT +λ∆·C = BAT , wherethe operation · represents the element-wise multiplication,and ∆ij = (µMj − µNi )2, where µMl and µNl respectivelycorrespond to the eigenvalues of ∆M and ∆N . It is theneasy to see that this amounts to a separate linear system forevery row ci of C :

(AAT + λdiag((µ1j − µ2

i )2))ci = Abi (5)

where bi stands for ith row of B.In total, if k is the number of eigenvectors used for rep-

resenting the functional map, this operation amounts to in-verting k different k × k matrices. Since inverting a linearsystem is a differentiable operation, which is already imple-mented e.g. in TensorFlow, this allows us to estimate thefunctional map in a robust way, while preserving differen-tiability.

4.5 . The supervised spectral loss

Our method also uses a loss with respect to the groundtruth functional map in the spectral domain. This is similarto the energy used in [11], but is different from the loss of

the original FMNet work [25], which converted a functionalmap to a soft correspondence matrix and imposed a losswith respect to the ground truth point-wise map, relying onexpensive geodesic distance matrix computation.

Specifically, calling C the functional map obtained bythe FMap block, and Cgt the ground truth spectral map, ourloss is defined as:

lspec =∥∥C−Cgt

∥∥2As mentioned above, we use a Frbenius norm to computethe distance between matrices.

It is important to note that whenever a pointwise groundtruth map is given it is trivial to convert it to the functionalmap representation. Conversely, the ground truth spectralmap is more general than the point-wise ground truth corre-spondence. Indeed, with just a few precise landmarks onecan recover a functional map accurate enough to make thisloss efficient, for instance through the original pipeline of[32], but also with more recent follow-up works, such as[40] or [29], which we will further describe as baselines toour method in Section 5.

This is useful, e.g., in the case of re-meshed datasets. In-deed, complete ground truth correspondences between twoshapes of these datasets are not fully known. One can onlyhave access to the (often partial and not bijective) groundtruth pointwise map from a template mesh T to each re-meshed shape Si. Ecah such map can be converted to afunctional map Ci and a very good approximation of thespectral ground truth Cgti→j between Si and Sj can be ex-pressed as C†jCi.

4.6 . Postprocessing

Once our model is trained, we can then test it on a pair ofshapes and get a functional map between these shapes. Thismap can either directly be converted to a point to point map,or refined further. We use a very recent and efficient re-fining algorithm, called ZoomOut [29] based on navigatingbetween spatial and spectral domains while progressivelyinceasing the number of spectral basis functions. This effi-cient postprocessing technique allows us to get state-of-the-art results, as described in Section 5.

4.7 . Implementation

We implemented our method in TensorFlow [1] byadapting the open-source implementation of SURFMNet[43] and KPConv [47].

Our feature extraction network is based on a residualconvolutional architecture of [47], consisting of 4 convolu-tional blocks with leaky linear units, with successive pool-ings and dimension augmentation from 128 to 2048, fol-lowed by a 4 up-sampling blocks with shortcuts from cor-responding pooling layers, and dimension reduction from

2048 back to 128. Please see the Supplementary materials,part A, in [47] for more details. Following the pipeline ofKPConv, we start with a sub-sampled version of our pointclouds with a grid subsampling of step 0.03. The poolinglayers are therefore obtained with grid samplings of param-eters 0.06, 0.12, 0.24 and 0.48.

Similarly to FMNet [25] and SURFMNet [43], our net-work is applied in a Siamese way on the two shapes, usingthe same learned weights for the feature extractor.

In the case of fully automatic spectral methods such asBCICP [40] and ZoomOut [29], or the deep learning basedFMNet [25, 19] (supervised or unsupervised) and SURFM-net [43], all results are invariant by any rigid transforma-tion of the input shapes. However, in the case of meth-ods using the 3D coordinates of the points to generate in-formation about the input shape, this does not remain true.Consequently, both 3D-CODED [16] and our method avoidthis dependency through data augmentation to be as closeas possible to the generality of fully spectral methods. Tothat end, assuming the shapes are all aligned on one axis(e.g. on human the natural up axis), both 3D-CODED andour method perform data augmentation by randomly rotat-ing the input shapes along that axis.

4.8 . Parameters

In addition to the architecture above, our method has twokey hyper-parameters: the size of the functional basis andthe regularizer λ in Equation 5. For the size of the basis,we discovered if this number is too high, for instance, with120 eigenvectors as in FMNet and SURFMNet, it can easilylead to overfitting. However, by reducing this number to30, the results of SURFMNet on FAUST re-meshed (herereported in Table 1) go from 0.15 to 4.5. As a consequence,we choose the number of eigenvectors to be 30 in all ofour experiments on our method. Regarding the weight λ inEquation (5), we observed that setting it to λ = 10−3 helpsgetting good results while drastically reducing the numberof training steps, as pointed out in the ablation study. Weuse this value throughout all experiments.

We train our network with a batch size of 4 shape pairsfor a number of epochs depending on the number of shapesin the dataset. We use a learning rate of .001 and graduallydecreasing it to 0.0001 with ADAM optimizer [13].

5. ResultsDatasets

We test our method on a wide spectrum of humandatasets: first, the re-meshed versions of FAUST dataset [7]containing 100 human shapes in 1-1 correspondence, andof SCAPE [3], made publicly available by Ren et al. [40].These re-meshed datasets offer significantly more variabil-ity in terms of shape structures and connectivity, including

Method \ Dataset F S F on S S on FBCICP 15. 16. * *ZoomOut 6.1 7.5 * *SURFMNet 15. 12. 32. 32.SURFMNet+icp 7.4 6.1 19. 23.Unsup FMNet 10. 16. 29. 22.Unsup FMNet+pmf 5.7 10. 12. 9.3FMNet 11. 17. 30. 33.FMNet+pmf 5.9 6.3 11. 14.3D-CODED 2.5 31. 31. 33.Ours 3.1 4.4 11. 6.0Ours+zo 1.9 3.0 9.2 4.3

Table 1. Comparative results (×100) of the different methods onExperiment 1.

for instance point sampling density, making them harderto match for existing algorithms. We also highlight thatthe SCAPE dataset is slightly more challenging since theshapes are less regular, and two shapes never share the samepose. This is not true for FAUST, wherein all the posespresent in the test set also exist in the training set, with thevariation coming from body type only, making the pose re-covery easier at test time.

We also use the re-meshed version of the more recentSHREC’19 dataset [28], which, in theory, is the most chal-lenging of the test sets, because of stronger distortions in theposes, the presence of an incomplete shape, and the numberof test pairs (430 in total, so two times the number of testpairs of FAUST or SCAPE). At last, we also use the generictraining dataset of 3D-CODED [16], originally consistingin 230K synthetic shapes generated using Surreal [50], withthe parametric model SMPL introduced in [26]. We useit only for training purposes in our second experiment, toshow that our method can generalize well to changes in con-nectivity, being able to train on a synthetic, very smooth,identical triangulation for the whole training set, and stillproduce results of excellent quality on re-meshed datasets.

Ablation studyOur method is built with a number of building blocks,

all of which we consider essential to achieve optimal per-formance. To illustrate this, in the supplementary materialswe provide an extensive ablation study of all the key com-ponents of our algorithm.

BaselinesWe compare our method to several state of the art meth-

ods: the first category includes fully automatic methodswithout any learning component [40, 29]. These methodsare simply evaluated on the test sets without any training.The second category includes FMNet [25] and its unsu-pervised versions, referred to as Unsup FMNet [19] andSURFMNet [43], with and without post-processing (PMF

0 1000 2000 3000 4000 5000elements in training set

0

5

10

15

20

25

30av

erag

e ge

odes

ic er

ror (

*100

)FAUST re-meshed

3D codedOurs

0 1000 2000 3000 4000 5000elements in training set

0

5

10

15

20

25

30

aver

age

geod

esic

erro

r (in

cm

)

SCAPE re-meshed3D codedOurs

0 1000 2000 3000 4000 5000elements in training set

0

5

10

15

20

25

30

aver

age

geod

esic

erro

r (in

cm

)

SHREC re-meshed3D codedOurs

Figure 3. Comparison with 3D-CODED while varying training size of SURREAL dataset and simultaneously testing on other datasets.

[52] for FMNet, and standard functional map refinement[32], referred to as ICP, for SURFMNet). All these variantsof FMNet give similar results, but SURFMNet is the onlyone to train within a few hours, without requiring too muchspace. This is due to the fact SURFMNet only operates inthe spectral domain, in contrast to other methods. Lastly,we compare to the supervised 3D-CODED [16], describedearlier in more details in Section 3. For conciseness, we re-fer to our method as Ours in the following text. We showour results with and without ZoomOut [29] refinement, re-ferred to as ZO, in order to prove that our method standsout even without post processing. We compare these differ-ent methods in two main settings named Experiment 1 andExperiment 2 below.

Experiment 1 consists of evaluating the different meth-ods in the following setting: we split FAUST re-meshedand SCAPE re-meshed into training and test sets contain-ing 80 and 20 shapes for FAUST, and 51 and 20 shapesfor SCAPE. We obtain results for training and testing onthe same dataset, but also by testing on a different dataset.For instance, by training on SCAPE re-meshed train set andtesting on FAUST re-meshed test set. This experiment aimsat testing the generalization power of all methods to smallre-meshed datasets, as well as its ability to adapt to a differ-ent dataset at test time.

Experiment 2 consists of sampling 100, 500, 2000, and5000 shapes from the SURREAL dataset to be used fortraining. We then test the trained models on the test sets ofFAUST re-meshed, SCAPE re-meshed, and SHREC19 re-meshed. This experiment aims at testing the robustness andgeneralization power of the different methods in the pres-ence of varying amounts of training data, as well as adapt-ability to train on a perfect synthetic triangulations and stillget results on challenging re-meshed shapes.

Quantitative results

To evaluate the results, we use the protocol introducedin [22], where the per-point-average geodesic distance be-tween the ground truth map and the computed map is re-

ported. All results are multiplied by 100 for the sake ofreadability.

As we can see in Table 1, our method performs the bestoverall on Experiment 1. Fully automatic methods do notprovide competitive results compared to the learning meth-ods (except on crossed settings because they did not train onanything and are thus not influenced by the training shapes).As reported in the Section 3.1 , this highlights that hand-crafted features can easily fail. It is noticeable that spectralmethods (FMNet variations, and Ours as a hybrid method)get reasonable, or even good results in our case, with thesesmall datasets. In comparison, 3D-CODED seems to fail inalmost all cases. It is remarkable that it can learn on such asmall dataset as the training set of FAUST re-meshed. Oneexplanation for that is that FAUST contains the same set ofposes in the test set as in the train set.

Contrary to other baselines, our method gives good re-sults on all settings, even without refinement, showing goodresilience to a really low number of shapes, even with re-meshed geometry. We would like to stress that no othermethod is able to achieve such a generalization with thislow number of shapes.

For a fair comparison with 3D-CODED, we completeour study with a second experiment, in which the trainingset is now made of the same shapes 3D-CODED uses fortraining in their paper, namely SURREAL dataset. The aimof this experiment is to further showcase the generalizationpower of our method when compared to 3D-CODED. First,by training on a very smooth synthetic dataset, on whichprevious fully spectral methods tend to easily overfit due tothe obvious mismatch in triangulation in training and testset. Our second goal is to observe the dependence of differ-ent methods on size of the training set.

We report the results (multiplied by 100) of 3D-CODEDand Our method in Figure 3, as they are the only two com-petitive algorithms in Experiment 2. These results onceagain demonstrate that our method can achieve impressiveresults even with a low number of training shapes. OnSHREC re-meshed, we achieve state of the art results withan average error of 0.048 with only 500 training shapes. We

Source

ground truth

Ours+zo Ours 3D-CODED FMnet+PMF FMnet

2000 shapes

100 shapes

Figure 4. Qualitative results obtained with texture transfers for the different methods on Experiment 2, training on two different numbersof shapes in the SURREAL dataset, and testing on SHREC re-meshed shapes.

provide additional quantitative comparisons in the supple-mentary materials.

It can be observed in Figure 3 that our results are consis-tent and unaltered even with the drop in number of trainingshapes. 3D-CODED, on the other hand, always suffers froma reduced training set.

Qualitative results

In Figure 4 we show the results of our method (with andwithout ZoomOut refinement [29]), 3D-CODED [16], FM-Net [25] (with and without PMF refinement [51]), trainedon respectively 2000 and 100 shapes, as presented in Ex-periment 2, via texture transfer.

With 2000 training shapes, both our method and 3D-CODED lead to good or even excellent texture transfers,while fully spectral methods fail due to the change of con-nectivity from training to test set. However, with only 100training shapes, 3D-CODED fails to get a good reconstruc-tion in many cases, leading to bad texture transfer as in Fig-ure 4. This highlights the fact that our method performs bet-ter than any other existing method when only a few trainingshapes are provided.

6. Conclusion, Limitations & Future Work

We presented a method for improving the robustnessand reducing overfitting in learning shape correspondences.Key to our approach is a hybrid network structure, made ofa raw-data feature extractor that learns descriptors on a pairof shapes, and a novel robust functional map layer. Our net-work can thus operate in both the spectral and the spatialdomain, thus taking advantages of both representations.

Our approach has several limitations: first, as a super-vised method it requires at least partial correspondences(as discussed in Section 4.5 ) between the training shapes.Also, it requires data augmentation to be able to predictnon-aligned shapes, which can be costly and unstable.

In the future, we plan to work towards an unsupervisedspectral loss, similar in spirit to SURFMNet [43], whileavoiding the symmetry ambiguity problem. We also planto try other, invariant feature extractors such as [18], or [35]to avoid data augmentation.

Acknowledgements This work was supported by KAUSTOSR Award No. CRG-2017-3426, a gift from Nvidia andthe ERC Starting Grant No. 758800 (EXPROTEA). We aregrateful to Jing Ren, Simone Melzi and Riccardo Marin forthe help with re-meshed shapes, and SHREC’19 dataset.

References[1] Martın Abadi, Ashish Agarwal, and Paul Barham et al. Ten-

sorFlow: Large-scale machine learning on heterogeneoussystems, 2015. Software available from tensorflow.org. 5

[2] Yonathan Aflalo, Anastasia Dubrovina, and Ron Kimmel.Spectral generalized multi-dimensional scaling. Interna-tional Journal of Computer Vision, 118(3):380–392, 2016.2

[3] Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Se-bastian Thrun, Jim Rodgers, and James Davis. SCAPE:Shape Completion and Animation of People. In ACM Trans-actions on Graphics (TOG), volume 24, pages 408–416.ACM, 2005. 6

[4] Matan Atzmon, Haggai Maron, and Yaron Lipman. Pointconvolutional neural networks by extension operators. ACMTransactions on Graphics, 37, 03 2018. 2, 3, 4

[5] Mathieu Aubry, Ulrich Schlickewei, and Daniel Cremers.The Wave Kernel Signature: A Quantum Mechanical Ap-proach to Shape Analysis. In Proc. ICCV Workshops, pages1626–1633. IEEE, 2011. 2, 3

[6] Silvia Biasotti, Andrea Cerri, A Bronstein, and M Bronstein.Recent trends, applications, and perspectives in 3d shapesimilarity assessment. In Computer Graphics Forum, vol-ume 35, pages 87–119, 2016. 2

[7] Federica Bogo, Javier Romero, Matthew Loper, andMichael J. Black. FAUST: Dataset and evaluation for 3Dmesh registration. In Proceedings IEEE Conf. on Com-puter Vision and Pattern Recognition (CVPR), Piscataway,NJ, USA, June 2014. IEEE. 1, 6

[8] Davide Boscaini, Jonathan Masci, Emanuele Rodola, andMichael M. Bronstein. Learning shape correspondence withanisotropic convolutional neural networks. In Proc. NIPS,pages 3189–3197, 2016. 2

[9] Oliver Burghard, Alexander Dieckmann, and ReinhardKlein. Embedding shapes with Green’s functions for globalshape matching. Computers & Graphics, 68:1–10, 2017. 2

[10] Mingjia Chen, Changbo Wang, and Hong Qin. Jointly learn-ing shape descriptors and their correspondence via deeptriplet cnns. Computer Aided Geometric Design, 62:192–205, 2018. 2

[11] Etienne Corman, Maks Ovsjanikov, and Antonin Chambolle.Supervised descriptor learning for non-rigid shape matching.In Proc. ECCV Workshops (NORDIA), 2014. 1, 2, 3, 5

[12] Theo Deprelle, Thibault Groueix, Matthew Fisher,Vladimir G Kim, Bryan C Russell, and Mathieu Aubry.Learning elementary structures for 3d shape generation andmatching. arXiv preprint arXiv:1908.04725, 2019. 2, 3

[13] J. Ba D.P. Kingma. Adam: A method for stochastic opti-mization. In ICLR, 2015. 6

[14] Davide Eynard, Emanuele Rodola, Klaus Glashoff, andMichael M Bronstein. Coupled functional maps. In 3D Vi-sion (3DV), pages 399–407. IEEE, 2016. 2

[15] Danielle Ezuz and Mirela Ben-Chen. Deblurring and denois-ing of maps between shapes. In Computer Graphics Forum,volume 36, pages 165–174. Wiley Online Library, 2017. 2,3

[16] Thibault Groueix, Matthew Fisher, Vladimir G Kim,Bryan C Russell, and Mathieu Aubry. 3d-coded: 3d cor-respondences by deep deformation. In Proceedings of theEuropean Conference on Computer Vision (ECCV), pages230–246, 2018. 1, 2, 3, 6, 7, 8, 14

[17] Thibault Groueix, Matthew Fisher, Vladimir G Kim,Bryan C Russell, and Mathieu Aubry. Unsupervised cycle-consistent deformation for shape matching. In ComputerGraphics Forum, volume 38, pages 123–133. Wiley OnlineLibrary, 2019. 1

[18] Paul Guerrero, Yanir Kleiman, Maks Ovsjanikov, andNiloy J. Mitra. PCPNet: Learning local shape proper-ties from raw point clouds. Computer Graphics Forum,37(2):75–85, 2018. 8

[19] Oshri Halimi, Or Litany, Emanuele Rodol‘a, Alex Bronstein,and Ron Kimmel. Unsupervised learning of dense shape cor-respondence. In CVPR, 2019. 1, 2, 3, 4, 5, 6, 14

[20] Qixing Huang, Fan Wang, and Leonidas Guibas. Functionalmap networks for analyzing and exploring large shape col-lections. ACM Transactions on Graphics (TOG), 33(4):36,2014. 2

[21] Ruqi Huang and Maks Ovsjanikov. Adjoint map representa-tion for shape analysis and matching. In Computer GraphicsForum, volume 36, pages 151–163. Wiley Online Library,2017. 2

[22] Vladimir G Kim, Yaron Lipman, and Thomas Funkhouser.Blended intrinsic maps. In ACM Transactions on Graphics(TOG), volume 30, page 79. ACM, 2011. 7, 12, 13

[23] Artiom Kovnatsky, Michael M Bronstein, Xavier Bresson,and Pierre Vandergheynst. Functional correspondence bymatrix completion. In Proceedings of the IEEE conferenceon computer vision and pattern recognition, pages 905–914,2015. 2

[24] Artiom Kovnatsky, Michael M Bronstein, Alexander MBronstein, Klaus Glashoff, and Ron Kimmel. Coupled quasi-harmonic bases. In Computer Graphics Forum, volume 32,pages 439–448, 2013. 2

[25] Or Litany, Tal Remez, Emanuele Rodola, Alexander M.Bronstein, and Michael M. Bronstein. Deep functional maps:Structured prediction for dense shape correspondence. 2017IEEE International Conference on Computer Vision (ICCV),pages 5660–5668, 2017. 1, 2, 3, 4, 5, 6, 8, 11, 14

[26] Matthew Loper, Naureen Mahmood, Javier Romero, GerardPons-Moll, and Michael J. Black. Smpl: A skinned multi-person linear model. ACM Trans. Graph., 34(6):248:1–248:16, Oct. 2015. 3, 6

[27] Jonathan Masci, Davide Boscaini, Michael Bronstein, andPierre Vandergheynst. Geodesic convolutional neural net-works on riemannian manifolds. In Proceedings of theIEEE international conference on computer vision work-shops, pages 37–45, 2015. 2

[28] S. Melzi, R. Marin, E. Rodol, U. Castellani, J. Ren, A. Poule-nard, P. Wonka, and M. Ovsjanikov. Matching Humans withDifferent Connectivity. In Silvia Biasotti, Guillaume Lavou,and Remco Veltkamp, editors, Eurographics Workshop on3D Object Retrieval. The Eurographics Association, 2019.3, 6

[29] Simone Melzi, Jing Ren, Emanuele Rodola, Maks Ovs-janikov, and Peter Wonka. Zoomout: Spectral upsamplingfor efficient shape correspondence. ACM Transactions onGraphics (Proc. SIGGRAPH Asia), 2019. 2, 3, 5, 6, 7, 8, 11,14

[30] Federico Monti, Davide Boscaini, Jonathan Masci,Emanuele Rodola, Jan Svoboda, and Michael M. Bronstein.Geometric deep learning on graphs and manifolds usingmixture model cnns. In CVPR, pages 5425–5434. IEEEComputer Society, 2017. 2

[31] Dorian Nogneng and Maks Ovsjanikov. Informative descrip-tor preservation via commutativity for shape matching. Com-puter Graphics Forum, 36(2):259–267, 2017. 2

[32] Maks Ovsjanikov, Mirela Ben-Chen, Justin Solomon, AdrianButscher, and Leonidas Guibas. Functional Maps: A Flexi-ble Representation of Maps Between Shapes. ACM Transac-tions on Graphics (TOG), 31(4):30, 2012. 2, 4, 5, 7, 12

[33] Maks Ovsjanikov, Etienne Corman, Michael Bronstein,Emanuele Rodola, Mirela Ben-Chen, Leonidas Guibas,Frederic Chazal, and Alex Bronstein. Computing and pro-cessing correspondences with functional maps. In ACM SIG-GRAPH 2017 Courses, SIGGRAPH ’17, pages 5:1–5:62,2017. 2

[34] Maks Ovsjanikov, Etienne Corman, Michael Bronstein,Emanuele Rodola, Mirela Ben-Chen, Leonidas Guibas,Frederic Chazal, and Alex Bronstein. Computing and pro-cessing correspondences with functional maps. In ACM SIG-GRAPH 2017 Courses, pages 5:1–5:62, 2017. 4

[35] Adrien Poulenard, Marie-Julie Rakotosaona, Yann Ponty,and Maks Ovsjanikov. Effective rotation-invariant pointcnn with spherical harmonics kernels. arXiv preprintarXiv:1906.11555, 2019. 8

[36] Adrien Poulenard, Primoz Skraba, and Maks Ovsjanikov.Topological function optimization for continuous shapematching. In Computer Graphics Forum, volume 37, pages13–25. Wiley Online Library, 2018. 2

[37] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas.Pointnet: Deep learning on point sets for 3d classificationand segmentation. In Proc. CVPR, pages 652–660, 2017. 2,3, 11

[38] Charles R. Qi, Li Yi, Hao Su, and Leonidas J. Guibas.Pointnet++: Deep hierarchical feature learning on point setsin a metric space. In Proceedings of the 31st Interna-tional Conference on Neural Information Processing Sys-tems, NIPS’17, pages 5105–5114, USA, 2017. Curran As-sociates Inc. 2

[39] Jing Ren, Mikhail Panine, Peter Wonka, and Maks Ovs-janikov. Structured regularization of functional map com-putations. In Computer Graphics Forum, volume 38, pages39–53. Wiley Online Library, 2019. 2

[40] Jing Ren, Adrien Poulenard, Peter Wonka, and Maks Ovs-janikov. Continuous and orientation-preserving correspon-dences via functional maps. ACM Transactions on Graphics(TOG), 37(6), 2018. 2, 3, 5, 6

[41] Emanuele Rodola, Luca Cosmo, Michael M Bronstein, An-drea Torsello, and Daniel Cremers. Partial functional corre-spondence. In Computer Graphics Forum, volume 36, pages222–236. Wiley Online Library, 2017. 2

[42] Emanuele Rodola, M Moeller, and Daniel Cremers. Point-wise map recovery and refinement from functional corre-spondence. In Proc. Vision, Modeling and Visualization(VMV), 2015. 2, 3

[43] Jean-Michel Roufosse, Abhishek Sharma, and Maks Ovs-janikov. Unsupervised deep learning for structured shapematching. In Proceedings of the IEEE International Con-ference on Computer Vision, pages 1617–1627, 2019. 1, 2,3, 4, 5, 6, 8

[44] Yusuf Sahillioglu. Recent advances in shape correspon-dence. The Visual Computer, pages 1–17, 2019. 2

[45] Jian Sun, Maks Ovsjanikov, and Leonidas Guibas. A Con-cise and Provably Informative Multi-Scale Signature Basedon Heat Diffusion. In Computer graphics forum, volume 28,pages 1383–1392, 2009. 2, 3

[46] Zhiyu Sun, Yusen He, Andrey Gritsenko, Amaury Lendasse,and Stephen Baek. Deep spectral descriptors: Learning thepoint-wise correspondence metric via siamese deep neuralnetworks. arXiv preprint arXiv:1710.06368, 2017. 2

[47] Hugues Thomas, Charles R. Qi, Jean-Emmanuel Deschaud,Beatriz Marcotegui, Francois Goulette, and Leonidas J.Guibas. Kpconv: Flexible and deformable convolution forpoint clouds. Proceedings of the IEEE International Confer-ence on Computer Vision, 2019. 2, 3, 4, 5, 6, 10, 11

[48] Federico Tombari, Samuele Salti, and Luigi Di Stefano.Unique signatures of histograms for local surface descrip-tion. In International Conference on Computer Vision(ICCV), pages 356–369, 2010. 2, 3, 11

[49] Oliver Van Kaick, Hao Zhang, Ghassan Hamarneh, andDaniel Cohen-Or. A survey on shape correspondence. InComputer Graphics Forum, volume 30, pages 1681–1707,2011. 1, 2

[50] Gul Varol, Javier Romero, Xavier Martin, Naureen Mah-mood, Michael J. Black, Ivan Laptev, and Cordelia Schmid.Learning from synthetic humans. In CVPR, 2017. 3, 6

[51] Matthias Vestner, Zorah Lahner, Amit Boyarski, Or Litany,Ron Slossberg, Tal Remez, Emanuele Rodola, Alex Bron-stein, Michael Bronstein, Ron Kimmel, and Daniel Cre-mers. Efficient deformable shape correspondence via kernelmatching. In Proc. 3DV, 2017. 8, 14

[52] M. Vestner, R. Litman, E. Rodola, A. Bronstein, and D. Cre-mers. Product manifold filter: Non-rigid shape correspon-dence via kernel density estimation in the product space. InProc. CVPR, pages 6681–6690, 2017. 6

[53] Lingyu Wei, Qixing Huang, Duygu Ceylan, Etienne Vouga,and Hao Li. Dense human body correspondences using con-volutional networks. In Proceedings of the IEEE Conferenceon Computer Vision and Pattern Recognition, pages 1544–1553, 2016. 1, 2

7. SupplementA . Additional details on KPConv [47]

Here, we review briefly KPConv method and describethe architecture we used in our implementation.

The input to this network is a 3D point cloud equippedwith a signal, such as the 3D coordinates of the points. Let

P ∈ RN×3 be a point cloud in R3. Let F ∈ RN×D be aD-dimensional feature signal over P .

The goal of point cloud convolutional networks is to re-produce the architecture of convolutional neural networkson images. It boils down to transferring two key operationson the point cloud structure: the convolution and the pool-ing operators.

First, we define a convolution between F and a kernelg at point x ∈ R3. Since we only want a signal over thepoint cloud at each layer, we only need these convolutionsat x ∈ P .

The kernel will be defined as a local function centeredon 0 depending on some learnable parameters, taking aD-dimensional feature vector as input and yielding a D′-dimensional feature vector.

More specifically, the kernel is defined in the followingway : let r be its radius of action. Let Br3 be the correspond-ing 3d ball. LetK be the number of points, thus the numberof parameter matrices in this kernel. Let {zk|k < K} ⊂ Br3be these points, and {Wk|k < K} ⊂ M(D,D′)(R) be thesematrices. Then the kernel g is defined through the formula :

g(y) =∑k<K

h(y, zk)Wk

where we simply set h(y, z) = max(0, 1 − ‖y−z‖σ ), soeach point of the kernel has a linear influence of range σaround it.

Then the convolution simply becomes :

(F ∗ g)(x) =∑

i|xi∈Br3

g(xi − x)fi

Where the learnable parameters are the matricesWk. Weset the points zk of the kernel to be uniformly organized inBr3, so as to better encompass the variations of the convo-luted signal at a given point of the point cloud, and a givenscale (see [47] supplementaries, Section B for more details).

For the pooling operator, we use a grid sampling thatallows us to get the point cloud at an adjustable density.The network can then build hierarchical features over thepoint clouds by both adjusting the radius of influence ofits kernels and the density of the mesh they are performedupon.

Once these two operations are set up, it is easy to builda convolutional feature extractor over the point cloud P . Inour work, we use the following architecture :

• Four strided convolutional blocks, each down-sampling the point cloud to half its density, and takingthe feature space to another feature space (correspond-ing to higher-level features) two times larger.

• Four up-sampling layers, taking the signal back on thewhole point cloud, through skipping connections fol-lowed by 1D convolutions.

Method No Ref RefFMNet 17. 13.PointNet 18. 14.Old FMap 4.5 1.9Ours 3.4 1.9

Table 2. Comparative results for the different ablations of ourmethod.

B . Ablation study

This section presents the extensive ablation study of allthe vital components of our algorithm.

We train all these ablations on 100 random shapes amongthe 230K proposed by 3D-CODED, as in experiment 2. Wetest them on the 20 test shapes of FAUST re-meshed, so thatthe connectivity differs from train to test. The different partsto ablate are :

• The point cloud feature extractor : as explained in theprevious sections, it learns descriptors from raw datawithout relying too much on connectivity. The firstablation consists of our method, but with FMNet [25]feature extractor instead of ours. Similar to FMNet,we use SHOT [48] descriptors. However, we use thesame number of eigenvectors as in our method, namely30. As a general remark, we noticed that lowering thisnumber can often help prevent overfitting in the caseof FMNet-based architectures.

• The choice of KPConv [47]. The second ablation studyreplaces KPConv sampling and feature extractor blockwith that of PointNet [37]. For this ablation, we userandom sampling to 1500 points instead of KPConvgrid sampling. Indeed, grid sampling does not provideany guarantee on the number of points after sampling,so it can only be used in a network built to overcomethis issue, with batches of adaptable size, which is notthe case of PointNet.

• The regularized functional map layer. This third abla-tion simply consists in replacing our regularized func-tional map layer by the old functional map layer orig-inally introduced by FMNet. Our layer is in theorymildly heavier than the original one, but in practicefor less than 50 eigenvectors the computation times re-main the same.

• Lastly, we remove the post-processing refinement step.Here we show the results of every ablation with andwithout refinement, thus proving it helps in getting bet-ter results. As a refinement method, we use the state-of-the-art ZoomOut [29], as mentioned in the mainmanuscript.

0 5000 10000 15000 20000 25000 30000 35000number of training steps

0.0

2.5

5.0

7.5

10.0

12.5

15.0

17.5

20.0su

perv

ised

loss

Training loss with and without FMap RegRegNoReg

5000 10000 15000 20000 25000 30000 35000number of training steps

5

10

15

20

25

avg

geod

esic

erro

r (on

FAU

ST re

-mes

hed)

Evaluation error with and without FMap RegRegNo RegReg (refined)No Reg (refined)

Figure 5. Comparison of convergence speed with and without Laplacian Regularization in the FMap block. Left: Training loss evolution,Right: Evolution of geodesic error on test set with the number of epochs. Notice how the regularized fmap layer helps drastically with theconvergence speed. It gives optimal results within only 500 epochs.

Table 2 shows the ablation study of our method. Itdemonstrates the importance of all individual blocks andascertains that all these components are needed to achieveoptimal performance with our solution.

However, just looking at the results of the ablation studyone does not see the importance of the FMap Reg addition.To prove its efficiency, we compare the learning and evalu-ation curves of our method, with and without this addition.As can be seen in Figure 5, the models converge much fasterwith our regularized functional map layer. The models aretrained on 100 shapes of the surreal dataset of 3D-CODEDas in Experiment 2, and tested on FAUST re-meshed.

In addition, our regularized functional map layer is morerobust, and does not result in a Cholesky Decomposition fa-tal error when computing the spectral map. In comparison,the previous functional map layer gave that fatal error insome experiments, and the model had to be relaunched.

Graphically, as reported in the original functional mappaper [32], a natural functional map should be funnelshaped. Our regularized functional map layer naturallycomputes maps that almost commute with the Laplacianson the shapes. These maps will naturally be close to diago-nal matrices (as are funnel shaped maps) in the eigenbasis ofthe Laplacians, thus reducing the space of matrices attain-able by this layer. We believe it helps the feature extractorblock focus on setting the diagonal coefficients of the func-tional map as in the ground truth, rather than on trying toget its funnel shape in the first thousand iterations, which iswhat a model with the original functional map layer does.

C . More quantitative results

Figures 6 and 7 summarize the accuracy obtained by ourmethod and some baselines on the different settings of thetwo experiments we conducted, using the evaluation pro-tocol introduced in [22]. Note that in all cases but one(trained on 2000 shapes of SURREAL, tested on SCAPEre-meshed), our network achieves the best results even com-

pared to the state-of-the-art methods. As explained morethoroughly in the main manuscript, this proves our methodis able to learn point cloud characterizations with only asmall amount of data, and by projecting these descriptors ina spectral basis can retrieve accurate correspondences fromthem. Our method does not need any template and is thusmore general than 3D-CODED, in addition to the fact thatit trains faster and does not need a big training set.

We even believe the superiority of our method with a lownumber of training shapes shapes is partially due to this factthat 3D-CODED uses a template and operates in the spatialdomain, unlike our approach which is template-free, andpartly operates in the spectral domain, making it easier toadapt to any new category of 3D shapes.

The relatively low performance of our method onSCAPE in Experiment 2 (see Figure 7) is due to the pres-ence of back-bent shapes in this dataset. These shapes areseen by the network through their truncated spectral approx-imation, as discussed in section D , making it unable to ex-ploit refined features such as the face or hands, that couldhelp getting descriptors able to differentiate left from right.Consequently, as there are no back-bent shapes in the train-ing sets of this experiment, these shape are often mappedwith a left-to-right symmetry, resulting in a huge error forthese particular shapes, increasing the mean error for thewhole SCAPE test set.

D . Visualization of some descriptors learned by ourmethod

Our method aims at building descriptors on both inputshapes (that are often labeled source and target shapes) fromtheir raw point cloud data. These descriptors are then pro-jected on the eigen basis of the respective Laplace-Beltramioperators of the source and the target shapes. We outputthese projections, that we call spectral descriptors, and wevisualize some of them in Figure 8.

It is remarkable that the descriptors learned on a para-

0.00 0.02 0.04 0.06 0.08 0.10error

0.0

0.2

0.4

0.6

0.8

1.0

fract

ion

of c

orre

spon

denc

es b

ello

w

Geodesic error on FAUST re-meshed (trained on FAUST re-meshed)

Ours : 0.0315Ours + ZO : 0.0188FMNet : 0.1268FMNet + PMF : 0.058Unsup FMNet : 0.1032Unsup FMNet + PMF : 0.0572SURFMNet : 0.1479SURFMNet + ICP : 0.07393d Coded : 0.0192

0.00 0.02 0.04 0.06 0.08 0.10error

0.0

0.2

0.4

0.6

0.8

1.0

fract

ion

of c

orre

spon

denc

es b

ello

w

Geodesic error on SCAPE re-meshed (trained on SCAPE re-meshed)

Ours : 0.0439Ours + ZO : 0.0303FMNet : 0.1581FMNet + PMF : 0.0704Unsup FMNet : 0.1639Unsup FMNet + PMF : 0.0992SURFMNet : 0.1246SURFMNet + ICP : 0.06073d Coded : 0.3095

0.00 0.02 0.04 0.06 0.08 0.10error

0.0

0.2

0.4

0.6

0.8

1.0

fract

ion

of c

orre

spon

denc

es b

ello

w

Geodesic error on FAUST re-meshed (trained on SCAPE re-meshed)

Ours : 0.0599Ours + ZO : 0.0429FMNet : 0.3346FMNet + PMF : 0.1398Unsup FMNet : 0.2211Unsup FMNet + PMF : 0.0931SURFMNet : 0.3152SURFMNet + ICP : 0.22513d Coded : 0.3265

0.00 0.02 0.04 0.06 0.08 0.10error

0.0

0.2

0.4

0.6

0.8

1.0

fract

ion

of c

orre

spon

denc

es b

ello

w

Geodesic error on SCAPE re-meshed (trained on FAUST re-meshed)

Ours : 0.1102Ours + ZO : 0.0924FMNet : 0.3003FMNet + PMF : 0.1078Unsup FMNet : 0.2865Unsup FMNet + PMF : 0.1218SURFMNet : 0.3162SURFMNet + ICP : 0.18693d Coded : 0.3067

Figure 6. Quantitative results of the different methods using the protocol introduced in [22], on all the settings of Experiment 1

0.00 0.02 0.04 0.06 0.08 0.10error

0.0

0.2

0.4

0.6

0.8

1.0

fract

ion

of c

orre

spon

denc

es b

ello

w

Geodesic error on FAUST re-meshed (trained on 100 SURREAL shapes)

Ours : 0.0346Ours + ZO : 0.0191FMNet : 0.3564FMNet + PMF : 0.1606Unsup FMNet : 0.3517Unsup FMNet + PMF : 0.14743d Coded : 0.0921

0.00 0.02 0.04 0.06 0.08 0.10error

0.0

0.2

0.4

0.6

0.8

1.0

fract

ion

of c

orre

spon

denc

es b

ello

w

Geodesic error on SCAPE re-meshed (trained on 100 SURREAL shapes)

Ours : 0.1657Ours + ZO : 0.1554FMNet : 0.3787FMNet + PMF : 0.1709Unsup FMNet : 0.3983Unsup FMNet + PMF : 0.26593d Coded : 0.3064

0.00 0.02 0.04 0.06 0.08 0.10error

0.0

0.2

0.4

0.6

0.8

1.0

fract

ion

of c

orre

spon

denc

es b

ello

w

Geodesic error on SHREC re-meshed (trained on 100 SURREAL shapes)

Ours : 0.0809Ours + ZO : 0.0677FMNet : 0.547FMNet + PMF : 0.4755Unsup FMNet : 0.5341Unsup FMNet + PMF : 0.44883d Coded : 0.2984

0.00 0.02 0.04 0.06 0.08 0.10error

0.0

0.2

0.4

0.6

0.8

1.0

fract

ion

of c

orre

spon

denc

es b

ello

w

Geodesic error on FAUST re-meshed (trained on 2000 SURREAL shapes)

Ours : 0.0331Ours + ZO : 0.0188FMNet : 0.3624FMNet + PMF : 0.1409Unsup FMNet : 0.3951Unsup FMNet + PMF : 0.18363d Coded : 0.0417

0.00 0.02 0.04 0.06 0.08 0.10error

0.0

0.2

0.4

0.6

0.8

1.0

fract

ion

of c

orre

spon

denc

es b

ello

w

Geodesic error on SCAPE re-meshed (trained on 2000 SURREAL shapes)

Ours : 0.1132Ours + ZO : 0.1015FMNet : 0.3701FMNet + PMF : 0.2057Unsup FMNet : 0.3947Unsup FMNet + PMF : 0.27463d Coded : 0.0569

0.00 0.02 0.04 0.06 0.08 0.10error

0.0

0.2

0.4

0.6

0.8

1.0

fract

ion

of c

orre

spon

denc

es b

ello

w

Geodesic error on SHREC re-meshed (trained on 2000 SURREAL shapes)

Ours : 0.0703Ours + ZO : 0.0579FMNet : 0.5174FMNet + PMF : 0.4862Unsup FMNet : 0.5414Unsup FMNet + PMF : 0.45773d Coded : 0.0896

Figure 7. Quantitative results of the different methods using the protocol introduced in [22], on two of the settings of Experiment 2. Toprow: 100 shapes (low number). Bottom row: 2000 shapes (high number).

metric dataset such as the one used in 3D-CODED stillgeneralize well to shapes with entirely different mesh con-nectivity and number of points. This is made possible bytwo components of our method. Firstly, it down-samplesthe input shapes through grid sampling before buildingthese descriptor functions with convolutional neural net-

works. This allows for regularity in the input point cloudsat all different hierarchies (see Figure 10 for an exampleof such grid sampling). Secondly, the spectral projectionstake these point cloud descriptions to the shapes intrinsicspace, adding some comprehensive surface-related infor-mation without depending too much on the connectivity,

mesh 2

mesh 25

mesh 21

mesh 30

Figure 8. Visualization of spectral descriptors learned by our method (with 2000 surreal shapes) on a test pair of SCAPE re-meshed. Thesource shape is shown in the first row, and the target shape in the bottom row. Notice how the descriptors are localized and seem to highlightone specific part of the body (first column for shoulder, second for scalp, third for right thigh, fourth for right side of the torso, fifth forelbow).

like with SHOT descriptors. Without this intrinsic trans-lation, the network could have trouble differentiating twogeometric components close in euclidean space, such as forinstance the arms in mesh 25 of Figure 8.

Additionally, these descriptors seem to capture somesegmentation information, such as for instance head, arms,body and legs for humans, as can be observed in Figure 8.More precise or complex descriptors such as hand or facialdescriptors can not appear with only 30 eigen vectors. Itis due to the fact that a spectral reconstruction of a humanshape with only 30 eigenvectors does not show small de-tails such as hands, feet or facial features. One would needto push the number of eigenvectors above 100 to see suchdescriptors appear, and used correctly by the algorithm toproduce even better correspondences. However, this couldalso more easily lead to overfitting.

E . Additional Texture transfer on SHREC’19 re-meshed

In Figure 9 are shown additional qualitative results ofour method (with and without Zoomout refinement [29]),3D coded [16], FMNet [25] and Unsupervised FMNet [19](with and without PMF refinement [51]), trained on respec-tively 2000 and 100 shapes, as presented in the Results sec-tion, Experiment 2, of the main manuscript.

These results show again the failure of FMNet, due tothe change in connectivity. It can be seen more thoroughlyin the quantitative graphs provided in Figure 7.

This pair of shapes in Figure 9 represents a challeng-ing case for both 3D-CODED and our method. Indeed,these networks are not rotation invariant, as discussed inthe implementation section of the main manuscript. Here,the source shape is bent over and its head is really low com-

Source

ground truth

Ours+zo Ours 3D-CODED FMnet+PMF FMnet

2000 shapes

100 shapes

Figure 9. Qualitative results on Experiment 2 through texture transfer, showing cases where our method is the only one that can give goodcorrespondence with only 100 training shapes.

pared with the rest of the body. 3D-CODED and our methodare made robust to rotation around the vertical axis throughdata augmentation, but here the source shape is slightly ro-tated along another axis. As we can see, this resulted in poorreconstructions in the case of 3D-CODED algorithm, evenwith 2000 training shapes, whereas our method was able toyield good results with both a high and a low number ofshapes.

F . General Pipeline

We also provide a visual illustration of our generalpipeline in Figure 10 to complement the textual descriptionof our method provided in the main manuscript.

Supervised Loss

f-mapGround-truth

f-map

Descriptorshierarchical Point Clouds

Shapes

CNN

CNN

Figure 10. Pipeline of our method: 1) Down-sample source and target shapes with grid sampling (providing the pooling at different scales).2) Learning point cloud characterizations and project them in the Laplace-Beltrami eigen basis. 3) Compute the functional map fromsource and target spectral descriptors, with our regularized fmap layer. 4) Compute the loss by comparing the computed functional mapwith the ground truth map.