292 |
B. Bustos and I. Sipiran |
Fig. 7.8 (a, b) Matching with shapes from the same category. (c) Matching between shapes from different categories. Figure courtesy of [52]
dataset. Also, an important aspect of this technique is that it delivers a set of correspondences between salient points, in addition to the matching score. It is valuable because two problems (matching and correspondences) can be addressed with the same approach.
7.3.3.5 Complexity Analysis
Let S be a 3D object with n vertices. In the following, we list the computational complexity associated with the various stages of the presented approach.
•Compute the Laplacian matrix: O(n2).
•Compute eigenvalues and eigenvectors of the Laplacian matrix: O(n3).
•Interest point detection: O(nk), if k eigenvectors are chosen.
•Interest point description: O(mq3), where m is the number of detected interest points and q is the number of vertices in the patch associated to each interest point.
•Matching: Let suppose two objects P and Q with N and M interest points, respectively. The computational complexities of each step of the matching are:
–Computation of matrix C: O(N M).
–Computation of matrix H: O(N 3M2 log N ). The matrix H corresponds with all the possible combinations of correspondence pairs. For each pair, the computation of the geodesic distance takes O(N log N ).
–Solve the IQP problem: O(N 2M2).
The total complexity of this method is dominated by the calculation of the matrix H, due to the computation of the geodesic distances between interest points. Therefore, the complexity of this method is O(N 3M2 log N ).
7.3.4 Heat Kernel Signatures
We observed in the previous approach the importance and applicability of the Laplace-Beltrami operator and its spectrum in shape description tasks. In this section, we describe the application of heat kernel signatures which are based on the
7 3D Shape Matching for Retrieval and Recognition |
293 |
Fig. 7.9 Process to obtain a descriptor for a shape using the heat kernel signatures and bag of features. (a) Input shape, (b) local descriptors (heat kernel signatures) are extracted, (c) clustering of descriptor space (black stars are centroids of resulting clusters), and (d) Vector quantization, using local descriptors and clusters, results in shape descriptors
intimate relation between the heat diffusion process and the Laplace-Beltrami operator. In addition, the presented technique utilizes a widely-used approach for describing information entities based on their components and frequencies known as Bag-of-Features [83]. Figure 7.9 summarizes the approach.
The heat diffusion process over a compact manifold S, possibly with boundary, is governed by the heat equation
Su(x, t ) = − |
∂u(x, t ) |
(7.42) |
∂t |
where S is the Laplace-Beltrami operator of S and u(., t ) is the heat distribution over S over time t .
The fundamental solution of Eq. (7.42) is Kt (x, y) called the heat kernel. This represents a solution with a point heat source at x and can be considered as the amount of heat transferred from x to y after time t . For compact manifolds, the heat kernel can be expressed using the eigenvalues and eigenvectors of the LaplaceBeltrami operator as follows:
∞ |
|
|
|
Kt (x, y) = exp(−λi t )vi (x)vi (y) |
(7.43) |
i=0
where λi is the i-th eigenvalue and vi (·) is the i-th eigenvector’s entry corresponding to a given point.
Sun et al. [97] formally proved that the heat kernel is an isometric invariant, informative (redundant information exists), multi-scale, and stable against perturbations of the surface. In addition, restricting the heat kernel to the temporal domain and fixing the spatial variables, we can obtain a representation for each point on the manifold:
∞ |
|
|
|
Kt (x, x) = exp(−λi t )vi (x)2 |
(7.44) |
i=0
294 |
B. Bustos and I. Sipiran |
Fig. 7.10 Heat kernel signatures calculated on two isometric shapes. At top, signatures in corresponding points look very similar. At bottom, signatures in different points on the mesh differ
In Fig. 7.10, we show heat kernel signatures for two isometric shapes. Given a shape S, we need to calculate the heat kernel signature for a point on S. In practice, the heat kernel signature p(x) of a point x S is an n-dimensional descriptor vector with each bin corresponding to some value of t :
|
i (x) = |
|
α |
− |
t0 |
|
p(x) = |
|
p1(x), . . . , pn(x) |
(7.45) |
|||
p |
|
c(x)K i |
1 |
(x, x) |
(7.46) |
|
where c(x) must be selected in order to have p(x) 2 = 1. Note that we need to restrict the number of eigenvalues and eigenvectors to be considered in Eq. (7.44). As a result, we obtain a descriptor for each vertex on the mesh.
Once we have computed the descriptors for each shape in the database, these must be grouped in a huge collection of local descriptors which will be called the descriptor space. Next, it is necessary to quantize the n-dimensional descriptors space. The idea is to find a point set in the descriptor space in order to better cluster the whole descriptor set. Unsupervised techniques from machine learning field can be used such as k-means and its variants [38]. In order to make this section relatively self-contained, we briefly describe k-means clustering in the descriptor space.
Let D be the huge set of n-dimensional descriptors and k be the number of clusters we want to find. The algorithm can be summarized as follows:
1.Initial centroids selection: Select k points in the n-dimensional space. This step can be performed in different ways, for instance, selecting random points in the n-dimensional space, selecting random descriptors from D, or using in-
formation about the distribution of descriptors in D, just to name a few. Let M = {m1, . . . , mk } be the set of selected centroids.
2.Cluster assignment: Assign each descriptor d in D to the closest cluster Ci
Ci = d D : d − mi ≤ d − mj , j = 1 . . . k |
(7.47) |
7 3D Shape Matching for Retrieval and Recognition |
295 |
3. Centroids update: Compute the new centroids for each cluster |
|
|||||
1 |
|
d |
|
|||
mi = |
|
(7.48) |
||||
|Ci | |
d |
|
Ci |
|||
|
|
|
|
|
|
|
4.Stop criterion: If centroids remain unchanged after update step, stop and return M. Otherwise, go to step 2.
Using the set of centroids, M, and the heat kernel signatures previously calculated for a shape, P, we need to compute a single descriptor for P, so it is necessary to combine the local descriptors in a shape descriptor. To tackle this problem, we calculate the feature distribution at a vertex x P as θ (x) = (θ1(x), . . . , θk (x))T where
i |
|
= |
|
− |
2σ 2 |
|
|
||
θ |
(x) |
|
c(x) exp |
|
|
p(x) − mi |
2 |
|
(7.49) |
|
|
|
|
|
|
||||
where c(x) is a constant selected such that θ (x) 2 = 1, p(x) is the heat kernel signature of x, mi is the centroid of cluster Ci , and σ is constant. Each bin in θ (x) can be considered as the probability that x belongs to the cluster corresponding to such a bin. This is a soft version of quantization because the classic bag of features approach considers placing a one in the bin corresponding to the closest cluster and zeros in the rest. Although the classic way can be performed here, the soft version has proved to be effective in experiments.
To obtain a shape descriptor, the feature distributions are simply added to obtain
a shape descriptor of size k, the vocabulary size: |
|
|
f (S) = |
|
|
θ (x) |
(7.50) |
|
x S
and the matching between two shapes S and T is performed by using the L1 distance
d(S, T) = f (S) − f (T) 1 |
(7.51) |
|
|
|
|
Nevertheless, during the quantization process, the spatial information is lost. Obviously, this information could be useful in the matching process. To address this problem, Ovsjanikov et al. [83] proposed a feature distribution among pairs of descriptors using a weighting factor related to spatial information. Again, we can use the heat kernel Kt (x, y) as a spatial factor which will have high values for close points x and y. Therefore, the following definition for descriptors should be used:
F(S) = |
|
|
θ (x)θ T (y)Kt (x, y) |
(7.52) |
x S y S
This descriptor results in a k × k matrix and the distance between two shapes can be done with the L1 distance, as usual.