7 3D Shape Matching for Retrieval and Recognition |
289 |
As the eigenvalues are increasing and the first eigenvectors are related to large scales, taking the scale as previously defined seems to be a good approximation, although the relation between the eigenvalues and the real spatial scale of the interest points is not clear.
7.3.3.2 Local Descriptors
To create local descriptors associated with each interest point, the Laplace-Beltrami operator can be used in a straightforward way. The process starts by selecting a patch based on the scale information of each vertex. Given an interest point, pi , we select a mesh patch formed by the vertices with geodesic distance from pi not exceeding the radius r × sf(pi ) (in the experiments, r was 1.7). The extracted patch is an open boundary surface, so that we need to attach another patch in order to apply the Laplace-Beltrami operator with successful results. The authors proposed to attach the same patch but in the opposite direction. Once we have a closed local surface, we apply the aforementioned method in order to obtain the spectrum.
The local descriptor is defined as the set {λ1, λ2, . . . , λm} for some value m. For similarity comparison, the spectrum must be normalized in order to allow scale invariance. This can be done by dividing each element in the local descriptor with the first eigenvalue λ1.
7.3.3.3 Shape Matching
From the preceding stages, each mesh is represented as a set of descriptors, so now we are interested in comparing two objects based on this information. Given two descriptor sets P = {pi } and P = {pi }, the problem is to find a transformation function φ such that:
i |
) |
= |
i |
(7.31) |
φ (p |
|
p |
Thus, the similarity function is defined as:
Sim(φ) = Sims (φ) + Simp (φ) |
(7.32) |
Sims (φ) refers to the feature-to-feature comparison and Simp (φ) to the cluster- to-cluster similarity. The feature-to-feature similarity is defined as follows:
|
Sims (φ) = ωs |
|
|
|
|
|||
|
C i, i |
|
(7.33) |
|||||
|
|
|
|
φ (i)=i |
|
|
|
|
where |
|
= |
− |
2σs2 |
|
|
|
|
|
|
|
|
|||||
C |
i, i |
|
exp |
|
f (i) − f (i ) 2 |
(7.34) |
||
|
|
|
||||||
where f (.) is the local descriptor of a point and σs |
is a constant. |
|||||||
290 |
B. Bustos and I. Sipiran |
In addition, the cluster-to-cluster similarity is defined as:
|
|
|
Simp (φ) = ωp |
|
|
|
|
|
|
|
H |
i, j, i , j |
|
|
|
||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
i ,φ (j ) |
= |
j |
|
|
|
|
|
|
||
|
|
|
|
|
|
|
|
|
|
φ (i)= |
|
|
|
|
|
||||||||
where |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
H i, j, i |
, j |
|
|
= |
exp |
|
−(dpg (i, j, i , j ) + βdps (i, j, i , j |
))2 |
|
||||||||||||||
|
|
|
|
|
|||||||||||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
2σp2 |
|
|
|
|
|
||||||
where |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
dpg i, j, i , j |
|
|
= |
|
g(i, j ) |
− |
g(i , j ) |
|
|
|
|
|
|||||||
|
|
|
|
sf(i) |
|
sf(i ) |
|
|
|
|
|||||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
and |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
dps |
i, j, i , j |
|
= |
|
|
|
|
sf(j ) |
|
|
sf(j |
) |
|
|
||||||||
|
|
log sf(i) − log sf(i ) |
|
||||||||||||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
(7.35)
(7.36)
(7.37)
(7.38)
where g(·, ·) is the geodesic distance between two vertices, sf(·) is the vertex scale factor calculated in the feature detection stage and σp is a constant.
In order to solve the matching problem using the defined similarity measure, we need to define an indicator variable
|
x |
i, i |
|
|
|
|
1 |
if φ (i) = i exists |
|
|
(7.39) |
|
|
|
|
|
= |
0 |
otherwise |
|
|
|
|||
|
|
|
|
|
|
|
||||||
We can reformulate the similarity measure using the indicator variable as |
||||||||||||
|
|
|
|
|
|
+ ωs |
|
|
|
|
|
|
Sim(x) = ωp |
C i, i |
x i, i |
|
|
H i, j, i , j |
x i, i |
x j, j |
(7.40) |
||||
i,i |
|
|
|
|
|
|
|
i,j,i ,j |
|
|
|
|
subject to i x(i, i ) ≤ 1 and |
|
|
|
i x(i, i ) ≤ 1. These expressions impose the con- |
||||||||
each point in the first descriptor set has at most one correspondence in |
||||||||||||
straint that |
|
|
|
|
|
|
|
|
|
|
||
the second descriptor set. To numerically solve this problem, we can use Integer Quadratic Programming (IQP) as follows
max |
Sim(x) = x |
T H |
x + |
CT |
x subject to |
A |
x ≤ b |
(7.41) |
x |
|
|
|
We now describe some details for implementation. Let N be the number of descriptors in P and M be the number of descriptors in P . After calculating the matrix C, we have to convert it in a column vector concatenating each column in the matrix so that the dimension of C is N M × 1. Note that the size of H is N M × N M because we need to calculate H(i, j, i , j ) between each pair of descriptor from P with each pair from P . With respect to the constraints on the number of correspondences, these have to be located in the matrix A and a vector b. The geodesic distances can be approximated using a shortest path algorithm on the mesh, however another clever algorithm can also be used such as the fast marching method
7 3D Shape Matching for Retrieval and Recognition |
291 |
proposed by Kimmel and Sethian [62]. In order to solve the aforementioned optimization problem, there are several alternatives such as the MATLAB’s optimization toolbox or the CGAL library [2].
In summary, the method can be described as follows.
1.Solving the Laplace-Beltrami eigenproblem: We need to compute the eigenvectors and eigenvalues of the Laplace-Beltrami operator for each shape in the dataset.
2.Feature points detection: Computing the geometric energy given in Eq. (7.30), it is possible to select the interest points along with their scales.
3.Construction of local descriptors: For each interest point, the method extracts a local patch over which the Laplace-Beltrami operator is calculated. A few normalized eigenvalues of this operator are considered as a local descriptor.
4.Matching: The problem of multi-feature matching is considered as an integer quadratic programming problem, as given in Eq. (7.41).
7.3.3.4 Evaluation
In the original work by Hu and Hua [52], several tests were accomplished for shape correspondence and shape retrieval over the SHREC’07 3D shape database available at http://www.aimatshape.net/event/SHREC. The authors proposed to extract 10 shapes from each category of the dataset. Figure 7.7 shows the precision-recall plot for the presented approach compared to the best technique in the SHREC’07 watertight retrieval contest. In addition, Fig. 7.8 present an example of matching between shapes in the same category and between shapes in different categories.
From the results, it is important to note that the method performs well on nonrigid transformations. In fact, it outperforms the best technique in the SHREC’2007
Fig. 7.7 Precision-recall plot for the salient spectral features approach. Figure courtesy of [52]