A Local Density Shape Context Algorithm for Point Pattern Matching in Three Dimensional Space

Three dimensional space point pattern matching technology shows significant usage in many scientific fields. It is a great challenge to match pairwise with rigid transformation in three dimensional space. In this paper, we propose an effect of Local Density Shape Context algorithm (LDSC). In LDSC, the point local density is firstly used for cutting down the negative impacting on extracting the feature descriptor. And the optimization of pairwise matching is firstly used in LDSC for improving the effectiveness. To demonstrate the performance of LDSC, we conduct experiments on synthetic datasets and real world datasets. The experimental results indicate that LDSC outperforms the three compared classical methods in most cases. LDSC is robust to outliers and noise.


INTRODUCTION
Point pattern matching plays important roles in many scientific and application fields, such as fingerprint recognition and iris recognition [1]. It is also an indispensable technology in computer vision, image matching and moving object recognition [2]. The goal of the point pattern matching problem is to find a spatial optimal correspondence from images [3]. Feature points are extracted from images and used for finding matching pattern. Point pattern matching is a NP hard problem. And there is not available prior information between point patterns. Outliers and noise are included in point sets within the extracting procedure which increases the complexity of solving the problem. The study of point pattern matching in three dimension space has become a challenging task [4].
The rest of this paper is organized as follows. In Section 2, we review related work. In Section 3, we present the proposed novel Local Density Shape Context algorithm (LDSC). We report experimental results with discussions in Section 4. Finally, we conclude the paper in Section 5.

RELATED WORK
Aiming at matching point pattern with different rigid transformation, many matching algorithms have been proposed in different methodologies and used in various fields. The existing algorithms are divided into five types and discussed in the flowing subsections.

Spectra Context Based Algorithms
Spectra Context algorithm is firstly introduced in [5], they used the matching rate of point pairwise to construct a proximity matrix, and then get final matching result by using singular value decomposition of the proximity matrix. Spectra Context algorithm is easy to implement, but it cannot work well when large angle rotation exists in point pattern. Shapiro et al. put forward to get the proximity matrix inside the point sets and get better result [6]. Carcassoni et al. combined the proximity matrix with EM algorithm for improvement [7]. These improved methods enhance the robustness of Spectra Context algorithm. However, this kind of algorithms can only be used for two dimension point pattern matching and cannot work well in three dimensional situation.

Biological Model Based Algorithms
A kind of biological model based algorithms was proposed in the last decade and such as Particle Swarm Optimization algorithm [8,9] and Genetic Algorithm [10] were used to find the corresponding relationship between two point sets. Ansari et al. proposed a Genetic Algorithm based method, the correspondence between point sets is regarded as strings, the strings are updated iteratively according to genetic options (selection, cross, mutation) [10]. Particle Swarm Optimization model is also used for matching point pattern, the solution space is directly represented by transform parameters [8]. This kind of algorithm can be applied to the n-dimension point pattern matching, but with the dimension increasing, it is easy to fall into local optimal value. High computational cost is needed, which means time consumption algorithm.

Iterative Optimization Algorithms
Iterated Closest Point (ICP) is the best known point pattern matching algorithm [11], it assigns a correspondence based on the nearest neighbor relationships, then uses the estimated transformation to obtain new correspondence iteratively until algorithm converges. ICP is sensitive to dispersed distribution of points, and the initial position of two point sets is required close to each other. To overcome the shortcomings of ICP, improved algorithms were proposed. For example, LM-ICP introduces nonlinear optimizer in iteration [12]. TPS-RPM is an algorithm based on soft assignments and deterministic annealing [13]. This kind of algorithms is sensitive to the initial position.

Shape Context Based Algorithms
The Shape Context algorithm is firstly put forward by Belongie et al. [14]. They used the spatial distribution of point sets as features of each point, but it is not good for dealing with large angle rotation between two point sets. However, this kind of algorithms is designed for two dimensional problem and cannot work well in three dimension situation [15,16]. Positive Bipartite Graph Shape Context (PSC) algorithm was proposed especially for three dimensional point pattern matching problem [17]. The space division and feature descriptor for three dimensional space were firstly introduced in PSC. However, PSC does not deal well with low density points and sensitive to outliers. The performance of PSC declines greatly because of the influence of low density points when extracting point features.

Other Algorithms
Some other kind of algorithms was proposed in the last decade, such as methods based on quadratic programming [18,19], Markov random field [20,21] and Geometric invariant features [22,23] etc.
In this paper, we propose a novel Local Density Shape Context algorithm based on the local points density for three dimensional point pattern matching. The local point density and optimization of pairwise matching are firstly proposed in LDSC. LDSC is robust to outliers and noise, it is also a fast and efficient algorithm. The detailed description of LDSC is introduced in section 3.

DENSITY SHAPE CONTEXT ALGORITHM 3.1 Problem Description
A general definition of pattern matching about two point sets is defined as Definition 1. The point pattern matching algorithm is to find a transformation T between P and Q.

Definition 1
Given two point sets P and Q, where P = {P i : P i ∈ R N , i = 1, 2, …, n} and Q = {Q j : Q j ∈ R N , j = 1, 2, …, m}, m ≠ n. Point pattern matching algorithm is to find an assignment T : P k → Q k , where P k ⊆ P and Q k ⊆ Q, T is similarity transformation. The best matching result is to ensure the matching error minimized between T(P k ) and Q k .

The Detailed Steps of LDSC Algorithm
LDSC is proposed for robust matching point pattern in three dimensional space. It uses local point density and optimization of pairwise matching. In order to understand the steps of the LDSC easily, the psuedocode of our method is shown in Algorithm 1.

Eliminate Low Density Point
Low density point negatively impacts point feature descriptor while using shape context descriptor, especially it is redundant point. Low density points are eliminated from point set in LDSC. Given a n points set P, we use L i as the variable of low density point, which is defined in Eq. (1), where p i is low density point when L i is 1, ρ i is the local density of p i and defined in Eq. (2), d c is the cutoff distance and defined in Eq. (3). In these equations, where d ij is the Euclidean distance of p i and p j ,

The Shape Context Descriptor
In LDSC, the shape context descriptor is used to calculate features of each point. For any point p i , p i is moved to the origin, the spatial distribution of other points p j is used as the feature of point p i . The detailed processes are shown in the following subsection. The method of calculating descriptor is the same for P and Q.

Translation
For any point p i ∈ P, in order to facilitate to determine the spatial distribution of the remaining points in point set P, we first translate the point set P and ensure that p i is translated to the origin. The coordinate of other point p j is calculated by Eq. (4).

Rotation
In three dimensional space, three necessary points are needed to uniquely determine the rotation transformation. For any point p i , the other two points are necessary to be chosen, the relative position of these two points does not change when the point set rotates. We use M i and C i as these two points for point p i , the definitions are shown in Eq. (5) and Eq. (6), where μ is an user defined parameter which influences the robust ability of outlier, μ = 2 is used in this paper. The coordinates of p i and p j are updated by Eq. (4).
The transformations are taken by calculating the angle ∂ between vector p i M i and XOY plane according to Eq. (7). We rotate clockwise ∂ degrees around X-axis to put M i on XOY plane. β is the angle between p i M i and Xaxis right direction which is defined in Eq. (8). We rotate P clockwise with β degrees around Z-axis and put M i on X-axis. According to Eq. (9), we calculate angle γ between p i C i and XOY plane, then we rotate P with γ degrees around Z-axis and put C i on XOY plane.

Three Dimensional Space Division
We divide three dimensional space into multiple regions by using two parameters D_plane and D_spheres. The XOY plane is divided into isometric sectors regions, where D_plane = 30° in this paper as shown in Fig. 1a. Then we extend all sectors vertically to the entire spherical area as shown in Fig. 1b, D_spheres is the number of spheres and D_spheres = 6 in this paper. The feature descriptor of each point p i is a 144 dimensions vector. In order to get the values of feature descriptor in each dimension, we calculate the number of points in each region. For each point p j , p p i j ϑ is the angle between p i p j and XOY plane based on Eq. (10). We use R(p j ) to calculate the regions of point p j which is defined in Eq. (11). f(x) is used to calculate the value of point p j in which is located spherical area as shown in Eq. (12), where [x] is downwards. g(x) is used to calculate the value of point p j in which are located regions sectors as shown in Eq. (13). Avg(p i ) is the average Euclidean distance between p i and other points p j which is defined in Eq. (14).
The similarity of two points p i and q j is defined in Eq. (15), where K is the number of divided regions and 144 in this paper, O k (x) is the value of the k th region of point x. The smaller w ij means more similarity between p i and q j .

Point Pairwise Matching
A similarity matrix between P and Q can be calculated by w ij . To find the best matching result is equivalent to finding the best matching relation, which can guarantee S is the minimum value shown in Formula (16), q j is the corresponding point of p i .
For the convenience of solving the problem, we assume that edge E(p i , q j ) exists between any pairwise in P and Q, and w ij is the weight of E(p i , q j ), so the problem is degenerated to the minimum weighting of a bipartite graph. For the minimum weight matching, we use Kuhn-Munkres [24] algorithm to find the point pattern matching result.

Point Pairwise Matching Optimization
We use maximal Qvalue as the best matching result. The equation of Qvalue is Eq. (17), where d is the number of pairwise in rel_i, |x| is the number of points in set x, RMSD is root mean square deviation and defined in Eq. (18), where s is scale, r is rotation, t is translation. The number of pairwise points and RMSD are taken into account in Qvalue, which means good result with bigger d and smaller RMSD.
In each iterative step, we calculate Euclidean distance of rel_i pairwise and arrange in ascending order. The pairwise with maximal Euclidean distance is removed or does not depend on Qvalue. The optimization is trying to find the maximal Qvalue. After iterative procedure, we get a point pattern matching result rel_i and maximal Qvalue.

EXPERIMENTAL RESULTS AND ANALYSIS
In this section, we use synthetic datasets and real world datasets to assess the effectiveness of the proposed LDSC. We take three algorithms as baselines: PSC [15], ICP [9] and TPS-RPM [11]. We use Accuracy as the evaluation criterium and describe it in subsection 4.1. We give a detailed method of synthetic datasets generation. The accuracy results are shown in figures and tables in subsections 4.2 and 4.3. We conducted these experiments on the same device (RAM 8G, NVIDIA GeForce GTX 1050 Ti).

Algorithm Evaluation Criteria
In all experiments, the target point set Q is generated from template point set P by rigid transformation. The ground truth of matching pairwise is known in advance. The Accuracy is defined as Eq. (19), where the Number(x) is a function for calculating pairwise number of set x, SetG is a pairwise set of ground truth, SetM is an algorithm output pairwise set, ∩ is intersection.

Synthetic Datasets
We use three synthetic datasets cases to evaluate the performance of four mentioned algorithms. For three cases, the template point sets of each datasets are generated with the same pattern. The noise and outlier are generated following the corresponding parameters and are described in each case part. Case1 and Case2 are used for illustrating noise measurement of four algorithms. Case3 is used for illustrating outlier measurement.
Given two point sets P and Q, P is the template point set, Q is the target point set and generated by transformation of P. The transformation includes translation, rotation and scale. The number of points of P and Q is the same and take 100. Each point is randomly generated in three axis. The range of values in each axis is [-100, 100]. The parameters of transformation are t x , t y , t z , s, r x , r y and r z , t x , t y and t z are the translation in x-axis, yaxis and z-axis separately. The range of translation is [-100, 100]. s is scale factor. The range of s is [0. 5,3]. r x , r y and r z are three rotation angles in three axes separately. The range of rotation is [-180°, 180°]. In each case, we generate 100 synthetic datasets and demonstrate the average results.

Case1: Different Noise Percentage Experiments
In this case, for the noise percentage of point set Q is chosen different value from 0 to 100 percentage. We randomly choose percentage points in Q and add noise to these points. The perturb range of noise is [-5, 5] in three axes. The result is shown in Fig. 2, where the column is accuracy and the row is noise percentage. LDSC and PSC achieve good results on increasing noise percentage. ICP and TPS-RPM cannot work well for point pattern matching in three dimensional space. The performance of LDSC is almost a slight decline and superior to PSC. It proves that LDSC algorithm has better ability to resist noise than other algorithms.

Case 2: Different Noise Level Factor Experiments
In this case, noise is added to all points in point set Q. The noise measurement is decided by the noise level factor. We add noise in three axis, the range of noise is [ ] where f is noise level factor. The range of f value is taken as [0, 10] in this case. The average accuracy of four algorithms is shown in Fig. 3, where the column is accuracy and the row is noise level factor. This case indicates that LDSC algorithm works better than other algorithms.

Case 3: Different Outlier Percentage Experiments
In this case, an amount of outliers is put into target point set Q. Therefore, the point number of P and Q is different. Outliers are randomly generated, the scopes of three coordinates are chosen in [-200, 200]. Outlier percentage is the number of outliers divides number of points. The relationship between n and m is shown as Eq. (20), where n is point number of P and m is point number of Q. The average accuracy of four algorithms is shown in Fig. 4, where the column is accuracy and the row is outlier percentage. These experimental results demonstrate that LDSC could get more than 60 correct matching rates even if there are 50 percentage outliers. LDSC is more robust on outlier than other algorithms.

Real World Datasets
We use three real world datasets test performances of four algorithms in accuracy and time consumption. These three datasets are consisted of three models with different shapes. First model represents a Bunny, the second model represents a Cow and the third model represents a Wolf. We did six experiments on three real world datasets.
Three of six are rigid transformation experiments. The detailed parameters are shown in Tab. 1. The other three experiments are rigid transformation with outliers which are randomly generated. The transform parameters are the same, as shown in Tab. 1. In addition, we add 60 outliers to the target point set. The original point set is seen as template point set, and the target point set is generated by rigid transformation of the template point set. In this paper, the initial temperature parameter of TPS-RPM and ICP is slightly bigger than the largest distance of all point pairs, the other parameters are the same as the original algorithms. The registration results of four algorithms are shown in Fig. 5 and Fig. 6. In Fig. 5, the target point set is generated by rigid transformation. In Fig. 6, the outlier points are put into target point set. The detailed information of accuracy is recorded in Tab. 2 and Tab. 3 respectively. LDSC gives the best matching results. In addition, the accuracy of TPS-RPM based on deterministic annealing and thin plate splines sometimes varies greatly.
The accuracy of four algorithms is the average value of hundred experiments. In Tab. 2, the accuracy of ICP and TPS-RPM is low and changes greatly, and they are sensitive to the distribution of outlier points. LDSC performs the best results. In Tab. 3, in the case of redundant points, ICP and TPS-RPM almost do not work, the performance of PSC declines significantly, LDSC always maintains a high accuracy, the results demonstrate that LDSC is robust.

CONCLUSION
In this paper, we proposed the effect of Local Density Shape Context algorithm, namely LDSC. We carried out experiments on a lot of synthetic datasets and real world datasets to assess the effectiveness of LDSC. We mainly make the following contributions: (1) we firstly use the local point density for cutting down the negative impact on extracting the feature descriptor. (2) the optimization of pairwise matching is also firstly used in LDSC for improving the effectiveness.
LDSC works well on point pattern matching in three dimensional space. LDSC is a powerful and promising tool and we believe that it can be applied to many research fields. Multiple point pattern matching should be further studied in our future work.