Abstract: In order to enhance the robustness of tracking fast moving IR small target when we extract grey scale and local standard deviation of pixel as feature vector and treat tracking as two classification problems in target and local background feature vector pattern recognition, a new target tracking algorithm based on local background feature vectors Gaussian Mixture Model (GMM) clustering can be proposed. The method combines k-mean clustering with Expectation Maximization (EM) clustering algorithm, so the probability density parameters of the model are precisely determined and the GMM modeling speed is improved which are both important for real-time tracking. At the same time we can reform the model by using target feature vectors to improve the classification capability between the feature vectors of the target and background and then the classification model of the target and background is constructed. In the process, the Weighted Information Entropy (WIE) is applied as the discrimination criterion of the local background complexity and it can be used to guide the updating of the model adaptively. The validity of this algorithm is verified by the actual experiment.
INTRODUCTION
Now-a-days there are two strategies for target tracking. One is constructing the target models, such as kernel histogram (Comaniciu et al., 2000; Zuo et al., 2008) and invariant moment (Taekyu and Sukbum, 2008), for matching and tracking; the other is taking the differences between the target and background as the features. Here, the centre image on which the tracking process is performed, is referred to a confidence image (Avidan, 2005), likelihood image (Chen et al., 2004) or weighted image (Collins et al., 2005). The values of image pixels represent the probability of the coordinate pixels belonging to some backgrounds or target (Kaawaase et al., 2011; Benoudnine et al., 2009; Kim, 2005).
The mean shift algorithm based on kernel histogram gradient matching, a model matching and tacking algorithm, is widely applied in the practice. It takes the kernel histogram of the features in the regions as the template for tracking and the displacement for the centre of the targets is determined by the gradient matching algorithm. The result has reliable performances under the case of rigid moving object in visible light (Wu and Yang, 2007). For the small target tracking in IR image on FLR, Yilmaz et al. (2003) proposed a useful method. It describes the kernel histogram using pixel grayscale and local standard deviation as 2-D features and takes the one-dimensional kernel as the weighted factor of the histogram. Applying the feature to the experiment has performed well in target tracking under the mean-shift frame. However, the mean-shift algorithm has its implicit deficiency in tracking the small IR target. Compared to the visible light image, the quality of the IR is lower and this can be seen in the following two aspects: (1) For IR image is of lower contrast, small SNR and fewer grayscales (Asaduzzaman et al., 2010), kernel histogram cannot depict the features of the small target with weak texture accurately; (2) The noise, inhomogeneity in space and stray light can have great influence on the gray level distribution of the small target to be tracked as well as the grayscale stability in different frames which resulting the degeneration of the target model. The points above set some limitations when we employ the mean-shift algorithm based on model gradient matching in the small target tracking. Further more, if the object to be tracked is fast moving small target we must set the kernel window big enough as to cover its moving range. Thus the influence of stray light and complex background is increased and the respective features of kernel histogram and background can cause aliasing which definitely diverges the algorithm and bring about bad tracking results.
The second strategy has relatively strong stability, that is, if the selected feature can distinguish the target from the background and other targets effectively, it cannot be confused when we carry out the classification for the targets and background. Collins takes the kernel density estimation on targets and background grayscale histogram and forms grayscale probability density curve. Take the log likelihood value in the curve as likelihood function which is coordinate to the weighted image and the pixel value represents the probability of the points to be discriminated as target pixel. Whereas in the process of tracking the target, the weighted image in every frame is taken as data source and the mean-shift algorithm is adopted to seek the optimal model as the gradient matching points (Collins et al., 2005). The method has good tracking performance when the objects in the visible light image are of rich color and of definite shapes but the performance is not good when the IR objects are only of grayscale information. Besides, the implement is based on the stability of grayscale value of the target image and thus it is not suitable for small target tracking with the big SNR in space-time domain.
In this study, a new approach is proposed to enhance the robustness of tracking fast moving IR small target with unstable grayscale. Here we adopt the idea of feature classification (Jun and Wang, 2008; Abed and Zaoui, 2011; Ranjan and Khalil, 2007; Zhou et al., 2007), propose a fast small target tracking algorithm based on local background feature and mixture gussian models after extracting the features of the small target and background. The Weighted Information Entropy (WIE) is applied as the discrimination criterion of the local background complexity to guide the updating of the model adaptively. The validity of this algorithm is verified in the tracking experiment for the small targets in IR image.
Tracking algorithm based on gussian mixture clustering: The algorithm is summarized in the flowchart in Fig. 1, in which selecting the features for classification, implementing clustering method and classification method effectively are the key procedures.
Selecting the features for classification: The characters of IR image with complex background fB (x, y) can be summarized as follows (Anderson and Iltis, 1997).
| • | Be of longer relevance length and usually of constant distribution in large area |
| • | For the inhomogeneity in the thermal distribution in the scene and sensors, the mean value of local grayscale may vary in a large range which is a two-dimension unstable random process with a slow change |
Compared to the background, the targets can be considered as a transient signal and the emergence of the target destroys the constant consistency in the background. If we neglect the effect of the optical aberration, the target image acquired from Infrared focal plane arrays (IRFPA), for the diffraction that are caused by the image size and atmospheric transmission, can be transformed into scattered light points and represented by two-dimension Gaussian distribution function. In order to describe the spatial distribution of small target fully, the study adopts the method of feature selection in study of Yilmaz et al. (2003) and the extracted feature in pixel level can be represented as:
where, f (x) is the grayscale value for the pixel point x in the search region;
g (x) is the grayscale standard deviation for the pixel point x in neighbor
regions.
| Fig. 1: | The tracking algorithm based on the classification for the targets and background |
GMM clustering based on the background feature: To complete the task of fast small target tracking, the search window should be enlarged and the pixel number in the background will be larger than that in the target pixel. Further more, the relevance of local background is strong, thus the background pixel in the feature space has good clustering feature as shown in Fig. 2.
Under the premise, the best classification criterion between targets and local background should be based on the clustering based on local background feature, that is we can use the distance between feature vector and background to discriminate the category the feature vector belongs to. The feature that is larger than some threshold is considered as the target feature, otherwise it is the background feature. For the background in the complex IR image is unstable in time-space domain we construct the new model using the Gaussian mixture model as to represent the local background accurately.
The EM algorithm is a supervised clustering method. It builds models of given sample on the idea of probability and statistics, estimates the parameters by maximizing the expectation of the model and acquires the description of the original data set finally. It has applied to many fields, such as image recovery, image classification and target tracking (He et al., 2007; Wang et al., 2007). Considering that constant iteration updating can affect the real-time featurv, the approach in this study first initialize the parameters for the data set by using the standard K-mean clustering algorithm and then model the clustering for every category by using EM algorithm, ie. determine the weighted value, mean value and covariance matrix for every category, finally the accurate description for the feature of the local background is attained.
For the selected local background B0 in the reference frame, the feature vector of the current pixel x ε B0 is ξ1 (x) = [f (x), g (x)}, the Gaussian mixture probability model of ξ (x) attributed the background pixel is as follows:
| (1) |
where, the ωj is the prior probability that the feature vector ξ of pixel x belong to the jth Gaussian component of the background and satisfy:
If we assume every mixture component distribution with mean vector μ and covariance matrix Σ, then:
| Fig. 2: | Classification for small targets and local background in the feature space |
| (2) |
where, d is the dimension of feature space.
Now, the Gaussian mixture model for the pixel feature in the local background B0 can be represented as:
where as, taking into account the idea of the second strategy for IR target tracking we should make sure that the established background model has the optimal capacity for distinguishing the targets and background, so the target feature should be added into the model. By Reallocating the ωj on the basis of the target feature, the similarity between the target feature and background is reduced. Let the feature vector of the pixel y ε T0 of the small target in the reference frame is η, we can have the formula as follows:
| (3) |
The υj represents the mean confidence level that the small target feature η belong to the jth Gaussian component of the background. For the component with low confidence level, the corresponding weighted factor should be increased; for that with high confidence level, the weighted factor should be reduced. In this way, the description for the differences between the feature of background and targets is formed and the adjustment for the prior probability of the model is as follows:
| (4) |
According to this formula we normalize
| (5) |
Thus, the Eq. 1 is transformed into:
| (6) |
Now, the Gaussian mixture model based on background feature can be represented as:
Classification and discrimination model for the feature of the targets and background: For the respective pixel feature of local background and small target, ie., ξ (x) and η (x), we can calculate the mean value and mean square deviation of the probability density in every category, respectively as:
and
by Eq. 1.
Set the test threshold of the probability density as:
where,
And the k is a set constant used to control the false alarm in the discrimination of small target feature.
For the feature vector ζ (x) in the search window of the following frames we construct two kinds of discrimination model for the classification, the model A0 being as:
| (7) |
Updating of the gaussian mixture model: The Eq. 7 represents the two classification and discrimination model constructed from two training samples according to the feature vectors of the targets and background. Use this discrimination criterion to classify the targets and background and locate the target points. For the inhomogeneity in the detector and the time-space complexity in actual scene (for example, the influence of the background and stray light), destroy the stability of the grayscale in the target itself and cause a gradual change in the statistics distribution of the feature of local background, so it will result in tracking deviation when we apply Eq. 7 in the frames. Therefore, to improve the robustness of the algorithm we employ the Weighted Information Entropy (WIE), the index of local background complexity, as the discrimination function of updating Gaussian mixture model, here WIE is defined as:
where, the S is the image of local back ground and ps is the proportion of the pixel with the grayscale s to all the pixels in the image.
Figure 3 represents a updating process for Gaussian mixture model when applying WIE. In the process we adopt the discrimination model A0 in the classification of the feature vectors in all search windows before the kth frame and WIE0 is the grayscale Weighted Information Entropy (WIE) of local background B0 in the training samples; for the image feature of the kth frame, ie., ζk (x), we use the model A0 to classify and discriminate its feature subset in the search window Qk-1.
Figure 4 represents the shifting process of the small targets in two continuous frames and the definition for the local background and search window in the tracking process. The updating process for the model is specified as follows.
If the Weighted Information Entropy (WIE) of the local ground B'k-1, ie. WIE'k-1 satisfies:
(
If the WIE of the local background B'k-1 satisfies
| Fig. 3: | The updating process for GMM |
| Fig. 4: | The shifting process of the small targets in (k-1)th and kth frames and the related |
We consider there is a great change in the statistics features from the current local background B'k-1 to the local background B'k-1 when the discrimination model for clustering is established and further updating process is needed; otherwise, the confidence level of the discrimination for clustering is reduced. Considering that there would be few differences between the statistics feature of local background in the two continuous frames including the targets we adopt k-2 frames to discriminate the features when updating the model and get the feature vector sample of local background B'k-1, ie., Vk–2, to conduct the clustering and classification process on Gaussian mixture model and then the Weighted Information Entropy (WIE) of A1 and B'k-1, ie., WIEk-1, is attained. Finally, combine the target feature vectors in k-1 frames into a discrimination model A1, which means a model updating process is completed and save the three parameters for clustering and tracking in the flowing frames.
EXPERIMENTAL RESULTS
To test the performance of the proposed algorithm for the small target tracking on the IR image sequence we use a uncooled infrared micro-bolometer with the size of 60x120 and conduct video collecting under the cloud background with the frequency in every frame of the FLR 25 frame per second. Figure 5 shows the reference image and local standard deviation image used in the training process for the classification algorithm and the selected target T0 and local background B0 are shown respectively in the two boxes.
Figure 6 represents the 4-dimension Gaussian mixture clustering results of the target feature scattered points and background feature points. We can see that the feature vectors of the pixels in background B0 are of higher clustering effect in every category while the dispersion in the target is bigger when compared to that of the local background.
| Fig. 5: | The data for reference frame: (a) the image (b) the local standard deviation image |
| Fig. 6: | The scatter diagram for the feature of local background B0 and target T0 with 4-dimension Gaussian mixture model |
| Fig. 7: | The tracking results of the fast moving small target |
| Table 1: | The prior probability before and after the adjustment |
The data for the prior probability before and after adjustment based on the target features, i.e., Pb (probability before adjustment) and Pa (probability after adjustment), is shown as in Table 1. It is noticeable that the algorithm reforms the prior probability effectively by the rule of the dispersion of the targets and local background.
The tracking results of the fast moving small target for the proposed algorithm are shown in Fig. 7. Obviously the target is a no-rigid body and it changes gradually in its shape and local background. The target is of 3x3 pixels size and have moved over 70 pixels in the time of 17 frames, i.e., the shifted pixels is more than its size and can be seen as a fast moving small target. Here the mean-shift algorithm cannot track the target effectively while the method proposed in this study has good performance in tracking this kind of fast moving small target with unknown motion model.
CONCLUSIONS
In this study, we first analyze the limitations of the mean-shift algorithm in tracking fast moving IR small target, then propose a tracking algorithm based on local background feature vectors Gaussian Mixture Model (GMM) clustering. The method extracts the grayscale of the small target and background as the feature vectors and treats the tracking process as two classification problems. In the process, the Weighted Information Entropy (WIE) is applied as the discrimination criterion of the local background complexity to guide the updating of the model adaptively. The experimental results show that this algorithm is verified effectively in the tracking for the small targets in IR image.