Abstract: Background and Objective: Corners detection plays an important role in finding accurate corners points especially in image processing fields. In this study, improvements of global and local curvature properties are implemented to detect true corners shape of an object. Materials and Methods: The corners detected are the potential key points to determine the shape corners of the shape object. A comparison of the corners detected was made between the proposed method with six other different corner detectors and descriptors. To evaluate the proposed improved global and local curvature properties method, experimental works were conducted by using the bench mark image data set. Results: The results obtained showed that improvement of the corners detected by the proposed method is viable for practical use. Conclusion: The improved corners detector is robust and suitable to be used in various image processing field such as shape matching, shape retrieval, object matching and pattern recognition.
INTRODUCTION
Recently, an important task in image processing to be solved is how to imitate human vision so that a computer is able to recognize the object in an image. Digital images represent an image in digital form which it can be the original visual images and its reproductions. Hence, important image information is critically essential to describe the images such as image features. There are many existing approaches which operate to find important features of an image such as corners detector. Corners can represent a lot of information of an images since they are points that have high curvature and lie at the junction of images. They are important features and recently have been used in a variety of image field including image matching1-3, image retrieval4-7, computer vision8 and pattern recognition9,10.
In 1988, Harris detector has been introduced by Harris and Stephens11, that is based on local auto-correlation function. Xiao et al.12 proposed an improved algorithm that combines Harris and circular boundary theory of corners which can eliminate all false corners. However Harris detector is not invariant to scale changes. Therefore, Mikolajczyk and Schmid13, proposed Harris Laplace detector in 2001. Meanwhile, smallest univalue segment assimilating nucleus (SUSAN) detector14 was proposed in 1997. It does not require derivative and works well in noise interruption. The main idea of SUSAN is the use of a mask to count the number of pixels that having the same brightness as the center pixel by comparing the number of pixel with a threshold to determine the corners. Zou et al.15, compared the performance of Harris and SUSAN detectors in terms of noise immunity, stability and complexity. The results showed that the anti-noise capability of SUSAN detector is worse than Harris detector and concluded that Harris detector is better than SUSAN detector. However, the Harris detector is very sensitive to changes in image scale.
To resolve the scaling problem, scale invariant feature transform (SIFT) descriptors was introduced by Lowe16, in 1999. SIFT has been proposed to detect and describe local features in images effectively and achieve scale invariance. Later, a more in depth development and analysis of SIFT was described by Lowe17 and proposed a number of improvements in stability and feature invariance. An improved SIFT descriptor has been proposed by Liu et al.18 and Zhou et al.19. The research in this area is still continuing and the image detector and descriptor had been progressively improved.
Curvature scale space (CSS) was proposed by Mokhtarian et al.20, for robust image corners detection. Based on CSS technique, an improved multi-scale corners detector with dynamic region of support has been proposed by He and Yung21 and the results showed that it offers a robust and effective solution to images containing widely different size features. The work continued in the 2008 where they22 proposed a curvature-based corners detector that detects both fine and coarse features. The proposed corners detector have better capability to detect better corners with high number of true corners as compared to other conventional method, however, some of the true corners are still undetected. This finding has motivated the authors of this paper to further explore the possibility of finding the true corners of an object.
In this study, some modification has been made to the original global and local curvature properties22, so that true corners can be detected in a single object binary shape data sets.
MATERIALS AND METHODS
Corner detection: Corners detection is one of the approaches to extract features information in images. After detecting the corners, the image features could be extracted at these particular key points. It could provide a lot of useful information including the position (location) of the key points in the given image. In this study, corners detector based on the improvements of original global and local curvature properties22 will be used to determine a meaningful shape of image object.
A curvature, K of a planar curve23 can be defined as:
| (1) |
Where:
and ⊗ is the convolution operator, while
Global and local curvature properties: He and Yung22, proposed the global and local curvature properties which is the enhanced version of the original CSS and can be described as follows:
| • | Detect edges using the likes of a canny edge detector to obtain a binary edge map |
| • | Extract contours as in the CSS method |
| • | After contour extraction, compute the curvature at a fixed low scale for each contour to retain the true corners and regard the local maxima of absolute curvature as corners candidates |
| • | Compute a threshold adaptively according to the mean curvature within a region of support. Round corners are removed by comparing the curvature of corners candidates with the adaptive threshold |
| • | Based on dynamically recalculated region of support, evaluate the angles of the remaining corners candidates to eliminate any false corners |
| • | Finally, consider the end points of open contours and mark them as corners unless they are very close to other corners |
The main objective of the proposed steps by He and Yung22, is to utilize the global and local curvature properties and balance their influence when extracting corners. The steps that used by the original CSS algorithm to detect corners of an image can be found in details by He and Yung22.
Initial list of corners candidates: In listing the initial corners candidates, first define the jth extracted contour as defined in Eq. 2:
| (2) |
Where:
are pixels on the contour, N is the number of pixels on the contour and
| (3) |
where, the threshold T is used to determine whether two end points are close enough. A typical value of T is 2 or 3 pixels.
For a closed contour, circular convolution can be applied directly to smooth the contour. For an open contour, however a certain number of points should be symmetrically compensated at both ends of the contour when it is smoothed. The contour convolved with the Gaussian smoothing kernel,
| (4) |
where,
| (5) |
From Eq. 5, all the local maxima of the curvature function are included in the initial list of corners candidates.
Improved global and local curvature properties: In this study, global and local curvature properties proposed by He and Yung22, has been modified to determine the true corners by determining the key points of an object or shape. All the six steps that proposed by He and Yung, are followed with some modification that has been described in the subsequent section.
Corner evaluation: The list of corners candidates includes true corners, round corners and sometimes simply noise. To determine only true corners, round corners and noise must be removed and to do that, the following two steps were used by He and Yung22:
| • | Compare the corners candidates using an adaptive threshold instead of a single global threshold to remove the round corners |
| • | The angles of the remaining corners candidates are evaluated to eliminate any false corners due to quantization noise and trivial details |
The evaluation is based on a dynamic region of support which varies from corners to corners according to adjacent corners candidates. Region of support (ROS) of corners is defined by the segment of the contour bounded by the corners of two nearest curvature minima. In determining a meaningful shape image descriptor, the entire true corners are important including obtuse corners. In that case, some modification and improvement has been done to classify the true corners shape of an object.
Improved corners detection: In this section, the method for improved corners detector is illustrated in Fig. 1.
The proposed new corners detection steps are as follows:
| • | Edge detection by using Canny edge detector |
| • | Contour extraction by using CSS method |
| • | Curvature computation: To retain the true corners, compute the curvature at a fixed low scale for each contour and regard the local maxima of absolute curvature as corners candidates |
| • | Threshold computation: Compute a threshold adaptively according to the mean curvature within a region of support |
| • | Corners final evaluation: Evaluate the angle of the remaining corners candidates to eliminate any false corners such as a noise. Consider the end points of open contours and mark as corners and highlight the final true corners |
The proposed corners detector has been tested by using images with many corners. The experiment has been implemented by using MATLAB® Version 7.12.0 and executed on Intel (R) CPU 2.00 GHz with 3.0 GB RAM under the Microsoft Windows XP operating system. To illustrate the process, Fig. 2 shows the process of detecting corners.
| Fig. 1: | Block diagram of the improved corners detector |
| Fig. 2(a-d): | Process of corners detection, (a) Original image, (b) Contour image, (c) Detected corners and (d) Highlighted detected corners |
The results of the image at different steps to determine the final corners of an image is shown in Fig. 2. Firstly, a query image or the original image of a rat image is chosen as in Fig. 2a. Then by using step one and two as described earlier, contour image of the rat image is illustrated in Fig. 2b. After the computation in step three and four, Fig. 2c shows the detected corners. Finally, all true corners of the rat image are highlighted as in Fig. 2d. For the rat image, all 28 true corners were detected by the proposed method and later compared with the original global and local curvature properties method that had only detected 26 corners. The feasibility of the modified global and local curvature properties method has successfully proven to be better than the original method in detecting true corners of the shape will be further demonstrated in the following section.
RESULTS AND DISCUSSION
This work focuses on a single binary object. To evaluate the proposed method, the benchmark image dataset, MPEG-7 Core Experiment Shape-1 Part B was employed throughout the experiments. This image dataset contains a total of 1400 binary images that are grouped into 70 categories by their content and each category contains 20 image samples. This dataset was created by the Motion Picture Expert Group (MPEG) Committee which is a working group of ISO/IEC and has defined the standards for description and search of audio and visual content.
Since Harris and Stephens11, Harris Laplace13, SUSAN14, SIFT16, CSS20 and Global and Local Curvature22 are the most popular corner detectors and descriptor, a comparison in terms of true corners detection have been evaluated and compared with those of modified global and local curvature properties method. The results of corners detection by using six other methods on one of selected image dataset, i.e. hammer image is shown in Fig. 3. The blue square in the Fig. 3 indicates detected corners by the respective methods.
From Fig. 3(a-f), it shows that different location of a corners have been detected for all six corners detectors. The Harris corners detector detects the most corners for hammer image, i.e. with 17 corners as shown in Fig. 3(a). It should be noted that some of them are false corners such as highlighted in red circle line. 14 corners have been detected by Harris Laplace method while SUSAN method detected 15 corners, however some of the detected corners were fall on straight line of the image boundary and some of the corners are redundant on the same location or it may be located too close with each other as indicated in red circle line. SIFT detected 13 corners but some are false corners that can be seen in the red circle line. As observed in Fig. 3d, there are some corners points in hammer image but it is actually false corners. CSS detected only 4 corners as shown in Fig. 3e. Corners detector based on global and local curvature detected 7 corners as shown in Fig. 3f. However, there are still corners points that are undetected. Figure 4 shows 8 true and reliable corners of the hammer image by using the proposed modified global and local curvature properties corner detector.
| Fig. 3(a-f): | Comparison of corners detection results on the hammer image, (a) Harris, (b) Harris Laplace, (c) SUSAN, (d) SIFT, (e) CSS and (f) Global and local curvature properties |
| Table 1: | Comparison results of corners detection for MPEG-7 shape dataset |
| Fig. 4: | Corners detection results by using improved global and local curvature properties method |
The 8th corner point is a new detected corners point as compared to original global and local curvature properties method as can be observed in Fig. 3f, i.e. the original method only detects 7 corners. All the eight corners detected can be considered as reliable key points in determining a shape image descriptor. It is important to detect all the true corners of an object to fully represent the shape of an object.
In all of the experiments, the total number of detected corners were computed and the results produced for each corners detector are summarized in Table 1. From the results as shown in Table 1, it can be observed that Harris method detected the most number of corners with the highest total of corners detected in rat, ray and bird images. It follows by SIFT corners detector with the second highest total number of corners detected in brick and rat images. However, some of the corners detected lie on the straight part of the shape. In addition, most detected corners are not reliable and not suitable for shape representation since some of them are either located close to each other or missed to find the true corners of the object. Therefore, it could not fully represent the whole shape of the object. Moreover, the process of computing the corner will slow down the corner detection process. It was also observed that the CSS produces the lowest total number of detected corners which is not good enough for shape representation due to missing important shape points. The proposed modified global and local curvature properties method detected most reliable corners of the object shape where it could detect more corner point than its original counterpart. Therefore, the corners detected are more reliable and each detected corner points are important for better shape representation and thus suitable to be applied in many applications including image retrieval and pattern matching. To visualize the corners detected using the proposed modified method, Fig. 5 shows sample of images with corner points marked in blue. In terms of corners detection point of view, the modified global and local curvature properties method detected all the significant true corners and reliable key point of the object. However, all the methods investigated in this study failed to locate corner points on the top part of the bird and face image in Fig. 5. This failure is perhaps due to low curvature then the preset threshold i.e. weak corners. Hence further investigations is needed to overcome this limitation.
In the literature, there are not many researchers that investigates at the fundamental level of improving the localization of corners of an object. Most of the literatures focus in utilizing corner detection algorithms as part of their methods for broad range of applications such as telemedicine24, pattern recognition25, target tracking26, machine vision27 and image stabilization28. As this study focuses on the fundamental aspect of corner detection, the findings in this study could be applied in related research projects to further enhance their applications. Bhujle29, reported an improved version of Harris corner detector by constructing a nonlinear nonlocal structure tensor where he measured the relative positions between neighboring pixels and also intensity variations between them by considering a small patch around the candidate pixel. Further he proposed to use Gaussian smoothing with multiple scales to remove falsely detected corners.
| Fig. 5: | Corners detected using the improved global and local curvature properties method |
His implementation is on grey level images thus not relevant to compare with at this point. Recently, Xing et al.30 introduced a concept of mask with filled circle and outer ring to detect corners for a gray level images. The filled circle is used to filter out non-corner region and the complex function is used together with the inner circle and the outer ring to select the corner candidates. They reported that the corner detector has better detection accuracy, less sensitivity to noisy and fuzzy images, high computational efficiency and good repeatability. Most recently, Lin et al.31, introduced two novel corner detection algorithms to detect 3D mesh corners. Unfortunately, at this point of time, we do not have access to any of these algorithms to compare with. The research into corner detection is so much challenging as digital images could be manipulated in many forms and in various conditions.
CONCLUSION
In this study, we have enhanced the capability of the original global and local curvature properties method to detect true corners of a shape object. The improvement made is in term of number of true corner points detected. The modified method has been compared with six other popularly used method in the literature, i.e. Harris, Harris Laplace, SUSAN, SIFT, CSS and the original global and local curvature properties method. The main contribution in this research is the reliable detection of true corner points of an object that will lead to meaningful shape descriptor. The modified corner detector is robust and suitable to be used in various image processing field such as shape matching, shape retrieval, object matching and pattern recognition. In the future, the proposed method is suggested to be implemented on grayscale and color image. Further investigations can be continued especially on the usage of the proposed method so that it can be tested on many objects as well as overlapped objects.
SIGNIFICANCE STATEMENT
This study discover the possible shape corners of an object that can be beneficial for shape based image retrieval. This study help the researchers as well as industrial practitioners to uncover the critical areas of true corners that have not been explored before. Thus, a new theory on global and local curvature properties may be arrived at.
ACKNOWLEDGMENT
The authors gratefully acknowledge the Ministry of Education Malaysia (MOE) for providing the scholarship for the first author.