Face Recognition in Compressed Domain by Applying Wavelet Transform and Feature Vector Optimization
P. Ranjit Jeba Thangaiah
In this study, a novel method for extracting feature vector in the form of entropy points from compressed images was proposed. This efficient approach for performing face recognition systems directly into wavelet based compressed domain which involve wavelet based image compression/decompression for feature extraction, an efficient feature optimization technique and a method for image classification. This is accomplished by stopping the decompression process after entropy decoding and utilizing the entropy points as input to recognition systems. During the experiments, firstly, a standard recognition algorithms used like principal component analysis, independent component analysis and kernel PCA for optimizing feature vector and kd-tree based method for image matching. Secondly, an improved version of canonical correlation analysis method is applied for feature projection. Finally, cascade forward neural network based algorithm for better matching of facial images for image classification. The experimental results proved that the proposed approach is effective in achieving face recognition in compressed domain with additional reduction of computational time and storage requirements.
Received: December 16, 2012;
Accepted: February 11, 2013;
Published: April 22, 2013
Automatic face recognition is an extensively researched area in computer science
along with other biometric techniques like fingerprint verification and voice
recognition. Identification of person is the most important factor of security
applications and smart interaction applications. Problems to be addressed in
automatic face recognition are challenging due to the changes facial images
undergo in real life like variations in facial expressions, illumination, occlusion
etc. Researchers have used uncompressed high resolution still images for performing
face recognition until recently even though compression was realized as an important
issue in finger print verification and iris recognition previously (Rakshit
and Monro, 2007; Funk et al., 2005).
Previously researchers had tried to implement face recognition in compressed
domain with standard recognition systems such as Principal Component Analysis
(PCA) and Linear Discriminant Analysis (LDA) methods and Independent Component
Analysis (ICA). A study on human recognition in wavelet domain was done by Curtis
and Sabharwal (1997). Daubechies 2 wavelet was used in this experiment.
The obtained filter coefficients were used as input to the standard classification
technique PCA. Increase in recognition rate was noticed compared to PCA with
uncompressed facial images. Chien and Wu (2002) had
proposed a multi resolution wavelet transform based approach for face recognition.
The authors had calculated the approximation band for recognition algorithms
by applying two wavelet decompositions. Li and Liu (2002)
performed a study using a combination of wavelet techniques and eigenfaces to
extract features for face recognition. The experimental results show that using
DWT coefficients as input to PCA given superior performance compared to standard
PCA. Ekenel and Sankur (2005) study the effect of multi
resolution analysis for face recognition by using wavelet decomposition to find
the subbands which are least sensitive to variations due to expression changes
and illumination conditions. Delac and Grgic (2007) and
Delac et al. (2009) focused on the practicability
of compressed images face recognition by using feature vector as input to various
standard face recognition algorithms. Canonical Correlation Analysis (CCA) has
stimulated the interest of researchers in the field of pattern recognition and
signal processing recently which helps to relate two sets of observations by
describing different aspects of appearance. Lee and Choi
(2007) focused on CCA to give the high dimensional relationship between
two sets of multi-dimensional variables with a few pairs of canonical variables.
Hotelling (1936) also was intended to define relations
between CCA and two sets of one dimensional data sequences unlike PCA and LDA.
Jelsovka et al. (2011) had conducted two dimensional
face recognition based on CCA method and the obtained result showed high efficiency.
Oh (2005) proposed a standard LDA method using radial
basis function network in order to improve the robustness of the face recognition
system. The authors had studied the effect of the combination of two compensating
classifiers and testing ORL database which improve by 93.55 recognition rate.
In James and Arockiasamy (2011) study first analysis
of performing face recognition with partially compressed images using standard
recognition systems PCA and ICA. The preliminary results observed there are
largely expanded in the next studies by the same authors (James
and Arockiasamy, 2012a,b) where the impact on face
recognition using compressed images is analysed when canonical correlation based
algorithms are applied for feature projection. Standard yale data sets were
used for testing in these experiments.
In this study, a novel generic face recognition system that performs robustly in direct compressed domain in spite of the facial appearance variations. In addition, further expanded by implementing face recognition of compressed images using Kernel PCA based feature projection method with and kd-tree matching. A numerous experiments conducted with AR data bases which contain more than 4000 facial images which involve image data sets with varying expression, illumination changes and partial occlusion.
Principal Component Analysis (PCA): Alwakeel and
Shaaban (2010) proposed Principal Component Analysis (PCA) as a tool to
reduce the large dimensionality of the data space to a set of representative
projection vectors for describing the data efficiently. In this method image
elements used as random variables with Gaussian distribution and minimized second-order
statistics which keep lower-order principal components and ignoring higher-order
ones. Jolliffe (2002) proved that low-order components
often contain the most important aspects of the data. The most representative
vectors are the eigenvectors corresponding to the largest eigenvalues of the
covariance matrix (Turk and Petland, 1991). In addition,
PCA can be used for dimensionality reduction is data set by retaining those
characteristics of the data which has successful applications that have been
used in digital image processing and pattern recognition.
Independent Component Analysis (ICA): ICA is a generalization of PCA.
For any non-Gaussian distribution, largest variances would not respond to PCA
basis vectors. While PCA deals with second-order statistics (variance), ICA
captures both second and higher order statistics and projects the data onto
basis vectors that are statistically independent as possible. ICA implementation
was done based on the algorithm proposed by Bartlett et
Kernel Principal Component Analysis (KPCA): Kernel PCA is an extension
of PCA which firstly proposed as a nonlinear extension of PCA. Even though Principal
Component Analysis performs very well in dimensionality reduction of data, the
efficiency of PCA method is not that satisfactory for problems with high nonlinearity
such as face recognition. Hence, KPCA is developed to handle the nonlinearity
of face recognition problems (Timotius et al., 2010).
In Kernel PCA, a nonlinear kernel function is used for dimensionality reduction.
The face images are first transformed from image space into a feature space
though nonlinear mapping and then the principal components are computed in that
feature space. By applying kernel method, the principal components are computed
by high-order correlations of input pixels making up a facial image and hence
performance is improved (Kim et al., 2002).
Canonical Correlation Analysis (CCA): Canonical correlation analysis
is a a powerful feature projection approach which suitable technique for exploring
the relationships among multiple dependent and independent variables. CCA recognize
and measure the relationship between two sets of variables. For this purpose,
CCA uses two sets of linear combination of variables. A pair of linear combinations
with greatest correlation among all pairs is determined subsequently. Hence,
high dimensional association between two sets of variables with a few pairs
of canonical variables is represented in by CCA (Kukharev
and Kamenskaya, 2010). A detailed analysis of this method is given in our
work presented in James and Arockiaswamy (2012b).
Figure 1 sows the main framework to perform face recognition in direct compressed domain which involve new approaches for image pre-processing and DWT based feature extraction in compressed domain, efficient methods for feature vector optimization and classification (Recognition) of images.
Three classification algorithms are developed in the framework to be used for matching of images in the proposed approaches. A matching algorithm based on kd-tree technique is developed and used along with feature projection approaches based on PCA, ICA and KPCA. Mode based matching algorithm is used in experiments conducted with feature projection approach based on CCA. Also, a neural network based image classification algorithm is developed and applied for the face recognition approach based on canonical correlation analysis. The details of the algorithms are explained as follows:
Pre-processing of images: Pre-processing of images prior to Wavelet Transformation is essential. Pre-processing processes are performed on original images to maintain the size of the train and test image as same. The RGB image is converted into gray scale image and then cropped to size of 128x128 pixels. Elliptical masking is utilized to mostly remove the background. Also, images are originally transformed to obtain the eyes at the fixed points and histogram equalization was done to have better background intensity.
Extracting entropy points as feature vector: This stage is extracting features as DWT coefficients from the pre-processed image by applying Cohen Daubechies Feauveau (CDF) 9/7 wavelet on the image. All the preprocessed images used in the experiments were compressed according to JPEG2000 compression scheme with various compression ratios of 1, 0.5, 0.3 and 0.2 bpp (lower, moderate and higher compression levels). The proposed approach eliminates major part of the decompression phase as the decoding is interrupted after entropy decoding and all entropy points obtained are used as input to the proposed recognition method. Also, feature vector is extracted from image code block thus avoiding image decompression fully. During experiments the feature projection system is placed either at point B or at point A.
Feature vector optimization: In this stage, various algorithms based on PCA, ICA, Kernel PCA and improved canonical correlation analysis method are applied on features obtained in the previous stage for conducting experiments.
The block diagram of the experimental set up is given in Fig. 2. This diagram represents the exact description of the procedures carried out in the experiments. For the simulated experiments, the uncompressed face images were first transformed and then encoded using the EBCOT (Embedded block coding with optimal truncation) coding technique as per JPEG2000 compression standard. In the real time application the facial images will be captured through high resolution digital camera and will be stored in JPEG2000 compressed format. To test these images for face recognition, it needs to decode the code stream in order to extract the entropy points.
In the second part of the experiments the propose image labeling in the pre-processing
stage before compression. The input image is cropped in such a way to extract
only the particular components of the image. By doing this, the background of
the image and hair patterns are been eliminated and hence the focus only will
be in face. After this cropping process, a HIGH BOOST FILTERING as given by
James and Arockiaswamy (2012b) is applied to improve
the cropped image, so that it can improve the high frequency component in an
image to much higher level.
|| Experimental setup for the training and testing session of
enhanced CCA approach
The recognition rate for image data sets with varying expressions is improved
significantly. The other two image data sets (varying illumination and minor
occlusion) also showed further improvement.
RESULTS AND DISCUSSION
Various image data sets from the AR face database Martinez
and Benavente (1998) was used in the experiments and this include images
of individuals with varying expression (fb), varying illumination (fc) and partial
occlusion (fd). The AR database was collected at the Computer Vision Center
in Barcelona, Spain in 1998. It contains 4000 uncompressed colour images of
126 individuals (70 men and 56 women). The pictures were taken under strict
conditions in order to ensure that settings are identical across subjects. Facial
expression variations include normal, smile, anger, sad, sleepy and surprised
impression. Illumination variations include lighting in the center, left and
right position. Images of individuals with glasses and scarf are considered
as partially occluded images. Two sets of experiments were conducted in compressed
domain. In the first set of tests, entropy points are fed as input to PCA, ICA
and KPCA instead of image pixels and a kd-tree based algorithm is applied for
image classification. During the latter set of experiments, canonical correlation
analysis based approach is used for feature vector optimization. In this second
set of experiments for matching of images using two classification algorithms
based on mode based matching and cascade forward neural networks are applied.
First set of experiments, the recognition algorithms based on PCA, ICA and
KPCA were implemented in MATLAB following the description in studies (Alwakeel
and Shaaban, 2010; Bartlett et al., 2002;
Kim et al., 2002). In the pixel domain experiments,
first PCA was performed on the uncompressed images which were cropped to 128x128
pixels. After performing PCA, top 40% of eigenvectors were kept and this subspace
ws used for classification based on kd-tree as PCA face space and also as input
to ICA algorithm. Finally, KPCA was performed in the experiments to calculate
the features for the uncompressed face images. The results are noted in Table
1 (original images). All images used in experiments were compressed according
to JPEG2000 compression standards, with various compression ratios 1, 0.5, 0.3
and 0.2 bpp. To compress images using JPEG2000 standard, JPEG2000 toolbox in
MATLAB was used.
|| Comparison of recognition rate
|| Comparison of recognition rates in compressed domain for
various image data sets
|PCA: Principal Component Analysis, ICA: Independent Component
Analysis, fb: Varying Expression, fc: Varying Illumination, fd: Partial
The compressed images were partially decompressed to extract the transform
coefficients. The compression/decompression of images were required as the uncompressed
images from the AR Face database was used in the experiments.
From Table 1 pixel domain are shown as recognition rate of
original images and results for experiments as per existing methodology (Delac
et al., 2009) are given in first two rows (PCA Euclidean and ICA
Euclidean). From one glance itself it is clear that the proposed approach experience
no significant performance drop for all image data sets. The results for ICA
are comparatively better when compared to other approaches. The comparison of
Normalized Recognition Rate (NRR) of the image probe sets for the proposed methods
in compressed domain (PCA, ICA and KPCA) to that of same in pixel domain is
done. The computational time of the proposed approach using the kd tree method
is compared with the existing techniques described by (Delac
et al., 2009) using nearest neighbour algorithm which apply Euclidean
distance method. The kd-tree takes very less computation time when compared
to the recognition rate of the existing approaches which uses Euclidean matching.
Second set of experiments, canonical correlation Analysis based feature projection method was performed on the original uncompressed 128x128 images and the results for all AR database data sets were noted Table 2 (original images) using the three different image data sets. Significantly better results are observed for the CCA based method for images with varying illumination and partial occlusion in the first set of tests. The recognition rate of images with varying expressions is also improved by applying labeling of images in the second part of the experiments.
Normalized recognition rate is used to compare the results obtained in compressed domain to pixel domain of fc probe set for the various approaches are given above in Fig. 3. The curves marked NRR after decompression show result obtained in experiments where face images were first compressed to a certain compression ratio and then uncompressed before using them in recognition. These results show how NRR changes at various compression ratios when image degradation caused by compression procedure.
Comparison of the proposed methods with existing nearest neighbour algorithm which uses Euclidean distance method for matching is given in Fig. 4. All the proposed methods showed significant reduction in computational time when compared to the existing recognition method which uses nearest neighbour algorithm for matching of images.
||NRR evaluation of fc probe set in compressed domain, (a) PCA
kd-tree, (b) ICA kd-tree, (c) KPCA kd-tree (d) CCA with ANN classified
|| Comparison of computational time of the proposed approaches
The main advantage of this new approach is significant saving in computational
time as it excludes the computationally most expensive phases of image decompression.
Since, this method uses the arithmetic decoded data as feature vector, significant
reduction in time is achieved. An average of 1.54 sec only is taken for recognition
when compared to methodologies applying DWT coefficients and entropy points
as feature vector for which the computational time was 5.2 and 2.87 sec, respectively.
Even though the recognition rate is reduced when compared with previous approaches
using entropy points, it is proved that face recognition is possible in direct
compressed domain hence avoiding the decompression process completely. Future
research will include finding a method for intelligently extracting feature
vector from compressed image therefore completely excluding decompression of
images in face recognition. Also, an algorithm for three dimensional face recognition
in compressed domain which avoid the pitfalls of 2D face recognition algorithms
in recognizing images with varying expressions, illumination and occlusions.
Translation of the proposed methodology into video based face recognition is
also recommended as the focus of this research work was face recognition from
1: Rakshit, S. and D.M. Monro, 2007. An evaluation of image sampling and compression for human iris recognition. IEEE Trans. Inform. Forensics Secur., 2: 605-612.
2: Funk, W., M. Arnold, C. Busch and A. Munde, 2005. Evaluation of image compression algorithms for fingerprint and face recognition systems. Proceedings of the 6th Annual IEEE Information Assurance Workshop, June 15-17, 2005, United States Military Academy, West Point, NY., USA., pp: 72-78.
3: Curtis, W.S. and C.L. Sabharwal, 1997. Human Face Recognition in the Wavelet Compressed Domain, Smart Engineering Systems. In: Smart Engineering Systems: Neural Networks, Fuzzy Logic, Data Mining and Evolutionary Programming, Dagli, C.H., M. Akay, O. Ersoy, B.R. Fernandez and A. Smith (Eds.). ASME Press, New York, USA pp: 555-560.
4: Chien, J.T. and C.C. Wu, 2002. Discriminant wavelet faces and nearest feature classifiers for face recognition. IEEE Trans. Patt. Anal. Mach. Intell., 24: 1644-1649.
CrossRef | Direct Link |
5: Li, B. and Y. Liu, 2002. When eigenfaces are combined with wavelets. Knowledge-Based Syst., 15: 343-347.
6: Ekenel, H.K. and B. Sankur, 2005. Multiresolution face recognition. Image Vision Comput., 23: 469-477.
CrossRef | Direct Link |
7: Delac, K. and M. Grgic, 2007. Face Recognition. I-Tech Education and Publishing, Vienna.
8: Delac, K., M. Grgic and S. Grgic, 2009. Face recognition in JPEG and JPEG2000 compressed domain. Image Vision Comput., 27: 1108-1120.
CrossRef | Direct Link |
9: Lee, S.H. and S. Choi, 2007. Two-dimensional canonical correlation analysis. IEEE Signal Process. Lett., 14: 735-738.
10: Hotelling, H., 1936. Relations between two sets of variates. Biometrika, 28: 321-377.
11: Jelsovka, D., R. Hudec and M. Breznan, 2011. Face recognition on FERET face database using LDA and CCA methods. Proceedings of the 34th International Conference on Telecommunications and Signal Processing (TSP), August 18-20, 2011, Budapest, pp: 570-574.
12: Oh, B.J., 2005. Face recognition using radial basis function network based on LDA. World Acad. Sci. Eng. Technol., 7: 225-259.
13: James, M. and S. Arockiasamy, 2011. Face recognition in compressed domain based on wavelet transform and kd-tree matching. Global J. Comput. Sci. Technol., 11: 28-34.
Direct Link |
14: James, M. and S. Arockiasamy, 2012. Human face recognition in wavelet compressed domain using canonical correlation analysis. Int. J. Comput. Appl., 37: 36-40.
Direct Link |
15: James, M. and S. Arockiasamy, 2012. Face recognition in compressed domain by applying canonical correlation analysis based feature vector optimization and neural network matching. Int. J. Comput. Sci. Network Secur., 12: 77-84.
Direct Link |
16: Alwakeel, M. and Z. Shaaban, 2010. Face Recognition based on haar wavelet transform and principal component analysis via levenberg-marquardt backpropagation neural network. Eur. J. Sci. Res., 42: 25-31.
Direct Link |
17: Turk, M. and A. Pentland, 1991. Eigenfaces for recognition. J. Cognit. Neurosci., 3: 71-86.
CrossRef | Direct Link |
18: Jolliffe, I.T., 2002. Principal Component Analysis. 2nd Edn., Springer-Verlag, New York, USA.
19: Bartlett, M.S., J.R. Movellan and T.J. Sejnowski, 2002. Face recognition by independent component analysis. IEEE Trans. Neural Networks, 13: 1450-1464.
CrossRef | Direct Link |
20: Timotius, I.K., I. Setyawan and A.A. Febrianto, 2010. Face recognition between two person using kernel principal component analysis and support vector machines. Int. J. Electr. Eng. Inform., 2: 53-61.
Direct Link |
21: Kim, K.I., K. Jung and H.J. Kim, 2002. Face recognition using kernel principal component analysis. IEEE Signal Process. Lett., 9: 40-42.
CrossRef | Direct Link |
22: Kukharev, G. and E. Kamenskaya, 2010. Application of two-dimensional canonical correlation analysis for face image processing and recognition. Pattern Recogn. Image Anal., 20: 210-219.
23: Martinez, A.M. and R. Benavente, 1998. The AR face database. CVC Technical Report No. 24, June 1998.