Thermography is a noninvasive procedure for acquiring the thermal profile of a subject in the diagnosis and surgical procedures. The superimposition of thermograms with corresponding visual image of the subjects is a technique followed by clinicians to relate the thermal image with the surface anatomy to localize the diseased parts. Authentication is an essential security aspect in healthcare systems to avoid medical errors. This study focuses on the fusion and authentication of thermograms. The semiblind reversible watermarking system proposed in this study presents a watermarking scheme that embeds the visual facial image of the patient within the facial thermogram in an invisible manner to both authenticate and protect the privacy of the patient. The watermark is extracted later to identify the patient and to generate a composite image for cross reference. The experimental results illustrated for different pathologies demonstrate the efficacy of the proposed technique.
PDF Abstract XML References Citation
How to cite this article
Medical thermography is a non-invasive procedure employed in the pre-clinical diagnosis of many disorders. The efficiency and simplicity of the thermographic systems enable easy capture of the thermograms of different body parts and transmission of the same to remote medical practitioners and healthcare professionals.
Infrared thermal images of the faces and body parts can be captured by thermal cameras with or without the knowledge of the subjects. Thermography or Digital Infrared Thermal Imaging (DITI) systems comprise of high speed computer systems with high precision thermal cameras. Thermograms are generated by mapping the body temperature of the subjects into an image. Thermography is employed in face recognition systems for surveillance and monitoring to identify individuals in crowds. Being a noninvasive and a painless procedure, thermography has gained popularity among the physicians in screening pathologies. This procedure is widely adapted in the fields of Dentistry, Neurology, Rheumatology, Oncology, Physiotherapy, Sports medicine etc., as discussed in an article by Sandham (2005). Schulte et al. (1975) identified Facial thermography as a potential methodology in the screening of strokes as early as 1975. Out of a study conducted with 102 subjects, Gratt and Sickles (1995) identified that vascular heat emissions from the human face are reliable psychological indicators of diseases. They concluded that the thermal differences between specific facial regions can be used to identify subjects with asymptomatic disorders. Further, thermography has been identified as a robust technique to uniquely identify individuals and body parts when compared to any other biometric. The accuracies of detection from 85 to 98% have been reported in a study by Prokoski and Riedel (1998).
Authentication is an important security aspect in the present scenario of exchange of medical images between medical practitioners. The cost of medical errors due to erroneous patient identification has been elaborately discussed in different literatures. Instances of patient identification errors and consequences have been elaborately discussed, illustrating a case history by Hall (2008). This commentary emphasizes the usage of bar coding and radio frequency devices, for patient identification and developing systems for patient authentication. A research with the radiology (Turner et al., 2008) experts has shown that, including the photograph of the patient along with the radiology images improves the diafgnosis.
The need for fusion of the image in thermal spectrum and visual image has been emphasized in many literatures pertaining to face recognition. Image registration is one of the most important medical image processing techniques and is used to geometrically align two images taken from different sensors, viewpoints or instances in time. Both images are typically aligned through a combination of scaling, translation and rotation transforms. Registration is often used to verify the effects of treatment and make comparisons of patient data with anatomically normal subjects.
This study focuses on the aspects of photograph based authentication and fusion of thermograms. The proposed system embeds the visual facial image or surface anatomy image within the corresponding thermogram of the subject. The visual image serves as a watermark for authentication of the thermogram upon extraction and for alignment with the thermogram to localize the pathology.
MATERIALS AND METHODS
Medical applications of thermography: Thermal imaging finds applications in breast oncology, integrative medicine, plastic surgery, dentistry, orthopedics, acupuncture, occupational medicine, pain management, vascular medicine, cardiology and veterinary medicine. It has been identified to be the best technique for preliminary breast examinations. Clinical applications employ static and dynamic thermography methods for capturing the thermal profile of the subjects. The provision for both qualitative and quantitative assessment in thermography helps in the estimation of progression of the disease in a systematic manner. The results of clinical assessment by facial thermography are discussed in detail by Gratt et al. (1996). From the results it is observed that, hot, cold and normal facial thermograms give a clear indication of sinusitis, peripheral nerve mediated pain and psychogenic facial pain , respectively. Thermography is used in the diagnosis and treatment of chronic orofacial pain patients, Temporo Mandibular Joint (TMJ) disorders etc. (Sudhakar et al., 2011). The study concludes that thermography aids in the assessment and staging of various dysfunctions of the head and neck region. Medical thermography is followed as an adjunctive procedure along with other diagnostic procedures to help the clinician in making a thorough examination. Lately, the significance of selection of acupoints aided by facial thermography for treating facial paralysis has been reported by Nan and Hou (2012). The studies with subjects indicate that, acupuncture treatment provided at selected acupoints aided by thermography resulted in fewer sessions of short treatment durations.
Medical image authentication with watermarking: The significance of medical image watermarking for integrity and authenticity has been elaborately discussed in a study by Coatrieux et al. (2006). Zhou et al. (2001) presented a scheme for watermarking digital mammography images with a message digest generated with MD5 algorithm for enforcing integrity. Lossless or reversible (Tian, 2002) watermarking schemes have invited attention owing to the need for high quality images with no or less degradation for medical diagnosis, military investigation and forensic analysis etc. As medical images contain sensitive data which an expert cannot afford to lose, reversible watermarking schemes are applied to recover the medical images after the extraction of the watermark. The strength of a watermarking system can be improved by incorporating the paradigm of blind (Bartolini et al., 1999) watermarking which refers to the watermark extraction without the need for the original unmarked image during the extraction procedure. It prevents the unauthorized distribution and copying of the original images. A review of reversible watermarking techniques, their classification and applications have been elaborately discussed in an article by Caldelli et al. (2010). This article summarizes various reversible watermarking techniques which are based on simple integer arithmetic operations.
Thermal visual image fusion: Often the thermal and visual images are compared by the clinician to localize the pathology based on the thermal patterns. This procedure is followed for medial diagnosis as well as prognosis. It requires the clinician to perform a mapping between these images. However, a composite image of these modalities will help the clinician to cross reference the thermal profile of the subject with that of the surface anatomy. A method for overlay has been presented by Schaefer et al. (2006) based on skin detection and image registration. From medical literatures, once can identity the significance of fusion for treating subjects with Freys Syndrome (FS). This is characterized by excessive sweating of the forehead, upper lip, peri-oral region or sternum subsequent to intake of food. Isogai and Kamiishi (1997) have proposed thermography as a diagnostic procedure for FS. Localization of the foci of FS in the facial region is essential to administer the Botulinum Toxin (Botox) or performing parotid surgery. Kragstrup et al. (2011) presented an elaborate literature review on FS and a case report of a subject with FS. The authors conclude that facial thermograms and starch iodine tests supplement the clinical examination of the subject.
Proposed system: The proposed system is the extension of the work on watermarking a medical image with the facial image of the subject for improved telediagnosis and tamper detection (Velumani and Seenivasagam, 2010). The previous work is extended to demonstrate the fusion of the host and watermark images for enhanced diagnosis. The proposed systems is termed as semiblind as the watermark extraction algorithm needs a key for watermark extraction and reconstruction of the original host image. The algorithms for watermark embedding and extraction and superimposition are given below.
Watermark embedding and extraction: The algorithms are based on the application of a Triangular Number Generator (TNG) function. It is a function as in (1) which uniquely encodes a pair of positive integers (a, b) into a unique number TR. The number TR can be factored back to a and b exactly by applying the extraction procedure. The proposed system exploits this reversible and blind nature of the TNG in the watermark embedding and extraction algorithms. The properties of TNG are discussed in detail in an article by Gupta (2002).
Watermark embedding is performed by combining n number of LSB planes of the thermal and visual images by Eq. 1. As the combination leads to an integer whose magnitude cannot be expressed in n bits, modulo 2n operation is performed on the resultant integer. While the n bit reminder is appended with 8-n bit planes, the quotient is preserved as a key. Similarly, on watermark extraction, the combination is reconstructed by combining the key and the n LSB planes of the watermarked image. The watermark embedding and extraction algorithms are given in Algorithm 1 and 2, respectively assuming n = 4.
Algorithm 1 watermark embedding:
|Step 1:||Extract 4 MSBs and 4 LSBs of Thermal Image to form matrices MSBT and LSBT|
|Step 2:||Extract 4 MSBs of Visual Image to form a matrix MSBV and shift it right 4 times|
|Step 3:||Combine LSBT & shifted MSBV using TNG to form matrix TR|
|Step 4:||Perform Modulo 16 arithmetic operation on TR to generate remainder and quotient matrices RemTR and QuoTR, respectively|
|Step 5:||Combine MSBT and RemTR to form the watermarked image WMI; preserve QuoTR|
Algorithm 2 watermark extraction:
|Step 1:||Extract 4 MSBs and 4 LSBs of the watermarked image to form matrices MSBWMI and LSBWMI, respectively; LSBWMI is RemTR|
|Step 2:||Multiply key/quotient matrix QuoTR by 16 and add the resultant matrix with LSBWMI to recover matrix TR|
|Step 3:||Extract the LSBT and shifted MSBV from TR; Left shift MSBV>>4 for 4 times to extract the watermark|
|Step 4:||Combine with LSBT with MSBWMI to recover the Thermal image|
The above algorithms are illustrated for with Fig. 1 and 2, respectively. The facial profile of a subject with FS who has taken a starch iodine test is taken as the visual image V and the thermogram of the same area is taken as the host image T.
|Fig. 1:||Watermark embedding|
|Fig. 2:||Watermark extraction|
Visual and thermal image overlay: The superimposition of the visual and thermal images is performed by computing the weighted summation of the intensity values of the thermal and visual image pairs. As the purpose of overlay is to localize the area of pathology in the human anatomy, the pixel intensity values of the thermal and visual images are scaled down by a factor α and β, respectively. The resultant value is added with the pixel intensity value in the same spatial orientation of the visual image to get the composite intensity value. The overlay generation is given in Algorithm 3 as below.
Algorithm 3 thermal and visual image overlay:
|Step 1:||Select a bit plane of the Thermal Image T and Visual Image V|
|Step 2:||Select the values of α and β|
|Step 3:||Compute overlay (i, j) = T(i, j)*α+V(i, j)*β for each pixel position (i, j)|
|Step 4:||Repeat steps 1 to 3 for all 3 color planes of the Thermal and Visual images|
|Step 5:||Combine the 3 color planes of Overlay to generate the composite image|
RESULTS AND DISCUSSION
The proposed algorithms have been implemented in MATLAB with the visual and thermal images of healthy subjects and subjects with pathologies. For healthy subjects the facial images have been chosen as watermarks. For thermograms with pathologies, the digital images of the surface anatomy have been chosen as watermarks. All the images are of the size 512x512x3. The results are illustrated with performance metrics for healthy subjects and subjects with pathologies in Table 1 and 2, respectively. The Peak Signal to Noise Ratio (PSNR), a measure of the image perceptibility is evaluated for the watermarked and reconstructed images and the watermarks. The composite images are given in the last row of the tables. Invariably the values of α and β are assumed to be .5 and .7, respectively for all images to generate the overlays.
The thermal and visual images in Table 1, produced by Mikos Biotek in Virginia, USA are taken from the Science Photo Library database (Biotek, 2012). The visual and thermal images in Table 2 are taken from the gallery of IR images (Mercer, 2011).
|Table 1:||Test images and experimental results for healthy subjects|
|Table 2:||Test images and experimental results for pathologies|
|Table 3:||Comparison with earlier approaches|
|The above table compares the proposed scheme with the earlier schemes reviewed in the article by Caldelli et al. (2010) with respect to the embedding capacity and PSNR values|
The experiments have been implemented for the following pathologies:
|•||Abdominal region prior to breast reconstruction surgery|
|•||Post operative breast reconstruction surgery|
Exclusively, the visual image illustrating the Freys syndrome is one which shows black patches in the parotid area as a result of Minors Starch Iodine test.
It is seen from Table 1 that the PSNR values of the watermarked images are closer to 38 dB for facial thermograms. The watermarked thermogram images in Table 2 seem to have PSNR values around 36 dB. Though the PSNR values are not significantly high, 36 dB is an acceptable value for medical images. PSNR is also attributed by the size of the watermark. In this experiment both the visual and thermal images are resized to the same dimension before watermarking for pixel by pixel alignment for the generation of overlay. The PSNR values for the watermarked images will significantly improve on reducing the dimension of the watermark. However, it will necessitate another resize for overlay. It is clearly understood from the experimental results that the proposed system provides perfect reconstruction of the cover image and authentication of the thermogram and patient identification without ambiguity. It is evident from the PSNR values which are Infinity for both the recovered cover images and the watermarks for both healthy subjects and pathologies.
The proposed system based on spatial watermarking is compared with similar systems discussed in the review article by Caldelli et al. (2010). The performance of the proposed system is compared with the other schemes in respect of the PSNR values of the watermarked images. As arbitrary images have been taken in different studies, the comparison is based on the size of the cover images and the watermark. The details of the comparisons are tabulated in Table 3. Only the best PSNR values realized by the other schemes are taken for comparison.
From the Table 3, it is evident that the proposed system provides a theoretical embedding capacity of 4 bpp (bits per pixel) with an average PSNR of 37 dB. In contrast to the above schemes, the semi-blind nature of the proposed system requires the secured storage and exchange of the key which is of the size 512x512x3. Though, it is an overhead, it is worth in accomplishing the watermark extraction and cover image recovery in an intact manner. Further, as the key is a combination of the cover and the watermark images, guessing the key by an attacker is highly unrealistic and it provides a great deal of security.
In addition to the above, the method for superimposition presented in the study seems to be a promising one for generating the overlays. The proposed scheme for overlay generation is contrary to the scheme proposed by Schaefer et al. (2006), in which the final overlay image is weighted by 20 and 80% of the thermal and visual images, respectively. The pixel by pixel superimposition proposed in this study guarantees the perfect geometric alignment of the visual and thermal patterns. Particularly, in the case with Freys syndrome, the overlay generated by the proposed approach seems to exactly localize the region for injection of Botox than the conventional starch iodine test. Hence, the preciseness of the watermarking system complements the overlay generation.
The proposed semiblind reversible watermarking scheme for authenticating the thermograms contributes to generate a perfect overlay of the thermal and visual images. The proposed system is novel when compared to the earlier reversible watermarking systems in the approach selected for realizing reversibility and blind extraction. From the diverse nature of the thermal images chosen, it found that the proposed system can be extended to all thermogram based diagnosis and prognosis. The system can be further enhanced by eliminating the need for the key. However, the semiblind nature of the system protects the images from unauthorized access and also indicates incidental or accidental tampering. The efficacy of the proposed system can be tested further by experimenting with medical images related to fine scale dermatitis, melanoma and other syndromes which require exact localization for incision in surgical procedures.
- Alattar, A.M., 2003. Reversible watermark using difference expansion of triplets. Proceedings of the International Conference on Image Processing, Volume 1, September 14-17, 2003, Barcelona, Spain, pp: 501-504.
- Alattar, A.M., 2004. Reversible watermark using the difference expansion of a generalized integer transform. IEEE Trans. Image Process., 13: 1147-1156.
- Bartolini, F., G. Bini, V. Cappellini, A. Fringuelli, G. Meucci, A. Piva and M. Barni, 1999. Enforcement of copyright laws for multimedia through blind, detectable, reversible watermarking. IEEE Int. Conf. Multimedia Comput. Syst., 2: 199-203.
- Coltuc, D. and J.M. Chassery, 2007. Very fast watermarking by reversible contrast mapping. IEEE Signal Process. Lett., 14: 255-258.
- Schaefer, G., R. Tait and S.Y. Zhu, 2006. Overlay of thermal and visual medical images using skin detection and image registration. Proceedings of the 28th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, August 30-September 3, 2006, New York, USA., pp: 965-967.
- Gratt, B.M. and E.A. Sickles, 1995. Electronic facial thermography: An analysis of asymptomatic adult subjects. J. Orofacial Pain, 9: 255-265.
- Gratt, B.M., S.B. Graff-Radford, V. Shetty, W.K. Solberg and E.A. Sickles, 1996. A 6-year clinical assessment of electronic facial Thermography. Dentomaxillofacial Radiol., 25: 247-255.
- Isogai, N. and H. Kamiishi, 1997. Application of medical thermography to the diagnosis of Frey's syndrome. Head Neck, 19: 143-147.
- Caldelli, R., F. Filippini and R. Becarelli, 2010. Reversible watermarking techniques: An overview and a classification. EURASIP J. Inform. Secur.
- Schulte, B.P., M.A. Bomhof and N.J. Aarts, 1975. Facial thermography in the diagnosis of cerebrovascular disease and in evaluation of carotid endarterectomy. Clin. Neurol. Neurosurgery, 78: 118-130.
- Sudhakar, S., B. Kayshap and P.S. Reddy, 2011. Thermography in dentistry-revisited. Int. J. Biol. Med. Res., 2: 461-465.
- Kragstrup, T.W., J. Christensen, K. Fejerskov and A. Wenzel, 2011. Frey syndrome-an underreported complication to closed treatment of mandibular condyle fracture? Case report and literature review. J. Oral Maxillofacial Surgery, 69: 2211-2216.
- Velumani, R. and V. Seenivasagam, 2010. A reversible blind medical image watermarking scheme for patient identification, improved telediagnosis and tamper detection with a facial image watermarkc. Proceedings of the IEEE International Conference on Computational Intelligence and Computing Research, December 28-29, 2010, Coimbatore, pp: 1-8.
- Zhou, X.Q., H.K. Huang and S.L. Lou, 2001. Authenticity and integrity of digital mammography images. IEEE Trans. Med. Imag., 20: 784-791.