Abstract: Breast cancer is the highest incident cancer in women and a serious threat to a woman’s life. Early detection and treatment of breast cancer can reduce the mortality rate. Currently, mammography is widely employed for routine screening of breast cancer. Ultrasound imaging is used as an important adjunct to mammography, especially in the post-screening (diagnostic) phase. Irrespective of the imaging modality, several factors including the level of radiologists’ expertise affect the accuracy of breast cancer detection and diagnosis. Computer Aided Detection/Diagnosis (CAD) systems are objective in nature as opposed to the subjective analysis made by radiologists. Many studies show that the use of a CAD system as a second reader has the potential to improve the accuracy of breast cancer detection and diagnosis. Recently, integration of information from multiple sources is gaining wide popularity in data analysis. Information fusion in CAD systems would serve to mimic the radiologist’s practice of combining information from multiple mammographic views and from multiple imaging modalities like ultrasound imaging and mammography to arrive at better diagnostic decisions. This study reviews the literature on such CAD systems based on mammograms and ultrasound images for breast cancer detection and diagnosis.
INTRODUCTION
The World Health Organization (WHO) has estimated that breast cancer is the highest incident cancer among women, accounting for 23% of total cases. It is also the primary cause of death in women. Nearly 14% of female deaths due to cancer are caused by breast cancer. Early detection and subsequent treatment of breast cancer can save the life of a woman without the need for mastectomy (Jemal et al., 2011). A mammogram is a low-dose X-ray image of the breast that can depict the earliest sign of breast cancer even in asymptomatic women. Screening mammography is performed on women who do not exhibit any signs or symptoms of breast abnormality, in an attempt to detect breast cancer in the very early stages. On the other hand, diagnostic mammography aims in evaluating the clinically determined abnormalities or those found during screening mammography (Tang et al., 2009). The standard practice in mammography is to acquire breast images from two different angles. These include the mediolateral (MLO) and craniocaudal (CC) views. The MLO projection is a side-to-side view of the breast, taken at a 45° angle. This view shows a better perspective of the glandular portion. The CC projection is a top-down view of the breast. The focus of this view is on the central and inner portions of the breast. In screening mammography, both these views are acquired for both the breasts, whereas in digital mammography, they are acquired for the breast that is to be examined. Normally while analysing mammograms, radiologists analyze suspicious regions in these two projections in combination so as to improve the detection and diagnostic performance (Velikova and Karssemeijer, 2008).
Currently, mammography is the only widely accepted imaging modality for routine screening of breast cancer. It has been shown that screening mammography can reduce breast cancer mortality rates. On the negative side, the fact that mammographic abnormalities are subtle especially for dense breasts, limits the detection performance. Missed malignancies result in delayed treatment and severe implications including loss of life. It has been reported that radiologists fail to detect 10-30% of cancers. Also, the accuracy of mammography in distinguishing benign from malignant lesions is low. Benign cases diagnosed as malignant result in biopsies that cause unnecessary physical, emotional and financial discomfort to the patients. Biopsy is an invasive procedure which is considered to be the gold standard to determine whether a tumor is malignant. About 65-85% of biopsy operations are reported to be unnecessary (Sampat et al., 2005; Gupta et al., 2006; Tang et al., 2009).
Ultrasound imaging is an import adjunct to mammography for breast cancer diagnosis. It is useful in detecting breast lesions in women with dense breasts and those who are less than 35 years old, for whom mammogram is not effective. Statistics show that the use of ultrasound can enhance detection rate by 17%. It has been shown that ultrasound can reduce the number of unnecessary biopsies by 40%. Ultrasound has an important role in differentiating cysts (fluid-filled lesions that are almost benign) from solid lesions which is not possible with mammograms (Cheng et al., 2010; Prasad and Houserkova, 2007). Some studies have also reported that the accuracy of classifying solid benign lesions from malignant lesions is higher for ultrasound images than mammograms (Stavros et al., 1995; Prasad and Houserkova, 2007).
Despite these advantages ultrasound is not considered as a standard screening test and is used only in combination with mammography for diagnosing breast cancer. The reason is that ultrasound imaging involves a hand-held probe for scanning and hence it is much more operator-dependent than mammography. The nature of image acquisition might also result in some areas of the breast not being scanned. Ultrasound imaging has poor ability to capture deep lesions and is more useful only for diagnosing clinically palpable and/or mammographically detected lesions (Moon et al., 2011). Further, ultrasound cannot always detect microcalifications. Nevertheless, ultrasound imaging is finding increased use as a complementary imaging technique for mammography and can add value to breast cancer detection and diagnosis (Cheng et al., 2010; Egorov and Sarvazyan, 2008).
Double reading by two different radiologists has been recommended to surmount the drawbacks in interpreting breast images. It has been reported that double reading can improve the detection rate by 5-15%. However, the time, cost and work load involved is high and the resulting diagnostic decisions might be ambiguous. A Computer Aided Detection/Diagnostic system (CAD) can be used to replace the second human reader so as to aid radiologists in their interpretation. Many studies show that the use of a CAD system as a second reader has the potential to improve the accuracy of breast cancer detection and diagnosis (Rangayyan et al., 2007; Mencattini et al., 2010). The CAD systems are generally of two kinds: Computer Aided Detection (CADe) systems and Computer Aided Diagnosis (CADx) systems. The CADe systems determine suspicious regions called regions of interest (ROIs) and intend to improve the detection accuracy of radiologists. The CADe systems intended for mammogram analysis usually suffer from a large number of False Positives (FPs). Therefore, in these systems, a false positive reduction stage follows the detection phase, wherein the detected ROIs are classified as normal tissue or abnormal. Computer Aided Diagnostic (CADx) systems classify the abnormal regions as benign or malignant and intend to reduce the number of benign cases that are subjected to biopsy (Jalalian et al., 2013).
Many studies have shown that two-view mammographic analysis has improved the cancer detection rate and has also reduced the number of call-back examinations (Bassett et al., 1987; Blanks et al., 1999; Hackshaw et al., 2000). It has also been demonstrated that diagnostic performance of radiologists is higher when both ultrasound images and mammograms are used in combination (Malur et al., 2000). The CAD systems that adopt these principles have been suggested by many researchers.
CADe SYSTEMS BASED ON MULTIPLE MAMMOGRAPHIC VIEWS
The central idea behind all CADe systems which have utilized two-view mammographic information for false positive reduction is that if a suspicious region in one view has characteristics which are similar to that in another view, there is a higher probability that the region is abnormal, i.e., a True Positive (TP).
An adaptive system based on two-view analysis has been proposed by Qian et al. (2007) for false positive reduction in a CADe system. In this system, features are extracted from the suspicious regions in the two views to perform region matching. For an ROI in the MLO view, the projection distance between the ROI’s center and the nipple on the line perpendicular to the pectoral wall and passing through the nipple is computed. If this distance is comparable with the projection distance of an ROI in the CC view, then the two ROIs have more chances of being the projections of the same lesion. For CC views, chest wall is considered as the reference instead of the pectoral wall. This method for defining the search region is called the straight line-based method. If such a matching ROI is not available in the other view, pre-processing and segmentation of the other view is performed again, with focus on the region where the projection distance is matched. If this results in a matched region, it is used for further analysis; otherwise the ROI for which no matched region could be determined is rejected as an FP. On the contrary, if more than a matched region could be determined for an ROI, they are subjected to further analysis. After the spatial matching is performed, the matched ROIs are further checked for matching according to shape and other characteristic features. If matching according to these criteria could not be established, then the single view analysis is performed again to determine whether such matching could be made possible. If such regions could not be determined, the ROI is rejected as an FP. Thus an AND logic is employed in this work such that a region is considered to be suspicious only if it is found in both views. It has been shown that the improvement in mean Area Under Curve (AUC) achieved by the two-view system when compared to the single view system is 6.5%. In this system, each characteristic feature is individually tested for similarity between the two views by determining whether the ratio of their values for the ROIs in the two views is approximately equal to unity. The check for similarity has not utilized the interactions between the various features. Many other researchers including Paquerault et al. (2002), Wei et al. (2009), Van Engeland and Karssemeijer (2007) and Samulski and Karssemeijer (2011) have proposed more sophisticated systems that involve classification algorithms for determining the correspondence between ROIs of the two views.
Paquerault et al. (2002) have proposed a two-view mammographic CADe system for classification of ROIs as normal regions or masses. In this study, single view analysis is first performed wherein morphological and texture features are used to train two different Linear Discriminant Analysis (LDA) classifiers whose scores are averaged to produce the malignancy score for each ROI. This score is called the one-view detection score. Following this, two-view analysis is performed to combine information from MLO and CC views. In the two-view analysis, for every object in one view, a search region is determined in the other view based on a geometric model that uses the distance between the nipple and the candidate region. In this model, the distance to nipple is determined in one of the views. Following this, an annular region is defined in the other view which is the nipple to object distance in the second view ± a width representing the localization error which is estimated from the training set. This method for determining the search region is called the arc-based method. An ROI in a view is then paired with all ROIs in the search region of the second view. Similarity measures based on morphological and texture are then determined for all paired objects. These similarity features are used to train two different LDA classifiers separately in the morphological and texture space and the resulting scores are averaged to determine a single correspondence score for each object pair. Finally, decision fusion is performed wherein the detection score obtained from single view analysis is combined with the correspondence score obtained from two-view analysis, by averaging the two scores. This scheme for decision fusion, wherein scores from two classifiers are averaged is called the sum rule. It has been reported that this scheme improved the mass detection sensitivity at a False Positive Rate (FPR) of 1 FP/image over that of the single view analysis by 14%. This system did not address the issue of an ROI in a view being paired with more than one ROI in the other view. It also does not consider the scenario wherein an ROI in a view does not have a potential corresponding region in the other view which may arise if a lesion is missing in one view.
A solution has been provided for the first problem by Wei et al. (2009) who have proposed a dual CADe system with two-view analysis for classification of suspicious regions as masses or FPs. In this study, a dual system which comprises two single-view systems in parallel trained separately using two different training sets is employed in the first stage. Current mammograms which are less subtle comprised the first data set and prior mammograms whose subtlety is relatively high comprised the second data set. For a given object, the scores generated by the two systems are merged by a neural network to generate one score for the mass likelihood. The MLO and CC images are processed separately by the dual system to detect suspicious regions in these views. Single view scores for the detected regions in the two views are generated by the dual system. Following this, a two-view fusion strategy is employed to combine information from the objects in the two views. For this, regional registration is first performed to determine the approximate locations containing the corresponding ROIs. This registration technique is similar to the arc-based method used in the system proposed by Paquerault et al. (2002). An object in one view is paired with all objects within the search region of the other view. For all candidate pairs, a cross-correlation measure is determined. Following this, a similarity measure is also determined for paired regions using an LDA classifier that differentiates TP-TP pairs from TP-FP and FP-FP pairs. The inputs to this classifier are paired morphological features, Hessian features and texture features. The similarity LDA score is then weighted by the correlation measure. If an object in one view is paired with more than one object in the other view, the maximum weighted LDA score is considered to be the two-view fusion score. Finally, decision fusion is performed wherein the single-view dual system scores and the two-view fusion score are used as input to another LDA classifier to classify a given region as a normal region or a mass. For average subtlety masses the improvement in sensitivity achieved by the two-view dual system is 3.6 and 20.1% at 1 FP/image respectively, when compared to the dual system and single view system. The corresponding improvement for highly subtle cases has been observed to be 1.5 and 20.8%, respectively. Though the problem of multiple linking is addressed by Wei et al. (2009), the possibility of missing cancer in one of the views has not been explored.
Van Engeland and Karssemeijer (2007) have proposed the analysis of only those potential paired ROIs which are retained after a linking process based on the correspondence scores, thereby providing a solution for the scenarios of multiple linking as well as missed cancer. This system has been built using a cascade connection of single-view and two-view analysis. Features of suspicious regions that include shape, size, contrast, texture and relative location are used by an Artificial Neural Network (ANN) to determine the measure of suspiciousness of the regions. Following this, correspondence is established between masses in the two mammographic views. For this, the nipple to object distance is used to define the search region. Masses from the two views for which the difference in distance to nipple in the respective views is smaller than 2.4 cm are considered. A set of features based on the nipple distance difference, contrast difference and correlation are used as input to an LDA classifier which is trained to differentiate between correct and incorrect pairs. Correspondence scores for various candidate pairs are produced by an LDA classifier. Based on the correspondence score, masses in the two views are linked using a one-to-one correspondence strategy that maximises the correspondence scores. Following this, single view features, malignancy scores, two-view features and the correspondence scores of linked ROIs are merged by a ANN to determine the final likelihood of malignancy. The improvement in lesion-based sensitivity of the two-view system has been reported to be 4% at 0.1 FP/image when compared to the single view system.
Samulski and Karssemeijer (2011) have proposed a two-view CADe system which focuses on improving case-based detection performance rather than just the lesion-based performance. Pre-segmentation and segmentation steps are carried out to detect ROIs. Morphological features and mass likelihood score from the initial detection stage are used by a neural network to determine the single view suspiciousness measure for a region. Following this, regions in MLO and CC views are matched by defining the search region for which both the arc-based method and the straight line-based method are explored. Similarity features are then computed for all possible region pairs. These include the difference in distance to nipple, compactness difference, linear texture difference, entropy difference, single-view mass likelihoods of the two-views and their difference, histogram correlation and pixelwise correlation. These similarity features are fed as inputs to a k-NN classifier trained to classify the candidate pairs into one of the four classes TP-TP, TP-FP, FP-TP and FP-FP. The resulting correspondence score from the classifier is subjected to a threshold so that only those regions for which the score exceeds a threshold are matched to regions in the two views. For regions with no correspondence, the similarity features and TP-TP score have been set to zero and scores for all other pairs (TP-FP, FP-TP and FP-FP) are set to one third. Further, case-based sensitivity is improved by excluding from the training set, those TPs whose malignancy scores are much lesser than their counterparts in the other view. This would avoid a lower malignancy score for a TP after applying two-view classification. For two-view classification, single view features, single view malignancy scores, two-view features and four class probabilities of the correspondence classifier have been merged by a ANN which produces the final suspiciousness measure. To improve the detection performance at the case-level, those true positive regions for which the malignancy score are weaker than their counterparts are not included in the training set for training the two-view classifier. An increase of 4.7% in the mean case-based sensitivity in the range of 0.01-0.5 false positives per image has been reported.
Rather than generating correspondence scores using classifiers, few CADe systems (Velikova and Karssemeijer, 2008; Velikova et al., 2012) have made use of Logistic Regression (LR) models to determine link probabilities. Velikova and Karssemeijer (2008) proposed probabilistic causal model based on Bayesian framework to build a CADe system for differentiating normal regions from masses. In this work, single view systems are used to detect suspicious regions in both views. Following this, features based on breast and local area information are extracted from the detected regions. Using these features, an ANN is used to classify a region as normal or mass region. Links are then established between ROIs of the two views, where a link value is true if it links two TPs and false if it links two false positives FPs. Following this, LR is used to compute the link probabilities using features that include the ANN’s output for the single view CAD, morphological and location features and the binary link value. The causal model utilizes the link probabilities to determine the probabilities of the suspicious regions (region probabilities) being true, using the logical OR. In a similar manner region probabilities are combined using logical OR to obtain view probabilities. Two different methods have been explored to combine the view probabilities to obtain the probability of a given case (patient). These include the sum rule and the LR method. In the first method, a decision strategy where the case probability is equal to the average of the probabilities of the two views is employed. In the second method, an LR model with view probabilities as inputs is used to determine the case probability. The Receiver Operating Characteristics (ROC) analysis showed an improvement of 4.13% in mean AUC for the two-view system over the single view system.
Velikova et al. (2012) proposed another CADe system by slightly modifying their previous work (Velikova and Karssemeijer, 2008). The major difference is that instead of defining binary values for links in the matching phase, they are assigned one of the four values, i.e., TP-TP, TP-FP, FP-TP and FP-FP. The authors reported an increase in sensitivities of 6.3 and 5.2% at 5 and 10% false positive rates for the two-view system when compared to single-view system.
CADx SYSTEMS BASED ON MULTIPLE MAMMOGRAPHIC VIEWS
Many researchers (Chan et al., 1998; Wei et al., 2005; Veldkamp et al., 2000) have combined information from MLO and CC views for classification of microcalcification clusters (MCCs) as benign or malignant. Chan et al. (1998) used morphological and texture features to characterize the MCCs and LDA for the classification task. Two decision fusion methods that include sum rule and maximum rule are used to combine the individual malignancy scores of the single view systems. The mean AUC for the sum rule has been observed to be 3% better than that obtained using the maximum rule and 4% better than that of the single view system.
Wei et al. (2005) also proposed different CADx algorithms for combining MLO and CC views for classification of MCCs of the breasts as benign or malignant. These include the direct, average and joint methods. The features used for classification represent the morphological characteristics of the MCCs. In the direct method, both views of a case are included either in the training set or the testing set. Even if the ROI in one view is diagnosed as malignant, the final decision for the case is malignant, i.e., a decision fusion based on OR rule is employed for decision making. In the average method, the parallel feature fusion technique is employed wherein the final feature vector is obtained by averaging the feature vectors from the two views. In the joint method, the serial feature fusion strategy based on concatenating feature vectors from the two views is adopted. All these methods are performed using both ANN and SVM classifiers. The performance of the SVM classifier has been reported to excel that of the ANN classifier. Research results indicated that the average method outperformed the other two methods when either classifier is used. All the methods have been evaluated using the AUC as the performance measure. With SVM classifier, the average method improved the mean AUC by 2.2% over the direct method and by 6.2% over the joint method.
Analysis of MCCs was performed only at the lesion-level by Chan et al. (1998) and Wei et al. (2005). Veldkamp et al. (2000) targeted at improving the patient-level performance in classifying MCCs. In this work, corresponding lesions in the two views are matched using one-to-one correspondence strategy that minimizes the squared distance between the clusters in the two views. Following this, features including distribution features, cluster shape features and cluster location features are extracted from clusters in either view. The features from a given view are given as input to a k-nearest neighbor (k-NN) classifier that results in a lesion-level malignancy score. In this way, two k-NN classifiers are employed to obtain the malignancy scores from the two views at lesion level. Finally, the lesion-level scores are used to obtain a patient-level score for malignancy. For determining the patient-level score, two different decision fusion approaches are employed. In the first approach, the patient-based score is considered to be the maximum lesion-based score which would correspond to the most suspicious cluster. In the second approach, the patient-based score is determined by first computing the mean value of the scores of corresponding lesions. The maximum mean cluster score is then considered to be the patient-level malignancy score. The second approach in which information is combined from both views resulted in 10% improvement in mean AUC over the former method and 20% over that of experienced radiologists.
Gupta et al. (2006) proposed combining MLO and CC information for classification of masses. In this work, the use of BI-RADS descriptors and patient age from both MLO and CC views has been adopted to design a two-view CADx system to classify masses as benign or malignant. The use of different decision fusion strategies including maximum, minimum and sum rule are explored to produce two-view score. The sum rule has been reported to outperform all the other methods in terms of the mean AUC. The improvement has been observed to be 3.9% when compared to the single view system.
CADx SYSTEMS BASED ON MULTIPLE MODALITIES INCLUDING MAMMOGRAMS AND ULTRASOUND IMAGES
Few researchers (Horsch et al., 2006; Jesneck et al., 2006, 2007) have analyzed automated analysis ultrasound and mammogram images in combination for classification of breast lesions. The performance of observers in classifying lesions with and without the aid of a CADx systems been analyzed by Horsch et al. (2006). In this study, two independent CADx systems based on mammogram and ultrasound images are developed. Both the systems employed an automatic segmentation of lesion starting with a manually indicated lesion center, followed by automatic feature extraction. The features extracted from the mammographic lesion included gray level in lesion neighborhood, texture, shape, margin sharpness and spiculation. The ultrasound features included posterior acoustic behavior, texture, shape and margin. Following feature extraction, each CADx system employed a Bayesian neural network to determine the probability of malignancy of the lesion. The performance improvement achieved by the observers due to the multimodal CADx system’s aid of has been reported to be 6, 44 and 6% in terms of AUC, partial AUC and sensitivity. However, this study has not employed computerized fusion techniques to combine the information from the two modalities.
Jesneck et al. (2006) have proposed a CADx system that is used to combine ultrasound and mammogram features of lesions so as to classify them as benign or malignant. Mammographic features included size, shape, density, calcification and architectural distortion associated features and comparison with prior findings. Ultrasonic features including texture, shape, orientation, margin, posterior acoustics, echogenicity, calcification and cystic component related features, vascularity and patient history features. The mammographic and ultrasonic features are subjected to serial feature fusion and used for lesion classification. Both LDA and ANN are employed for classification. It has been observed that there was no significant performance difference between the two classifiers. The performance of LDA has been observed to match that of radiologists.
Jesneck et al. (2006) employed decision fusion using the product rule to combine the two modalities. The features extracted from either modality are the same as in the above-mentioned work (Jesneck et al., 2006). Following this, classification is performed using likelihood test followed by two variants of decision fusion. One of the variants (DF-A) optimized the AUC and the other (DF-B) optimized the normalized partial area under curve (pAUC) optimized. These methods are compared with the LDA and ANN classifier based systems proposed by Jesneck et al. (2006) that employ feature fusion. It has been reported that DF-A significantly outperformed the other two systems in terms of AUC (p<0.02). Also, when compared to LDA, the DF-P system significantly improved the specificity at both 98 and 100% sensitivity (p<0.04).
CONCLUSION AND FUTURE SCOPE
Most of the studies combining the two mammographic views as well those combining ultrasound and mammogram modalities to improve the performance of CAD systems employ different feature level fusion and/or decision level fusion strategies. Feature fusion is effective when the features across the modalities to be combined are correlated, whereas decision fusion can be useful if the information provided by the different modalities is uncorrelated. However, since multiple sources of information provide a mixture of correlated and uncorrelated features in general, the performance of these schemes are highly dependent on the nature of the datasets involved and the best performing scheme could not be predicted before the actual testing process. This necessitates a systematic approach in which the features from the different sources are transformed in such a way that the subsequent fusion is optimized. From this point of view, one possible direction is the application of Canonical Correlation Analysis (CCA) on the features from different sources so as to maximize the correlation between them. This in effect would serve to optimize the performance of feature fusion which is highly suitable for correlated features.