Abstract: Textual pattern mining is one of the major research areas in the field of data mining. The data mining is a emergent technique which adopts many approaches and methods from other fields of study. The data mining is implemented in other areas to learn hidden knowledge. In this study, Artificial Neural Network (ANN) is used for learning textual pattern in the Metadata conceptual mining model. The proposed learning algorithm is called as, analysis of bilateral intelligence, which is used to identify and classify the synonymy of the sentences. The proposed method provides efficient learning by identifying the patterns which have synonymy. The results of the proposed work show that the convergent of the training algorithm is very fast than existing methodology. From the results, it is concluded that the performance of proposed ABI is optimized. Hence, the proposed Metadata conceptual mining model with ABI learning will provide optimality than existing clustering algorithm.
INTRODUCTION
Artificial Neural Network (ANN) is the concept of parallel distributed processing. ANN is a field of study which is framed based on the neural system of the human being. The term neural is mapped into neuron in the human body. The interconnection between neuron (Bouzenada et al., 2007) with input connections called ‘axon’ and output connections called ‘dendrites’ are called the neural network. The entire neural network in the human being is controlled by the Soma.
This neural network is imitated in the computer science in order to achieve artificial intelligence. The ANN is acted as (Kordik et al., 2010) parallel processing and distributed processing in nature.
ANN is a directed graph in which the neurons are nodes or vertices, the inputs and connections between neurons are edges or links or arcs of the directed graph. ANN, also a labelled graph, has the connections (Bouzenada et al., 2007; Hsieh et al., 2010) like dendrites associated with weights and bias. The Soma initiates some computation efforts, which cause defined results for each set of actions. The weight of the interconnections are used to differentiate the input. If the inputs or interconnections are more important then higher value is assigned as weight value and lesser importance connections and inputs are assigned with lower value weights.
The ANN can be simply modelled with the working nature of zener diode. The working voltage/breakdown voltage of the zener diode is the threshold value of the ANN. If the sum of computational efforts (Khashei and Bijari, 2010) taken by the soma for the given inputs and interconnections is exceeding the threshold the ANN performs the action defined by the system otherwise the ANN is in stable state.
ANN is classified in the following types:
| • | Feed-forward (FFN) networks |
| • | Back-propagation neural (BPN) networks |
| • | Counter propagation neural (CPN) networks |
| • | Hop-field neural (HFN) networks |
| • | Self-organization neural (SON) networks |
In the above, the BPN is mostly used ANN model, especially for the field of data mining (Hui et al., 2010). The fifth generation computers, knowingly Artificial Intelligence (AI), is a well known example for ANN. Therefore, ANN is a growing rapidly to meet the requirement of modern computing industry. The problems of convergence, stability, local minimum and parameters adjustment are various critical issues in the design of ANN.
In the above issues, the problem of convergence is most critical research issue which severely affects the system performance (Ghwanmeh et al., 2006; Hsieh et al., 2010). The convergence is a problem, which leads to slow processing of training. The delay in training may lead to local minimum value, which may be sometimes deviated more from the optimum value. Therefore, solving convergence is important research task for the past few decades. To improve the convergence, some researchers proposed hybrid model which combines the intelligence of ANN with genetic gene algorithms and fuzzy logic. This hybrid model proved better result.
Dede and Sazli (2010) reviewed various ANN methods and concluded that three recent implementations, namely multilayer back propagation (MBPN), Elman Neural Networks (ENN) and Probabilistic Neural Networks (PNN) are proved better performance in terms of convergence. The authors developed a model which is applied for speech recognition. The speech recognition problem is a branch of pattern recognition field of study.
The Metadata Conceptual Mining Model (MCMM) is an effective text clustering method which leads to more number of classifications per unit time (Koteeswaran et al., 2012). In this study, ANN is used as learning method which identifies and classifies the synonymy. The proposed learning method is called Analysis of Bilateral Intelligence (ABI). The proposed ABI is explained briefly in this communication.
MATERIALS AND METHODS
The ANN model developed by Jolai and Ghanbari (2010) is applied for well-known problem, Travelling Salesman Problem (TSP). This method is applied the HFN. The method used Data Transformation Techniques (DTT) for improving the accuracy of the system. The DTT helps to achieve better results. Whereas, to improve the performance, Z-score and logarithmic approaches may be integrated with HNN.
These powerful unified methods have recently culminated (Rojanavasu et al., 2009) with the HNN method. It is innovative across various scientific and engineering fields. For example: Huang and Liu (1997) employed HNN and genetic algorithms together for the purpose of pattern recognition. Shen and Wang (2008) is used HNN and fuzzy systems for broadcast scheduling in wireless sensor networks.
Hence, it is derived that the knowledge representation, network architecture, convergence are the most important design issues for developing ANN. The knowledge representation is a process of acquiring knowledge in the form of inputs given to the ANN and weights assigned to the connected links. The links may be input link, hidden link and output link. The values of weights associated with these links are acting major role in the working principle of the ANN. By the perfect training method, these weight values are optimally chosen.
There are many reviews are recommended for further studies (Dam et al., 2008; Scarselli et al., 2009; Rojanavasu et al., 2009).
Predicting business failure (Hui et al., 2010) and speech recognition (Dede and Sazli, 2010) are some of the prediction model based implementation in various engineering domain.
A recent study on isolated Malay digit recognition reports dynamic time warping and hidden Markov modelling techniques to have recognition rates of 80.5 and 90.7%, respectively. Meanwhile, recognition rates obtained by neural networks for similar applications—as in this study—are often above. Due to this aspect, ANN appears to be a convenient classifier for the speech recognition problem (Dede and Sazli, 2010).
The design of the architecture for ANN is most important for the successful implementation. The Feed Forward Neural Network (FFNN) is a highly desirable network model for the researcher due to its simple design, less hardware cost and relatively high performance. The improved FFNN called Feed Forward Functional Link Neural Link (FFFLNN) (Garcia and Kirschen, 2006) is developed which defines each neural network as a functional element.
The architecture of the proposed ANN based unsupervised learning, training and testing methodologies, the sample data set and ratio of training and testing dataset are the important factors for achieving optimal result in a neural network based learning model.
PROPOSED ANALYSIS OF BILATERAL INTELLIGENCE LEARNING METHOD
The proposed ANN based unsupervised learning, is termed as, Analysis of Bilateral Intelligence (ABI). The ABI applies learning process to identify two equivalent terms which has same meaning. ABI contains text documents as dataset, improving accuracy of text clustering which is the required output and achieving error free clustering in shorter time is the goal.
The working model of proposed ABI Learning method: The following sigmoidal function is applied in the proposed ABI:
| (1) |
where, XA is the output in the hidden and output layer. Where the inputs are ‘x’ which is connected to the hidden layer from input layer. The connection has weights ‘rai’, between input to hidden layer. And the output of the neurons referred as ‘sba’ is computational values between output and hidden layer. Where, ‘b’ neurons in the output layer, ‘a’ neurons in the hidden layer and ‘i’ neurons in the input layer.
Algorithm of proposed ABI learning method
Initial phase: The proposed ABI has implemented from well known initial
phase. In the initial phase, values for the weights are assigned. Let the values
are ‘R’ and ‘S’. ‘R’ is a value of the hidden
layer and input layer. ‘S’ is a value of output layer-hidden layer,
respectively.
The other constants are penalty constant, which is defined as μ; and the number of iterations, which is called epoch, are initialized in the system. The weight vectors ‘R’ and ‘S’ are to be optimized in order to minimize the error function.
The generalised delta rule is imposed in the proposed ABI, which involves two stages of operation. In the first stage of operation, the input ‘x’ is presented and propagated in forward direction through the network is to compute the output values ‘y’ for each output unit. This output is compared with its desired value ‘do’, resulting in an error signal (the difference between the actual value and the desired value), for each output unit.
The second stage involves a backward transmission, which passed through the network after the error was computed. The error signal is passed to each unit in the network and the appropriate weight changes are calculated.
Weight adjustments phase: This weight adjustments step is processed based on sigmoid activation function, shown in the first phase.
The weight of a connection is adjusted by an amount proportional to the product of an error signal calculated in the second stage of the first phase.
On the neuron, the unit ‘k’ receiving the input and the output of the unit ‘j’ sending this signal along the connection.
Optimization of output layer weights:
| (2) |
where:
| (3) |
| (4) |
where, ‘ZP’ = scalar output of the hidden neuron of training data ‘p’, ‘A’ and ‘B’ are output of the hidden layer and output layer, respectively, ‘a’ and ‘b’ are neurons in the hidden layer and output layer, ‘i’ is neuron in the input layer and ‘t’ is transaction function.
The concept of state is fundamental to this description. The state vector or simply state, denoted by ‘xk’, is defined as the minimal set of data that is sufficient to uniquely describe the unforced dynamical behaviour of the system; the subscript ‘k’ denotes discrete time. In other words, the state is the least amount of data on the past behaviour of the system that is needed to predict its future behaviour. Typically, the state ‘xk’ is unknown. To estimate it, use a set of observed data, denoted by the vector ‘yk’.
Test for completion: RMS error (ERMS) was then calculated comparing the ‘Rtest’ matrix with ‘Soptimum’ matrices calculated in Step 3:
a. ERMS<E
The hidden layer weight matrix ‘R’ is updated ‘R’= ‘Rtest’. Decrease the influence of the penalty term by decreasing ‘μ’, Proceed to Step 5:
b. ERMS≥E
Increase the influence of ‘μ’ and repeat Step ‘4’.
Process termination: If the RMS error is not within the desired range, repeat Step 3, else the training process is ceased. After the successful completion of the training phase, the sample real time data is given as input of the system. The system will choose comparatively best path. This thesis used 60% dataset for training and 40% dataset for testing.
RESULTS AND DISCUSSION
This ANN based learning model is implemented using Neural Network Tool Box in MATLAB (MatLab). In the training phase, the goal is assigned as “0.01” and the epoch is assigned as 250.
Table 1 shows the %RMS error in Estimation and Elimination of FFFLNN, dynamic model available in MATLAB and the proposed learning model. The estimation error identifies number of documents and or terms identified in the clustering model. The elimination error defines the mismatch ratio for document clustering.
The Fig. 1 shows that the proposed learning model reaches the performance 0.021 in 250 epoch (number of iteration), whereas the existing FFFLNN reaches only 0.038 which is lesser than the proposed system.
| Fig. 1: | Error in proposed vs. existing FFFLNN based on number of epoch |
| Table 1: | RMS errors of various ANN models |
From the results shown in the Table 1 and performance shown in the Fig. 1, it is concluded that the performance of proposed ABI learning model always performs better than existing methodology.
The Fig. 1 shows the proposed ABI learns the synonymy better than the existing systems. From this it is concluded that the proposed ABI performs better than existing systems. The ABI shows 3-4% improvements in the estimation and 1-5% improvements in the elimination.
The convergence of the proposed ABI and existing learning models are compared in Fig. 1. This shows that the proposed ABI provides optimal result within few iteration of training.
CONCLUSION
The proposed ABI learning method is to improve estimation, elimination and accuracy of the system. The estimation is improved 3-4% and elimination is improved 1-5% and the accuracy is improved which is shown in the error rate and learning rate based on epoch. Therefore, the proposed MCMM with ABI learning is optimal than existing model.