INTRODUCTION

Fuzzy Neural Networks (FNNs) have been widely used in economic and industry (Amjady, 2006; Pindoriya et al., 2008; Lin and Chou, 2009) that require modeling uncertain and imprecise system in recent years, due to the merge of Fuzzy Interference System (FIS) and Neural Network (NN) (Lin et al., 2005). Because of its logic inference and adaptive learning ability, FNNs have attracted a lot of attention. For example, A novel FNNs approach is proposed and it can adjust the structures by growing and pruning fuzzy rules during the learning process (Han and Qiao, 2010). In order to improve the computational efficiency, the record of each neuron’s firing strength for all data previously clustered was used (Coyle et al., 2009). Evolution computation was also applied to the parameters and structures of FNNs, such as the order of polynomial, the number of membership function (Roh et al., 2007).

While facing with practical problems, system outputs usually depend on spatial aggregation and temporal cumulative of input signals. However, FNNs of all the above models can only deal with spatial related inputs. To solve the issues in a time-varying system, common method has to be replaced by spatial relation (time series) methods. In the year 2000, a Process Neural Network (PNN) model (He and Liang, 2000) was proposed which can deal with spatial-temporal information synchronously. Later, a PNN and traditional neural network combined model (He and Xu, 2003) was presented with time-varied input and output function. Because of the good localization property of the wavelet transform in time domain and frequency domain, a wavelet process neural network (Gang et al., 2008) was introduced and applied to the time series prediction.

In order to improve the ability of dealing with time-varying system, this paper proposed a kind of fuzzy process neural network based on orthogonal basis function. The inputs of the new network could be the fuzzy process information, the time-varying numeric, or the combination of them. Meanwhile, the network can receive the time/space signals or sequences directly. This means that the input signal scopes are enlarged. Because the computation of time cumulative is complicated, a set of the orthogonal basis functions in the input space was used. By the nature of the orthogonal basis functions, it’s easy to simplify the computation and the aggregation operation process.

PROCESS NEURON

Process neuron is similar to the traditional neuron, it consists of three parts: weight function, converge and activation threshold. The differences between them are that the weight function, activation threshold of the PN are time-varying and the converging operation includes not only the multi-inputs space aggregation but also time cumulative. The topology of single process neuron is shown in Fig. 1.

where, X (t) = (x₂ (t), x₂ (t),…, x_n (t)) is the vector of inputs, y is the output, W (t) = (w₁ (t), w₂ (t),…, w_n (t)) is the vector of connection weighting function and f (.) is the activation function, [0 T] is the sample range and the symbols Σ and ∫ denote spatial and time aggregation operation, respectively.

Fig. 1:	Topology of single process neuron

According to the topology, the relation between inputs X (t) and output y can be written as:

(1)

In a function space, there are many sets of orthogonal functions (infinite dimensional), such as the trigonometric function space system in L² (0, T) function space, the Legendre polynomials in L² (-1, 1) function space, Walsh function systems in L² (0, 1) function space and so on.

If the input space of process neuron is [0 T], {b_l (t)}^L_{l = 1} is a set of orthogonal basis function which is also included in the input function space of the model, the input vector X (t) can be feature expanded into the following form:

(2)

where, a_il is a corresponding coefficient in the expanded equation. Let:

which is also in the input function space. According to the nature of the orthogonal functions, the flowing equations can be established:

(3)

(4)

STRUCTURE OF FUZZY PROCESS NEURAL NETWORK

Here, introduces the structure of the FPNN. The proposed network structure is shown in Fig. 2 which has a total of five layers. The following sections present mathematical functions of each layer in detail.

Layer 1 (input layer): No computation is performed in this layer. Each node in this layer corresponds to one input variable. The input vector is X_k (t) = (x_k1 (t), x_k2 (t),…, x_kn (t)), where k = 1, 2,…, k is the number of the samples and n is the number of input variables. And one of the input x_ki (t) can be feature expanded as followed:

(5)

where, a^l_ki is a corresponding coefficient in the expanded equation. The inputs of this layer can be numerical time-verified function or the fuzzy variable with process information.

Layer 2 (fuzzification layer): Layer 2 acts as the fuzzification layer of the PFNN, where the values of the activated fuzzy Membership Functions (MFs) for a given current values are calculated. Each node in this layer represents a membership function. For input variable x_ki, the following Gaussian membership function is used:

(6)

where, μ_ij (x_kj) is the output of this layer, c_ij(t) and σ_j denote the center and width of the Gaussian membership functions, respectively and m is the number of rules. Only one width σ_j is assigned to each fuzzy rule j, so the network size could be reduced.

With the same orthogonal basis functions of input vector, the center c_ij can be feature expanded to the following form:

Fig. 2:	The structure of fuzzy process neural network

(7)

where, q^l_ij is a corresponding coefficient in the expanded equation. The Eq. 5 can be rewritten as:

(8)

Layer 3 (rule layer): Each node in this layer performs a t-norm operation on inputs from Layer 2 using an algebraic product operation to obtain a spatial firing strength. The number of nodes in this layer equals the number of fuzzy rules. So there are m nodes. The output of the jth node is φ_j which can be written as:

(9)

Layer 4 (normalized layer): This layer consists of normalized nodes. The number of nodes is equal to that of Rule layer:

(10)

where, Ψ_j is the output of the jth node in this layer.

Layer 5 (output layer): This layer performs defuzzification (weighted average) of the output as follows:

(11)

where, y_k (t) is the output of the network, ω_j denotes the connecting weight between the output layer and the normalized layer. The weight can be expressed as follows:

(12)

Suppose (α_j0 (t), α_j1 (t),…, α_jn (t)) are also in the input space, each of them can be feature expanded into:

(13)

where, w^l_ji is a corresponding coefficient in the expanded equation. The corresponding coefficient α^l_kj in Eq. 5 is replaced by α^*l_ki. The weight can be rewritten as:

(14)

Equation 11 can be rewritten as:

(15)

LEARNING FOR FUZZY PROCESS NEURAL NETWORK

The back propagation algorithm is used as learning algorithm. Assume that we have K learning functions samples and each learning function has n inputs:

where, the input function or sequence is x_ij (t), y_i (t) is the desired output. Suppose is the corresponding actual output function of the FPNN, then the mean square error of the FPNN output can be written as:

(16)

According to the gradient descent method, the modifications of the network connection weights and corresponding coefficient are:

(17)

(18)

(19)

where, β, γ, λ are the learning rates constants and the modifications Δw^l_ij, Δq^l_ij, Δσ can be calculated as follows:

(20)

(21)

(22)

In general, the major steps of the FPNN learning algorithm are as follows:

SIMULATION

To validate the proposed fuzzy process neural network algorithm, this section gives some numerical results.

In Fig. 3, continuous process neural networks are used to validate the effectiveness of the FPNN. The input internal is [0 1]. The sets of simulated input signals are generated by the following function: {sin (2kπt); cos (2kπt)}. Where k = 1, 2…, 10 is the number of the input set. The output is k for the corresponding kth input set.

Fig. 3:	Learning error curse

Using the FPNN shown in Fig. 2 and taking n = 2, m = 4, the number of each FPNN layer is 2-8-2-2-1, respectively. The learning rate constants are β = 0.005, γ = 0.07; the maximal learning time is 200. The basis function is the Walsh orthogonal function and the number of basis function is 32. The mean square error of the FPNN output is 5.7732e-007 after the 200th iteration. The learning error curse is shown in Fig. 3.

The result shows that there is powerful approximation ability of the fuzzy process neural network. As shown in the Fig. 3, there is a little glitch at the beginning of the curve. This means that the fuzzy process neural network may be sensitive to some parameters and this will be verified in Fig. 4.

In Fig. 4, discrete process neural networks are used to validate the number of fuzzy rule’s effect on the FPNN. Construct 9 input sample function belonging to 3 sequences with process input interval [0 1]. The first sequence has 4 sample functions: sin (2π (t-0.5)), sin (2.1π (t-0.5)), sin (2.2π (t-0.5)), sin (2.4π (t-0.5)). Suppose that the corresponding expected output is 0.3333. The second sequence has 4 sample functions: 1.2 sin (3π (t-0.667)), 1.2 sin (3.2π (t-0.667)), 1.2 sin (3.4π (t-0.667)) and 1.2 sin (3.6π (t-0.667)). Suppose that the corresponding expected output is 0.6667. The third sequence has 4 sample functions: 1.4 sin (4π (t-0.25)), 0.14 sin (4.3π (t-0.25)), 1.4 sin (4.6π (t-0.25)) and 1.4 sin (4.8π (t-0.25)). Suppose that the corresponding expected output is 1.0000. The sample function are dispersed as {sin (2kπ (t_i-0.5))}, where t_i = i/128 for i = 0, 1,ÿ, 127. Use the FPNN shown in Fig. 2. The number of the input and output are 3 and 1, respectively. The learning rate constants are β = 0.006, γ = 0.055, λ = 0.08; the maximal learning time is 200. A Walsh transform is implemented for discrete data and the transformed data are submitted to the network for training.

Fig. 4:	Learning error curse with different number of the fuzzy rules

Table 1:	The mean square error of the FPNN output

FPNN with the different number of the fuzzy rules are trained. After the 200th iteration, the mean square error of the FPNN output is shown in Table 1. The learning error curses are shown in Fig. 4.

The results show that the number of fuzzy rules is very important to the generalization ability of the FPNN model. If the number of the rules is large (k = 7), the learning error may decrease at fist but at some point it would increase. If it is small (k = 2), the amplitude of the oscillate part on the learning curse may be large. This indicates that the FPNN is sensitive to the number of the fuzzy rules. So we can choose suitable number of the fuzzy rules to optimize the performance, for example, here we choose k = 4.

CONCLUSION

In this study, fuzzy process neural network is proposed which combine the advantage of fuzzy neural network and process neuron, so the fuzzy neural network has the ability of achieving space aggregation and time cumulative. The network structure is introduced in detail, the orthogonal functions are used as time integral operators and the learning algorithm is given. Simulation results show that the fuzzy process neural network has strong approximation ability and the network is sensitive to the number of the fuzzy rules. Finally some issues which the FPNN may have are discussed, such as the unequal signal duration, rule extraction and the possible solutions are also considered.

ACKNOWLEDGMENT

Present study is supported by the National Natural Science Foundation of China under Grant No. 61074127.