**Cheng Ma**

Control and Simulation Center, Harbin Institute of Technology, Harbin, China

Ye San

Control and Simulation Center, Harbin Institute of Technology, Harbin, China

In order to widen the range of processing signals for fuzzy neural network, a kind of fuzzy process neural network based on orthogonal basis function is proposed. By inducting the orthogonal basis function into input space, the input function can be feature expanded. Meanwhile the weight vectors are also expanded under the same orthogonal basis function. The operation process of space aggregation and time cumulative can be simplified by using the orthogonality of basis function. The back propagation algorithm is used as learning algorithm. Simulation results show the great approximation ability of the fuzzy process neural network. And further experiments show that the network is sensitive to the number of fuzzy rules. The best approximation accuracy can be obtained only by choosing the proper values, such as k = 4 in this study.

PDF Abstract XML References Citation

Cheng Ma and Ye San, 2011. Fuzzy Process Neural Network based on Orthogonal Basis Function. *Information Technology Journal, 10: 1999-2003.*

**DOI:** 10.3923/itj.2011.1999.2003

**URL:** https://scialert.net/abstract/?doi=itj.2011.1999.2003

Fuzzy Neural Networks (FNNs) have been widely used in economic and industry (Amjady, 2006; Pindoriya *et al*., 2008; Lin and Chou, 2009) that require modeling uncertain and imprecise system in recent years, due to the merge of Fuzzy Interference System (FIS) and Neural Network (NN) (Lin *et al*., 2005). Because of its logic inference and adaptive learning ability, FNNs have attracted a lot of attention. For example, A novel FNNs approach is proposed and it can adjust the structures by growing and pruning fuzzy rules during the learning process (Han and Qiao, 2010). In order to improve the computational efficiency, the record of each neuron’s firing strength for all data previously clustered was used (Coyle *et al*., 2009). Evolution computation was also applied to the parameters and structures of FNNs, such as the order of polynomial, the number of membership function (Roh *et al*., 2007).

While facing with practical problems, system outputs usually depend on spatial aggregation and temporal cumulative of input signals. However, FNNs of all the above models can only deal with spatial related inputs. To solve the issues in a time-varying system, common method has to be replaced by spatial relation (time series) methods. In the year 2000, a Process Neural Network (PNN) model (He and Liang, 2000) was proposed which can deal with spatial-temporal information synchronously. Later, a PNN and traditional neural network combined model (He and Xu, 2003) was presented with time-varied input and output function. Because of the good localization property of the wavelet transform in time domain and frequency domain, a wavelet process **neural network** (Gang *et al*., 2008) was introduced and applied to the time series prediction.

In order to improve the ability of dealing with time-varying system, this paper proposed a kind of fuzzy process **neural network** based on orthogonal basis function. The inputs of the new network could be the fuzzy process information, the time-varying numeric, or the combination of them. Meanwhile, the network can receive the time/space signals or sequences directly. This means that the input signal scopes are enlarged. Because the computation of time cumulative is complicated, a set of the orthogonal basis functions in the input space was used. By the nature of the orthogonal basis functions, it’s easy to simplify the computation and the aggregation operation process.

**PROCESS NEURON**

Process neuron is similar to the traditional neuron, it consists of three parts: weight function, converge and activation threshold. The differences between them are that the weight function, activation threshold of the PN are time-varying and the converging operation includes not only the multi-inputs space aggregation but also time cumulative. The topology of single process neuron is shown in Fig. 1.

where, X (t) = (x_{2} (t), x_{2} (t),…, x_{n} (t)) is the vector of inputs, y is the output, W (t) = (w_{1} (t), w_{2} (t),…, w_{n} (t)) is the vector of connection weighting function and f (.) is the activation function, [0 T] is the sample range and the symbols Σ and ∫ denote spatial and time aggregation operation, respectively.

Fig. 1: | Topology of single process neuron |

According to the topology, the relation between inputs X (t) and output y can be written as:

(1) |

In a function space, there are many sets of orthogonal functions (infinite dimensional), such as the trigonometric function space system in L^{2} (0, T) function space, the Legendre polynomials in L^{2} (-1, 1) function space, Walsh function systems in L^{2} (0, 1) function space and so on.

If the input space of process neuron is [0 T], {b_{l} (t)}^{L}_{l = 1} is a set of orthogonal basis function which is also included in the input function space of the model, the input vector X (t) can be feature expanded into the following form:

(2) |

where, a_{il} is a corresponding coefficient in the expanded equation. Let:

which is also in the input function space. According to the nature of the orthogonal functions, the flowing equations can be established:

(3) |

(4) |

**STRUCTURE OF FUZZY PROCESS NEURAL NETWORK**

Here, introduces the structure of the FPNN. The proposed network structure is shown in Fig. 2 which has a total of five layers. The following sections present mathematical functions of each layer in detail.

**Layer 1 (input layer): **No computation is performed in this layer. Each node in this layer corresponds to one input variable. The input vector is X_{k} (t) = (x_{k1} (t), x_{k2} (t),…, x_{kn} (t)), where k = 1, 2,…, k is the number of the samples and n is the number of input variables. And one of the input x_{ki} (t) can be feature expanded as followed:

(5) |

where, a^{l}_{ki} is a corresponding coefficient in the expanded equation. The inputs of this layer can be numerical time-verified function or the fuzzy variable with process information.

**Layer 2 (fuzzification layer): **Layer 2 acts as the fuzzification layer of the PFNN, where the values of the activated fuzzy Membership Functions (MFs) for a given current values are calculated. Each node in this layer represents a membership function. For input variable x_{ki}, the following Gaussian **membership function** is used:

(6) |

where, μ_{ij} (x_{kj}) is the output of this layer, c_{ij}(t) and σ_{j} denote the center and width of the Gaussian membership functions, respectively and m is the number of rules. Only one width σ_{j} is assigned to each fuzzy rule j, so the network size could be reduced.

With the same orthogonal basis functions of input vector, the center c_{ij} can be feature expanded to the following form:

Fig. 2: | The structure of fuzzy process neural network |

(7) |

where, q^{l}_{ij} is a corresponding coefficient in the expanded equation. The Eq. 5 can be rewritten as:

(8) |

**Layer 3 (rule layer): **Each node in this layer performs a t-norm operation on inputs from Layer 2 using an algebraic product operation to obtain a spatial firing strength. The number of nodes in this layer equals the number of fuzzy rules. So there are m nodes. The output of the jth node is φ_{j} which can be written as:

(9) |

**Layer 4 (normalized layer): **This layer consists of normalized nodes. The number of nodes is equal to that of Rule layer:

(10) |

where, Ψ_{j} is the output of the jth node in this layer.

**Layer 5 (output layer): **This layer performs defuzzification (weighted average) of the output as follows:

(11) |

where, y_{k} (t) is the output of the network, ω_{j} denotes the connecting weight between the output layer and the normalized layer. The weight can be expressed as follows:

(12) |

Suppose (α_{j0} (t), α_{j1} (t),…, α_{jn} (t)) are also in the input space, each of them can be feature expanded into:

(13) |

where, w^{l}_{ji} is a corresponding coefficient in the expanded equation. The corresponding coefficient α^{l}_{kj }in Eq. 5 is replaced by α^{*l}_{ki}. The weight can be rewritten as:

(14) |

Equation 11 can be rewritten as:

(15) |

**LEARNING FOR FUZZY PROCESS NEURAL NETWORK**

The back propagation algorithm is used as learning algorithm. Assume that we have K learning functions samples and each learning function has n inputs:

where, the input function or sequence is x_{ij} (t), y_{i} (t) is the desired output. Suppose is the corresponding actual output function of the FPNN, then the mean square error of the FPNN output can be written as:

(16) |

According to the gradient descent method, the modifications of the network connection weights and corresponding coefficient are:

(17) |

(18) |

(19) |

where, β, γ, λ are the learning rates constants and the modifications Δw^{l}_{ij}, Δq^{l}_{ij}, Δσ can be calculated as follows:

(20) |

(21) |

(22) |

In general, the major steps of the FPNN learning algorithm are as follows:

Step 1: | Select the standard orthogonal basis functions in the input space. If the inputs vectors are continue variables, the number of basis function should make the expansion of the basis function satisfy the required precision. Otherwise, divide the input interval [0 T] equally and determine the partition points t_{1}, t_{2},…, t_{l} |

Step 2: | Initialize the connection weights, the center and the width of the Gaussian functions, the maximal learning times is MaxGen, the learning time is gen which is equal to 1 at the beginning |

Step 3: | Calculate the error function according to Eq. 16. If gen>MaxGen, go to step 5 |

Step 4: | Modify the connection weight, the center and the width of the Gaussian functions according to Eq. 20-22. Then gen = gen+1, go back to step 3 |

Step 5: | Output the learning result and stop the progress |

**SIMULATION**

To validate the proposed fuzzy process **neural network** algorithm, this section gives some numerical results.

In Fig. 3, continuous process **neural networks** are used to validate the effectiveness of the FPNN. The input internal is [0 1]. The sets of simulated input signals are generated by the following function: {sin (2kπt); cos (2kπt)}. Where k = 1, 2…, 10 is the number of the input set. The output is k for the corresponding kth input set.

Fig. 3: | Learning error curse |

Using the FPNN shown in Fig. 2 and taking n = 2, m = 4, the number of each FPNN layer is 2-8-2-2-1, respectively. The learning rate constants are β = 0.005, γ = 0.07; the maximal learning time is 200. The basis function is the Walsh orthogonal function and the number of basis function is 32. The mean square error of the FPNN output is 5.7732e-007 after the 200th iteration. The learning error curse is shown in Fig. 3.

The result shows that there is powerful approximation ability of the fuzzy process neural network. As shown in the Fig. 3, there is a little glitch at the beginning of the curve. This means that the fuzzy process **neural network** may be sensitive to some parameters and this will be verified in Fig. 4.

In Fig. 4, discrete process **neural networks** are used to validate the number of fuzzy rule’s effect on the FPNN. Construct 9 input sample function belonging to 3 sequences with process input interval [0 1]. The first sequence has 4 sample functions: sin (2π (t-0.5)), sin (2.1π (t-0.5)), sin (2.2π (t-0.5)), sin (2.4π (t-0.5)). Suppose that the corresponding expected output is 0.3333. The second sequence has 4 sample functions: 1.2 sin (3π (t-0.667)), 1.2 sin (3.2π (t-0.667)), 1.2 sin (3.4π (t-0.667)) and 1.2 sin (3.6π (t-0.667)). Suppose that the corresponding expected output is 0.6667. The third sequence has 4 sample functions: 1.4 sin (4π (t-0.25)), 0.14 sin (4.3π (t-0.25)), 1.4 sin (4.6π (t-0.25)) and 1.4 sin (4.8π (t-0.25)). Suppose that the corresponding expected output is 1.0000. The sample function are dispersed as {sin (2kπ (t_{i}-0.5))}, where t_{i} = i/128 for i = 0, 1,ÿ, 127. Use the FPNN shown in Fig. 2. The number of the input and output are 3 and 1, respectively. The learning rate constants are β = 0.006, γ = 0.055, λ = 0.08; the maximal learning time is 200. A Walsh transform is implemented for discrete data and the transformed data are submitted to the network for training.

Fig. 4: | Learning error curse with different number of the fuzzy rules |

Table 1: | The mean square error of the FPNN output |

FPNN with the different number of the fuzzy rules are trained. After the 200th iteration, the mean square error of the FPNN output is shown in Table 1. The learning error curses are shown in Fig. 4.

The results show that the number of fuzzy rules is very important to the generalization ability of the FPNN model. If the number of the rules is large (k = 7), the learning error may decrease at fist but at some point it would increase. If it is small (k = 2), the amplitude of the oscillate part on the learning curse may be large. This indicates that the FPNN is sensitive to the number of the fuzzy rules. So we can choose suitable number of the fuzzy rules to optimize the performance, for example, here we choose k = 4.

In this study, fuzzy process **neural network** is proposed which combine the advantage of fuzzy **neural network** and process neuron, so the fuzzy **neural network** has the ability of achieving space aggregation and time cumulative. The network structure is introduced in detail, the orthogonal functions are used as time integral operators and the learning algorithm is given. Simulation results show that the fuzzy process **neural network** has strong approximation ability and the network is sensitive to the number of the fuzzy rules. Finally some issues which the FPNN may have are discussed, such as the unequal signal duration, rule extraction and the possible solutions are also considered.

Present study is supported by the National Natural Science Foundation of China under Grant No. 61074127.

- Amjady, N., 2006. Day-ahead price forecasting of electricity markets by a new fuzzy neural network. IEEE Trans. Power Syst., 21: 887-896.

CrossRefDirect Link - Pindoriya, N.M., S.N. Singh and S.K. Singh, 2008. An adaptive wavelet neural network-based energy price forecasting in electricity markets. IEEE Trans. Power Syst., 23: 1423-1432.

CrossRefDirect Link - Lin, F.J. and P.H. Chou, 2009. Adaptive control of two-axis motion control system using interval type-2 fuzzy neural network. IEEE Trans. Ind. Electron., 56: 178-193.

CrossRefDirect Link - Lin, C.T., W.C. Cheng and S.F. Liang, 2005. An on-line ica-mixture-model- based self-constructing fuzzy neural network. IEEE Trans. Circuits Syst. I: Regul. Pap., 52: 207-221.

CrossRefDirect Link - Han, H. and J. Qiao, 2010. A self-organizing fuzzy neural network based on a growing-and-pruning algorithm. IEEE Trans. Fuzzy Syst., 18: 1129-1143.

CrossRefDirect Link - Coyle, D., G. Prasad and T.M. McGinnity, 2009. Faster self-organizing fuzzy neural network training and a hyperparameter analysis for a Brain-computer interface. IEEE Trans. Syst. Man Cybern. Part B: Cybern., 39: 1458-1471.

CrossRefDirect Link - Roh, S.B., W. Pedrycz and S.K. Oh, 2007. Genetic optimization of fuzzy polynomial neural networks. IEEE Trans. Ind. Electron., 54: 2219-2238.

CrossRefDirect Link - He, X.G. and J.Z. Liang, 2000. Some theoretical issues on procedure neural networks. Eng. Sci., 2: 40-44.

Direct Link - Gang, D., Z. Shi-Sheng and L. Yang, 2008. Time series prediction using wavelet process neural network. Chin. Phys. B, 17: 1998-2003.

CrossRefDirect Link