Abstract: There are many forecasting models, but not all of them are able to monthly inflow forecasting. In this study, the abilities of static and dynamic artificial neural network model for Dez reservoir inflow forecasting compared. The 47-years monthly discharges used, so that first 42 years and last 5 years used for model training and models forecasting phase, respectively. Different structures of the static and dynamic models of artificial neural network model investigated in terms of RMSE index. Firstly, by using data from October 1960 to September 2002 the best structures of static and dynamic neural networks are determined. Therefore, the Dez reservoir monthly inflow are forecasted and compared with observed data October 2002 to September 2007 based on optimized data. Furthermore, two transfer functions, radial and sigmoid and different neurons in hidden layer were evaluated. The results show that the best Dez reservoir inflow forecasting model is autoregressive artificial neural network model using sigmoid activity function with 17 neurons in hidden layer. Autoregressive artificial neural network model with sigmoid activity function were able to forecast next 5 years Dez inflow.
INTRODUCTION
Autoregressive artificial neural network model has been used in many fields such as economy, crisis management, mathematics, etc. but has not been used for monthly inflow forecasting. More accurate estimation of the monthly inflow to the reservoir is an important issue in water resources management since it plays important roles in management and exploitation of reservoirs, hydroelectric energy generation and designing controlling structures. One famous black box model that has successfully applied to forecasting river flow in recent decades is artificial neural network model. Artificial neural network model is among the intelligent dynamic model-free systems, which are based on experimental data and transmit knowledge or the rule beyond the data to the network structure by processing this data (Menhaj, 2012). Artificial neural network model in various fields of hydrology is widely applied, which some of them will be described in the following.
Teschl and Randeu (2006) predicted short term river flow using a neural network model successfully. Jia and Culver (2006) using bootstrapped artificial neural network model suggested that even a small set of periodic instantaneous observations of stage from a staff gauge, which can easily be collected by volunteers, can be a useful data set for effective hydrological modeling.
Sahoo et al. (2006) using neural network predicted flash flood and attendant water qualities of a mountainous stream on Oahu, Hawaii. The results showed that predictive performance of artificial neural network model for the estimation of stream flow is improved if weather data, rainfall and evapotranspiration are included in the input data set. Karunasinghe and Liong (2006) predicted chaotic time series using an artificial neural network model. The overall results showed the superiority of global artificial neural network model over the widely used local prediction models. Tawfik (2003) examined contrasts in linear and nonlinear forecasting in Nile River flows. Since Egypt is dependent upon the Nile River for supplying about 95% of its needed water, Aswan High Dam at the highest point on the river plays an important role. So that any decision to release lost water is failed and a considerable amount of water will be lost. Therefore, long-term and short-term forecasting of Nile flows to achieve better management of the reservoir is very important. Various autoregressive models are used to forecast inflow to the Aswan Dam. Nevertheless, most of these models failed to forecast, because inflow peak in the months July, August and September had changed drastically. According to Tawfik, this lack of precision is related to the linearity of autoregressive models. Therefore, a static neural network model for nonlinear structure can be used. The results showed that the nonlinear neural network models in some cases would lead to improved forecasting accuracy. Sahoo and Ray (2006) forecast the inflow for a Hawaii stream using rating curve and neural networks. The results indicated that artificial neural network model is superior to rating curve, especially when the behavior of river flow will change. Karamouz et al. (2004) also determined basic neural network models for forecasting floods. This was done as a part of Ahvaz flood warning system. Three different types of artificial neural network model including multilayer perceptron neural network as a static network, Elman recurrent neural networks and time delay neural network as the network dynamics were studied. Comparison of forecast results showed that the neural network dynamics is substantially better. Results indicated that artificial neural network model were effective tools in forecasting flood. Toth et al. (2000) used the artificial neural network model and ARMA (Auto Regressive Moving Average) models to forecast rainfall. The results showed the success of both short-term rainfall-forecasting models for forecast floods in real time. Kisi and Cigizoglu (2005) using dynamic artificial neural network model forecast the monthly inflow, storage and evaporation on Canak Dere basin. The results were satisfactory for the monthly storage and evaporation, but the prediction of the monthly inflow was less accurate than monthly storage and evaporation. They used both radial and sigmoid activity function in neural network dynamics. Their results showed that the sigmoid activity function is superior to the radial activity function. Banihabib et al. (2008) also forecast the inflow to the Dez reservoir at daily and monthly time scales using static artificial neural network model and simple linear regression model based on discharge data from hydrometric stations located upstream of the desired station on minor and major river branches. This research showed that static artificial neural network model is better than linear regression models. The results indicated that static artificial neural network model has a better performance than linear regression models in forecasting daily and monthly inflows into dam reservoir. Mohammadi et al. (2005) forecast Karaj reservoir inflow using data of melting snow and artificial neural network model and ARMA methods and regression analysis. Sixty percent of inflow in dam happens between April until June, so forecasting the inflow in this season is very important for dam’s performance. The highest inflows were in the spring due to the snow melt because of draining in threshold winter. The results showed that artificial neural network model has lower significant faults as compared with other methods.
Thus, according to the research which some mentioned above, it could be mentioned that artificial neural network model are useful in forecasting and modeling hydrological processes than other statistical models including linear and nonlinear regression. Dynamic artificial neural network model based on autoregressive monthly reservoir inflow forecast have not used in previous studies, therefore, this study aims to forecast reservoir inflow using the dose static and dynamic artificial neural network model and to compare these two models according Taleh Zang station discharge statistics located on the Dez reservoir. In this study, the monthly inflow to the reservoir by two models of dynamic and static autoregressive artificial neural network model has been forecast.
MATERIALS AND METHODS
In this study, the autoregressive static and dynamic neural network models were used to forecast monthly reservoir inflow for Taleh Zang station and all necessary steps have been programmed in MATLAB software. Artificial neural network model used in this study is composed of an input layer, a middle layer and output layer and it has been chosen due to its high capacity to estimate the complex relationship in the three-layer structure (Chegini, 2012). Static and dynamic neural networks are different in terms of the use of model output as input. In dynamic networks, the output enters into the network with a delay, but output in the static artificial neural network model is not used as input. In this research, neural networks with bias and hidden and output layer with sigmoid and radial activity function are used. Choosing initial weights and biases has a significant effect on network speed and finding a satisfactory answer. Nguyen and Widrow proposed a method by which the choice of initial weights and biases values is better than their random selection (Avarideh, 2012). To do this, they analyzed the approximation method of functions by a two-layer network. This technique provides a better network in terms of training rate (less iteration for training). So Nguyen-Widrow function was applied to initialization of weights due to the less iteration for training (Avarideh, 2012). If n number of input layer neurons, p the number of hidden layer neurons and β is a factor as follows:
| (1) |
For each hidden layer, the initial value (Wijold) is available randomly between -0.5 and 0.5. Then the amount of improvement is as follows due to provide a better network (Avarideh, 2012):
| (2) |
In order to classify data using artificial neural network model, monthly discharge statistics from October 1960 to September 2007 were arranged Q1 to Q564. Q1 specifies the discharge in October 1960 and the rest is the same manner. Considering the artificial neural network model need two educational (training) and test (forecasting) sets. Each set contains two sets of data: inputs and the target. Therefore, data division was as follows: in the training set for input data, Q1 to Q264 were used; in target data, Q445 to Q504 in the test set (forecasting) for input data, Q265 to Q444; and for the target data Q505 to Q564 were used. Since considered Q445 to Q564 as forecasting phase (last 5 years) in this drawing could be compared both training and testing data sets (Table 1). In addition, the number of input data in both training and test sets should be equal, therefore, for final implementation of models, each input matrix contained 180 data, which has been identified as three vectors with 60 rows in the following:
So in training set for calculating the first target data (Q455), Q85, Q145 and Q205 were used that all belongs to a special month and for remaining data and also testing set the same method was used. The cycle of training in artificial neural network model is as follows: first, initial weights and biases are allocated to the training set input data and by their entry into the hidden layer (the effect of number of neurons) and using training function and network education activity functions, a comparison is made between the input data and target data and a communication is established between them. Then forecast is carried out for the new data (test set of input data) and by modifying the weights and biases and the effect of output delay (dynamic network), mentioned cycle is repeated until an acceptable error (Avarideh, 2012).
| Table 1: | The final comparison of used models to forecasting Dez reservoir inflow in Taleh Zang station |
The number of neurons in the hidden layer to solve a problem in general is not known in artificial neural network model and should be determined by experimental methods. However, if their numbers are lower than a standard level (as it was also observed in this study), it is possible that learning does not occur in total, which this is called under fitting and it means that there are not weighting and biases though which network can produce logical outputs near to the correct answers. In contrast, over fitting possibly occurs and necessary neurons used in this case are excessive. Of course, by increasing the level of target errors, it is possible to prevent severe and unfavorable variations in learning and in turn cancel over training in network. More neurons in the hidden layer cause more degrees of freedom in the network. The more variables are optimized, time of training lengthens and weight matrix and bias vector become higher. In addition, the higher number of neurons results in increase possibility of finding answer and much chance for preventing falling in local minimums. This study reports the results of under fitting and over fitting states. On the other hand, it has been proven that if the number of hidden layer neurons is smaller than or equal to the number of inputs, better results are given (Avarideh, 2012). Therefore, in this study, the best number of neurons in the hidden layer was obtained by different tests. So that, for fixed structures, the number of middle layer neurons were considered 10, 20 and 30 to 59. Then, for the best structure, about 20 neurons were tested respectively. To avoid more deviation from main topic, all structures are not investigated (for selecting the best number of neurons) and due to the superiority of the sigmoid activity function to radial activity function in this study, only the results concerning the structures of static and dynamic autoregressive neural network model, with sigmoid activity functions. Figure 1 shows a flowchart of calculation steps by the autoregressive dynamic neural network model.
Algorithm of calculation steps by the dynamic autoregressive artificial neural network model as follows:
| Fig. 1: | A flowchart of calculated steps by the autoregressive dynamic neural network model |
First, dynamic neural network structure is designed in MATLAB software. Then, the training function and activity functions are defined. The next steps include providing network with training and testing sets and normalizing of input data (automatically by artificial neural network model toolbar) according to the activity function. Then, initial weights and biases are allocated to the training set input data and by their entry into the hidden layer (the effect of number of neurons) and using training function and network education activity functions, a comparison is made between the input data and target data and a communication is established between them. Then forecast is carried out for the new data (test set of input data) and by modifying the weights and biases and the effect of output delay (dynamic network), mentioned cycle is repeated until an acceptable error (Avarideh, 2012). The algorithm of calculation steps by static autoregressive artificial neural network model is exactly similar to dynamic autoregressive artificial neural network model with this difference that in the correction step of weights and biases, the model output cannot be entered as input to the network. All mathematical methods made by default MATLAB functions for artificial neural network model.
Criterion to select the best structure of artificial neural network model: For more accurate study of used models five criteria were used to select the best structure of artificial neural network models as follows:
In order to select the best structure between artificial neural network models, the root mean square error and the mean bias error were used as follows:
| (3) |
| (4) |
where, RMSE is the root mean square error, MBE is the mean square error, i
is the number of months, Qci is the computational discharge in month
i, the Qoi observational discharge in month i and n is the number
of data. Finally, for being comparable the results with other similar studies,
RMSE/
| (5) |
| (6) |
| (7) |
where, Ei is the relative error in month i, Fi is the
average of cumulative relative error in the month i,
Dez basin encompasses some part of the middle peaks of Zagros. The basin ranges between 32°, 35' to 34°, 07' North latitude and 48°, 20' to 50°, 20' east longitude and is located in southwestern Iran. Dez basin is limited from west to Karkheh basin, from north to Ghareh Chay basin and from east and south to Karun basin (Sadrolashrafi et al., 2008; Afkhami et al., 2007; Heidarnejad and Gholami, 2012). In this research for forecasting irrigation of entrance station in Dez reservoir, Taleh Zang station data is used and for this reason, models used in this study are named autoregressive. In order to forecast the goal station discharge (Taleh Zang station at the entrance to the Dez reservoir) at the monthly scale, the station's monthly discharge period from water year 1960-1961 water year 2006-2007 has been selected. In reality, the used data involved 564 data that began from October 1960 and end in September 2007.
RESULTS AND DISCUSSION
Table 1 shows training and forecasting RMSE/
Table 3 shows the minimum E and F indexes, the month of occurrence of these values and the Cv index for forecasting period. For a better comparison of E and F indexes in forecasting period, Fig. 2 and 3 could be used, respectively. To investigate the forecasting time changes, Eq. 5, 6 and 7 were used and the best forecasting time for the models obtained. Table 3 and Fig. 2 and 3 show lower values of the E and F indexes for autoregressive artificial neural network model with sigmoid activity function into the autoregressive artificial neural network model with radial activity function.
| Table 2: | The effect of number of hidden layer neurons in the forecasting data accurately |
| Table 3: | The minimum of E and F indexes and month of their occurrence in forecasting period |
| Fig. 2: | E index changes of models in forecasting period |
| Fig. 3: | F index changes of models in forecasting period |
Figure 2 shows that the changes in relative error in autoregressive artificial neural network model with sigmoid activity function into the autoregressive artificial neural network model with radial activity function is lower and according to Table 3 the coefficient of variation of these models is also lower. Figure 3 shows that the mean of cumulative relative error have less fluctuation in autoregressive artificial neural network model with sigmoid activity function. In other words, autoregressive artificial neural network model with sigmoid activity function have reached stagnation in error. Also, lower value of mean relative error and coefficient of variation of relative error in activity artificial neural network model with sigmoid activity function into the autoregressive artificial neural network model with radial activity function represents less error changes for autoregressive artificial neural network model with sigmoid activity function and implies the superiority of autoregressive artificial neural network model with sigmoid activity function to the autoregressive artificial neural network model with radial activity function. Given the above discussion, the autoregressive artificial neural network model with sigmoid activity function and 17 neurons in the middle layer is selected as the top model. Figure 4-6 compare results of the top models in the training and forecasting periods with observed data, respectively.
According to Table 3, Emin and Cv indices in static autoregressive artificial neural network model with sigmoid activity function has decreased nearly 50% into the static autoregressive artificial neural network model with radial activity function.
| Fig. 4: | Comparison of used models in this study in training period |
| Fig. 5: | Comparison of used models in this study in forecasting period |
Also,
| Fig. 6: | Comparison of observed data with the top model ANN in forecasting period regardless the occurrence time |
Two models with different structures and two different activity functions are also comparable and since the only difference between static and dynamic structures is related to output delay, it is possible to discover more effectiveness of activity function in forecasting accuracy compared to the output delay.
Finally, dynamic artificial neural network model with sigmoid activity function and 17 neurons in hidden (middle) layer equal to 0.6380 was chosen as the best model to forecast Dez reservoir inflow at Taleh Zang station. After this model, static artificial neural network model with 6 neurons in the middle layer and with sigmoid activity function and then dynamic artificial neural network model with radial activity function and 27 neurons in middle layer was placed in second and third category of forecasting, respectively. Finally, static artificial neural network model with radial activity function and four neurons in the hidden (middle) layer was in fourth place. Thus, dynamic autoregressive artificial neural network model models used in this study, are superior to the static autoregressive artificial neural network model in both training and forecasting stages, due to the effectiveness of output delay as an input to the network and increase in network training power. As it is shown in Fig. 4-6, the disadvantage of artificial neural network model is significant relatively error in forecasting maximum monthly discharge. In the artificial neural network model, because input data are entered into the network as three vectors 60-ary (each vector continues previous vector statistically), where each vector is related about five years, which starts with the October of the first year and ends in the September of the fifth year. Thus, for forecasting each month, not only once-in-a month relationship of each 60-ary vector, but also the communication between each data in these categories of 60-ary with other two categories is considered. It could be better said that if the relationship between the data is regarded as a hydrological parameter, the artificial neural network model uses two hydrological parameters (once-in-a-month communication between each category of 60-ary and communication of every data in this category with two other categories). This shows that artificial neural network model have a better performance in long-term forecasting. This is confirmed in Fig. 4 and 5. Because moving further forward to the points of ending period in these figures, more accuracy of forecasting peak points. According to Fig. 3 to 6 and Table 3, it is difficult to select the best forecasting time, but as we move further toward the end of five-year forecast period, fluctuations decrease and even the accuracy of forecasts increases (especially in the peak). Therefore, it is concluded that by dynamic autoregressive neural network model with sigmoid activity function, discharge for the next 5 years could be forecast with a proper accuracy.
Compared with previous researches, the achievements of this study can be outlined as:
Using Dynamic artificial neural network model, accuracy of forecasting increased than Banihabib et al. (2008) that used only Static artificial neural network model. In both observed and simulated stages accuracy of forecasting increased than Sadrolashrafi et al. (2008). Accuracy of modeling decreased in observed period into the Jia and Culver (2006) but forecasting was more accurate due to large sample data (42 years data). Amount of RMSE reduced in Taleh Zang station than Heidarnejad and Gholami (2012). Accuracy of forecasting increased due to more neurons in hidden layer (17 neurons in this study into the 5 neurons used by Kisi and Cigizoglu (2005)). Time horizon increased to 5 years than Teschl and Randeu (2000) in hourly scale, Baareh et al. (2006) in monthly scale, Sahoo et al. (2006) in daily scale, Karunasinghe and Liong (2006) in monthly scale, Pulido-Calvo and Portela (2007) in monthly scale, Balaguer et al. (2008) in weekly scale, Coulibaly et al. (2000) in daily scale, Chiang et al. (2007) in hourly scale and Abrahart et al. (2001) in hourly scale.
CONCLUSION
In this study, ability of static and dynamic autoregressive artificial neural network model is compared in forecasting Dez reservoir inflow at the Taleh Zang stations. Monthly discharge data for a period of 42 years were collected from Taleh Zang hydrometrical station and used for training models. Then, the accuracy of forecasting models was investigated by recent 5 years data. To summarize, it could be concluded that:
Dynamic autoregressive artificial neural network model used in this study are superior to static autoregressive artificial neural network model, in both training and forecasting phases, due to the effect of output delay as an input to the network and increasing network training power.
The effect of selecting the proper activity function and the number of middle layer neurons were well identified in this study as well. Because by selecting appropriate activity function and number of neurons, forecasting is nicely performed, but by selecting activity function and number of inappropriate neurons forecasting accuracy decreases significantly. Appropriate number of middle layer neurons for static and dynamic networks is 6 and 17, respectively and an appropriate activity function is sigmoid.
Changes in relative error, cumulative mean relative error and variation coefficient of relative error in autoregressive artificial neural network model with sigmoid activity function was less than autoregressive artificial neural network model with radial activity function; this indicates the superiority of autoregressive artificial neural network model with sigmoid activity function to other models. By investigating these changes, it will be clear that the forecasting dynamic autoregressive artificial neural network model with sigmoid activity function could suitably applied to forecasting discharge of the next 5 years.
NOTATIONS
| β | = | Factor of Nguyen-Widrow function |
| p | = | Number of hidden layer neurons |
| Wijold | = | Initial value of weight |
| Wijnew | = | Amount of improvement weight |
| RMSE | = | Root mean square error |
| MBE | = | Mean square error |
| I | = | Number of months |
| Qci | = | Computational discharge in month i |
| Qoi | = | Observational discharge in month i |
| n | = | Number of data and number of input layer neurons |
| = | Average of observational discharge | |
| Ei | = | Relative error in month i |
| Fi | = | Average of cumulative relative error in the month i |
| = | Average of relative error | |
| Cv | = | Variation coefficient of relative error |