Application of ANN and ANFIS Models on Dryland Precipitation Prediction (Case Study: Yazd in Central Iran)

The purpose of this research is to evaluate the applicability of two artificial intelligence techniques including Artificial Neural Networks (ANN) and Adaptive Neuro-Fuzzy Inference Systems (ANFIS) in prediction of precipitation amount before its occurrence. In fact, this paper presents the application of these models to predict precipitation in Yazd meteorological station in central Iran with a hyper arid climate condition and very low and highly variable annual rainfall. In this study, different architectures of ANN and ANFIS models as well as various combinations of meteorological parameters including 3-year precipitation moving average, maximum temperatures, mean temperatures, relative humidity, mean wind speed, maximum wind direction and evaporation have been used as inputs of the models. According to the results, among different architectures of ANN, dynamic structures including Recurrent Network (RN) and Time Lagged Recurrent Network (TLRN) showed better performance for this application. Final results show that the efficiency of TLRN and ANFIS for this application are almost the same, although in different tests with different input patterns the results produced by these two methods are slightly different. In general, it was found that both ANN and ANFIS models are efficient tool to model and predict precipitation amounts 12 months in advance.

INTRODUCTION

Characteristics and amount of precipitation is not often easily known until it occurs. In the other hand, as prediction of precipitation plays a crucial role on evaluation and management of drought and flood events, it is very important to be able to predict precipitation before it occurs. Most of the methods used to predict precipitation in the past, are regression or auto-regression linear models which their ability is limited in dealing with natural phenomenon with generally non-linear trend (Gholizadeh and Darand, 2009). However, in recent decades some data driven techniques such artificial intelligence varieties have shown great ability to deal with non-linear hydrology and water resources problems. Two main varieties of artificial intelligence technique which have been widely used to predict natural phenomenon are Artificial Neural Networks (ANN) and Adaptive Neuro-Fuzzy Inference Systems (ANFIS). Most of the previous investigations have indicated that ANN is an efficient tool with superior abilities and is widely used in different areas of water-related research (Dastorani et al., 2009). In this regard, Hsu et al. (1995) and Minns and Hall (1996) used ANN to model rainfall-runoff relationship. In these studies some physical and meteorological characteristics of the catchment including drainage area, slope, precipitation, temperature and evaporation were used to predict flow at the outlet of the catchments. It was stated that the results were satisfactory. In addition, the superior performance of the ANN for short-term stream flow forecasting in the Winnipeg River system (Canada) within a stochastic-deterministic watershed model was described by Zealand et al. (1999). Jain et al. (1999) compared ARIMA time series model and ANN for streamflow forecasting in India and again concluded in favor of the ANN approach. Thirumalaiah and Deo (1998) demonstrated the ability of ANN to accurately prediction of hourly flood runoff and daily water stage in real-time. Birikundavyi et al. (2002) used ANN as a conventional conceptual model in forecasting of daily streamflow in the Mistassibi River in Quebec, Canada. Wang et al. (2006) used ANN to forecast daily streamflow from streamflow records alone, without employing exogenous variables of runoff generating process such as rainfall. Hall et al. (1999) applied ANN for rainfall forecasting in Texas. Kuligowski and Barros (1998) used ANN to predict 6 h rainfalls in two drainage basins in Pennsylvania. Luk et al. (2000) employed ANN for short-term rainfall forecasting within a flood warning system. In other research project, Jain (2001) predicted suspended sediment load of the Mississippi river and recommended the applicability of multi-layer perceptron ANN for this purpose. Dastorani and Wright (2002) used Artificial neural network for real-time river flow prediction in a multi-station catchment. Dastorani and Wright (2003) completed a research project on flow estimation for ungauged catchments using a neural network method. Using conjunctively dyadic wavelet transforms and an Artificial Neural Network (ANN), Kim and Valdes (2003) employed PDSI to forecast droughts in the Conchos River basin in Mexico. Dastorani and Wright (2004) employed artificial neural networks to optimize the results of a hydrodynamic approach for river flow prediction. Using SPI as a drought index, Mishra and Desai (2005) employed stochastic models for forecasting droughts in the Kansabati River basin in India. Sarangi and Bhatlacharya (2005) compared the application of regression methods and ANN models to predict the rate of erosion and sediment and mentioned the superiority of ANN models over the regression methods. Maria et al. (2005) used ANN model for daily rainfall forecasting. Mishra and Desai (2006) used ANN technique to predict drought in Kansabati catchment in India. In this research, they also used ARIMA and SARIMA models and compared the results to those of ANN, then recommended more efficiency of ANN over other used methods. Morid et al. (2007) carried out an investigation on drought prediction using ANN models. In fact, it was tried to predict two drought indexes including EDI and SPI with 12 months lead time (12 months ahead) In Tehran, Iran. Mishra et al. (2007) completed the research project on drought forecasting using a hybrid stochastic and neural network model and stated that the hybrid model which was a combination of statistical linear and non linear models, is a suitable method to model and predict drought events. Dastorani et al. (2009) used neural network as well as neuro-fuzzy models to reconstruct flow data series and compared the results of these new techniques to some traditionally used methods and mentioned superiority of the new techniques (especially neuro-fuzzy system) over traditional methods. Hung et al. (2009) employed ANN for rainfall forecasting in Bangkok and then forecasts by ANN model were compared to the convenient approach namely simple persistent method. Results showed that ANN forecasts had superiority over the ones obtained by the persistent model. Tektas (2010) Used ANFIS and ARIMA MODELS for weather forecasting and the results were evaluated according to prediction performance, reliability and efficiency. The performance comparisons of ANFIS and ARIMA models due to MAE (Moving Average Error), RMSE (Root-Mean-Square error) and R² criterion, indicate that ANFIS yields better results. Bustami et al. (2007) used ANN for precipitation data reconstruction and found that backpropagation ANN developed for this purpose performed very well in simulating missing precipitation. Wong et al. (2003) used Self-Organising Map (SOM), backpropagation neural networks (BPNN) and fuzzy rule systems to perform rainfall spatial interpolation based on local method and stated that results were satisfactory. Gholizadeh and Darand (2009) used ANN for precipitation forecasting in Tehran. They compared the results to the related observations and obtained the maximum r of about 0.84 and recommended the applicability of ANN for precipitation prediction.

Present research describes the application of ANN as well as ANFIS models to predict future precipitation in hyper arid region of Yazd (with about 50 mm annual precipitation and more than 3500 mm potential evapotranspiration) in Iran. The main purpose is to specify the best type and structure of the ANN and ANFIS models and also the most appropriate input variables to have a reliable and accurate prediction of the future rainfall.

MATERIALS AND METHODS

Study area and data: The related research project was designed in Faculty of Natural Resources, Yazd University and started in Autumn 2008 and completed in early Spring 2010. The study area is Yazd meteorological station located in Yazd city in Iran with geographical longitude of 54°, 17' and latitude of 31°, 54' with a hyper arid climate condition. Various combinations of climate factors including previous monthly precipitation, evaporation, wind speed, intensive wind direction, relative humidity, maximum temperature and mean temperature for the period of 1975-2007 were used as inputs of the models. It must be added that historical precipitation data was used in different forms such as normalized rainfall data, SPI (Standardized Precipitation Index), seasonal and 3-year moving average of precipitation data. Monthly precipitation data of the next year (12 month before it occurs) was the output of the models in this research. Different types of ANN and ANFIS models were used and evaluated to choose the most appropriate one. Table 1 indicates the type of variables used as inputs of the models.

Artificial Neural Networks (ANN): The first model used in this research project is an Artificial Neural Network (ANN) based approach.

Table 1:	Type and code of the variables used to predict precipitation
Data of 1975 to 2001 was used for training purpose and the data of 2002 to 2007 was used to test the model performance

An ANN is an interconnected group of artificial neurons that uses a mathematical model for information processing based on a connectionist approach to computation. In most cases an ANN is an adaptive system that changes its structure based on external or internal information that flows through the network. In more practical terms neural networks are non-linear statistical data modelling tools. They can be used to model complex relationships between inputs and outputs or to find patterns in data (Lucio et al., 2007). In many applications, modelling tools have provided better results when used in hydrological time series analysis (Elshorbagy et al., 2002). Neural networks must be trained with a set of typical input/output pairs of data called the training set. The final weight vector of a successfully trained neural network represents its knowledge about the problem. As different types of neural network deal with the problems in different ways, their ability varies depending on the nature of the problem in hand. Therefore, various types of ANN were used in this study including Multi Layer Perceptron (MLP), Generalized Feed Forward (GFF), Modular Neural Network (MNN), Principal Component Analysis (PCA), Recurrent Network (RN) and Time Lagged Recurrent Network (TLRN). Figure 1 shows a simple MLP architecture with the related inputs and output used in this research. Prediction networks usually take the historical measured data and after some processing stages future condition is simulated. Among the ANN models, after evaluation and testing of different ANN structures (mentioned above), TLRN and RN networks were selected due to their higher performance and then between these two, TLRN network showed slightly higher abilities. Therefore, TLRN was the final selected ANN type for precipitation prediction in this study. In all ANN models three transfer functions including linear, tangent hyperbolic and sigmoid were used and tested in hidden and output layers and then in each case the results were compared to the measured values to select the best structure for ANN models. For statistical comparison of the outputs to the measured values, coefficient of efficiency (R ) and root mean square error (RMSE) were employed.

Fig. 1:	A simple MLP architecture used in this research

Adaptive Neuro-Fuzzy Inference System (ANFIS): The second model used in this research was Adaptive Neuro-Fuzzy Inference System (ANFIS), a new improved tool and a data-driven modeling approach for determining the behaviour of imprecisely defined complex dynamical systems (Kim and Kasabov, 1999). ANFIS model has human-like expertise within a specific domain -adapt itself and learns to do better in changing environments (Kurian et al., 2006). An ANFIS aims at systematically generating unknown fuzzy rules from a given input-output data set (Abraham et al., 2003). Figure 2 represents a typical ANFIS architecture based on following layers:

Layer 1: Every node in this layer is an adaptive node with a node function that may be a generalised bell membership function (Eq.1), a Gaussian membership function (Eq. 2), or any membership functions:

(1)

(2)

where, a_i, b_i and c_i are premise parameters. Also x is the input to node i and Ai is the linguistic label (for example, low and high) associated with this node function. Premise parameters change the shape of the membership function.

Layer 2: Every node in this layer is a fixed node labelled Π, representing the firing strength of each rule and is calculated by the fuzzy AND connective of product of the incoming signals by using Eq. 3.

Fig. 2:	A typical ANFIS architecture used in this study

(3)

where, μA_i (x) and μB_i (x) are membership grades of fuzzy sets A, B and also w_i is firing strength of each rule.

Layer 3: Every node in this layer is a fixed node labelled N, representing the normalised firing strength of each rule. The ith node calculates the ratio of the ith rule’s firing strength to the sum of two rules’ firing strengths by using Eq. 4.

(4)

where, is normalized firing strength that is the ratio of the i-th rule’s firing strength (w_i) to the sum of the first and second rules’ firing strengths (w₁, w₂).

Layer 4: Every node in this layer is an adaptive node with a node function (Eq. 5), indicating the contribution of ith rule toward the overall output:

(5)

where, z_i is equal to (p_i^x+q_i^y+r_i) and also p_i, q_i and r_i are consequent parameters.

Layer 5: The single node in this layer is a fixed node labelled Σ, indicating the overall output as the summation of all incoming signals calculated by Eq. 6.

(6)

where, Z is the summation of all incoming signals.

What is important when inspecting the above layers is principally three different types of components that can be adapted as follows (Lughofer, 2003):

•	Premise Parameters as nonlinear parameters that appear in the input membership functions
•	Consequent Parameters as linear parameters that appear in the rules consequents (output weights)
•	Rule structure that needs to be optimised to achieve a better linguistic interpretability

In this study, three Gaussian membership functions were used for input variable. There are a wide variety of algorithms available for training a network and adjusting its weights. In this study, an adaptive technique called momentum Levenberg-Marquardt based on the generalized delta rule was adapted (Rumelhart et al., 1987). In this scheme, the adaptive learning rates were used for adapted increasing the convergence velocity throughout all ANFIS simulations.

To compare the outputs of the simulations to the measured values and evaluate the applicability of different ANN and ANFIS models as well as type of input variables and combinations, RMSE and R² were calculated using following equations:

(7)

(8)

In which P_m is the measured value, P_es is the predicted (estimated) value and P¯ is the measured values mean.

RESULTS AND DISCUSSION

As mentioned earlier, this study focuses on evaluation of the application of different ANN and ANFIS models to predict precipitation in a hyper arid region (Yazd in central Iran) 12 months in advance. Although, between the used ANN models, both RN and TLRN architectures in some cases presented quite acceptable results but the accuracy of the predictions made by TLRN is higher. Therefore it was decided to compare the outputs of this type of ANN to those presented by the ANFIS model. In different tests (using different combinations of the inputs) the results of TLRN and ANFIS are slightly different although these differences are not considerable. Figure 3 shows the results produced by two techniques (TLRN and ANFIS) against the observed values in a test where the inputs of the models are 3-year moving average precipitation and pan evaporation data of the previous year and the out is the amount of precipitation 12 months later. Scatter plots showing results of each model against the observed values are also shown in Fig. 4a and b.

Fig. 3:	The results produced by two techniques (TLRN and ANFIS) against the observed values in test 1

As it is seen from the figures, although the quality of the results produced by the models is not considerably different but the accuracy of the outputs of the TLRN are higher than those of ANFIS model. The values of the coefficient of efficiency (R) for the results produced by TLRN and ANFIS are respectively 0.92 and 0.86.

Results produced by two techniques (TLRN and ANFIS) against the observed values in another test where the inputs of the models are 3-year moving average precipitation, mean wind speed, intensive wind direction and relative humidity are shown in Fig. 5. To be able to have a better comparison for the results presented by these two techniques the scatter plots showing results of each model against the observed values are also shown in Fig. 6a and b. As Fig. 6 indicate, in opposite to the previous test in this test the quality of the results produced by the ANFIS model is higher than those of TLRN model. The values of R (coefficient of efficiency) for the results produced by TLRN and ANFIS are, respectively 0.88 and 0.96.

For the third test the results produced by two techniques (TLRN and ANFIS) against the observed values using inputs including 3-year moving average precipitation and maximum temperature of the previous year are shown in Fig. 7. The scatter plots showing results of each model against the observed values are also presented in Fig. 8a and b.

Fig. 4:	(a, b) Scatter plots showing results of each model against the observed values in test 1

Fig. 5:	The results produced by two techniques (TLRN and ANFIS) against the observed values in test 2

Fig. 6:	(a, b) Scatter plots showing results of each model against the observed values in test 2

Fig. 7:	The results produced by two techniques (TLRN and ANFIS) against the observed values in test 3

As the figures show, in this test the quality of the results produced by the ANFIS model is slightly lower than those produced by the TLRN model. The values of R (coefficient of efficiency) for the results produced by TLRN and ANFIS in this test are respectively 0.95 and 0.82.

As figures show the most accurate predictions have been produced when 3-year moving average precipitation data has been used as one of the inputs to the models (ANN and ANFIS). Other forms of precipitation data including SPI, normalized and seasonal did not make considerable improvement on results accuracy.

It must be mentioned that the results presented in this paper show only a part of simulations which their outputs have been relatively acceptable (as samples for different input variables and model structures). As mentioned earlier, input data have been used in different forms including measured values (without scale change), SPI, seasonal and normalized values. Using 3-year moving average precipitation as input of the models considerably improved the results. Apparently this variable has the most important role on prediction of the future precipitation.

It must be mentioned that precipitation is a highly variable, randomic and complicated phenomena in the study area where is a hyper arid region and therefore is quite difficult to predict especially with enough lead time and acceptable accuracy.

The accuracy of predictions in this research has been improved step by step by changing the type and number of input variables. The final obtained results of this study is encouraging, as precise prediction of a phenomenon like precipitation is quite a difficult task due to its complexity and variability. Comparing the results of this research to those carried out by Morid et al. (2007) and also Mishra and Desai (2006) indicates that although the study area of the present research has been located in a hyper arid climate condition where rainfall amount and distribution is extremely variable but the obtained predictions are quite acceptable. In Morid et al. (2007) the best prediction had the R² value of 0.79 (R= 0.89) for the lead time of 6 months, in an area where mean annual precipitation varies from 700 to 120 mm (in different stations), however they have mentioned efficiency of ANN in precipitation prediction which is in support with the present study. About the results of Mishra and Desai (2006) the highest R for the predictions with 6 months lead time has been 0.631 (for one month lead time it is 0.925) and recommended the that ANN is an efficient tool for rainfall prediction which is in support with the findings of present study. Study area of Mishra and Desai (2006) is Kansabati catchment in India with mean annual precipitation of about 1268 mm.

Fig. 8:	(a, b) Scatter plots showing results of each model against the observed values in test 3

However, in the present study where mean annual precipitation is about 60 mm and for prediction lead time of 12 months the highest R for the predictions is about 0.96 (by the ANFIS model) which shows the higher quality of predictions in comparison to both mentioned studies. It is quite clear that normally as lead time increases the accuracy of predictions decreases and also in humid climate conditions the variability of precipitation decreases and therefore in general the accuracy of predictions increase. Hung et al. (2009) used ANN for rainfall forecasting in Bangkok, according to the results, it was stated that ANN forecasts have had superiority over the local traditional model. Therefore the findings of the research is also in support with the results of this study. In addition, the results taken by Wong et al. (2003), Bustami et al. (2007), Gholizadeh and Darand (2009) and Tektas (2010) are all in support with the findings of this research recommending good performance of ANN and ANFIS tools for precipitation prediction.