Application of ANN and ANFIS Models for Estimating Total Infiltration Rate in an Arid Rangeland Ecosystem

INTRODUCTION

Arid and semi arid areas constitute over 30% of the world land surface (Saco et al., 2007). These areas function as tightly coupled ecological-hydrological systems with strong feedbacks and interactions occurring across scales (Noy-Meir, 1973; Wilcox et al., 2003). Generally, the vegetation of semiarid and especially arid rangelands consist of mosaics or patterns composed of patches with high biomass cover interspersed within a low cover or bare soil components (interpatch) that have profound effect on infiltration rate (Lotfi-Anari et al., 2010). In semi arid ecosystems, it’s already well established that hydrology exerts a profound influence over the abiotic components of landscape primarily erosion (Weinwright et al., 2000).

Infiltration is one of the critical processes of the hydrologic cycle as it controls the spatio-temporal partitioning of rainfall into surface and subsurface flow and the subsequent movement in and between these two general components of the hydrologic cycle. Proper understanding of infiltration behavior is essential for accurate modeling of the hydrological response of a watershed (Sarkar et al., 2008).

Application of regression techniques however finds a limitation in the necessity to define explicitly the mathematical form of the link between independent and dependent variables. To partly overcome such difficulty, a new interpolation method based on Artificial Neural Networks (ANN) has been recently applied to the determination of operating rules. The ANN are able to approximate a wide range of multivariate linear and non-linear functions, while maintaining very good generalization capabilities and therefore they are ideal candidates when the relationship among the variables is unknown (Cancelliere et al., 2002). Reviews of the state of the art about ANN applications in hydrology were recently published in ASCE Task Committee (Govindaraju, 2000; Govindaraju and Rao, 2000), showing the increasing interest in neural networks during last years in many hydrology related areas, such as rainfall-runoff modeling, streamflow forecasting, ground water, precipitation forecasting and water quality issues. In water management, ANN has been applied for deriving reservoir operating policies with respect to different types of water supply systems (Raman and Chandramouli, 1996; Cancelliere et al., 1998, 2002; Rossi et al., 1999; Jain et al., 1999; Chandramouli and Raman, 2001). Nestor (2006) tried to model the infiltration process with a multi-layer perceptron artificial neural network.

This research was designed to investigate the capabilities of new data-driven techniques for modeling the infiltration rate in an arid rangeland ecosystem. This study deals with the estimation of total infiltration rate, using first and second 5 min time period infiltration rates by new machine learning techniques including six types of ANN as well as ANFIS models. The results are then compared with each other and also with those obtained from two new linear models (LLR and DLLR). According to the application of the new techniques (ANN and ANFIS) reported from studies completed in other aspects of hydrology and water resources, the main hypothesis of this research was created. The main hypothesis of this study is: appropriate performance of ANN and ANFIS models in estimation of total infiltration rate. This hypothesis would be accepted or rejected according to the final results.

MATERIALS AND METHODS

Study area: The study area is located in Nodoshan arid rangeland ecosystems in the Yazd Province in the center of Iran (31°46’85” N, 53°43’04” W). According to Emberger method arid frigid climate with warm summers and cold winters prevails in the study area. The study site receives an average of 188 mm rainfall annual falling as rain and snow, concentrated in the period of autumn and winter of the year. The rainfall erosivity caused by the frequent storms of high intensity and short duration is high. Virtually no water exists on the surface, except locally after infrequent, heavy rainfall. The area consists primarily of sandy loam entisols with a low degree of development and moderate depth (30-70 cm). The elevation of the study site ranges from 1500 to 1900 m above sea level. The Maximum and minimum mean temperatures of the hottest and coolest month are 36 and -15°C, respectively. This study has been done in April, 2008.

The study area includes shrubland in gently sloping alluvial fans that are dominated by Artemiaia sieberi and Astragalus achrochlarus, both are native species that have expanded considerably in extent and density and each has its unique growth pattern and distribution. Some other plant species are: Astragalus candolleanus, Iris songarica, Acantholimun sp., Acanthphilum sp., Stachys inflate, Lactuca glaucifolia, Poa sinaica, Stipa barbata and Agropyron desertorum.

Infiltration rate measurement: Different kinds of patches and interpatch were reconnoitered in the rangeland study site. Plant species and interspaces between them were considered as microenvironments that have different functions on rangeland hydrological processes, such as infiltration. In this study area, three kinds of patches were observed: shrub, grass, iris and one kind of interpatch: bare soil that was included spaces between vegetated patches. Shrub patches were included Artemisia sieberi, Artemisia aucheri and Astragalus achrochlarus. Grass patches were included Stipa barbata and Agropyron desertorum. Iris patch was Iris songarica. In all of these patches and interpatch, infiltration rates were determined by using mono ring infiltrometer. Ring diameter was 15 cm that is optimum size for ring method. Infiltration rate was determined in 5 min periods for 30 min as total infiltration rate. Infiltration rate in first 5 min period was considered as initial infiltration (sorption). And last 5 min period was considered as final infiltration (steady state flow). Various patches and interpatch sites were considered as ecohydrologycal sites, then Initial infiltration rates and total infiltration rates were measured with 8 replications for every kind of patches and interpatch.

Modeling total infiltration rate: WinGamma software pack was mainly used for preliminary analysis of data. For this purpose, we used two options of this software: (1) genetic algorithm option and (2) M-Test option. The best input composition was determined using genetic algorithm option. For this purpose results of genetic algorithm have been compared using gamma value and standard error. We also used M-test option to determine optimum size of training set.

Then total infiltration rate of different patches and interpatch estimated using first and second 5 min time period infiltration rates as input data. For this purpose, Local Linear Regression (LLR), Dynamic Local Linear Regression (DLLR), Artifitial Neural Networks (ANNs) and neuro-fuzzy system (ANFIS) were applied. The performances of the models applied in this study were assessed using three standard statistical performance evaluation criteria. The statistical measures considered were Nash-Sutcliffe model efficiency coefficient (E), Root Mean Square Error (RMSE) and coefficient of determination (R²). The efficiency coefficient (E) between the observed and simulated Infiltration rates is defined as:

(1)

where, X_obs is observed data and X_est is simulated data is average of the observed data.

Nash-Sutcliffe efficiencies can range from -8 to 1. An efficiency of 1 (Eq. 1) corresponds to a perfect match of simulated data to the observed one (Nash and Sutcliffe, 1970). The Nash-Sutcliffe efficiency coefficient is used to assess the predictive power of hydrological models. The RMSE is defined as:

(2)

where, n is the number of data points, X_obs is the observed value and X_est is the predicted value. There are several different definitions of determination coefficient (R²) which are only sometimes equivalent. One class of such cases includes that of linear regression. In this case, R² is simply the square of the sample correlation coefficient between the measured and their predicted values.

Models
LLR and dynamic LLR: To make a prediction for a given query point in input space Local Linear Regression (LLR) first finds the k nearest neighbours of the query point from the given data set (where the number k is supplied by the user) and then builds a linear model using these k data points. Finally the model is applied to the query point thus producing a predicted output. Because of the way WinGamma analysis the data to compute the Gamma statistic the k nearest neighbours of any point in input space can be found very rapidly. Thus local linear regression is a very fast and capable predictive tool. Dynamic Local Linear Regression (DLLR) is basically identical to LLR with the additional feature that as new data is seen for the first time it is incorporated into the model. As new test data is encountered (but after the attempt at prediction of stage) dynamic LLR will make steadily better predictions (Jones, 2001).

Artificial neural networks: The ANNs is a relatively new nonlinear statistical technique. It can be used to solve problems that are not suitable for conventional statistical methods. A neural network consists of simple synchronous processing elements, called neurons, which are inspired by biological nerve system (Malinova and Guo, 2004). Neurons having similar characteristics in an ANN are arranged in groups called layers. The strength of connection between the two neurons in adjacent layers is represented by what is known as a connection strength or weight (Jain and Kumar, 2006). The most commonly used neural network structure is the feed forward hierarchical architecture. Feed Forward Neural Network (FFNN) has a parallel and distributed processing structure. It is composed of three layers: an input layer, which is used to present data to the network; an output layer, which is used to produce an appropriate response to the given input and one or more intermediate layers, which are used to act as a collection of feature detectors (Luk et al., 2001).

Neural networks must be trained with a set of typical input/output pairs of data called the training set. There are a wide variety of algorithms available for training a network and adjusting its weights. An adaptive technique called Levenberg-Marquardt was used to train. The trained network is then validated on the testing data set, which it has not seen before. Once an ANN has been trained and tested, it can be used for prediction or modeling the physical system for which it is has been designed.

As different types of neural network deal with the problems in different ways, their ability varies depending on the nature of the problem in hand. Therefore, six types of ANN as well as ANFIS models were used in this study. To determine the best architecture of the models, many structures were tested and the results were considered. The number of hidden layers, number of processing elements in hidden layers, type of transfer and output functions have been considered and evaluated.

Multilayer perceptron neural network: The most common neural network model is the Multi-Layer Perceptron (MLP). This type of neural network is known as a supervised network because it requires a desired output in order to learn. The goal of this type of network is to create a model that correctly maps the input to the output using historical data so that the model can then be used to produce the output when the desired output is unknown.

An advantage of MLP in terms of mapping abilities is its capability of approximating arbitrary functions. This is an important point in the study of nonlinear dynamics and other function mapping problems. In this study, different types of transfer and output functions for hidden and output layers as well as different numbers of hidden layers were used to find the best structure of MLP for this application. From these trials, it was found that the log-sigmoid function was the most compatible one for the hidden layer (4 neuron). However, for the output layer (1 neuron), the linear function was the most suitable one. In this study, only one hidden layer was the most suited number of this type of layers for the ANN model.

Two layer back propagation: We use the standard backpropagation algorithm to produce a two-layer feedforward neural network. Backpropagation, or propagation of error, is a common method of teaching artificial neural networks how to perform a given task. However, if the target MSError is much less than the Gamma statistic on the training data then (I) the network may end up being overtrained resulting in poor predictions, or (II) the training algorithm may never be able to reach the (possibly) unrealistic target MSError (Jones, 2001).

Conjugate gradient descent: This network is a variation and improvement on two-layer vanilla back propagation, it is generally more effective but requires more memory. The procedures for set up are very similar (Jones, 2001).

BFGS neural network: Probably the fastest and most efficient neural network training algorithm offered by WinGamma is a modified version of the Broyden-Fletcher-Goldfarb-Shanno learning algorithm. This algorithm uses second differences and is sometimes degraded by very noisy data, but generally it is probably best to use this option first when trying to produce a neural model (Jones, 2001).

Recurrent neural network: This type of network can be divided into fully and partially recurrent. Having a memory element distinguishes this network from the previous one. Although, recurrent networks are more powerful than feed forward networks, they are more difficult to train and their properties are not as well understood. The training of a recurrent network is much more sensitive to divergence. After using different types of transfer and output functions for hidden and output layers and comparison of the results, it was found that a log-sigmoid function is the most suitable one for the hidden layer (4 neuron). However, for the output layer (1 neuron), the linear function is a more compatible function.

NARX network: The nonlinear autoregressive network with exogenous inputs (NARX) is a recurrent dynamic network, with feedback connections enclosing several layers of the network (MATLAB Help Ver. 7.0.1). There are many applications for the NARX network. It can be used as a predictor, to predict the next value of the input signal. The use of the NARX network is demonstrated in another important application, the modeling of nonlinear dynamic systems. In this study, the NARX network with two hidden layer was applied. Log-sigmoid and tan-sigmoid transfer functions were selected for first (4 neuron) and second (3 neuron) hidden layers, respectively and also linear transfer function was employed for output layer.

Adaptive neuro-fuzzy inference system: A specific approach in neuro-fuzzy development is the Adaptive Neuro-Fuzzy Inference System (ANFIS), which has shown significant results in modeling nonlinear functions (Jang et al., 1997). The ANFIS is a new improved tool and a data driven modeling approach for determining the behavior of imprecisely defined complex dynamical systems (Kim and Kasabov, 1999).

Fig. 1:	A typical ANFIS architecture (Jang, 1993). Here, x and y are the inputs and z is the final output, A1, A2, B1 and B2 are the linguistic label (small, large, etc.) associated with this node function, wi is the normalized firing strength that is the ratio of the ith rule’s firing strength (Wi) to the sum of the first and second rules’ firing strengths (W1 and W2) and Π is the node label

The ANFIS model possess human-like expertise within a specific domain-adapt itself and learns to do better in changing environments (Kurian et al., 2006). An ANFIS aims at systematically generating unknown fuzzy rules from a given input/output data set (Abraham et al., 2003). Figure 1 shows a typical ANFIS architecture. Every node in layer 1 is an adaptive node with a node function that may be a Gaussian membership function or any membership functions. every node in layer 2 is a fixed node labeled Π, representing the firing strength of each rule. Every node in layer 3 is a fixed node labeled N, representing the normalized firing strength of each rule. Every node in Layer 4 is an adaptive node with a node function The single node in layer 5 is a fixed node labeled Σ, indicating the overall output (Z) as the summation of all incoming signals (Dastorani et al., 2009). In this study, Gaussian membership function was used for the input variable and Levenberg-Marquardt algorithm was employed for network training and adjusting its weights.

RESULTS

Infiltration rate: Infiltration rate in different patches and interpatch are shown in Fig. 2. Different responses of patches and interpatch areas in infiltration process, during 30 min time period, obviously observed.

Genetic algorithm: Results of using genetic algorithm (gamma values and standard errors) to determination of the best input composition are shown in Table 1.

Gamma value (0.00026) and standard error (0.0014) of the input composition includes both the first and second 5 min time period infiltration rate parameters (11) in comparison with the other input compositions (01: 0.0041, 0.0031 and 10: 0.0058, 0.0028) are lower. So, this input composition used for modeling.

M-test: Figure 3 shows winGamma M-test curve applied to determine optimum size of training set in modeling.

M-test curve get stabilized between data number 17 and number 20. According to this result, approximately 70% of data (20) were appropriate for training the models.

Fig. 2:	Infiltration rate curves of different patches and interpatch

Fig. 3:	M-test curve for determination optimum size of training set

Table 1:	Results of WinGamma genetic algorithm
11: Infiltration rate of first and second 5 min time period, 01: Infiltration rate of second 5 min time period, 10: Infiltration rate of first 5 min time period

Models: After determining the best input composition and optimum size of training set, different models were used to relating infiltration rates of first and second 5 min time period and total infiltration rates. List of the prediction accuracy of the models using three statistical indices (RMSE, R² and E) at training and validation stages is shown in Table 2.

According to Table 2, The NARX model was the best model in the training stage in comparison with other models, because of the lowest RMSE (0.06) and the highest E (0.99) and R² (0.99). After the NARX model, ANFIS and MLP models were the best trained models, respectively. The ANFIS, RNN and MLP models were the most appropriate models in the validation stage.

Table 2:	The statistical criteria of RMSE, R2 and efficiency coefficient (E), used for performance assessment of the applied models

Fig. 4:	(a) Comparison between the RNN predicted values and measured values and determination coefficient (R2) between them in training stage and (b) comparison between the RNN predicted values and measured values and determination coefficient (R2) between them in validation stage

However, results obviously showed that the ANFIS model performed a bit better than both the RNN and MLP models for predicting total infiltration rate from first and second 5 min time period infiltration rates. From the other hand, most of the neural network models were used were able to find the appropriate relationship between inputs and outputs in training stage, but had made some mistakes in validation stage.

Fig. 5:	(a) Comparison between the ANFIS predicted values and measured values and determination coefficient (R2) between them in training stage and (b) comparison between the ANFIS predicted values and measured values and determination coefficient (R2) between them in validation stage

Between these models the RNN model had better performance. Comparison between measured total infiltration rate and RNN simulation are shown in Fig. 4a, b.

According to results combination of the neural network and the fuzzy logic (ANFIS) had the most capability to generate total infiltration rate from the known initial (first and second time period) infiltration rates. Comparison between measured total infiltration rate and ANFIS simulation are shown in Fig. 5a, b.

DISCUSSION

Infiltration rate in interpatch and different patches: Considering the infiltration rate curves of different patches and interpatch (Fig. 2), the areas treated with vegetated patches had higher initial and total infiltration rates than the bare soil. Infiltration rate decreased in the iris, shrub, grass patches and the bare soil, respectively. Blackburn (1975), Wood and Blackburn (1981), Knight et al. (1984), Dunkerley (2002), Bhark and Small (2003) and Ludwing et al. (2005) confirmed infiltration rates are often observed to be different under different life forms. Higher infiltration rates of the shrub patches in comparison with the grass patches are in contradiction with results that indicated by Pressland and Lehane (1982). This inconsistency is related to shrub root system in arid environments. In limiting water ecosystems, plant roots have wide extension especially in shrub life form and include higher rate of plant biomass in comparison with grass life form, lead to high infiltration rates.

Total infiltration rate modeling: Turning to the results, Table 2 provides a summarized basis to judge the behavior of the evaluated models in dealing with measured data. ANFIS performed well with the highest accuracy of the predictions. Between six types of the ANN models applied in this study, RNN, MLP and BFGS networks have had more accurate results, respectively. LLR and DLLR showed week performance with the lowest accuracy of the predictions. Thus, the accuracy of outputs decreases gradually from ANFIS to ANNs and ANNs to the linear models (LLR and DLLR), respectively. The lack of linear models appropriate accuracy in present research was related to non-linear relation of infiltration rates of first and second 5 min time period with total infiltration rates. These kinds of models are most effective in regions of the input space with a high density of data points. If data points are few and far between in the vicinity of the query point and underlying function we are trying to model is truly non-linear, then the LLR and DLLR models will not be very effective (Jones, 2001). The reason of low performance of the ANNs in comparison with the ANFIS is their lack of explanatory power, also referred to as their black box problem. Neuro-fuzzy techniques remove some of the shortcomings of ANNs. They merge neural networks and fuzzy logic into an integrated system. ANFIS had the best applicability in this study; that is stated by Aqil et al. (2007) and Dastorani et al. (2009).

Nestor (2006) employed ANN multilayer perceptron to model infiltration process and showed the model was unable to estimate the first few minutes of the process, but improved significantly in the later time intervals. Because of applying first and second 5 min time period infiltration rates as input data; so MLP neural network in this study had better performance (R² = 0.93) than the obtained results in Nestor (2006) study (R² = 0.91).