Abstract: This study shows the ability of Artificial Neural Network (ANN) technology to be used for the prediction of the correlation between common lambsquarters (Chenopodium album L.) population, corn (Zea mays L.) population and planting pattern in different days after planting (as inputs) with common lambsquarters biomass production (as output). The number of patterns used in this study was 60 which were randomly divided into 45 and 15 data sets for training and testing the neural network, respectively. The results showed that a very good performance of the neural network is achieved. Some explanation of the predicted results is given. The multi layer perceptrons with training algorithm of backpropagation (BP) was the best one for creating nonlinear mapping between input and output parameters. The mean training of root mean square error (RMSE) was equal to 0.0156. ANN model predicted the common lambsquarters biomass with maximum RMSE, t-value, average prediction error and correlation coefficient of 0.0091, 0.985, 2.6% and 0.989, respectively. The ANN model, predicted common lambsquarters biomass within ± 5% of the measured biomass for 59.8% of the samples indicates that the ANN can potentially be used to estimate plant biomass. Adjusting ANN parameters such as learning rate, momentum, number of patterns and number of hidden nodes/layers affected the accuracy of biomass production predictions.
INTRODUCTION
Specific mechanisms that result in enhanced competitiveness of the crop with weeds are not well understood. A more equidistant spatial arrangment of corn plants, by optimizing planting pattern and plant population, is thought to play a role in reducing the potential for weed interference by enhancing the competitiveness of the crop (Fischer and Miles, 1973; Holt, 1995; Teasdale, 1995). Studies have shown that agronomic practices, such as planting pattern and plant population, type and time of tillage, crop rotation and cover crops that promote competitive crops are suitable tools for weed management (Gill et al., 1997; Swanton and Weise, 1991). Studing the effects of weed management techniques on weed growth and reproduction help to researchers for using the effective long-term weed managements.
The use of models in the decision-making process is a central component of Integrated Weed Management (IWM) (Swinton and King, 1994; Wilkerson et al., 1990). However, little research has quantified the effect of weed management techniques on weed survival, biomass and seed production. Weed biomass loss models are one of the important tools for investigation weed management efficiency (Begna et al., 2001; Håkansson, 2003). Some researchers showed that the ability of weed biomass to predict crop yield loss (Zimdahl, 2004; Askew and Wilcut, 2001; Baziramakenga and Leroux, 1998; Clewis et al., 2001), plant biomass (Scursoni and Satorre, 2005) and weed seed production (Draper and Smith, 1998; Bosnic and Swanton, 1997; Kropff et al., 1995) was very accurate. Thus, weed biomass can as an effective predictor use to estimate crop and weed seed production.
Various linear and nonlinear techniques have been used to perdict plant biomass and yield (France and Thornley, 1984; Cousens, 1985; Dieleman et al., 1995; Håkansson, 1997). Clewis et al. (2001) reported a linear relationship between weed density and weed aboveground dry biomass. Bosnic and Swanton (1997) applied nonlinear regression related weed shoot biomass or fecundity to weed density and time of cohort emergence. Kropff et al. (1995) applied nonlinear regression related weed shoot biomass or fecundity to weed relative leaf area or weed relative volume. Plant biomass is a complex interaction involving some parameters which have various degrees of effect on the biomass production. The relationships between biomass and related variables are almost always very complicated and highly non-linear which are hard to describe with mathematical models. Thus, it is important to researchers to find a model-free stimator model that incorporates a large number of variables. One of the most appropriate methods to illustrate it, seems to be Artificial Neural Networks (ANNs) (Almeida, 2002). In fact, this method is very powerful in dealing with non-linear relationships. Artificial neural network (ANN) models are powerful empirical modeling approaches and relatively simple compared to mathematical models (Hornik et al., 1989). Neural networks could be used for modeling nonlinear, accommodating multivariate and nonparametric data. Neural network approach, unlike the mechanistic model, is a model-free estimator; they don’t require any external manifestation of parametric relationship. Hence, the relationship between the parameters is automatically incorporated into the network model in an implicit manner during the training process (Drummond, 1998). So, it eliminates the difficulty of extracting the parameters for a mechanistic model. Accordingly, the use of ANNs has gained increasing popularity for application where a mechanistic description of the dependency between dependent and independent variables is either unknown or very complex (Almeida, 2002). Neural networks have been employed in a wide variety of applications, including the prediction of soil moisture content (Chang and Islam, 2000), crop yield (Cerrato and Blackmer, 1990; Drummond et al., 2003, Feng Lei, 1999; Heinzow and Tol, 2003; Safa et al., 2004; Kaul et al., 2005; Liu et al., 2001; O’Neal et al., 2002; Shearer et al., 1999; Simpson, 1994; Uno et al., 2001; Yang et al., 2003), seeding dates (Major et al., 1996), organic matter content in soils (Ingleby and Crowe, 2001), maturity of spring wheat (Hill et al., 2002) and physical and physiological damage to wheat seeds (Khazaei and Shahbazi, 2005). In conclussion, the neural network modelling is suitable for simulations of correlations which are hard to describe by mathematical models. Some other studies have also reported that ANN’s were better than traditional statistical methods when estimated soil water content based on soil physical properties, nitrogen leaking below the root zone of turf grass and soybean rust (Batchelor et al., 1997; Pachepsky et al., 1996; Starrett and Adams, 1997).
A neural network has two components: the node and the connection. A node consists of a neuron with positioning and connecting information. A connection consists of a weight with node addressing information. Neurons are single processing elements, which connected to neurons in the next layer, therefore forming different types of ANN. The parameter of weight is associated with each connection between two neurons, thus each cell in the upper layer receives weighted inputs from each node in the layer below. Neural networks are mainly characterized by the type of learning rule, neurons used (transfer function) and the way that they are organized, number of layers and number of neurons per layer. These specifications are related to the number of training points and to the nature of the function (Weiss et al., 2000). The learning algorithm is a procedure for modifying the weights and biases of the network. This procedure may also be referred to as a training algorithm. The learning algorithm is applied to train the network to perform some particular task. Among the many learning algorithms of neural networks, the backpropagation (BP) has been shown to be theoretically sound and has demonstrated excellent capability for various complex classification and prediction problems (Shearer et al., 1999; Pao, 1989). Transfer functions for the neurons are needed to introduce nonlinearity into the network. Without this nonlinearity, neurons would perform in a linear fashion and the ANN would not be able to map non-linear input/output relationships. For the output neuron(s), one should choose a transfer function suitable to the distribution of the target values. Many transfer functions have been introduced over the last few years by researchers specialized in ANN. However, only three of these transfer functions are commonly used: Linear, Sigmoid and Tang hyperbolic (Ripley, 1996).
The objectives of this research were (1) to build up and evaluate the predictive performance an ANN to approximate a nonlinear function relating common lambsquarters biomass in different days after planting respons to common lambsquarters, corn population and corn planting patternand (2) to evaluate the effects of the ANN model parameters on model performance.
MATERIALS AND METHODS
Dataset and input/output parameters: A neural network is usually trained using a large number of input with corresponding output data (input/output pairs). This means that for reliable training and performance of any neural network, we need an appropriate database. Using such a database, we can train neural network to perform complex functions. The common lambsquarters biomass data at different common lambsquarters population, corn population, planting pattern and different days after planting were used to develop and assess the ANN models. The variable levels used in this study are shown in Table 1. Three replications were made for each combination of the input variables. The averages of the treatments were used for training and testing the ANN models.
Field experiments were conducted at the field experiment of Tehran University (With 33: 28´ N, 51: 46´ E and 1180 m altitude) in 2003. The soil was loam silt which it was chiseled plowed in the fall and field cultivated in the spring. Based on the soil test results, Nitrogen and phosphorus (P2O5) fertilizers were applied broadcast at a rate of 180 and 120 kg ha-1, respectively.
| Table 1: | Summary of input and output variables ranges |
| Fig. 1: | Topology of the 4-layers, feedforward backpropagation NN with
four input for calculating common lambesquarters biomass |
The experimental design was a randomized complete block in a factorial arrangement with three replications. Plant population treatment was at two levels: recommended and 1.5 times recommended plant population. Planting pattern treatment was at two levels: one and two rows planting (planting on both of ridge sides) and the common lambsquarters was established in an addition series experiment (Radosevich et al., 1997) at four densities. Each plot was 7x3 m and consists of 4 rows. The corn hybird single cross 704 corn and common lambsquarters were planted to 0.75 m rows on the 23th and 24th of May. After corn and common lambsquarters establishment, both of them were thinned to achieve the desired population ratios. Except of common lambsquarters, all other weeds in each plot were controlled by hand throughout the growing season. Biomass of common lambsquarters were determined biweekly 43 days after planting in five stages. In each sampling, common lambsquarters plants that were placed in 0.225 m2 (0.30x0.75 m) harvested. These samples were oven dried at 75°C for 48 h and then weighed. The statistical analyses were performed using the General Linear Model (GLM) procedure of SAS (SAS Institute, 1994). The data obtained from the experiments were used to train and test the ANN models.
Artificial neural network model development: Feed-forward artificial neural networks were used as a modeling technique to model correlations between common lambsquarters population, corn population, planting pattern in different days after planting with common lambsquarters biomass. In this study, both multi-layer perceptron trained by BackPropagation (BP) and Radial Basis Function (RBF) neural networks to predict common lambsquarters biomass were developed. The networks were developed by using experimental biomass data for common lambsquarters. All of the ANN models had four input nodes and one output node corresponding to common lambsquarters biomass (Fig. 1). There were a total of 60 patterns each with 4 components (x1, x2, x3, x4; Y,) four of which are the input variables whereas the Y is the output variable. Initially 45 of samples were randomly selected to train the ANNs and the remaining 15 were used to test the accurcy of the developed models.
Adjustment of ANN parameters included the number of hidden layers and neurons, the type of transfer function, learning rate, momentum and a number of patterns. Preliminary trials indicated that two hidden layer networks performed better results than one hidden layer networks. Figure 1, shows the topology of the 4-layers, feedforward backpropagation neural network for calculating common lambsquarters biomass based on the four input variables.
The number of hidden nodes selected for ANN models was equal to one-half the total number of inputs plus outputs. The number of neurons were then increased and decreased by two to improve model performance.
The performances of the ANNs were compared using the Root Mean Square Error (RMSE) and the t-statistics that measures the scattering around the line (1:1), (Khazaei and Shahbazi, 2005). When t is close to 1.0, the fitting is very good. Meanwhile, the accuracy of the trained ANN was evaluated by calculating an individual absolute error for each of the examples reserved for testing (Khazaei and Shahbazi, 2005).
In order to achieve fast convergence to minimal RMSE, the input and output data were normalized with respect to the corresponding maximum and minimum values. As a result of normalization, all variables acquire same significance (importance) during the learning process. It must be pointed out that the same normalization process should be used for both training and prediction data sets to ensure that all the data items lie over the same range. In the present study, the transformation was performed as follows:
Where Xt is the transformation of the data point Xi; Xmin the overall minimum in training and prediction data sets; and Xmax the overall maximum in training and prediction data sets. The value of Xt lies between 0.05 and 0.95, corresponding to Xt = Xmin and Xt = Xmax, respectively. The Neuralworks professional ii/plus Simulator, version 5.23, was the software package used in this study.
RESULTS
Common lambsquarters biomass production: The results showed that the common lambsquarters biomass production in different days after planting was influenced by its own population, corn population and corn planting pattern. Significant effects of Common Lambsquarters population, corn population and planting pattern on common lambsquarters biomass occurred in different days after planting (Table 2). Thus, Common Lambsquarters population, corn population, corn planting pattern and days after planting applied as input variables for predicting common lambsquarters biomass.
Neural network modelling: Different ANN models were developed and tested for common lambsquarters biomass production based on the four input variables (Fig. 2). Neural network models were built directly from experimental data. Using this method, the best neural network model was obtained to predeict the correlation between the input and output parameters.
Results showed that BP neural networks were able to create a good nonlinear mapping between input and output parameters. The configuration that had a minimal dimension and minimum error gaving satisfying results, was retained with trial and error method. Table 3 shows the best BP neural network model and the best related parameters value to predict the common lambsquarters biomass. Before arriving at this optimum, several tests were carried out with different configurations of the neural network. The range of neural networks parameters tried were: Number of hidden layers: one and two layers; neurons/hidden layer: from 3 to 40; activation function: sigmoid, linear and tanh; learning rate: 0.01-0.9; momentum: 0.01-0.9; number of epochs: 1000-100,000.
| Fig. 2: | Normalized predicted common lambsquarters biomass data using
BP NN versus the normalized measured data |
| Table 2: | The main effects of common lambsquarters population and corn
population and planting pattern on common lambsquarters biomass production,
in different days after planting |
| ** Difference significant at 1%, * Difference significant at 5%, ns: Not significantly diffrent, DAP: Days After Planting | |
| Table 3: | The best BP structure and optimum values of the ANNs used
to predict the common lambsquarters biomass data |
| η = Learning rate α = Momentum Pmean = Mean predicted error (%) | |
| Fig. 3: | Prediction error histogram for the output points generated
by the RBF NN model |
| Fig. 4: | Error distribution of the neural network model for the prediction
of the common lambsquarters biomass |
| Fig. 5: | Average of the root mean square error versus the number of epochs |
Figure 2 shows the testing results for the final BP NN model. Comparisons between the 15 predicted common lambsquarters biomass data (test set data) versus the same set of measured data are visually presented using the 45° line of graph and two deviation lines, the ±5% deviation from the 45° line, as shown in pictures. On the y-axis, the network output is represented for the 15 cases and compared, on the x-axis, with the target (measured data).
| Fig. 6: | Effects of learning rate and momentum value on training RMSE
for the final network |
Agreement between the predicted and measured common lambsquarters biomass data within ±5% can be observed.
Figure 3, represents a histogram of the actual predicted error of the output points generated by the BP NN model corresponding to the experimental test data presented to the network during the testing phase. The error magnitudes are represented for all the 15 data points in the test set. The study of the relationship between the predicted error and values estimated by the final BP model showed complete independence (Fig. 4). The results obtained from this study showed that the network parameters affected the BP NN significantly. The learning rate, momentum term and epoch size were adjusted to achieve the least error ( Fig. 5, 6 and Table 3).
DISCUSSION
Common Lambsquarters biomass production: Table 2 showed that Common Lambsquarters population, corn population and corn planting pattern effect on Common Lambsquarters biomass in different days after planting. So that, the significant effects of Common Lambsquarters population, corn population and corn planting pattern on common Lambsquarters biomass detected in 43, 71 and 99 days after planting, respectively. Common Lambsquarters biomass in high level of its population, low level of corn population and one-row planting pattern of corn was higher than that at low level of its population, high level of corn population and two-rows planting pattern, respectively (data not shown).
The decline common lambesquater biomass under two-rows planting pattern and high population compared to one-row planting and low population of corn can be explained by accelerated leaf senescence and decreased photosynthetic rate due to shading effect. Some researchers (Board and Harville, 1992; Ottman and Welch, 1989) have reported increased light interception by crops such as corn and soybean due to higher plant population and narrower row spacing. There are some researches which confirm our results about the effects of weed population, Planting patterns and crop population on biomass production. For example, weed biomass was reduced 50% by increasing corn density twofold in Ontario (Tollenaar et al., 1994). Increased crop density not only reduces weed biomass (Hashem et al., 1998; Tanji and Zimdahl, 1997) but may also reduce quantity of weed seed produced. Planting patterns that favor crops in terms of better light interception should also result in the crops accumulating more biomass than the potentially competing weeds (Fischer et al., 2004). Relationship between weed population and weed biomass was also proved by some researches (Spitters and Aerts, 1983; Scursoni and Satorre, 2005).
Neural network modelling: Based on the RMSE, t-value, R2 and predicted errors, the results showed that among the various BP models, model of good performance was produced by the 4-25-10-1 structure. This BP neural network produced the smallest RMSE (0.0156) in training. This result implies that the designed ANN was able to properly learn the relationship between the input and output parameters. In deed, a well-trained ANN model is the key to design and analysis the inputs and outputs relations. In this study, the well trained ANN model was able to predict biomass production data with RMSE of 0.0156, t-value of 0.985, (Table 3) and the highest correlation coefficient of 0.989 between the actual and predicted data (Fig. 2). Ideally, the RMSE values should be close to zero, indicating that, on average, there is no difference between the predicted and measured values. This again confirm that given sufficient hidden units, multi-layer feed-forward network architectures can approximate virtually any function of interest to any desired degree of accuracy (White et al., 1992). Haykin (1999) has also reported that one or two hidden layers with an arbitrarily large number of neurons may be enough to approximate any function.
Radial Basis Function (RBF) neural networks were also employed to predict the common lambsquarters biomass. However, it did not produce any meaningful model for the biomass production estimation. The RBF models produced very small t-value and large RMSE between the actual and the predicted biomass data than the BP models. Compared to the 3-layer models, almost 4-layer models produced better performance. This indicates that increasing the number of hidden layers increased the modeling capability. In general, a higher non-linear level in the function corresponds to a larger number of neurons (a more powerful network). Previous study has also reported that two hidden layers generally perform better for continuous data (Heinzow and Tol, 2003).
Goel et al. (2003) used artificial neural networks to identify weed stress and nitrogen status of corn. They tried the models with one to two hidden layers and reported that the models with two hidden layers were useful. Park et al. (2005) found that ANN without cross-validation shows very high modelling accuracy, with r-values mostly exceeding 0.95 when this techniques to predict crop yield response under varying soil and land management conditions. Zaidi et al. (1999) indicate that neural networks were able to model lettuce plant growth with high correlation between the predicted and measured values (R2 = 0.92-0.99).
Figure 2 shows agreement between the predicted and measured common lambsquarters biomass data within ±5%. It is obvious that BP NN have successfully provided accurate prediction of the common lambsquarters biomass data.
Statistical comparisons between experimental and the predicted common lambsquarters biomass values using BP model were also performed by using the student t-distribution. The comparison was based on a 1% level of significance. The analyses showed that there was no significant difference between the predicted and experimental values (data not shown). These results again confirm that BP NN is function approximation models that can be trained by examples to implement a desired input-output mapping. These results indicate that the network successfully learned the relationship between the input factors and common lambsquarters biomass as output for all ranges of biomass data equally well (Fig. 3). As shown in Fig. 3 the histogram approached a normal distribution, except that there were a few over-predictions of common lambsquarters biomass. As clear the predicted error data ranged from -25.8 to 19.8. The mean predicted error from the ANN model was approximately 2.6%, which indicates good model performance. The picture shows that the BP NN model, predicted data within ±5% of the measured biomass data for 59.8% of the samples which indicates the ANN can potentially be used to estimate common lambsquarters biomass.
The study of the relationship between the predicted errors and values estimated by the final BP model showed some overe and under-stimates of some weak values were possibly observed (Fig. 4). The coefficient of determination was negligible (R2 = 4x10-7) and the slope of correlation between estimated values and residuals close to 0 (y = -0.02x+2.6); the residuals were well distributed on either side of the horizontal line (ordinate) representing the residual mean. This was the consequence of the scarcity of low values in the database for an effective learning of the model. These results indicate again that a BP neural model can predict all ranges of common lambsquarters biomass data equally well.
Figure 5 shows that the convergence of the RMSE of the network during training and represents the number of iterations performed by the network was achieved until the target RMSE. With epochs near to 25000 iterations, the final neural network structures give a good estimate of the plant biomass production. And so, in the final BP network, 25000 epochs were used. Because it is possible to obtain a near perfect fitting, the error on the training set is always decreasing with increasing complexity. Here, the testing error was also decreased continuously with increasing the epoch size from 1000 to 100000 and it does not show any changes at epochs higher than 25000. One of the important problems related to ANN is over fitting which was not occurred here. At this case, the error on the testing set at first decreases as the fitting improves, but it increases again when the epochs increase.
The lower training and testing RMSE and the best distribution of the predicted error were found for learning rate and momentum values of 0.2 and 0.3, respectively. This indicates that using these values, the necessary weight adjustments are appropriate.
Finally, the results showed the lower the training and testing RMSE the better distribution of the predicted error for learning rate and momentum values of 0.2 and 0.3, respectively.
The neural network models have the ability to re-learn and they improve their performance if new data are available. This ability is important to make a powerful model based on the several years data. Thus, characteristics of ANN such as high performance, accuracy of the prediction and re-learning showed that this approach has a high potential to use in weed management programs.
ACKNOWLEDGMENTS
The authors would like to thank the University of Tehran for funding this research work.