Research Article
Consumer-Grade GPS Receiver Measurement Accuracy in Varying Forest Conditions
Forest Engineering, Resources and Management, Oregon State University, Corvallis, Oregon, (541) 737-4009, USA
Consumer-grade Global Positioning System (GPS) receivers are typically sold for $400 or less and can provide an affordable technology for locating features on the earth's surface. GPS receivers were once considered to be primarily for use in settings that featured no forest canopy or other overhead features that obstructed satellite communication. Increased satellite availability and recent improvements in GPS receiver hardware and software have created conditions where GPS receivers are now being used for a variety of natural resource applications in forested environments (Wing, 2008b). The measurement accuracy and reliability of six consumer-grade GPS receivers were tested including several that were recently introduced as commercial products. The testing was conducted in three forested test courses that represented a range of conditions. Measurement were synchronized so that all receivers collected data simultaneously at each test course. Primary objectives were to examine the measurement accuracies of the six consumer-grade GPS receivers and to determine whether accuracies varied at the forested test courses. A secondary objective was to determine whether a single GPS measurement would be as accurate as positions that were averaged from repeated measurements. While several previous studies have evaluated the influence of repeated measurements (Bolstad et al., 2005; Wing et al., 2005; Wing, 2008a), studies have typically focused on a maximum average of 60 measurements. The influence of averages that were taken from groups of 10, 60 and 120 points were evaluated.
Wing (2008b) provides a succinct description of GPS operations and assumptions that incorporates a natural resource perspective. This information is summarized below but readers are also encouraged to see Leick (2004) and Van Sickle (2008) for additional detail. All GPS receivers rely on satellite communication in order to determine a position on the earth's surface. GPS satellites are continually sending signals in the microwave portion of the electromagnetic spectrum that contain information about the transmitting satellite and the positions of other satellites. A GPS receiver uses information within the satellite signal to calculate a distance, known as a range, to the satellite. Ranges are based upon the time it takes a signal to reach a receiver and the speed of light constant. A longitude and latitude position can be derived through simultaneous communication with three satellites through the process of trilateration. The calculation of elevation in addition to longitude and latitude requires communication with four satellites. A GPS receiver depends on an open pathway between its antenna and a satellite in order to receive a reliable signal. Anything that obstructs a signal pathway to a receiver antenna, such as canopy, landforms, or vegetation has the potential to prevent a range from being calculated. In addition, smooth surfaces such as metallic structures or automobiles have the potential to redirect a signal toward a receiver, creating a condition known as multipathing. Multipathing results in an error in the calculation of a range and consequently, an error in position determination.
GPS receivers are typically grouped into one of three categories or grades: consumer, mapping, and survey. Consumer-grade GPS receivers are the most affordable and vary in price from approximately $100 to $1000 depending on capabilities. Consumer-grade GPS receivers are usually limited in capability and have few controls for users to improve measurement accuracy and reliability. Mapping-grade GPS receivers vary in price from approximately $1,000 to $12,000 depending on hardware and software. Many mapping-grade GPS receivers enable users to set minimum quality thresholds, such as Dilution Of Precision (DOP) and minimum mask angle that take into account the strength of satellite geometry and communication. Mapping-grade GPS receivers also typically allow differential correction of calculated positions. A differential correction is based on the position of a stationary GPS receiver. As calculated positions based on satellite signals vary from a stationary GPS receiver position, the differences can serve as a time dependent correction factor that can be applied to other GPS receivers that are operating nearby. The corrections can be applied in real-time through radio or satellite communication. The corrections can also be applied following data collection in an office setting by linking to an established GPS base station. GPS base stations that make corrections available through Internet connections are available throughout the U.S. and in many other countries. Differential corrections typically improve GPS receiver measurements; in some cases the improvements can be dramatic. Mapping-grade GPS receiver accuracies are typically in the 1 to 10 m range depending on environmental conditions.
Survey-grade GPS receivers range from approximately $12,000 to $50,000 depending on configuration. Survey-grade GPS receivers, as with mapping-grade, feature the ability to establish minimum satellite communication quality standards. A primary difference from mapping-grade receivers is the enhanced hardware and software often associated with survey-grade receivers. Given prolonged measurement periods of two hours or more, survey-grade GPS receivers can derive locations that can be accurate to within a cm of actual position. This heightened measurement capability makes survey-grade GPS receivers capable of establishing measurement control points for engineering and other applications that require highly accurate measurements. The relatively high cost of survey-grade GPS receivers, however, is prohibitive for many organizations. In addition, some GPS users are hesitant to take survey-grade receivers into forested terrain, for fear of damaging equipment that is expensive to replace.
GPS receivers are now used for a variety of applications that require location measurements. Examples of typical GPS applications include determining the spatial variability of soil characteristics (Yasrebi et al., 2008), providing measurement needs for building construction (Adewumi and Olorunfemi, 2005) and tracking land degradation (Al-Quraishi, 2004). More novel GPS applications include positional tracking to reduce vehicular collisions with animals (Zahrani et al., 2011), establishing positional locations for camera images (Hsien-Chou and Pao-Tang, 2009) and associating elevations to fig growing areas to investigate influences on food quality (Simsek and Kuden, 2010).
Consumer-grade GPS receivers have received increased attention during recent years as a potential measurement tool for forestry and natural resource applications. A primary impediment for these applications has been forest canopy and topography that blocks or obstructs satellite signals from reaching a receiver. An increasingly robust collection of satellites has produced greater opportunities for satellite reception with promise of additional satellite numbers and signals becoming available in the near future. In addition, GPS users have traditionally been encouraged to collect multiple measurements at a location and to the use the average measurement coordinates to determine a position. Consumer-grade GPS measurements typically default to collecting a measurement every second. Although a general rule of thumb has been to collect and average 60 measurements (meaning that a consumer-grade GPS receiver operator would spend approximately one minute at a location with sustained satellite communication), there remains little evidence that 60 measurements is a reliable standard to improve measurement quality.
Several recent studies have examined consumer-grade GPS receiver accuracies in varying forested settings. In some cases, studies have also considered the influence of point averaging (collecting multiple measurements at a single location) on measurement accuracy. Bolstad et al. (2005) tested the measurement accuracy of two consumer-grade GPS receivers under varying forest canopy and reported average measurement errors of 6.5 and 7.1 m. They also examined the influence of the number of positional fixes on measurement accuracy. One of the two consumer-grade GPS receivers showed a significant decrease in errors when a larger number (up to 300) of positions were used to determine a location. Wing et al. (2005) examined six consumer-grade GPS receivers in several forest settings and found that measurement accuracies varied among the receivers. The best receivers were capable of measurements within 5 m of actual position in open sky settings, within 7 m in young forest conditions and within 10 m under dense canopy. The average of 25 one-second measurements was used to determine GPS-derived positions. Wing and Eklund (2007) examined the performance of a consumer-grade GPS under dense forest canopy and determined average positional errors of 8.9 m. The average of 60 measurements was used to determine positions. Wing (2008a) tested the accuracy of six consumer-grade GPS receivers in several distinct forest settings. The most accurate receiver was within 2 m from true position under open skies, within 3 m in a young forest setting, and within 9 m under closed canopy. The affect of the number of measurements on positional accuracy was tested using groupings of 1, 30 and 60 measurements. Two of the receivers demonstrated a statistically significant correlation between positional accuracy and the number of averaged positions.
Danskin et al. (2009) tested two consumer-grade GPS receivers in a hardwood forest during leaf-on and leaf-off conditions. Root Mean Square Error (RMSE) values at a 95% confidence interval were reported for the consumer-grade receivers and ranged from 12.8 to 38.7 m in leaf-on conditions and from 8.2 to 36.8 m in leaf-off conditions. Andersen et al. (2009) tested a single consumer-grade GPS receiver at seven forested sites in Alaska. RMSE values for the consumer-GPS receiver ranged from 3-7 m across the sites and no examination of the affect of repeated measurements was conducted. Wing (2009) evaluated the performance of six identical consumer-grade GPS receivers in open and urban forest settings. Measurement accuracies for all receivers combined were 1.8 m in an open field and 3.8 m at an urban forestry course. Wing (2009) also tested the influence of 1, 10 and 30 point averages and found no statistical correlation with measurement accuracy. Bettinger and Fei (2010) tested the accuracy of a single consumer-grade GPS over the course of a year at three forested courses. Significant differences were observed between the measurements collected at each test course. Average RMSE was reported to be 11.9 m at a young pine course, 6.6 m at an older pine course and 7.9 m at a hardwood course. All GPS measurements represented the average of 50 position fixes.
The performance of six consumer-grade GPS receivers was examined including several that were recently introduced. The testing took place in three forested test courses across a range of conditions. The primary objectives were to determine the measurement accuracies of the six GPS receivers and examine whether accuracies varied at the test courses. A secondary objective was to test whether single GPS measurements were as accurate as positions averaged from repeat measurements. To this end, the influence of averages that were taken from groups of 10, 60 and 120 points was evaluated.
Three test courses were established in a western Oregon forest in three distinctly different settings. An open sky test course was established within a forest clearing and had a generally unobstructed view of the sky overhead but had lodgepole pine scattered on the perimeter of the clearing. A second course was established within a young forest stand, comprised of predominately 20-30 year old lodgepole pine and Douglas-fir but also with some deciduous trees (bigleaf maple and alder). A third course was established within a mature forest stand characterized by mainly 50-60 year old grand fir but also included some Douglas-fir.
Each test course was established on areas that had ground slopes of less than two percent and included seven test stations established in a similar geometric pattern. The test stations were located at even intervals along a circle with a radius of five meters around a central test station. The five meter diameter radius was selected so that GPS operators could stand at any test station and talk with operators at other test stations. Test stations were established by driving a 1.6 cm diameter iron rod 0.6 m into the forest floor. Two survey-grade GPS receivers collected control measurements in a forest clearing within 100 m of the test courses. A closed traverse with a digital total station from the control measurements associated coordinates to all test stations.
Six consumer-GPS receivers produced by three different manufacturers were tested (Table 1). The study intent was to only involve units that were capable of measurement averaging as it was of interest to test the influence of different averages on measurement accuracy. Measurement averaging allows a GPS user to save a single measurement that is based on the average coordinates of multiple measurements. While five of the test receivers that we tested were able to automatically average points, a sixth (Endura Sierra) was not. This particular receiver was included since a previous version had performed well in an earlier study and was able to average measurements (Wing, 2008a).
Table 1: | Consumer-grade GPS receivers involved in testing |
* Manufacturer suggested retail price |
Fig. 1: | Consumer-GPS receivers involved in measurement testing: Dakota 10, Earthmate PN-60 Endura Sierra, GPSMAP 62, GPSMAP 78 and Oregon 450 |
It was discovered that the Endura Sierra was unable to average measurements after the receiver was delivered by the manufacturer. Nonetheless, the receiver was included in testing since it was available. The Endura Sierra was included in testing single-second measurements but was not included in point averaging comparisons in this study.
Each of the receivers has a compact case although there were some pronounced differences in form factor between the receivers (Fig. 1). During all testing, each receiver was mounted on top of a 1.4 m tall wooden staff using a rubber fastener- metal was avoided to reduce multipathing affects. Each receiver was associated with a single operator during all testing and all operators faced south during data collection in order to maximize satellite reception.
A consistent data collection protocol was followed at each measurement course. Each receiver was placed at one of the measurement test stations and a verbal count was used to synchronize data collection among all receivers during the collection of 1, 10, 60 and 120 measurements at each station. For multiple measurements, the receiver stores the average measurement. This sequence was repeated three more times before the receiver was moved to the next station. This pattern continued until all seven stations had been visited at each course for a total of 336 final measurements being recorded by each receiver. The lone exception to this protocol was the Endura Sierra which was not able to calculate average measurements. All GPS receiver measurements were collected on Nov. 20, 2010 under cloudy skies.
Statistical analysis: Average measurement errors were derived by calculating the straight-line distance between each receiver measurement and the coordinates of the test station that the receiver occupied during a measurement. Measurement Standard Deviations (SD) were also calculated as a means of examining receiver measurement reliability. Analysis of variance (ANOVA) statistical procedures were applied to examine whether measurement error differences among receivers were meaningful. Measurement errors were adjusted using a natural logarithmic transformation in order to approximate a normal distribution so that normality assumptions were met for ANOVA procedures.
The average measurement errors and deviations varied among the GPS receivers at each of the test courses (Table 2). Average error and SD were 2.5 and 1.6 m, respectively at the open sky course, 5.5 and 4.2 m at the young forest course, and 3.8 and 2.1 m at the mature forest course. The GPS receiver with the overall lowest average error was the GPSMAP 78 with a 2.5 m average. The GPSMAP 78 also had the lowest overall SD (2.0 m). In addition, the GPSMAP 78 also had the lowest average error when comparing receivers separately at each measurement course. The GPSMAP 78 had an average error of 1.5 m at the open sky course, 3.4 m at the young forest course and 2.6 m at the mature forest course. The SD of the GPSMAP 78 was either the lowest or tied for the lowest among all receivers at each of the measurement courses when all data were combined. The Oregon 450 had the same SD (0.9 m) as the GPSMAP 78 at the open course whereas the Earthmate PN-60 had the same SD as the GPSMAP 78 (2.6 m) at the young forest course. The measurement performance of the receivers besides the GPSMAP 78 was uneven. The Oregon 450 was the second best performer at the open course with an average error of 1.8 m (SD 0.9). Two other receivers, the Endura Sierra and GPSMAP 62 had errors that exceeded 3.7 m or more at the open course. The Earthmate PN-60 had the second best accuracy at the young forest course (4.6 m) whereas three other receivers (Dakota 10, GPSMAP 62 and the Oregon 450) had errors that exceeded 6.0 m. The Dakota 10 was the second best performer at the mature forest course with an average accuracy of 3.2 m (SD 1.9). The Endura Sierra and GPSMAP 62 had errors of 4.9 and 5.0 m, respectively, at the mature forest course.
All grouped receiver measurements were tested together and subsequent tests considered each receiver separately to determine whether the number of averaged measurements influenced accuracy for all measurements combined and for the measurements at each course. There were no significant differences for the number of measurements when all receiver measurements were combined, or when each receivers measurements were examined separately from all other receivers at each measurement course. Results for combined receiver measurements, however, did come much closer to statistical significance at the open sky course (p = 0.15) as opposed to the young (p = 0.87) and mature (p = 0.80) forest courses. In comparing receivers separately, only the Earthmate PN-60 was found to have statistically significant differences between the number of averaged measurements (p = 0.01). A Tukey range test identified the one second and 60 sec measurements for the Earthmate PN-60 as being significantly different but only at the open sky course. The average error for a one second measurement was 3.2 m while a 60 sec measurement had an average error of 1.7 m for the Earthmate PN-60 at the open sky course.
Table 2: | Average measurement errors (m) and standard deviations (SD) for consumer-grade GPS receivers. |
Testing results revealed the GPSMAP 78 to be the most accurate and reliable performer among the six tested receivers. This assessment includes whether the measurements for all three measurement courses were analyzed together, or whether the measurements were analyzed separately for each course. GPSMAP 78 average errors were 1.5 m at the open sky course, 3.4 m at the young forest course and 2.6 m at the mature forest course. The GPSMAP 78 and all but one other receiver had higher measurement errors at the young forest course in comparison to the mature forest course. These differences are likely due to the presence of several deciduous trees near the young forest course. Deciduous leaves typically have a greater potential to obstruct satellite signals than do branches and needles associated with coniferous trees. In addition, although the tested GPS receivers could not track DOP values, mission planning revealed that potential DOP values would be at their highest (worst) levels during data collection at the young forest course. These high DOP values might also have influenced the young forest measurement results. Nonetheless, the measurement errors that were determined for the GPSMAP 78 are at least comparable to one previous study that included measurements under forest canopy (Wing et al., 2008) but represent improved accuracies reported by several other previous studies (Bolstad et al., 2005; Wing et al., 2005; Wing and Eklund, 2007). The increased accuracies are a noteworthy trend and it will be interesting to see if accuracies continue to improve in future years. The predecessor to the GPSMAP 78 was the GPSMAP 76 and had been included in a previous study (Wing, 2008a). The GPSMAP 76 receiver was a solid performer in the previous study but was not exceptional. Beyond the GPSMAP 78, the performance of other GPS receivers was less consistent across the measurement test courses with no receiver standing out in terms of measurement accuracy.
The influence of point averaging on measurement accuracy was examined and compared 10, 60 and 120 averaged measurements to that produced by a single point. Previous studies have considered 60 point averages and have found either no influence (Wing, 2008a) of point averages on measurement accuracy or that only a subset of the GPS receivers demonstrated a significant influence (Wing, 2009). Only a single peer-reviewed study was discovered (Bolstad et al., 2005) that has reported on the influence of more than 60 points on measurement accuracy. Bolstad et al. (2005) found that one of two tested consumer-GPS receivers had significantly improved accuracies when as many as 300 data points were collected. It was anticipated that the average of 120 measurements might also demonstrate improved measurement accuracies but not one of the GPS receivers tested showed a significant influence of 120 averaged measurements in comparison to other measurements. The lone significant finding for the influence of point averaging involved the Earthmate PN-60 and indicated that a single measurement was different from the average of 60 points at the open sky course.
It may be that improvements in future GPS technologies including both satellite communications and consumer-grade GPS hardware increase the spatial resolution of repeated measurements but no statistical evidence was found to suggest that 120 average measurements substantially improves measurement accuracy. Future studies might consider prolonged observations (i.e., 300 or more averaged points) to determine a threshold for a minimum number of averaged points that improves consumer-grade GPS accuracies.
During testing, a single operator was assigned to each consumer-grade GPS receiver and was asked for their impressions of the receiver they used following testing. These impressions and other observations of each receiver are included below.
Dakota 10: The Dakota 10 was the most compact among the GPS receivers tested. The Dakota 10 does include an on/off button on its side but relies on a touch screen for primary operator input. A stylus was used during testing as conditions were relatively cold and the touch screen input areas were difficult to manipulate with gloves. The operator of the Dakota 10 reported that the touch screen interface was user friendly and intuitive to control, although it would be an improvement if point (waypoint) measurement tool icons that control features such as point averaging and point storage could be located more closely to one another. The Dakota 10 collected data throughout testing without pause or interruption, a quality that was sometimes absent in other receivers that were tested.
Earthmate PN-60: The Earthmate PN-60 was among the fullest featured of the six consumer-grade GPS receivers tested and is a recent update to previous models. The ability to customize data collection parameters and other choices was very robust. It was discovered that four of the 336 measurements that we collected with the Earthmate PN-60 were off by several hundred meters and that each of the points was a single (one-second) measurement. These four points were removed from analysis. The cursor mechanism for collecting points can sometimes save a point that is referenced to the cursor position on a supporting basemap rather than a current position if a user is not careful of initiating point storage. Although informed of this possibility prior to testing and subsequently instructing the Earthmate PN-60 operator to avoid this problem, it may be that this occurred during testing procedures. It is noted that a previous version, the Earthmate PN-20, was part of an earlier study (Wing, 2008a) and was also subject to measurements that were considerably distant from actual location. The Earthmate PN-60 was the only receiver tested that required a proprietary cable in order to download data onto a PC. The buttons on the Earthmate PN-60 are conducive to use with gloves, making it a good candidate for cold weather use. The Earthmate PN-60 lost satellite fix on two occasions during testing and was momentarily unable to collect data.
Endura sierra: As previously mentioned, the Endura Sierra was included in this study since a previous version had favorable measurement results (Wing, 2008a). Unlike the previous version, it was unexpectedly found that the Endura Sierra was unable to automatically collect point measurement averages but the receiver was included in testing since the receiver was available. The Endura Sierra is unique in that its screen is touch sensitive but controls are also still available to manipulate menu choices. The Endura Sierra however seemed slow to respond to user input and use of the touch screen sometimes resulted in the receiver locking up and not responding to operator input. This condition typically required that the receiver be restarted. Downloading data from the Endura Sierra also resulted in the mouse on the host PC freezing.
GPSMAP 62: The GPSMAP 62 is a recent update of the GPSMAP 60 but still bears a striking resemblance to a walkie-talkie given its protruding antenna located on top of the receiver. The GPSMAP 60 was the top performer in an earlier consumer-grade GPS accuracy test (Wing, 2008a). The screen registration of the GPSMAP 62 automatically switched to a night-time mode coincident with sunset during testing (sunrise and sunset times can be viewed within one of the menu groups within the GPSMAP 62) that made reading the screen easier as darkness fell. The GPSMAP 62 operator reported that the receiver was comfortable to operate even when wearing gloves. The GPSMAP 62 stopped collecting measurements at least five times during testing and had to be restarted in order to continue.
GPSMAP 78: The GPSMAP 78 is a new release of a popular GPS receiver (GPSMAP 76) but its unusual physical layout remains the same: the primary user input buttons are located above the screen which creates the potential for the hands of an operator to obscure the screen. While this layout might seem undesirable, the operator of the GPSMAP 78 remarked that it does enable relatively comfortable single-handed control of the user input buttons. However, the input interface was described as being slightly awkward and the receiver quit collecting measurements during testing on at least five occasions. Similarly to the GPSMAP 62, the GPSMAP 78 switched to a night time screen registration mode at sunset that made reading the screen much easier in darkness, The GPSMAP 78 has two other qualities that also distinguish it from other tested receivers. First, it supports a relatively inexpensive ($35) external antenna with a magnetic base. The external antenna plugs into a gasket-covered port in the back of the GPSMAP 78 that does not appear to be very robust. The antenna base can be affixed to a vehicle or other metallic surface to offer more measurement versatility. Previous testing of mapping-grade GPS receivers indicates that external antennas provide increased accuracy and in some cases an enhanced ability to receive and sustain satellite communication (Wing et al., 2008). Some brief testing of the influence on the external antenna on measurement accuracy was conducted. It was discovered that accuracy actually decreased. Second, the GPSMAP 78 is reported to float when placed in liquid. Although this capability was not confirmed, this would be a very desirable quality for potential users that work on or near water resources.
Oregon 450: The Oregon 450 is very similar to the Dakota 10 in terms of use in that it also relies on a touch screen and appears to have nearly identical control icons. The Oregon 450 operator also reported that the user interface was relatively easy to operate and operations such as deleting saved points were straightforward. The chassis of the Oregon 450 is larger than the Dakota 10 and was reportedly physically comfortable to handle and operate. As with the Dakota 10, a touch screen stylus was relied upon to indicate user inputs. The Oregon 450 did seem to hesitate slightly at times to respond to user input but recorded data throughout testing without interruption.
Consumer-grade GPS receivers offer an affordable technology for natural resource measurements. Testing in this study considered six consumer-grade GPS receivers and included some models that have only recently become available. Testing results indicated that the best performing consumer-grade GPS was capable of measurements within 2 m of actual locations in open settings and measurements within 4 m of actual position when operating under forest canopy. The acceptability of these measurement accuracies for particular purposes depends on project goals and applications. For a variety of natural resource applications that involve basic mapping and inventory activities, accuracies within 4 m are likely adequate. Other applications that require greater accuracies should consider mapping-grade GPS receivers or other measurement technologies.
The author gratefully acknowledges the following students for assisting in data collection: Brady N. Do, Chad T. Gilbreath, Trisha A. Hubbard, Kelsey M. Lee, Mark D. Mathre, and Jake R. Thompson.