Uncertainty [MGD Sections]

UNFCCC decisions and requirements
IPCC good practice guidance
Relationship to UNFCCC
GHGI coverage, approaches, methods and tiers
Design decisions relevant to national forest monitoring systems
Land cover, land use and stratification
Forest reference emission levels and forest reference levels
Quality assurance and quality control
Guiding principles – Requirements and design decisions
Estimation methods for REDD+ activities
Integration frameworks for estimating emission and removals
Selecting an integration framework
Activity data x emission/removal factor tools
Fully integrated tools
Practical considerations in choosing an integration tool
Guiding principles – Methods and approaches
Remote sensing observations
Coarse resolution optical data
Medium resolution optical data
High resolution optical data
L-band Synthetic aperture radar
C-band and X-band SAR
Global forest cover change datasets
Ground-based observations
National forest inventories
Auxiliary data
Guiding principles – Remote sensing and ground-based observations
Activity data
Methods for estimating activity data
Maps of forest/non-forest, land use, or forest stratification
Detecting areas of change
Additional map products from remote sensing
Estimating uncertainty of area and change in area
Estimating total emissions/removals and its uncertainty
REDD+ requirements and procedures
Reporting forest reference emission levels and forest reference levels
Technical assessment of forest reference emission levels and forest reference levels
Reporting results of REDD+ activities
Technical analysis of the REDD+ annex to the BUR
Additional advice on REDD+ reporting and verification
Guiding Principles – Reporting and verification of emissions and removals
Financial considerations
Country examples – Tier 3 integration
Use of global forest change map data
Relative efficiencies
Developing and using allometric models to estimate biomass

Record Keeping [MGD Sections]

Integration + Estimation [MGD Sections]

Ground Based Observations [MGD Sections]

4.1.7   Global forest cover change datasets Previous topic Parent topic Child topic Next topic

Global maps of land cover, including tree cover, are readily available(1). Work led by the University of Maryland (UMD) (Hansen et al., 2013)(2), provides tree cover, and cumulative tree cover gains and annual losses. The UMD maps are produced on a consistent basis and are updated annually. While there are plans to produce annual tree cover maps, currently the maps do not track multiple changes in tree cover through time, meaning that regrowth following loss is not tracked. This limitation needs to be considered when deciding to use the data to produce emissions and removals estimates.
The maps have 30m x 30m spatial resolution and are based on Landsat data. Depending on national circumstances they may provide countries with change maps if suitable maps do not already exist for their territories. This section provides advice on the use that can be made of this type of data for REDD+, mainly for activity data estimation, and discusses the issues that can arise in doing so.
Accuracy of global products varies regionally due to factors including differential sensitivity of detection at biome and ecoregional scales; change dynamics (e.g. at smallholder to industrial scale), and data richness (affected e.g. by cloud cover; better quality observations, and more observations will improve accuracy). In general, use of global maps will produce activity data estimates with lower accuracy and precision than are attainable by national mapping of comparable quality, because the latter can be tuned to national forest definitions and make use of knowledge and auxiliary data available at the national level. However as set out in Chapter 5, Section 5.1.5, correcting for estimated bias at a given level of precision depends on the combination of mapping (whether global or national) and reference data. Because of this, when correcting for estimated bias, lower accuracy associated with global datasets can be compensated by using more ground or other reference data, and global datasets may enable progress to be made until national mapping capacity is established. Global datasets and national mapping capacity can therefore be seen as complementary. There are country examples demonstrating the joint use of global data and national expertise (Box 18; Appendix D).
Whether using a national or global map, the process is the same, namely:
  • decide the precision required. This is likely to depend on the policy context including expectation of results-based payments (Chapter 2, Section 2.3). Discussion between technical experts and policy colleagues on what can be achieved cost effectively may be needed.
  • obtain an initial, exploratory reference data set(3).
  • based on the results of using the exploratory reference data and the map to indicate the precision obtainable as a function of sample size, gather additional reference data to correct for estimated bias and obtain the precision required.
Relative efficiency is a measure of the improvement in precision obtainable by using map data and reference data in combination. Box 19 shows how relative efficiency is defined and draws some overall conclusions about typical relative efficiencies obtainable from national and global mapping
Consideration of relative efficiency can help decide upon cost effectiveness (e.g. the cost of collecting more reference observations versus establishing a national mapping capability, and costs of establishing the relationship between global maps and national forest definitions). National assessment of the relative advantages of global and national maps to generate national level estimates of forest area and change are also related to:
  • preferences for national ownership of the process, to respond to technical developments
  • the need for information on the drivers of forest and land cover change, particularly when this information is required for results-based payments
  • whether national mapping capacity already exists – countries with mapping capacity are likely to want to use it
  • national needs for a land cover map (e.g. related to forest definition and land cover classifications, for integration with domestic planning).
The relationship between global data and the national forest definition is important and in comparing national estimates and global products the user should ensure that both products cover the same geographic extent and time period(4) and that the forest areas and area changes derived from the global data correspond as nearly as possible to the national definition. Common inconsistencies between global data and national forest definitions are related to the minimum canopy cover thresholds(5), detailed consideration of land use (e.g. the status of shifting cultivation, oil palm or other plantations), the minimum size of forested areas, and the minimum tree height required by the definition. The global maps available from UMD indicate three main characteristics: (i) percentage crown cover for vegetation over 5 metres in height(6),(7) (ii) tree cover loss (areas where tree cover has been removed entirely) and (iii) tree cover gain (areas where tree cover has been established where previously there was no tree cover).
Rules to map the extent of the minimum percentage crown cover(8) specified in the national forest definition could be implemented automatically in the case of the UMD data, because percentage crown cover is a pixel-level attribute. However some studies indicate given crown cover (say 30%) in the national forest definition may not correspond to 30% as estimated in the global dataset (Sannier, et al., 2016; McRoberts et al., 2016). This would necessitate an adjustment or compensation, using either auxiliary data to establish the relationship, or by treating the adjustment as part of the bias correction via the reference data set. Other criteria to define forest, such as a different height specification, or specific land use requirements, imply the need for supplementary national mapping (with significant associated cost) to correct for areas either erroneously included or excluded by the global maps. To help achieve this, the NFMS could identify areas that would otherwise meet the forest definition, but are under predominantly agricultural or urban land use, and identify ecosystems where trees do not meet the height definition.
Accommodating the minimum area, tree height, width, and canopy cover requirements of a forest definition is non-trivial with pixel-based maps, whether global or nationally produced. Although object-based and GIS methods may be useful, pixel elimination and aggregation rules(9) must be applied for consistency with the applied definition, which may degrade the spatial resolution of the map and involve complicated averaging methods to estimate percent canopy cover for the aggregated units. In practice, straight-forward and easily implemented techniques to do this are not readily available(10).
Global map products indicating areas where tree cover has been removed entirely can be used to help map forest/non-forest land cover change(11). However, areas where complete overstorey removal is indicated will not necessarily correspond to deforestation as a process of change in land use in accordance with the national forest definition, because:
  • deforestation, consistent with the national forest land definition, entails land use change and occurs when areas previously meeting the forest definition fall below the minimum tree cover, height or area thresholds without prospect of recovery. This is not necessarily the same as complete removal of tree cover.
  • tree cover may fall temporarily to zero (or below the minimum threshold specified in the national forest definition) because of harvest or natural disturbance (e.g. fire, wind, disease or landslides), but this does not indicate a change from forest land use if replanting, or natural or assisted regeneration will take place so that forest according to the national definition will be re-established.
Use of global datasets to estimate deforestation therefore needs to take into account factors other than simply using the global analysis of removal of tree cover below the minimum level that is estimated by the global data set classification algorithm. This is likely to require auxiliary information to identify areas subject to harvesting where replanting will take place, and information on the extent of any disturbances, and whether they have been followed by land use change, or not. Time series analysis has the capacity to be extremely helpful. The auxiliary information required should be obtained by interaction with stakeholders via the NFMS or other institutional arrangement responsible for land use. Modifications introduced via auxiliary data need to be treated consistently over time, or significant error may be introduced into mapping and area estimation. UMD provides Landsat image mosaics for the years 2000 and 2015 Opens in new window which could be interpreted by the user to provide a map that includes information on land use change drivers.
Reference observations consistent with the national forest definition can also be used with an unmodified global map. The reference data are used to adjust for estimated bias resulting from map prediction error when using global map products as the basis for estimation, but the amount of reference data needed to achieve given precision is likely to be greater in this case. If the reference data are stratified, e.g. by forest type, accessibility, or biomass quantity(12), strata should be applied consistently over time irrespective of whether national or global map products are being used.
The methods for using reference data, described in Chapter 5, Section 5.1.5, yield area estimates of land classes (e.g. forest, non-forest, forest loss and forest gain) that are adjusted for estimated bias. However these methods are not designed to determine which pixels are misclassified(13). This means that the act of area (or area change) estimation with reference observations does not improve site-specific mapping accuracy (at the level of individual pixels or minimum mapping units). Consequently if maximum possible site-specific accuracy is needed (e.g. for interacting with stakeholders, identifying drivers of deforestation(14), or associating ground-based with remotely sensed data for development of emission/removal factors) it may be better to develop national mapping using classification methods designed for national circumstances(15). Achieving a particular accuracy in either case is likely to require an initial trial followed by additional reference observations sampling or improvements to the classification technique until the desired result is obtained.
The choices are summarized in the decision tree Figure 11 below. Although national mapping should be more accurate and precise, global maps have value as a cross check because differences should be understandable, e.g. in terms of the factors discussed here.
Figure 11: Guidance on the use of global data sets for estimating forest cover and cover change
Considerations at the decision points in the tree are as follows:
Decision Point 1: Is there existing national mapping capacity to apply the methods set out in the MGD?
The methods for generating national activity data from remote sensing are outlined in Chapter 4 and Chapter 5. All cases assume joint use of mapped and reference data.
Decision Point 2: Do you need maximum accuracy at the minimum mapping unit?
Maximum accuracy at the minimum mapping unit may be required for interaction with stakeholders, identifying drivers, associating remotely sensed and ground-based data or nesting of sub-national activities.

Box 18: Use of global data sets – Ethiopia

In the context of the “Implementation of a national forest monitoring and MRV function for REDD+ readiness in Ethiopia” project, the Federal Democratic Republic of Ethiopia has initiated a series of activities to: a) develop a forest definition and a classification system; b) generate land cover change maps and statistics; c) collect data from the field or develop allometric models.
To date, several institutions in Ethiopia have carried out preparation of land cover maps including the Ministry of Environment, Forests and Climate Change (MEFCC, 2013), the Ethiopian Mapping Agency (EMA 2003, 2008 and 2013) and the, Central Statistical Agency of Ethiopia (CSA, 2008). However, these products lack the accuracy or time dimension to provide statistics and spatial analysis of forest change over time needed for the establishment of FRL. The Global Forest Watch (GFW) product (Hansen et al., 2013) was used as a first step to indicate where potential losses and gains within forest lands have occurred at the national scale.
In the absence of reliable data sources indicating areas of changes at the national level, a preliminary training dataset was generated automatically from the GFW product. The GFW product was down sampled to 3x3 pixel kernel to reduce the inclusion of potentially false classifications in the training datasets. The resulting product was randomly sampled with 300 points for 3 classes (loss, gain and no change).
The points for losses and gains were carefully assessed by national remote sensing experts to ensure the samples were an accurate representation. Visual assessment using very high resolution imagery available in the Google Earth, Bing Maps, and Here maps repository was performed through the Collect Earth interface. Ethiopian remote sensing experts identified additional training data in order to meet the national definition of forest (i.e. Height: 2 meters; Canopy Cover: 20%; Area: 0.5 hectares), which differs from the definition in the global product (i.e. Height: 5 meters; Canopy Cover: 25%; Area: 0.09 hectares). Classification of the image involved compiling the spectral signature for all the training points, creating a model from this spectral library and applying the model to the entire imagery. Two models for supervised classification were tested, the CART algorithm (Breiman et al., 1984) and the Random Forest algorithm (Breiman, 2001). After the first classification, the training datasets were improved by visually assessing zones of obvious false change, stable classified as change and missed changes and change classified as stable. The training sites were added to the misclassified locations for the correct class. The new sites were entered in the spectral library with appropriate classification. The classification process was iteratively repeated by carefully checking the batch of results.
The processing chain, from classification of the change, iterative improvement of the training data, and export of the results was performed in the Google Earth Engine API, with the script available here Opens in new window(16) .
The accuracy of final change product change was assessed using the methodology described in Olofsson et al., 2014.
Manual cleaning was finally performed (filtering out of zones of change to match the national MMU=0.5 ha ~5 pixels and manual delineation of mis-classified zones) using the 2013 MEF land cover map to filter out loss detected to occur on the forest mask.
The GFW global data set provided Ethiopia with a starting point for identifying areas of change, however the land cover and land use dynamics in Ethiopia are extremely complex and not fully captured by the global product. Therefore, inputs from national experts was critical for a robust classification in respect of possible errors of omission and commission. Global datasets will likely continue to inform NFMS; however national input is needed to facilitate ownership of officially reported data and statistics.

Box 19: Relative efficiency

The ratio between the variances of the direct area estimate (based only on reference data) and the variances of estimates that rely on maps as auxiliary information gives relative efficiency (RE):
Equation 3
The same reduction in variance (i.e. increase in precision) could also be achieved by increasing the size of the sample in the reference data set by a factor of n1 = RE.
Use of the map will be economically efficient if the cost of collecting the additional samples is greater than the cost of using the map in the project, given by:
Equation 4
where n is the original sample size of the reference data, p is the cost of acquiring each additional sample observation and M is the cost of producing the map. The break-even value of the map depends on the relative costs of producing the map and acquiring sample observations which will vary according to circumstances.
However, a map provides more than an improvement of the statistical precision. Additional information on the location of the forest and other land uses is provided and the map may also be used to carry out other tasks, subject to the accuracy of the map. The value of this additional information must also be taken into account when assessing overall economic efficiency.
Although they may not be representative of all cases examples of relative efficiencies obtained for national and global maps for a limited number of forest types are given in Appendix E, which suggest the following conclusions about the reference data sample size needed to achieve the level of precision required, subject to other constraints such as having sufficient observations within individual activity classes:
  • Use of national rather than global maps can reduce the reference data sample size by 70% to 90% for area estimation, and by 50% to 80% for area change estimation (Table 22).
  • Compared with using with reference data sample alone, use of a national map to estimate forest area can reduce the sample size by over 95% whereas use of global mapping can reduce sample size by 85% to 95%. When assessing change in forest area the same study suggests a 10% reduction in sample size when national mapping is used, and no reduction from the use of global maps However, this is likely to be due to the very low level of change observed during the 2000-2010 period, a 62% reduction in sample size is observed when the national map is used during the 1990-2000 period (Table 23)
  • Use of global maps uncalibrated to local conditions in estimating forest area can reduce sample size by between zero and 35% whereas use of maps calibrated to national forest definition can reduce sample size by 30% to 50% (Table 23).
The relative efficiency of using remotely sensed data depends on many factors , e.g. the type of estimate being made (different activities – area estimates, different emission/removal factors), type and structure of the forest or the properties of the change and type of remotely sensed data. Generally, the more the property being estimated correlates with the remotely sensed data, the higher the relative efficiency is likely to be. This is an area where more research is needed.

This section is based largely on Use of global tree cover and change datasets in REDD+ Measuring, Reporting and Verifying (MRV) (GFOI MGD Module 2, published 28 March 2015 Opens in new window) plus material from the joint GFOI-GOFC-GOLD Expert workshop on using global datasets for national REDD+ measuring and monitoring, Wageningen University, November 2015 Opens in new window.
The UMD data are available via the World Resources Institute Global Forest Watch web-site Opens in new window. Other data sets are the ESA Land Cover Climate Change Initiative at 300m Opens in new window:,the Global Land Cover dataset at 30m from China for 2000 and 2010 Opens in new window, and Do-Hyung Kim et al, (2014). Global, Landsat-based forest-cover change from 1990 to 2000, Remote Sensing and the Environment 155: 178-193. For a review of global data sets see Tsendbazar, N.E., et al. (2014) Assessing global land cover reference datasets for different user communities. ISPRS J. Photogram. Remote Sensing.
Reference data are high quality ground or independent remotely sensed data that can be used with map data or independently to correct for estimated bias and estimate confidence intervals. Use of reference data is described in Chapter 5.
Any global or national map starts at a certain temporal reference point. For example, the UMD map quantifies change annually from the year 2000. In a national context countries may have different starting temporal baseline
Canopy cover thresholds would not necessarily fit with the national definition when the minimum forest area tends to be very different to the Landsat pixel size. In addition, there may also be calibration issues with the global data related to phenology or radiometric quality of the input data.
In the case of the UMD data the mapping is tuned to detect tree cover about 5 m height. Tree height measurement by remote sensing requires interpretation of stereoscopic images or a returned signal from SAR or LIDAR which are not commonly available.
Tree cover gain is defined as detected increase in crown cover over the entire period 2000 to 2012. This information could be used in mapping tree regrowth, replanting and establishment areas over the entire period. Because increase in crown cover over time is harder to detect than abrupt loss, the forest gain product is likely only to be of value when used in conjunction with some form of local field data or a sampling procedure. Therefore the points made here on the use of reference data and relationship to national mapping capacity apply with even greater relevance than in the case of deforestation estimates.
The relative performance of global and national classification methods may be a function of the crown cover threshold used in the national forest definition.
Rules need to be defined when contiguous pixels below the specified threshold should belong to the surrounding forest area or be considered as non-forest. Introducing the concept of Minimum Mapping Area (MMU) can be useful in this context. Rules also need to be defined when characterizing changes. It can be decided that changes below the minimum forest area are considered as long as they aggregate with forest areas that are greater than the set minimum forest area.
The Australian National Greenhouse Gas Inventory approach to reporting land use, land use change and forestry applies such methods.
At the time of writing the UMD data do not yet provide updated global forest cover maps.
Samples corresponding to the same strata drawn from global biomass maps may help in identifying corresponding biomass carbon densities, or for cross-checking biomass estimates from national sampling.
Misclassification of individual pixels or other sampling units corresponding to the reference sample can be determined, although these will generally be a very small proportion of the study area.
Or other REDD+ activities
See Section 4.1 for discussion of image classification.