baseline data infrastructure as mega-file data cube (mfdc) of … · 2014-06-21 · baseline data...

33
Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World Table of Contents 1. Goal ....................................................................................................................................... 1 2. Global Landsat GLS data ...................................................................................................... 2 2.1. Landsat BDI for year 2000 (LBDI 2000) ......................................................................... 2 2.2. Uncompression & Selection of Bands ............................................................................ 3 2.3. Clip Landsat TM Images Irregular Edge Strip ................................................................ 3 2.3.1 In Erdas ................................................................................................................... 4 2.3.2 In Arcgis .................................................................................................................. 4 2.4 Re-project to Geographic Projection (Using Erdas) ....................................................... 5 2.5 Covert DN to TOA reflectance ........................................................................................ 5 2.6 Calculation of NDVI ........................................................................................................ 9 2.7 Mosaicking...................................................................................................................... 9 2.7.1 In Erdas ................................................................................................................... 9 2.8 Scene-Selected Map: Best Set of GLS data ................................................................ 10 2.9 5 Mosaics GLS data for the 7 regions of the world ....................................................... 12 3 Global MODIS 2000 MFDC ................................................................................................. 15 3.1 MODIS BDI for year 2000 (2000-2002) ........................................................................ 15 3.2 Maximum Composition of B1, B2 and NDVI ................................................................. 16 4 References .......................................................................................................................... 33 1. Goal The overarching goal of this document is to clearly outline the processing steps involved in creating a baseline data infrastructure (BDI) for the NASA MEaSUREs funded global food security-support Analysis data @ 30 m (GFSAD30) project. First, we will outline the steps involved in creating Landsat GLS data mosaics of the world for the 1975, 1990, 2000, and 2010 epochs. We will include 4 Landsat bands from the thematic mapper (TM) and enhanced thematic mapper (ETM+) sensors of 1990, 2000, and 2001. These 4 bands are: red, near-infrared, shortwave infrared, and thermal infrared. For the 1975 multi-spectral scanner (MSS), we will include a red and a near- infrared band. An additional band will be normalized difference vegetation index (NDVI).

Upload: others

Post on 24-Apr-2020

16 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

Baseline Data Infrastructure as Mega-File Data

Cube (MFDC) of the World

Table of Contents 1. Goal ....................................................................................................................................... 1

2. Global Landsat GLS data ...................................................................................................... 2

2.1. Landsat BDI for year 2000 (LBDI 2000) ......................................................................... 2

2.2. Uncompression & Selection of Bands ............................................................................ 3

2.3. Clip Landsat TM Images Irregular Edge Strip ................................................................ 3

2.3.1 In Erdas ................................................................................................................... 4

2.3.2 In Arcgis .................................................................................................................. 4

2.4 Re-project to Geographic Projection (Using Erdas) ....................................................... 5

2.5 Covert DN to TOA reflectance ........................................................................................ 5

2.6 Calculation of NDVI ........................................................................................................ 9

2.7 Mosaicking ...................................................................................................................... 9

2.7.1 In Erdas ................................................................................................................... 9

2.8 Scene-Selected Map: Best Set of GLS data ................................................................ 10

2.9 5 Mosaics GLS data for the 7 regions of the world ....................................................... 12

3 Global MODIS 2000 MFDC ................................................................................................. 15

3.1 MODIS BDI for year 2000 (2000-2002) ........................................................................ 15

3.2 Maximum Composition of B1, B2 and NDVI ................................................................. 16

4 References .......................................................................................................................... 33

1. Goal The overarching goal of this document is to clearly outline the processing steps involved in

creating a baseline data infrastructure (BDI) for the NASA MEaSUREs funded global food

security-support Analysis data @ 30 m (GFSAD30) project.

● First, we will outline the steps involved in creating Landsat GLS data mosaics of the

world for the 1975, 1990, 2000, and 2010 epochs. We will include 4 Landsat bands from

the thematic mapper (TM) and enhanced thematic mapper (ETM+) sensors of 1990,

2000, and 2001. These 4 bands are: red, near-infrared, shortwave infrared, and thermal

infrared. For the 1975 multi-spectral scanner (MSS), we will include a red and a near-

infrared band. An additional band will be normalized difference vegetation index (NDVI).

Page 2: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

● Second, we will outline the steps involved in creating MODIS monthly maximum value

composite (MVC) normalized difference vegetation index (NDVI) data.

● Third, we will outline the creation of megafile data cubes (MFDCs) involving Landsat

GLS, MODIS monthly MVC NDVIs, and secondary data for each epoch.

● Fourth, we will outline the creation of MFDCs of time series AVHRR GIMMS data for

1982 to 2011.

The above sets of data will be used as primary baseline data infrastructure (BDI) in the

GFSAD30 project.

2. Global Landsat GLS data Global Landsat BDI for: 4 epochs: 1975, 1990, 2000, 2005, 2010 are summarized in Table 1.

The Table 1 shows the numbers of images, sensor from which they are acquired, and the total

storage volume occupied. These images will be used in GFSAD30 project. The GLS scenes are

band separate, in UTM coordinates, WGS-84 datum, are distributed in GeoTIFF format, and are

compressed using tar and gzip / bZip. Collectively, these datasets provide consistent

observations of global, orthorectified, leaf-on, “cloud free” data (Gutman, et al., 2008).

Epoch Scenes Size* ETM+ TM MSS ALI

2010 8453 1.5TB 3719 4734 n/a n/a

2005 9375 1.64TB 7087 2288 n/a n/a

2000 8755 2.18TB 8755 n/a n/a n/a

1990 7371 975GB n/a 7371 n/a none

1975 7592 250GB n/a n/a 7592 n/a

Table 1. Global coverage of the Landsat GLS Data for epoch 2010, 2005, 2000, 1990, 1975.

The table shows the sensor from which the data are acquired and the required storage volume.

The entire process described in section 2.0 and it’s sub-sections are used to convert the entire

global Landsat GLS images from DN to reflectance using NEX supercomputing platform located

in NASA AMES Research Center. It took a total of 130 hours to convert ~8700 images totally.

2.1. Landsat BDI for year 2000 (LBDI 2000)

First, we will describe the LBDI for the year 2000 (LBDI 2000). The LBDI 2000 will consist of:

● 5 Bands (RED, NIR, MIR, THM, NDVI) will be stored with singed 16 integer. RED, NIR,

MIR is top of atmosphere (TOA) reflectance (%) from Band 3, 4, 5. THM is the

radiometric temperature (K) from thermal bands. NDVI is a ratio of the red and near

infrared reflectance.

● Spatial resolution: 30m pixels or ~0.00026949 degree or 0.09 hectares.

● Projection / Datum: Geographic / WGS 84

● Composite is based on data quality and the maximum NDVI for the overlaping area.

Page 3: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

● A Scene-Selected Map providing geo-location and date-gather information of selected

scenes is available.

● Global domain is divided into 7 regions (Figure 1): North America, South America, Africa,

Europe, Middle East, Asia and Australia.

Figure 1. Landsat tiles over 7 regions over cropland areas and/or potential cropland areas.

Overall, there are 4990 Landsat tiles over croplands/potential croplands (above figure) of ~9000

Landsat tiles (see Table 1) covering the entire terrestrial Earth. The areas where there is

currently zero croplands and/or their chances of occurring in future are about zero (e.g., Sahara

desert, Antarctica), no Landsat images are selected to avoid processing unnecessary images

for cropland studies.

2.2. Uncompression & Selection of Bands

The primary GLS scenes (http://glcf.umd.edu/data/gls/) are band separate, in UTM

coordinates, WGS-84 datum, are distributed in GeoTIFF format, and are compressed

using tar and gzip / bZip. Thereby, in order to process them, uncompressing them and

select the bands we need (see section 2.10) for the GFSAD30 project. The bands we

need to study croplands are selected based on our best knowledge of how best to study

croplands as well as to avoid redundant data from multiple bands to ensure to keep data

volumes low and processing fast.

2.3. Clip Landsat TM Images Irregular Edge Strip

Sometime there is some irregular strips on the edge of landsat TM. It could be a problem when

you mosaic these files together. So we need to clip them with an user-defined AOI (Figure 2).

Page 4: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

2.3.1 In Erdas

Here are the steps for how to clip an image by a shapefile in ERDAS Image Compressor:

1. Display the Landsat TM file in Erdas viewer.

2. Create a new AOI layer (ERDAS Menu button > New > 2D View > AOI Layer). Draw a

polygon which can sepeate the strips area.

3. Export the AOI area to shapefile, named the main basename of Landsat TM data, endswith

shp.

4. Import the Shapefile in to an ERV file by using the `Import Shapefile Wizard'.

5. Use VR Wizard (Vector to Region Conversion) in order to convert the ERV file into the Raster

Image as a Region. Make sure that Output dataset should be your Raster image > Ok

6. Open the imagery in ER Mapper IC

7. Click the CR Wizard (The Clip Region Wizard)

8. In the `Clip Region' dialog box choose the `Create Customized Clip Region' > `Next'

9. Make sure that in the `New Map Composition' dialog the `Raster Region' radio button is

selected >Ok > A `Tools' dialog box will be opened.

10. In the Tools buttons click the Pointer icon (Select / Edit Point mode)

11. Click inside the region, it will highlight the region.

12. In the `Tools' dialog click the `Display/Edit object Attribute' an button with a ABC and a tick

mark.> In the `Map Composition Attribute' dialog, at the lower box name your region (Say "r1")>

Apply

13. Then in the "Tools" dialog click the Save File button. The region `r1' will

be saved in the Imagery > OK to all subsequent messages.

14. Click the Finish button in the Define Clip Region for Current Window dialog

15. Close everything except the image window and ERM IC main window.

16. Click the `CR wizard' again> `Add Clip Region' radio button > Next.

17. Type the exact name of your clip region> Next

2.3.2 In Arcgis

1. In ArcToolbox navigate to Spatial Analyst Tools > Extraction > Extract by Mask. Open this

tool.

2. Specify the raster desired to clip as the input raster.

3. Specify the polygon feature class/shapefile desired to clip the raster.

4. Specify an output raster. Click OK to run the tool.

Page 5: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

Figure 2: Remove Irregular Edge Strip of Landsat TM Images

2.4 Re-project to Geographic Projection (Using Erdas)

Open the image in the ERDAS Imagine Viewer and by using the Raster> Polynimial

Order>Reproject utility for reproject.

The default output format is supposed to be Geotiff or ERDAS Imagine, with Geographic / WGS

84 Projection, output cell size should be 0.00026949 degree (0.09 hectares).

When storing your raster output, you can choose LZW for compression. Also check the

BILINEAR option, bilinear interpolation, which determines the new value of a cell based on a

weighted distance average of surrounding cells.

2.5 Covert DN to TOA reflectance

The Landsat Thematic Mapper (TM) and Enhanced Thematic Mapper Plus (ETM+) bands store

data as digital number (DN) between 0 and 255. Two steps need to do to retrieve the real value:

1).convert the DNs to radiance values using the bias and gain values; 2).converts the radiance

data to TOA reflectance.

The formula to convert DN to radiance using gain and bias values is:

Which is also expressed as:

Where: Lλ = Spectral Radiance at the sensor's aperture in watts/(meter squared * steradian * μm) Grescale = Rescaled gain (the data product "gain" contained in the Level 1 product header or ancillary data record)

in watts/(meter squared * steradian * μm)/DN Brescale = Rescaled bias (the data product "offset" contained in the Level 1 product header or ancillary data record

) in watts/(meter squared * steradian * μm) DN = the quantized calibrated pixel value in DN LMINλ = the spectral radiance that is scaled to QCALMIN in watts/(meter squared * steradian * μm) LMAXλ = the spectral radiance that is scaled to QCALMAX in watts/(meter squared * steradian * μm) QCALmin = the minimum quantized calibrated pixel value (corresponding to LMINλ) in DN. 1 for LPGS products; 1 for NLAPS products processed after 4/4/2004 ; 0 for NLAPS products processed before

4/5/2004 QCALmax = the maximum quantized calibrated pixel value (corresponding to LMAXλ) in DN = 255

Page 6: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

The LMINs and LMAXs are the spectral radiances for each band at digital numbers 0 or 1 and

255 (i.e QCALMIN, QCALMAX), respectively. LPGS used 1 for QCALMIN while NLAPS used 0

for QCALMIN for data products processed before April 5, 2004. NLAPS from that date now uses

1 for the QCALMIN value. Other product differences exist as well. One LMIN/LMAX set exists

for each gain state. These values will change slowly over time as the ETM+ detectors lose

responsivity.

Table 2(a), 2(b) and 2(c) list sets of LMINs, LMAXs, ESuns, and earth sun distance. The first set

should be used for both LPGS and NLAPS 1G products created before July 1, 2000 and the

second set for 1G products created after July 1, 2000. Please note the distinction between

acquisition and processing dates. Use of the appropriate LMINs and LMAXs will ensure

accurate conversion to radiance units. Note for band 6: A bias was found in the pre-launch

calibration by a team of independent investigators post launch. This was corrected for in the

LPGS processing system beginning Dec 20, 2000. For data processed before this, the image

radiances given by the above transform are 0.31 w/m2 ster um too high. See the official

announcement for more details. Note for the Multispectral Scanner (MSS), Thematic Mapper

(TM), and Advanced Land Imager (ALI) sensors.

Table 2a. Spectral range, calibration parameters and exo-atmospheric solar irradiance for TM

Page 7: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

Table 2b. Spectral range, calibration parameters, and exo-atmospheric solar irradiance for

ETM+

The effective at-satellite apparent reflectance (p-unitless) is calculated using spectral radiance (Ri), earth-

sun distance (d) expressed in astronomical units (Au), solar zenith angle () (which is 90 degrees minus

the sun elevation or sun angle when the scene was recorded as given in the image header file), and solar

flux or exatmospheric irradiances (F0) (Markam and Barker, 1985, 1987). This provides the nadir

reflectance from both the surface and the atmosphere above it and normalizes the effects of solar

elevation, and earth-sun distance. This is also referred in literature variously as planetary albedoes or

exatmospheric reflectance. d (unitless) varies between 0.96 to 1.04 (Table 2c), can be obtained from

nautical handbook (see Markham and Barker, 1987), but is often assumed as 1. (degrees) is provided in

the image header file. F0 (W m-2

m-1

) is obtained from Figure 3. (e.g., for Landsat TM bands in Table

2d).

Where:

p = at-satellite exo-atmospheric reflectance (unitless)

L = radiance W / m2 Sr1 m1 or mW/cm2 sr m1 or mW/cm2 Sr m

d = see Table 2c. earth to sun distance in astronomic units obtained based on acquisition date from

standard tables (unitless) (see Markham and Barker, 1987)

ESUN = Mean solar exo-atmospheric irradiances (W / m2 Sr1 m1 or mW/cm2 sr m1 or mW/cm2 Sr

m) or solar flux data obtained from Neckel and Labs (1984) data. For IKONOS, ALI, and ETM+ bands

see data in Table 2a, b derived from Neckel and Labs (1984).

s = solar zenith angle in degrees (which is 90 degrees minus the sun elevation or sun angle when

the scene was recorded as given in the image header file; see sun elevation data from header files of

individual images or can be calculated based on latitude, logitude, and acquisition time).

Page 8: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

Figure 3. Solar Flux (mW cm-2m-1) (Nickel and Labs, 1981).

Table 2d. Solar flux or exatmospheric irradiances (mW cm-2 m-1) for Landsat-5 TM wavebands

(Markham and Barker, 1985).

Band Exatmospheric irradiances (W m-2 μm-1)

1 1946.48

2 1812.63

3 1545.95

4 1046.70

5 211.12

6 10.000

7 76.91

Page 9: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

2.6 Calculation of NDVI

The NDVI is calculated from spectral reflectance measurements acquired in the visible (red) and

near-infrared (NIR) regions, respectively:

2.7 Mosaicking

2.7.1 In Erdas

1. Open Erdas, from the Data Prep icon select Mosaic Images.

2. When the Mosaic Tool window appears, select Add Images from the Edit menu.

3. The Add Images for Mosaic dialog box will appear. Browse to the files containing the tiled

imagery and select each tile in order from the Northwest corner. Change Filename: to *.TIF

or *.img (must be all caps) depending on which format you are using. Each tile has the row

and column number in the naming convention.

4. Highlight the image to add it and click on the Add button.

5. Do this for each successive tile, adding left to right, top to bottom.

6. The Mosaic Tool should look like the screenshot below (Figure 4) , starting with the first tile at

the upper left and the last tile and the lower right.

Page 10: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

Figure 4: Mosaic Tool of Erdas

7. When you have added all of the tiles, in the Mosaic Tool window, select Process -> Run

Mosaic. In the resulting Run Mosaic window check the box Stats. Ignore Value: 0, enter an

output File Name, and select OK. A pop-up window appears displaying the Percent Done of the

data preparation. This step may take some time, depending on the size and number of tiles.

When complete, click OK to close the window.

2.8 Scene-Selected Map: Best Set of GLS data

Shapefile containing geometry of scenes, path-row, acquisition date and sensor-type is

generated.

Page 11: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

Figure 5: Sensor-ID of the GLS data used in mosaic maps for Region Africa

Page 12: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

Figure 6: Acquisition Dates of the GLS data used in mosaic maps for Region Africa

2.9 5 Mosaics GLS data for the 7 regions of the world

Epoch Sensor Band Units Scale Data Type

Fill Value

Range

GLS 2000

ETM Band

3 TOA reflectance (-) 0.001 int16 -9999 0 - 1000

ETM Band

4 TOA reflectance (-) 0.001 int16 -9999 0 - 1000

ETM Bnad

5 TOA reflectance (-) 0.001 int16 -9999 0 - 1000

ETM Band

6 Radiometric temperature (K) 0.100 int16 -9999 0 - 1000

ETM NDVI Normalized Difference Vegetation Index (-)

0.001 int16 -9999 -1000 - 1000

GLS TM/ETM Band TOA reflectance (-) 0.001 int16 -9999 0 - 1000

Page 13: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

2010 3

TM/ETM Band

4 TOA reflectance (-) 0.001 int16 -9999 0 - 1000

TM/ETM Bnad

5 TOA reflectance (-) 0.001 int16 -9999 0 - 1000

TM/ETM Band

6 Radiometric temperature (K) 0.100 int16 -9999 0 - 1000

TM/ETM NDVI Normalized Difference Vegetation Index (-)

0.001 int16 -9999 -1000 - 1000

GLS 1990

TM Band

3 TOA reflectance (-) 0.001 int16 -9999 0 - 1000

TM Band

4 TOA reflectance (-) 0.001 int16 -9999 0 - 1000

TM Bnad

5 TOA reflectance (-) 0.001 int16 -9999 0 - 1000

TM Band

6 Radiometric temperature (K) 0.100 int16 -9999 0 - 1000

TM NDVI Normalized Difference Vegetation Index (-)

0.001 int16 -9999 -1000 - 1000

GLS 1975

MSS Band

2 TOA reflectance (-) 0.001 int16 -9999 0 - 1000

MSS Band

3 TOA reflectance (-) 0.001 int16 -9999 0 - 1000

MSS NDVI Normalized Difference Vegetation Index (-)

0.001 int16 -9999 -1000 - 1000

Table 3: The individual bands chosen for mosaics for the 7 regions of the world (Figure 1).

Page 14: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

Figure 7: GLS-2010 Mosaic of the R-NIR-MIR bands combination in crop area for Region Africa,

Map projection: Geographic

Page 15: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

Figure 8: GLS-2010 Mosaic of the NDVI in crop area for Region Africa, Map projection:

Geographic

3 Global MODIS 2000 MFDC

<should we merge Parda’s document here?>

3.1 MODIS BDI for year 2000 (2000-2002)

MODIS BDI for year 2000 consists 52-band, nominal 250 m MFDC of the World, including 250m

MODIS red and NIR band, ndvi band which is using the maximum value composite results. We

collected the best 8-days MOD09 reflectance product from 2000 to 2002, which means we will

have the monthly composite from 12 single 8-days images, which would make sure we have

enough clear-sky images even in rain season.

● Band 1 to band 12: MODIS b1 (red) minimum value composite (MVC); 1 band per month

Page 16: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

● Band 13 to band 24: MODIS b2 (NIR) maximum value composite (MVC); 1 band per

month

● Band 25 to band 36: MODIS NDVI maximum value composite (MVC); 1 band per month

● Band 37 to band 48: AVHRR Skin Temperature (in degree kelvin); 1 band per month

(based on 20 yer. Average)

● Band 49: ASTER DEM project (resolution: 30m) or GDEM project (resolution: 1km); 1

band

● Band 50: Tree Cover > 75% (resolution: 8km); 1 band

● Band 51: 40-year-annual-mean (CRU) precipitation (resolution: 8km); 1 band

● Band 52: 40-year-mean from CRU (resolution: 8km); 1 band

3.2 Pre-processing of MODIS Data

3.2.1 Download MODIS Data

There are two ways to download MODIS data

1). Directly from http site (http://e4ftl01.cr.usgs.gov)

Go to http://e4ftl01.cr.usgs.gov/MOLT/MOD09Q1.005/

Then you will have sub folders with dates, each sub folder have every 8-day MVC data files in

HDF format for the whole globe. Select the required tiles for your study area.

For example: Australia covered in the following tiles:

h27v11,h27v12, h28v11, h28v12, h28v13, h29v10, h29v11, h29v12, h29v13, h30v10, h30v11,

h30v12, h30v13, h31v10, h31v11, h31v12, h31v13 and h32v10 as shown in the following

sinusoidal map.

Page 17: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

Figure 9. MODIS Tiles covering Region Australia

2). By ordering data through NASA’s earth observing system Data and Information system

Reverb tool.

Open http://reverb.echo.nasa.gov and follow the steps

Under step 1: Select search Criteria

Search terms

Enter MOD09Q1 in search terms on the right hand side

Spatial search

There are 5 options in spatial search

Select 2D Coordinate under spatial search

Select MODIS Tile SIN under Coordinate System

For e.g. Australia covers in MODIS tiles SIN: 27-31, 10-13

Start X coordinate is 27

END X coordinate is 31

Start Y coordinate is 10

END Y coordinate is 13

Page 18: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

Figure 10a. Reverb tool: Product Search Page

Temporal search:

Select the time period (1st Mar 2000 to 28th Feb 2003) to order 3 year data to make maximum

value composite from three year data (which is called nominal 2000)

Check at MODIS/Terra Surface Reflectance 8-Day L3 Global 250m SIN Grid V005 under step2

:select datasets

Check at MODIS/Terra Surface Reflectance 8-Day L3 Global 250m SIN Grid V005 under step3

:Discover Granules

Then start search granules, if total number of granules is more than 2000 then reduce the time

period and start search (maximum limit is 2000 granules).

Now you can view the selected granules in your search

Page 19: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

Check the granule IDS, if not require a specific granule remove from the selection, then add to

cart

View the Items in the cart

Check the shopping cart; still you can verify the granule ids, if you don’t require you can remove

the granules from the cart

Figure 10b. Reverb tool: Select Granules

You will see items in the shopping cart

In the shopping cart, click on order to order your data

Page 20: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

Figure 10c. Reverb tool: Ordering Cart

Set the delivery option to http pull and save

Page 21: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

Figure 10d. Reverb tool: Order Options

Page 22: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

Figure 10e. Reverb tool: Order Review

Click on submit order

Page 23: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

Figure 10e. Reverb tool: Order Receipt

You will receive Reverb Order Confirmation to your email and Followed by a LPDAAC ECS

Order Notification (Order ID: 0306569132) with host details and directory for your data. From

there you can download the data.

Here are details for the above order:

HOST: e4ftl01.cr.usgs.gov

DIR: /PullDir/0302921601mGBIo

Pull Download Links:

http://e4ftl01.cr.usgs.gov/PullDir/0302921601mGBIo/

Download ZIP file of packaged order:

http://e4ftl01.cr.usgs.gov/PullDir/0302921601mGBIo.zip

Page 24: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

3.2.2 Re projecting data from Sinusoidal projection:

Step 1: Download and organize Hierarchical Data Format (HDF): Download HDF files of the

study area and arrange into one folder (e.g., G/MOD09Q1/Australia). Ensure that the folder you

have created contains all the HDF files that will be processed being systematic in storing data is

critical in the entire process; therefore properly name your folder.

Step 2: Format conversion and re projection: The MODIS re projection tool (MRT) converts HDF

file into tagged image file (TIF) format. The tool also allows re projection and resampling of the

TIF files. Below are the steps we followed.

If you haven't done so, download the MRTool from

https://lpdaac.usgs.gov/tools/modis_reprojection_tool and install it. A user manual is also

available providing a more detail description of what MRTool is and how to use it (Note: system

must be installed Java software).

Add HDF files: Click on the Open Input File button (shown in Magenta in the diagram below).

The Open window will appear. Using the Look in input box, navigate to the folder (e.g.,

G/MOD09Q1/Australia) where the downloaded HDF files are stored.

Select the required images on the same date at a time. Since 16tiles cover the entire Australia,

16 HDF files of the same date has to be selected.

Page 25: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

Figure 11a. ModisTool MRT: Open MODIS Reflectance Products

Page 26: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

Once the HDF files are loaded, the source information is displayed in the Input File Info,

Available/Selected bands, Spatial Subset, and Bounding coordinates.

Figure 11b. ModisTool MRT: Parameters Setting

Select all the files in the input box to view in a map, click on view selected tiles to view the

selected tiles on the map.

Page 27: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

Figure 11c. ModisTool MRT: Selected tiles Overview

Figure 11d. ModisTool MRT: Basic Input Parameters

Band selection: By default, all available bands are selected and appear in the box to the right

(Selected Bands). We only need bands 1 and 2.

To exclude certain bands from processing, click on the unwanted bands and use the “<<” button

to deselect them (below figure). This will move it to the box on the left (Available Bands). To

move bands from the Available box to the Selected box, click on the desired band and use “>>”

button. Shift-Click and Ctrl-Click are useful for highlighting multiple bands for selection.

Spatial subset : Input Lat/Long and specify Latitude and Longitude of UL corner and LR

corner(e.g., UL (-10, 112.9),LR(-45, 158.96)

Page 28: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

c. Output file: Click on the Specify Output File button. Provide an output name similar to the

input filename (i.e. “MOD09Q1.A2002059”) to maintain consistency.

Figure 11e. ModisTool MRT: Output Filename

It is very important to include the file extension as part of the filename. The file extension

indicates the file format of the output image. Adding ".hdf" will tell MRT to output to HDF-EOS,

".tif" to output to GeoTIFF, and ".hdr" to output to raw binary.

Likewise, in Output File Type, select GEOTIFF from the drop box (GEOTIFF is a standard

image format in image processing software).

d. Resampling: Select the Resampling Type as “Nearest Neighbor”

Page 29: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

e. Reprojecting: This procedure transforms the sinusoidal equal area projection of the input

HDF into the geographic coordinate system. The Output Projection Type is selected from a

drop-down list as Geographic, and its parameters are entered using the Edit Projection

Parameters button. Select WGS 84 as datum.

Figure 11e. ModisTool MRT: Output Options

Page 30: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

The output pixel also has to be defined. Since our output image will be in geographic

coordinates, the pixel resolution is measured in degrees. MOD09Q1 is 232 m resolution; this is

equivalent to 0.0021 degrees at the equator.

Figure 11e. ModisTool MRT: Output Options (cont)

f. Executing the conversion: Click Run button to execute the conversion process. After

processing , the following status window appears “Finished processing”

Page 31: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

Page 32: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

Page 33: Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of … · 2014-06-21 · Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World 2.3.1 In Erdas Here are

Baseline Data Infrastructure as Mega-File Data Cube (MFDC) of the World

Figure 11f. ModisTool MRT: Running Logging Information

3.3 Maximum Composition of B1, B2 and NDVI

TBD

4 References ● Global Land Survey(GLS) website in UMD:

http://glcf.umd.edu/data/gls/

● Global Land Survey Activities and Land Cover Analysis from Landsat

http://www.codata.org/10Conf/abstracts-presentations/Sessions%20E/E4/E4-Franks.pdf

● Creating Cloud-Free Landsat ETM+ Data Sets in Tropical Landscapes: Cloud and

Cloud-Shadow Removal

http://www.fs.fed.us/global/iitf/pubs/iitf-gtr32.pdf

● Converting Digital Numbers to Top of Atmosphere (ToA) Reflectance

http://www.yale.edu/ceo/Documentation/Landsat_DN_to_Reflectance.pdf

● Summary of current radiometric calibration coefficients for Landsat MSS, TM, ETM+,

and EO-1 ALI sensors. Chander, G. (2009). Remote Sensing of Environment. Remote

Sensing of Environment, 113(5), 893–903. doi:10.1016/j.rse.2009.01.007

● Markham, B.L, and Barker, J.L., 1987, Radiometric properties of U.S. processed

Landsat MSS data. Remote Sensing of Environment, 22:39-71.

● Markham, B.L, and Barker, J.L., 1985, Spectral characterization of the LANDSAT

Thematic Mapper sensors. International Journal of Remote sensing, 6(5):697-716.

● Neckel, H., and Labs, D., 1981, Improved data of solar spectral irradiance from 0.33 to

1.25 m. Solar Physics, 74:231-240.

● Neckel, H., and Labs, D., 1984, The solar radiation between 3300 and 12500 A, Solar

Physics, 90:205-258.