- 2011-04-01T00:00:00 - Step 1, Ingestion, calibration, and navigation of GAC data - 1.1. Calibrate and convert AVHRR digital counts - for channels 1 through 5 to radiances 1.1.1. Obtain AVHRR Global Area Coverage (GAC) channels 1 through 5 radiometer count data from the original AVHRR L0/L1b data stored at the NOAA Comprehensive Large Array-data Stewardship System (CLASS). 1.1.2. For channels 1 and 2, use pre-launch calibration coefficients to perform a linear counts-to-radiance conversion, followed by a correction for temporal changes using sensor decay rate data and then a correction for inter-satellite differences using inter-satellite standardization data to a NOAA-9 reference, both of which use Libyan desert target area data. 1.1.3. For channels 3, 4, and 5 use both the above pre-launch calibration data and onboard blackbody (space view and sensor base plate) data to perform a non-linear counts-to-radiance conversion. 1.2. Navigation, Clock, and Attitude Corrections - 1.2.1. Apply satellite clock corrections using Earth time offset data based RSMAS comparison of DOMSAT HRPT downlink from the NOAA Wallops Island receiving station to a NIST standard time reference. 1.2.2. Apply attitude corrections made using coastline comparison data. 1.2.3. At this point, navigated, calibrated albedos/brightness temperatures are available for all five channels. Note that channels 1 and 2 are used in the Pathfinder Matchup Database decision trees [see Kilpatrick et al. (2001), Figure 7] and channel 3 is used only in assignment of a quality indicator (see step 2.4.1).
- 2011-06-01T00:00:00 - Step 2, SST Calculation - 2.1. Convert channel 4 and 5 brightness temperatures to SST in degrees Celsius using the Pathfinder algorithm (Equation 1 below), which requires a set of monthly coefficients. Equation 1 - SSTsat = a + bT4 + c(T4 -T5)SSTguess + d(T4 -T5)[sec(q) - 1] where SSTsat = the satellite-derived SST estimate, T4 and T5 = brightness temperatures in the 10.8 and 11.4 micrometer AVHRR bands channels 4 and 5, respectively), SSTguess = a first-guess SST value, q = the satellite zenith angle, and a, b, c, and d = coefficients estimated from regression between collocated and coincident in situ and satellite measurements. 2.2. These coefficients are derived using the Pathfinder Buoy Matchup Database, a set of in situ SST observations and collocated AVHRR data. The in situ data consist mainly of drifting buoys, but during 1981-1984 bias-corrected ship-based observations are also used. 2.3. The calculation of SST in step 2.1 also requires a first-guess SST field. This first-guess field is an augmented form of the DOISST, which is generated by the Matlab script /nodc/projects/satdata-interim/landmask/newref_v52.m according to the following method: 2.3.1. The 4km AVHRR Pathfinder Version 5.0 daily harmonic climatology (http://accession.nodc.noaa.gov/0071181 and http://data.nodc.noaa.gov/thredds/catalog/pathfinder/Version5.0_Climatologies/1982_2008/Daily/catalog.html) is binned down to 25km spatial resolution. If any single 4km pixel in the bin is water, the entire 25km pixel is classified as water. 2.3.2. A 25km land mask is laid on top of the 25km binned climatology to identify water pixels with missing SST values. The development of this land mask was based on three data sets: the Physical Shoreline database from Natural Earth, the Global Self-consistent Hierarchical High-resolution Shoreline (GSHHS) from the NOAA National Geophysical Data Center, and the Global Lakes and Wetlands Database from the World Wildlife Fund. These three data sets were combined into a polygon land mask layer using ArcGIS, and converted into grid format using the "polygon to raster" tool. The tool converts polygons into gridded images based on the maximum area method. The criterion for the maximum area was set as >50 percent. The final land mask has four classes: Land, Ocean, Inland lakes and reservoirs, and Inland rivers. 2.3.3. A daily mean sea ice climatology, generated from the 10km EUMETSAT Ocean and Sea Ice Satellite Application Facility (OSISAF) Global Daily Sea Ice Concentration Reprocessing Data Set (http://accession.nodc.noaa.gov/0068294), is binned to 25km spatial resolution. If any single 10km pixel in the bin is water, the entire 25km pixel is classified as water. 2.3.4. The 25km ice climatology field is filled with values of 100 percent sea ice cover over the permanent ice shelves in Antarctica. These permanent ice shelves are defined by the difference between the 25km OSISAF land mask and the 25km Pathfinder Version 5.2 land mask below -60 degrees latitude. 2.3.5. For pixels with no existing daily SST climatology value, AND a sea ice climatology value greater than zero, an SST value of -1.8 degrees Celsius is assigned. 2.3.6. Remaining water pixels with no SST values are assigned annual mean SST values derived from the daily harmonic climatology. 2.3.7. Any remaining water pixels are assigned latitudinal mean SST values derived from the daily harmonic climatology. The result is an effectively gap-free daily SST climatology. 2.3.8. The original NOAA Daily 25km Global Optimally Interpolated Sea Surface Temperature (DOISST) analysis is read from the NODC archive (http://data.nodc.noaa.gov/thredds/catalog/ghrsst/L4/GLOB/NCDC/AVHRR_OI/catalog.html). Any pixel that is missing an SST value in the DOISST but has an SST value in the gap-free daily climatology is assigned the value from the daily climatology. 2.3.9. This new gap-filled DOISST field is written out to netCDF and posted online to http://data.nodc.noaa.gov/pathfinder/UserRequests/RSMAS/ (/nodc/www/data.nodc/htdocs/pathfinder/UserRequests/RSMAS/ on the internal network and archived at http://accession.nodc.noaa.gov/0071180) to be downloaded by the Miami team and ingested by SeaDAS. 2.4. Quality Flag Assignment 2.4.1. A Channel 3, 4, and 5 brightness temperature test is performed. The brightness temperatures were calculated in step 1.1.3. 2.4.2. The viewing angle is then evaluated using a satellite zenith angle check. 2.4.3. Next, a reference field comparison check is made against the DOISST used in step 2.3. 2.4.4. A stray sunlight test is then performed which requires information on whether the data in question are to left or right of nadir. 2.4.5. An edge test is performed next, which checks the location of the pixel within a scan line and the location of the scan line within the processing piece (a "piece" is a subset of an entire orbit file). 2.4.6. Then, a glint test is performed which requires a glint index calculated according to the Cox and Munk (1954) formulation. 2.4.7. Finally, these steps are all combined into an overall quality flag assignment for each pixel.
- 2011-06-01T00:00:00 - Step 3, Spatial Binning - 3.1. The GAC pixels, converted to SST in Step 2, are then binned into an equal-area integerized sinusoidal grid. 3.2. To reduce discontinuities along the date line, a data-day based on a spatial data-day definition (Podesta, 1995) is used. 3.3. The new land mask, as described in Step 2.3, is applied to the data, identifying pixels that fall on land.
- 2011-06-01T00:00:00 - Step 4, Temporal Binning - 4.1. Step 4 begins by temporally binning the spatially binned pieces from processing step 3 into a single ascending (daytime) and single descending (nighttime) file for each day. In the case of overlapping satellite passes, only the highest quality pixels available for each are used. If there is more than one contributing pixel at the highest quality, these values are averaged. Note: NOAA-17 is the only morning satellite included in the Pathfinder time series; for this satellite, descending = daytime and ascending = nighttime. In PFV50 and PFV51, temporally binned files were also created for 5-day, 7-day, 8-day, monthly, and yearly periods, but these were not created for PFV52. 4.2. For Pathfinder V5.0 and V5.1, an internal Pathfinder reference check comparison is then made to an internal 3-week Pathfinder comparison field, and a sea ice mask based on weekly SSM/I and sea ice information in the OISSTv2 is used to exclude pixels from the computation of the internal reference field. Note that no such check is performed for Pathfinder Version 5.2. 4.3. The SST, quality, and related fields are reformatted from equal-area to equal-angle and output in HDF4-EOS format from the mixed SeaDAS version 6.1/6.2 environment, one file for each day and night in the collation period. Note: In PFV50 and PFV51, the equal-angle global grid had dimensions of 8192 by 4096. In PFV52, the grid is 8640 by 4320. This change was made to correct the "18-line problem" in PFV50 and PFV51, in which a persistent false gradient arose every 18 lines in the image due to a round-off error in the binning code.
- 2011-06-01T00:00:00 - Step 5, Addition of Ancillary Data, Reformatting to GDS2.0, and Quality Assurance - 5.1 NODC transfers files produced from Processing Step 4 from Miami to NODC, using rsync and utilizing checksums to ensure file integrity. 5.2. Next, NODC processes the data using a program called hdf2nc_PFV52_L3C to convert the data from HDF format to GDS2.0-compliant netCDF-4. The usage of hdf2nc_PFV52_L3C.x: hdf2nc_PFV52_L3C.x [-v] sst_file [-ref ref_file] [-si dailyOI_seaice_file] 5.3. This conversion brings in several sources of ancillary data, in addition to converting the data from HDF to netCDF and adding many attributes. The sources of ancillary data are described below along with any processing done to them prior to inclusion in the converted netCDF-4 files: 5.3.1. Marine Wind Speeds: The source of Marine Wind Speeds is NCEP/DOE AMIP-II Reanalysis (Reanalysis-2). The wind speeds represent winds at 10 metres above the sea surface. The original data are U, V wind components in global T62 gaussian grid (192x94). The interpolation to Pathfinder V5.2 grid and a combination of U, V wind components to full wind speeds are done before the data put into the final netCDF. There was a bug in the program (get_wnd.c) to read the original downloaded U, V components winds which caused the incorrect data for December. The bug was fixed and the reprocessing was conducted. 5.3.2. Aerosol Depth Indicator: Aerosol depth indicator (ADI) in Pathfinder V5.2 comes from two sources: Monthly averaged Pathfinder from AVHRR (AVHRRPF, obtained from CLASS) which covers Jul 1981 to Dec 2000, and weekly averaged Aerosol Optical Thickness (100 KM) (AERO100) which covers from Nov 1998 to present. The two sources have some overlaps. AVHRRPF is used in the overlapped period. Two standalone programs (retrvAeroAVHRRPF.c and retrvAeroAERO100.c) are used to do the retrievals and interpolations. 5.3.3. Sea Ice Fraction: The main source of Sea Ice Fraction in Pathfinder V5.2 is OSI/SAF Global Daily Sea Ice Concentration Reprocessing Data Set (http://accession.nodc.noaa.gov/0068294). When the OSI/SAF data is unavailable (there are sporadic data gaps in the early 1990's and no data after year 2007), NCDC DailyOI SST data is the replacement. This is an automatic selection in the hdf2nc_PFV52_L3C program. In the hdf2nc_PFV52_L3C command line, mandatory use of DailyOI SST data is provided as an option. Retrieving and interpolating OSI/SAF data are conducted by a standalone program (main program get_seaice.c). The processing of DailyOI data is integrated in hdf2nc_PFV52_L3C. 5.4. During this initial conversion the original Pathfinder quality levels are read from the HDF file, mapped to the 6 quality levels required by the GDS2.0, and written to the variable quality_level in the netCDF file. The quality levels are as follows: GDS quality_level 5 = native Pathfinder quality level 7 = best_quality; GDS quality_level 4 = native Pathfinder quality_level 4-6 = acceptable_quality; GDS quality_level 3 = native Pathfinder quality level 2-3 = low_quality; GDS quality_level 2 = native Pathfinder quality level 1 = worst_quality; GDS quality_level 1 = native Pathfinder quality level 0 = bad_data; GDS quality_level 0 = native Pathfinder quality level -1 = missing_data. The original Pathfinder quality level is written to the optional variable pathfinder_quality_level in the netCDF file. 5.5. River, lake, and land information from the Pathfinder Version 5.2 land mask, as well as sea ice presence information from the sea ice data described previously, are encoded in bits and written to the bit mask variable l2p_flags in the netCDF file and global attributes are written to the netCDF file according to GDS2.0 requirements. 5.6. After the initial conversion is complete, a problem with a hidden netCDF attribute is resolved to make the data usable within Matlab. The problem is corrected using the Matlab m-file function called fix_creation_order_issue.m, which was provided by John Evans of the Mathworks, Inc. This function fixes affected netCDF-4 files by deleting the '_Netcdf4Dimid' attribute. This attribute is written to netCDF-4 files created with the netCDF version 4.1.2 libraries, making them unreadable to clients using older netCDF libraries (including Matlab R2010b, which uses netCDF version 4.0.1). This function is run on every file, using the simple m-file called run_fix.m. The log containing the output for each year is held at the NODC. 5.7. Browse Graphic Generation: A collection of browse graphics is generated from each netCDF file using the Matlab script browse_graphic_gen_pfv52.m. Both high- and low-resolution PNG graphics are generated for the following variables in each file: sea_surface_temperature, dt_analysis, declouded_sst, sea_ice_fraction, aerosol_dynamic_indicator, wind_speed, pathfinder_quality_level, quality_level, and l2p_flags. A KML wrapper file is also generated for use in GoogleEarth, and the high-resolution PNGs have land set to transparent. The log containing the output for each year is held at the NODC. 5.8. Browse Graphic Visual Review: Each of the browse graphics is then reviewed manually by NODC staff, and anomalies are documented and corrected as needed. 5.9. A Rich Inventory is under development as of 2011.
- 2011-06-01T00:00:00 - Step 6, Archiving and Distribution - 6.4. After passing the visual inspections, the PFV52 data are subjected to archive review then formally archived by NODC. Archival procedures are extensive and conform to the principles of the ISO standard for digital archives (ISO 14721), known as the Open Archival Information System Reference Model (OAIS-RM). Each year of PFV52 data are archived as an Archival Information Package, also known an NODC accession. 6.5. A collection-level metadata record in ISO 19115-2 format is created and maintained to document all PFV52 accessions. In addition, an FGDC version is created from that ISO record. This collection level record is maintained and available at: http://pathfinder.nodc.noaa.gov/ISO-AVHRR_Pathfinder-NODC-L3C-v5.2.html. 6.6. The result of all six processing steps steps is the Pathfinder Version 5.2 SST product. The data are distributed by the NODC in a wide variety of ways; see the DISTRIBUTION SECTION of the metadata record(s) for details.