isofit.utils.template_construction

class Pathnames(args)[source]

Bases: object

Class to determine and hold the large number of relative and absolute paths that are needed for isofit and MODTRAN configuration files.

Parameters:: args – an argparse Namespace object with all inputs

make_directories()[source]: Build required subdirectories inside working_directory

stage_files()[source]: Stage data files by copying into working directory

class LUTConfig(lut_config_file=None, emulator=False)[source]

Bases: object

A look up table class, containing default grid options. All properties may be overridden with the optional: input configuration file path

Parameters:: lut_config_file (Optional[str]) – configuration file to override default values

get_grid(minval, maxval, spacing, min_spacing)[source]

get_angular_grid(angle_data_input, spacing, min_spacing, units='d')[source]

Find either angular data ‘center points’ (num_points = 1), or a lut set that spans angle variation in a systematic fashion.

Parameters:

angle_data_input (array) – set of angle data to use to find center points
spacing (float) – the desired angular spacing between points, or mean if -1
min_spacing (float) – the minimum angular spacing between points allowed (if less, no grid)
units (str) – specifies if data are in degrees (default) or radians

Returns:

angular data center point or lut set spanning space

class SerialEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)[source]

Bases: JSONEncoder

Encoder for json to help ensure json objects can be passed to the workflow manager.

default(obj)[source]

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return JSONEncoder.default(self, o)

get_grid(minval, maxval, spacing, min_spacing)[source]

get_angular_grid(angle_data_input, spacing, min_spacing, units='d')[source]

Find either angular data “center points” (num_points = 1), or a lut set that spans angle variation in a systematic fashion.

Parameters:

angle_data_input (array) – set of angle data to use to find center points
spacing (float) – the desired angular spacing between points, or mean if -1
min_spacing (float) – the minimum angular spacing between points allowed (if less, no grid)
units (str) – specifies if data are in degrees (default) or radians

Returns:

angular data center point or lut set spanning space

build_surface_config(macro_config, flight_id, output_path, wvl_file)[source]

Write a surface config file, using the specified pathnames and all given info.

Parameters:

macro_config (dict) – dictionary of macro options for surface model
flight_id (str) – string of instrument specific flight identification number
output_path (str) – output directory for surface config file
wvl_file (str) – directory of instrument wavelength file

build_presolve_config(paths, h2o_lut_grid, n_cores=-1, use_emp_line=False, surface_category='multicomponent_surface', emulator_base=None, uncorrelated_radiometric_uncertainty=0.0, segmentation_size=400, debug=False, inversion_windows=[[350.0, 1360.0], [1410, 1800.0], [1970.0, 2500.0]], prebuilt_lut_path=None)[source]

Write an isofit config file for a presolve, with limited info.

Parameters:

paths (Pathnames) – object containing references to all relevant file locations
h2o_lut_grid (array) – the water vapor look up table grid isofit should use for this solve
n_cores (int) – number of cores to use in processing
use_emp_line (bool) – flag whether or not to set up for the empirical line estimation
surface_category – type of surface to use
emulator_base (Optional[str]) – the basename of the emulator, if used
uncorrelated_radiometric_uncertainty (float) – uncorrelated radiometric uncertainty parameter for isofit
segmentation_size (int) – image segmentation size if empirical line is used
debug (bool) – flag to enable debug_mode in the config.implementation
lut_path – lut path to use; if none, presolve config will create a new file

Return type:

None

build_main_config(paths, lut_params, h2o_lut_grid=None, elevation_lut_grid=None, to_sensor_zenith_lut_grid=None, to_sun_zenith_lut_grid=None, relative_azimuth_lut_grid=None, mean_latitude=None, mean_longitude=None, dt=None, use_emp_line=True, n_cores=-1, surface_category='multicomponent_surface', emulator_base=None, uncorrelated_radiometric_uncertainty=0.0, multiple_restarts=False, segmentation_size=400, pressure_elevation=False, debug=False, inversion_windows=[[350.0, 1360.0], [1410, 1800.0], [1970.0, 2500.0]], prebuilt_lut_path=None)[source]

Write an isofit config file for the main solve, using the specified pathnames and all given info

Parameters:

paths (Pathnames) – object containing references to all relevant file locations
lut_params (LUTConfig) – configuration parameters for the lut grid
h2o_lut_grid (Optional[array]) – the water vapor look up table grid isofit should use for this solve
elevation_lut_grid (Optional[array]) – the ground elevation look up table grid isofit should use for this solve
to_sensor_zenith_lut_grid (Optional[array]) – the to-sensor zenith angle look up table grid isofit should use for this solve
to_sun_zenith_lut_grid (Optional[array]) – the to-sun zenith angle look up table grid isofit should use for this solve
relative_azimuth_lut_grid (Optional[array]) – the relative to-sun azimuth angle look up table grid isofit should use for this solve
mean_latitude (Optional[float]) – the latitude isofit should use for this solve
mean_longitude (Optional[float]) – the longitude isofit should use for this solve
dt (Optional[datetime]) – the datetime object corresponding to this flightline to use for this solve
use_emp_line (bool) – flag whether or not to set up for the empirical line estimation
n_cores (int) – the number of cores to use during processing
surface_category – type of surface to use
emulator_base (Optional[str]) – the basename of the emulator, if used
uncorrelated_radiometric_uncertainty (float) – uncorrelated radiometric uncertainty parameter for isofit
multiple_restarts (bool) – if true, use multiple restarts
segmentation_size – image segmentation size if empirical line is used
pressure_elevation (bool) – if true, retrieve pressure elevation
debug (bool) – if true, run ISOFIT in debug mode

Return type:

None

get_lut_subset(vals)[source]

Populate lut_names for the appropriate style of subsetting

Parameters:: vals – the values to use for subsetting

write_modtran_template(atmosphere_type, fid, altitude_km, dayofyear, to_sun_zenith, to_sensor_azimuth, to_sensor_zenith, relative_azimuth, gmtime, elevation_km, output_file, ihaze_type='AER_RURAL')[source]

Write a MODTRAN template file for use by isofit look up tables

Parameters:

atmosphere_type (str) – label for the type of atmospheric profile to use in modtran
fid (str) – flight line id (name)
altitude_km (float) – altitude of the sensor in km
dayofyear (int) – the current day of the given year
to_sun_zenith (float) – final altitude solar zenith angle (0→180°)
to_sensor_azimuth (float) – azimuth view angle to the sensor, in degrees (AVIRIS convention)
to_sensor_zenith (float) – sensor/observer zenith angle, in degrees (MODTRAN convention: 180 - AVIRIS convention)
relative_azimuth (float) – final altitude relative solar azimuth (0→360°)
gmtime (float) – greenwich mean time
elevation_km (float) – elevation of the land surface in km
output_file (str) – location to write the modtran template file to
ihaze_type (str) – type of extinction and default meteorological range for the boundary-layer aerosol model

load_climatology(config_path, latitude, longitude, acquisition_datetime, isofit_path, lut_params)[source]

Load climatology data, based on location and configuration

Parameters:

config_path (str) – path to the base configuration directory for isofit
latitude (float) – latitude to set for the segment (mean of acquisition suggested)
longitude (float) – latitude to set for the segment (mean of acquisition suggested)
acquisition_datetime (datetime) – datetime to use for the segment( mean of acquisition suggested)
isofit_path (str) – base path to isofit installation (needed for data path references)
lut_params (LUTConfig) – parameters to use to define lut grid

:Returns

tuple containing:: aerosol_state_vector - A dictionary that defines the aerosol state vectors for isofit aerosol_lut_grid - A dictionary of the aerosol lookup table (lut) grid to be explored aerosol_model_path - A path to the location of the aerosol model to use with MODTRAN.

calc_modtran_max_water(paths)[source]

MODTRAN may put a ceiling on “legal” H2O concentrations. This function calculates that ceiling. The intended: use is to make sure the LUT does not contain useless gridpoints above it.

Parameters:: paths (Pathnames) – object containing references to all relevant file locations
Return type:: float
Returns:: max_water - maximum MODTRAN H2OSTR value for provided obs conditions

define_surface_types(tsip, rdnfile, obsfile, out_class_path, wl, fwhm)[source]

copy_file_subset(matching_indices, pathnames)[source]

Copy over subsets of given files to new locations

Parameters:

matching_indices (np.array) – indices to select from (y dimension) from source dataset
pathnames (List) – list of tuples (input_filename, output_filename) to read/write to/from

get_metadata_from_obs(obs_file, lut_params, trim_lines=5, max_flight_duration_h=8, nodata_value=-9999)[source]

Get metadata needed for complete runs from the observation file (bands: path length, to-sensor azimuth, to-sensor zenith, to-sun azimuth, to-sun zenith, phase, slope, aspect, cosine i, UTC time).

Parameters:

obs_file (str) – file name to pull data from
lut_params (LUTConfig) – parameters to use to define lut grid
trim_lines (int) – number of lines to ignore at beginning and end of file (good if lines contain values that are erroneous but not nodata
max_flight_duration_h (int) – maximum length of the current acquisition, used to check if we’ve lapped a UTC day
nodata_value (float) – value to ignore from location file

Returns:

tuple containing:: h_m_s - list of the mean-time hour, minute, and second within the line increment_day - indicator of whether the UTC day has been changed since the beginning of the line time mean_path_km - mean distance between sensor and ground in km for good data mean_to_sensor_azimuth - mean to-sensor azimuth for good data mean_to_sensor_zenith - mean to-sensor zenith for good data mean_to_sun_zenith - mean to-sun zenith for good data mean_relative_azimuth - mean relative to-sun azimuth for good data valid - boolean array indicating which pixels were NOT nodata to_sensor_zenith_lut_grid - the to-sensor zenith look up table grid for good data to_sun_zenith_lut_grid - the to-sun zenith look up table grid for good data relative_azimuth_lut_grid - the relative to-sun azimuth look up table grid for good data

Return type:

(List, bool, float, float, float, array, List, List)

get_metadata_from_loc(loc_file, lut_params, trim_lines=5, nodata_value=-9999, pressure_elevation=False)[source]

Get metadata needed for complete runs from the location file (bands long, lat, elev).

Parameters:

loc_file (str) – file name to pull data from
lut_params (LUTConfig) – parameters to use to define lut grid
trim_lines (int) – number of lines to ignore at beginning and end of file (good if lines contain values that are erroneous but not nodata
nodata_value (float) – value to ignore from location file
pressure_elevation (bool) – retrieve pressure elevation (requires expanded ranges)

Returns:

tuple containing:: mean_latitude - mean latitude of good values from the location file mean_longitude - mean latitude of good values from the location file mean_elevation_km - mean ground estimate of good values from the location file elevation_lut_grid - the elevation look up table, based on globals and values from location file

Return type:

(float, float, float, array)

reassemble_cube(matching_indices, paths)[source]

Copy over subsets of given files to new locations

Parameters:

matching_indices (np.array) – indices to select from (y dimension) from source dataset
paths (Pathnames) – output file array set