sicor.Tools.EnMAP package

Submodules

sicor.Tools.EnMAP.LUT module

sicor.Tools.EnMAP.LUT.download_LUT(path_LUT_default)

Download LUT file from remote GIT repository.

Parameters:

path_LUT_default – directory where to store the LUT file

Returns:

directory of downloaded LUT file

sicor.Tools.EnMAP.LUT.get_data_file(module_name, file_basename)

Load data file.

Parameters:

• module_name – name of python module where data file is located

• file_basename – name of data file

Returns:

path to data file

sicor.Tools.EnMAP.LUT.interpol_lut(lut1, lut2, xnodes, nm_nodes, ndim, x_cell, vtest, intp_wvl)

Multidimensional LUT interpolation.

Parameters:

• lut1 – LUT containing path radiance

• lut2 – LUT containing downwelling direct and diffuse surface solar irradiance, spherical albedo and transmittance

• xnodes – gridpoints for each LUT dimension

• nm_nodes – overall number of LUT gridpoints (2**ndim)

• ndim – dimension of LUT

• x_cell – interpolation grid (nm_nodes x n_dim)

• vtest – atmospheric state vector (interpolation point); contains VZA, SZA, HSF, AOT, RAA, CWV

• intp_wvl – wavelengths for which interpolation should be conducted

Returns:

path radiance, downwelling direct and diffuse surface solar irradiance, spherical albedo and transmittance interpolated at vtest

sicor.Tools.EnMAP.LUT.interpol_lut_red(lut1, lut2, xnodes, nm_nodes, ndim, x_cell, vtest, intp_wvl)

LUT interpolation based in a reduced grid.

Parameters:

• lut1 – LUT containing path radiance

• lut2 – LUT containing downwelling direct and diffuse surface solar irradiance, spherical albedo and transmittance

• xnodes – gridpoints for each LUT dimension

• nm_nodes – overall number of LUT gridpoints (2**ndim)

• ndim – dimension of LUT

• x_cell – interpolation grid (nm_nodes x n_dim)

• vtest – atmospheric state vector (interpolation point); only surface elevation and CWV

• intp_wvl – wavelengths for which interpolation should be conducted

Returns:

path radiance, downwelling direct and diffuse surface solar irradiance, spherical albedo and transmittance interpolated at vtest

sicor.Tools.EnMAP.LUT.read_lut_enmap_formatted(file_lut)

Read MODTRAN® LUT.

Parameters:

file_lut – path to LUT

Returns:

LUT of atmospheric functions, x and y axes grid points, LUT wavelengths

sicor.Tools.EnMAP.LUT.reduce_lut(lut1, lut2, xnodes, nm_nodes, ndim, x_cell, gp, intp_wvl)

Reduce grid dimensionality of LUT based on fixed solar and view geometry as well as one single scene value of AOT.

Parameters:

• lut1 – LUT containing path radiance

• lut2 – LUT containing downwelling direct and diffuse surface solar irradiance, spherical albedo and transmittance

• xnodes – gridpoints for each LUT dimension

• nm_nodes – overall number of LUT gridpoints (2**ndim)

• ndim – dimension of LUT

• x_cell – interpolation grid (nm_nodes x n_dim)

• gp – state vector of fixed scene parameters (VZA, SZA, AOT, RAA)

• intp_wvl – wavelengths for which interpolation should be conducted

Returns:

2D LUT including path radiance, downwelling direct and diffuse surface solar irradiance, spherical albedo and transmittance interpolated at gp for varying surface elevation and CWV; reduced gridpoints for each LUT dimension; reduced overall number of LUT gridpoints; reduced dimension of LUT; reduced interpolation grid

sicor.Tools.EnMAP.a_priori module

sicor.Tools.EnMAP.conversion module

sicor.Tools.EnMAP.conversion.generate_filter(wvl_m, wvl, wl_resol)

Generate norm array for wavelength resampling.

Parameters:

• wvl_m – start wavelength grid

• wvl – final wavelength grid

• wl_resol – resolution of final wavelength grid

Returns:

resampling norm

sicor.Tools.EnMAP.conversion.table_to_array(k_wi, a, b, col_wvl, col_k)

Convert refractive index table entries to numpy array.

Parameters:

• k_wi – variable

• a – start line

• b – end line

• col_wvl – wavelength column in pandas table

• col_k – k column in pandas table

Returns:

arrays of wavelengths and imaginary parts of refractive index

sicor.Tools.EnMAP.first_guess module

sicor.Tools.EnMAP.first_guess._compute_wv(ind)

sicor.Tools.EnMAP.first_guess.wv_band_ratio(data, water_msk, fn_table, sol_model, vza, sza, dem, aot, raa, intp_wvl, intp_fwhm, jday, month, idx, processes, disable=False)

Band ratio water vapor retrieval.

Parameters:

• data – image dataset

• water_msk – water mask

• fn_table – path to radiative transfer LUT

• sol_model – dictionary containing solar irradiance model (‘wvl’, ‘sol_irr’)

• vza – viewing zenith angle

• sza – sun zenith angle

• dem – digital elevation model, same shape as data

• aot – aerosol optical thickness

• raa – relative azimuth angle

• intp_wvl – instrument wavelengths

• intp_fwhm – instrument fwhm

• jday – acquisition day

• month – acquisition month

• idx – indices of instrument channels, which should be used for retrieval (should be approx. 870, 900 and 940 nm)

• processes – number of CPUs for multiprocessing

• disable – if True, progressbar during retrieval is disabled; default: False

Returns:

water vapor image

sicor.Tools.EnMAP.first_guess.wv_band_ratio_snow(data, fn_table, vza, sza, dem, aot, raa, intp_wvl, intp_fwhm, jday, month, idx, disable=False)

Band ratio water vapor retrieval, adapted to the estimation of snow and glacier ice surface properties.

Parameters:

• data – image dataset

• fn_table – path to radiative transfer LUT

• vza – viewing zenith angle

• sza – sun zenith angle

• dem – digital elevation model, same shape as data

• aot – aerosol optical thickness

• raa – relative azimuth angle

• intp_wvl – instrument wavelengths

• intp_fwhm – instrument fwhm

• jday – acquisition day

• month – acquisition month

• idx – indices of instrument channels, which should be used for retrieval (should be approx. 870, 900 and 940 nm)

• disable – if True, progressbar during retrieval is disabled; default: False

Returns:

water vapor image

sicor.Tools.EnMAP.metadata module

sicor.Tools.EnMAP.metadata.varsol(jday, month)

Compute solar irradiance variance.

Parameters:

• jday – julian day

• month – julian month

Returns:

solar irradiance variance factor

sicor.Tools.EnMAP.multiprocessing module

class sicor.Tools.EnMAP.multiprocessing.SharedNdarray(dims, typecode='f', default_value=None)

Bases: object

Wrapper class, which collects all necessary instances to make a numpy ndarray accessible as shared memory when using multiprocessing, it exposes the numpy array via three different views which can be used to access it globally. __init__ provides the mechanism to make this array available in each worker, best used using the provided __initializer__.

Instance of SharedNdarray object.

C Type Python Type Minimum size in bytes

‘b’ signed char int 1 ‘B’ unsigned char int 1 ‘u’ Py_UNICODE Unicode character 2 (1) ‘h’ signed short int 2 ‘H’ unsigned short int 2 ‘i’ signed int int 2 ‘I’ unsigned int int 2 ‘l’ signed long int 4 ‘L’ unsigned long int 4 ‘q’ signed long long int 8 (2) ‘Q’ unsigned long long int 8 (2) ‘f’ float float 4 ‘d’ double float 8

Parameters:

• dims – tuple of dimensions which is used to instantiate a ndarray using np.zero

• typecode – typecode of values

• default_value – default fill value of array

_initializer(globals_dict, name)

This adds to globals while using the ctypes library view of [SharedNdarray instance].sh to make the numpy

view of [SharedNdarray instance] globally available.

Parameters:

• globals_dict – dictionary of global variables

• name – name of variable

sicor.Tools.EnMAP.multiprocessing.initializer(globals_dict, add_to_globals)

Globs shall be dict with name:value pairs, when executed value will be added to globals under the name name,

if value provides a _initializer attribute this one is called instead. This makes most sense when called as initializer in a multiprocessing pool, e.g. Pool(initializer=__initializer__, initargs=(globs,)).

Parameters:

• globals_dict – dictionary of global variables

• add_to_globals – name of variable dict to be added to globs

sicor.Tools.EnMAP.multiprocessing.mp_progress_bar(iter_list, results, bar)

Print progress bar for multiprocessing tasks.

Parameters:

• iter_list – multiprocessing iterable

• results – results object

• bar – progress bar object

sicor.Tools.EnMAP.optimal_estimation module

exception sicor.Tools.EnMAP.optimal_estimation.LinAlgError

Bases: OeCoreError

exception sicor.Tools.EnMAP.optimal_estimation.MissingInputError

Bases: OeCoreError

exception sicor.Tools.EnMAP.optimal_estimation.NonSquareMatrixError

Bases: OeCoreError

exception sicor.Tools.EnMAP.optimal_estimation.OeCoreError

Bases: Exception

sicor.Tools.EnMAP.optimal_estimation.RESULT

alias of result

exception sicor.Tools.EnMAP.optimal_estimation.WrongInputError

Bases: OeCoreError

sicor.Tools.EnMAP.optimal_estimation.approximate_jacobian_function(func: Callable) → Callable

Return numerical jacobian function for a given function ff.

If func: R^N –> R^M then jac: R^N –> R^(MxN)

IMPORTANT:

It is assumed, that func takes two arguments:

x: 1d np-array (the state) p: any kind of parameter object…

Similar to func, jac_func takes two arguments the state and params, where params is a 2-tupel:

params[0] the same as p for func params[1] is delta_x, a 1d np-array

of the same size as x, used for the numerical differentiation

sicor.Tools.EnMAP.optimal_estimation.check_increment(ix: Type[array], sri: Type[array], eps: float) → bool

See Rodgers for details…

input:

ix : increment of x_i+1 = x_i -ix sri: inverse of retrieval error co-variance

sicor.Tools.EnMAP.optimal_estimation.dof_infocont_retrnoise_smootherrr(sa: Type[array], se: Type[array], gg: Type[array], av: Type[array]) → Tuple[Type[array], Type[array], Type[array], Type[array]]

sicor.Tools.EnMAP.optimal_estimation.gain_aver_cost(x: Type[array], y: Type[array], k: Type[array], xa: Type[array], sei: Type[array], sai: Type[array], rec: Type[array]) → Tuple[Type[array], Type[array], Type[array]]

input:

x: state vector y: forward(x) - measurement k: jacobian(x) xa: prior state sei: inverse of measurement error co-variance sai: inverse of prior error co-variance rec: retrieval error covarince

return:

gain: gain matrix aver: averaging kernel cost: cost function

sicor.Tools.EnMAP.optimal_estimation.gauss_newton_increment_mform(x: Type[array], y: Type[array], k: Type[array], xa: Type[array], sei: Type[array], sai: Type[array], sa: Type[array]) → Tuple[Type[array], Type[array], Type[array], Type[array]]

input:

x: state vector y: forward(x) - measurement k: jacobian(x) xa: prior state sei: inverse of measurement error co-variance sai: inverse of prior error co-variance se: measurement error co-variance sa: prior error co-variance

return:

nx: optimal state respecting measurement and prior

and coresponding uncertainties all for for the linear case

inc_r: increment of x, ret_err_cov_i: inverse of retrieval error covariance ret_err_cov: retrieval error covariance

Equation: 5.10 (with negative sign put into brackets) This variant is a ‘m-form’ (m is Dimension of measurement).

sicor.Tools.EnMAP.optimal_estimation.gauss_newton_increment_nform(x: Type[array], y: Type[array], k: Type[array], xa: Type[array], sei: Type[array], sai: Type[array], sa: Type[array]) → Tuple[Type[array], Type[array], Type[array], Type[array]]

input:

x: state vector y: forward(x) - measurement k: jacobian(x) xa: prior state sei: inverse of measurement error co-variance sai: inverse of prior error co-variance se: measurement error co-variance sa: prior error co-variance

return:

nx: optimal state respecting measurement and prior

and coresponding uncertainties all for for the linear case

inc_r: increment of x, ret_err_cov_i: inverse of retrieval error covariance ret_err_cov: retrieval error covariance

Equation: 5.8 (with negative sign put into brackets) This variant is a ‘n-form’ (n = dimension of state).

sicor.Tools.EnMAP.optimal_estimation.internal_optimizer(y: Type[array], func: Callable, fparam: None | Type[array], jaco: Callable, jparam: None | Type[array], xa: Type[array], fg: Type[array], sei: Type[array], sai: Type[array], se: Type[array], sa: Type[array], dx: None | Type[array], eps: float, maxiter: int, full: bool, gnform: str) → Tuple[Type[array], Type[array], bool, int, Type[array], None | Type[array], None | Type[array], None | float, None | float, None | float, None | Type[array], None | Type[array]]

y: measurement func: function to be inverted fparam: parameter for func jaco: function that returns the jacobian of func jparam: parameter for jaco xa: prior state fg: first guess state sei: inverse of measurement error covariance matrix sai: inverse of prior error covariance matrix se : measurement error covariance matrix sa : prior error covariance matrix eps: if (x_i-x_i+1)^T # S_x # (x_i-x_i+1) < eps * N_x, optimization

is stopped. This is the original as, e.g. proposed by Rodgers

maxiter: maximum number of number_of_iterations

sicor.Tools.EnMAP.optimal_estimation.inverse(inn: Type[array]) → Type[array]

sicor.Tools.EnMAP.optimal_estimation.invert_function(func, **args)

This solves the following equation:

y=func(x,params) and returns a function which is effectively the inverse of func x=func⁻¹(y,fparam=params).

mandatory INPUT:

func = function to be inverted

y=func(x)

optional INPUT: jaco = function that returns jacobian of func,

if not given, numerical differentiation is used

fparam = default additional parameter for func jparam = default additional parameter for corresponding jacobian

(if no jacobian function is given, and numerical differentiation must performed internaly, then jparam shall be either a 2-tupel, where the first element is the same parameter object as for func and the second element is te delta_x. )

eps = default convergence criteria when iteration is stopped (Rodgers),xtol=0.001

xa = default prior x fg = default first guess x sa = default prior error co-variance se = default measurement error co-variance mi = default maximum number of iterations

clip = default lower and upper limit

gnform = default Gauss Newton form (‘n’ or ‘m’)

OUTPUT:

func_inverse inverse of func

every optional input can be overwritten in the function

sicor.Tools.EnMAP.optimal_estimation.invert_lower(lo: Type[array]) → Type[array]

Calculate the inverse of a lower left triangular square matrix by forward substitution.

sicor.Tools.EnMAP.optimal_estimation.invert_np(inn: Type[array]) → Type[array]

sicor.Tools.EnMAP.optimal_estimation.invert_pupy(inn: Type[array]) → Type[array]

Pure python inversion of a square matrix.

sicor.Tools.EnMAP.optimal_estimation.invert_upper(up: Type[array]) → Type[array]

sicor.Tools.EnMAP.optimal_estimation.left_inverse(inn: Type[array]) → Type[array]

sicor.Tools.EnMAP.optimal_estimation.lu_decomposition(ua: Type[array]) → Tuple[Type[array], Type[array], Type[array]]

Decomposes A into P*A=L*U, L is lower, U is upper, P pivot sorting.

sicor.Tools.EnMAP.optimal_estimation.pivot(a: Type[array]) → Type[array]

Return p, so that p*a is sorted.

sicor.Tools.EnMAP.optimal_estimation.right_inverse(inn: Type[array]) → Type[array]

sicor.Tools.EnMAP.segmentation module

sicor.Tools.EnMAP.segmentation.SLIC_segmentation(data_rad_all, n_pca, segs)

Perform superpixel segmentation by searching the image for a handful of geographically and compositionally representative locations. Create spatially-contiguous clusters of similar radiances.

Parameters:

• data_rad_all – radiance data array with shape (rows, cols, bands)

• n_pca – number of principle components

• segs – number of segments

Returns:

flattened radiance data array, number of segments, array of segment labels for each pixel

sicor.Tools.EnMAP.segmentation._compute_coefficients_for_label(label_idx)

sicor.Tools.EnMAP.segmentation.empirical_line_solution(X, rdn_subset, data_l2a_seg, rows, cols, bands, segs, labels, land_only, processes, disable_progressbars)

Empirical line solution to infer exact result of remaining spectra.

Parameters:

• X – flattened radiance data array

• rdn_subset – radiance data subset

• data_l2a_seg – segmented surface reflectance

• rows – number of image rows

• cols – number of image columns

• bands – number of instrument bands

• segs – number of segments

• labels – array of segment labels for each pixel

• land_only – if True, SICOR is applied to land surfaces only (this may result in edge effects, e.g., at coastlines); otherwise, all image pixels (land + water) are processed; default: False

• processes – number of CPUs for multiprocessing

• disable_progressbars – True if progress bars should be disabled; default: False

Returns:

extrapolated surface reflectance for each pixel

sicor.Tools.EnMAP.segmentation.empirical_line_solution_snow(X, rdn_subset, data_l2a_seg, rows, cols, bands, segs, labels, processes)

Empirical line solution to infer exact result of remaining spectra, adapted to the estimation of snow and glacier ice surface properties.

Parameters:

• X – flattened radiance data array

• rdn_subset – radiance data subset

• data_l2a_seg – segmented surface reflectance

• rows – number of image rows

• cols – number of image columns

• bands – number of instrument bands

• segs – number of segments

• labels – array of segment labels for each pixel

• processes – number of CPUs for multiprocessing

Returns:

extrapolated surface reflectance for each pixel

Module contents