NEON¶
This notebook is an excercise in executing ISOFIT on two dates from the NEON dataset and interpreting the outputs of ISOFIT.
Prerequisites:
- Have ISOFIT installed and sRTMnet configured.
- Download the subset_data.zip and place the unzipped
datadirectory into tutorial's repository root. Alternatively, useisofit download examples --neonto automate this process.
Note: If you downloaded the ISOFIT extra data via isofit download, both sRTMnet and the NEON data files will be installed correctly and available with default settings for this notebook.
# Jupyter magics
%matplotlib inline
# Builtin
import os
import shutil
from pathlib import Path
from types import SimpleNamespace
# External
import matplotlib.pyplot as plt
import matplotlib.patches as patches
import numpy as np
from spectral.io import envi
# Internal
import isofit
from isofit.data import env
from isofit.utils.apply_oe import apply_oe
from isofit.utils.surface_model import surface_model
# Below are the default values for the ISOFIT environment. Change these if your environment differs
env.load('~/.isofit/isofit.ini') # Ini file to load
env.changeSection('DEFAULT') # Section of the ini to use
# env.changeBase('~./isofit') # Base path for ISOFIT extras (data, examples, etc)
# env.changePath('srtmnet', '/path/to/sRTMnet_v120.h5') # Overwrite the path to sRTMnet - copy this line for other products such as sixs if in non-default locations
print('Using environment paths:')
for key, path in env.items():
print(f"- {key} = {path}")
Using environment paths: - data = /home/runner/work/isofit-tutorials/data - examples = /home/runner/work/isofit-tutorials/isofit-tutorials - imagecube = /home/runner/work/isofit-tutorials/imagecube - srtmnet = /home/runner/work/isofit-tutorials/srtmnet - sixs = /home/runner/work/isofit-tutorials/sixs - surface = /home/runner/work/isofit-tutorials/surface - plots = /home/runner/work/isofit-tutorials/plots - libradtran = /home/runner/work/isofit-tutorials/libradtran - srtmnet.file = sRTMnet_v120.h5 - srtmnet.aux = sRTMnet_v120_aux.npz - libradtran.version = libRadtran-2.0.6
Setup¶
ISOFIT needs at minimum three pieces as input:
1. Radiance measurements (rdn)
2. Observation values (obs)
3. Location information (loc)
This sample dataset from NEON has radiance and observation data, but no location values (more recent NEON datasets include the location file). However, we can 'fake' the location file with sufficient accuracy for ISOFIT to run successfully. Note that there are data available for two dates:
Radiance
āāā 173647
ā āāā NIS01_20210403_173647_obs_ort
ā āāā NIS01_20210403_173647_obs_ort.hdr
ā āāā NIS01_20210403_173647_rdn_ort
ā āāā NIS01_20210403_173647_rdn_ort.hdr
āāā 174150
āāā NIS01_20210403_174150_obs_ort
āāā NIS01_20210403_174150_obs_ort.hdr
āāā NIS01_20210403_174150_rdn_ort
āāā NIS01_20210403_174150_rdn_ort.hdr
These files have corresponding in situ data as well, and below we've encoded the locations of each, which we can use to help subset data files.
# Extract the image locations of each point of interest (POI)
# These are defined in the NEON report as pixel locations, so we round here to convert to indices
report = {}
report['173647'] = { # Upp L Y | Low R Y | Upp L X | Low R X
'WhiteTarp': np.round([2224.9626, 2230.9771, 316.0078, 324.9385,]).astype(int),
'BlackTarp': np.round([2224.9626, 2231.0032, 328.0086, 333.9731,]).astype(int),
'Veg' : np.round([2245.0381, 2258.8103, 343.9006, 346.9423,]).astype(int),
'RoadEW' : np.round([2214.9905, 2216.9978, 348.9902, 373.0080,]).astype(int),
'RoadNS' : np.round([2205.9580, 2225.9612, 357.9536, 359.9608,]).astype(int)
}
report['174150'] = { # Upp L Y | Low R Y | Upp L X | Low R X
'WhiteTarp': np.round([653.9626, 659.9771, 3143.0078, 3151.9385]).astype(int),
'BlackTarp': np.round([653.9626, 660.0032, 3155.0086, 3160.9731]).astype(int),
'Veg' : np.round([674.0381, 687.8103, 3170.9006, 3173.9423]).astype(int),
'RoadEW' : np.round([643.9905, 645.9978, 3175.9902, 3200.0080]).astype(int),
'RoadNS' : np.round([634.9580, 654.9612, 3184.9536, 3186.9608]).astype(int)
}
# Which NEON date to process - change this to process a different date
neon_id = list(report.keys())[0]
neon_str = f"NIS01_20210403_{neon_id}"
# Select the locations from the neon id -- roi == Regions of Interest
roi = report[neon_id]
# Set the paths for this tutorial
base = Path(env.path('examples', 'NEON'))
raws = base / 'data'
data = base / 'neon_subset'
paths = SimpleNamespace(
rdn = str(data / f'{neon_str}_rdn_ort'),
loc = str(data / f'{neon_str}_loc_ort'),
obs = str(data / f'{neon_str}_obs_ort'),
insitu = raws,
output = base / 'output',
working = base / f'output/NIS01_20210403_{neon_id}',
surface = str(base / 'output/surface.mat'),
surface_config = env.path('examples', '20171108_Pasadena', 'configs', 'ang20171108t184227_surface.json')
)
paths.output.mkdir(exist_ok=True, parents=True)
# If you are missing either an OBS file or a LOC file, use these to create faked versions based off the radiance file
# This should not be needed if using the provided data
# Using this may cause the below plots to not generate the same results
# from isotuts import neon_utils
# paths.obs = neon_utils.fakeOBS(
# f"{paths.rdn}.hdr",
# sea = 153.4481201171875,
# sez = 178.3806858062744,
# soa = 39.8218994140625,
# soz = 39.8218994140625,
# slope = 31.813383102416992
# )[:-4] # Remove the .hdr extension from the return
# paths.loc = neon_utils.fakeLOC(
# rdn = f"{paths.rdn}.hdr",
# lon = -105.237000,
# lat = 40.125000,
# elv = 1689.0
# )[:-4]
Apply OE¶
The next part walks through running the ISOFIT utility script isofit/utils/apply_oe.py. This is the first step of executing ISOFIT and will generate a default configuration.
# First build a surface model
surface_model(
config_path = paths.surface_config,
output_path = paths.surface,
wavelength_path = f"{paths.rdn}.hdr"
)
0 ['/home/runner/work/isofit-tutorials/data/reflectance/surface_model_ucsb']
1 ['/home/runner/work/isofit-tutorials/data/reflectance/surface_model_ucsb']
2 ['/home/runner/work/isofit-tutorials/data/reflectance/surface_model_ucsb'] 3 ['/home/runner/work/isofit-tutorials/data/reflectance/surface_model_ucsb']
4 ['/home/runner/work/isofit-tutorials/data/reflectance/surface_model_ucsb'] 5 ['/home/runner/work/isofit-tutorials/data/reflectance/surface_model_ucsb']
6 ['/home/runner/work/isofit-tutorials/data/reflectance/surface_model_ucsb'] 7 ['/home/runner/work/isofit-tutorials/data/reflectance/surface_model_ucsb']
# For reference, all of the available parameters to the apply_oe script
print(apply_oe.__doc__)
Applies OE over a flightline using a radiative transfer engine. This executes
ISOFIT in a generalized way, accounting for the types of variation that might be
considered typical.
Observation (obs) and location (loc) files are used to determine appropriate
geometry lookup tables and provide a heuristic means of determining atmospheric
water ranges.
Parameters
----------
input_radiance : str
Radiance data cube. Expected to be ENVI format
input_loc : str
Location data cube of shape (Lon, Lat, Elevation). Expected to be ENVI format
input_obs : str
Observation data cube of shape:
(path length, to-sensor azimuth, to-sensor zenith,
to-sun azimuth, to-sun zenith, phase,
slope, aspect, cosine i, UTC time)
Expected to be ENVI format
working_directory : str
Directory to stage multiple outputs, will contain subdirectories
sensor : str
The sensor used for acquisition, will be used to set noise and datetime
settings
surface_path : str
Path to surface model or json dict of surface model configuration
copy_input_files : bool, default=False
Flag to choose to copy input_radiance, input_loc, and input_obs locally into
the working_directory
modtran_path : str, default=None
Location of MODTRAN utility. Alternately set with `MODTRAN_DIR` environment
variable
wavelength_path : str, default=None
Location to get wavelength information from, if not specified the radiance
header will be used
surface_category : str, default="multicomponent_surface"
The type of ISOFIT surface priors to use. Default is multicomponent_surface
aerosol_climatology_path : str, default=None
Specific aerosol climatology information to use in MODTRAN
rdn_factors_path : str, default=None
Specify a radiometric correction factor, if desired
atmosphere_type : str, default="ATM_MIDLAT_SUMMER"
Atmospheric profile to be used for MODTRAN simulations. Unused for other
radiative transfer models.
channelized_uncertainty_path : str, default=None
Path to a wavelength-specific channelized uncertainty file - used to augment Sy in the OE formalism
instrument_noise_path : str, default=None
Path to a wavelength-specific instrument noise file, used to derive per-wavelength NEDL / SNR
dn_uncertainty_file: str, default=None
Path to a linearity .mat file to augment S matrix with linearity uncertainty
model_discrepancy_path : str, default=None
Modifies S_eps in the OE formalism as the Gamma additive term, as:
S_eps = Sy + Kb.dot(self.Sb).dot(Kb.T) + Gamma
lut_config_file : str, default=None
Path to a look up table configuration file, which will override defaults
choices
multiple_restarts : bool, default=False
Use multiple initial starting poitns for each OE ptimization run, using
the corners of the atmospheric variables as starting points. This gives
a more robust, albeit more expensive, solution.
logging_level : str, default="INFO"
Logging level with which to run ISOFIT
log_file : str, default=None
File path to write ISOFIT logs to
n_cores : int, default=1
Number of cores to run ISOFIT with. Substantial parallelism is available, and
full runs will be very slow in serial. Suggested to max this out on the
available system
presolve : int, default=False
Flag to use a presolve mode to estimate the available atmospheric water range.
Runs a preliminary inversion over the image with a 1-D LUT of water vapor, and
uses the resulting range (slightly expanded) to bound determine the full LUT.
Advisable to only use with small cubes or in concert with the empirical_line
setting, or a significant speed penalty will be incurred
empirical_line : bool, default=False
Use an empirical line interpolation to run full inversions over only a subset
of pixels, determined using a SLIC superpixel segmentation, and use a KDTREE of
local solutions to interpolate radiance->reflectance. Generally a good option
if not trying to analyze the atmospheric state at fine scale resolution.
Mutually exclusive with analytical_line
analytical_line : bool, default=False
Use an analytical solution to the fixed atmospheric state to solve for each
pixel. Starts by running a full OE retrieval on each SLIC superpixel, then
interpolates the atmospheric state to each pixel, and closes with the
analytical solution.
Mutually exclusive with empirical_line
ray_temp_dir : str, default="/tmp/ray"
Location of temporary directory for ray parallelization engine
emulator_base : str, default=None
Location of emulator base path. Point this at the 3C or 6C .h5 files.
sRTMnet to use the emulator instead of MODTRAN. An additional file with the
same basename and the extention _aux.npz must accompany
e.g. /path/to/emulator.h5 /path/to/emulator_aux.npz
segmentation_size : int, default=40
If empirical_line is enabled, sets the size of segments to construct
num_neighbors : list[int], default=[]
Forced number of neighbors for empirical line extrapolation - overides default
set from segmentation_size parameter
atm_sigma : list[int], default=[2]
A list of smoothing factors to use during the atmospheric interpolation, one
for each atmospheric parameter (or broadcast to all if only one is provided).
Only used with the analytical line.
pressure_elevation : bool, default=False
Flag to retrieve elevation
prebuilt_lut : str, default=None
Use this pre-constructed look up table for all retrievals. Must be an
ISOFIT-compatible RTE NetCDF
no_min_lut_spacing : bool, default=False
Don't allow the LUTConfig to remove a LUT dimension because of minimal spacing.
inversion_windows : list[float], default=None
Override the default inversion windows. Will supercede any sensor specific
defaults that are in place.
Must be in 2-item tuples
config_only : bool, default=False
Generates the configuration then exits before execution. If presolve is
enabled, that run will still occur.
interpolate_bad_rdn : bool, default=False
Flag to perform a per-pixel interpolation across no-data and NaN data bands.
Does not interpolate vectors that are entire no-data or NaN, only partial.
Currently only designed for wavelength interpolation on spectra.
Does NOT do any spatial interpolation
interpolate_inplace : bool, default=False
Flag to tell interpolation to work on the file in place, or generate a
new interpolated rdn file. The location of the new file will be in the
"input" directory within the working directory.
skyview_factor : str, default=None
Flag to determine method to account for skyview factor. Default is None, creating an array of 1s.
Other option is "slope" which will approx. based on cos^2(slope/2).
Other option is a path to a skyview ENVI file computed via skyview.py utility or other source.
Please note data must range from 0-1.
resources : bool, default=False
Enables the system resource tracker. Must also have the log_file set.
retrieve_co2 : bool, default=False
Flag to retrieve CO2 in the state vector. Only available with emulator at the moment.
eof_path : str, default=None
Add 1 or 2 Empirical Orthogonal Functions to the state vector. File is a 1-2 column text file
with one number per instrument channel.
terrain_style : str, default=dem
Flag to set the terrain style. dem uses provided obs values, flat sets the surface to the spheroid
References
----------
D.R. Thompson, A. Braverman,P.G. Brodrick, A. Candela, N. Carbon, R.N. Clark,D. Connelly, R.O. Green, R.F.
Kokaly, L. Li, N. Mahowald, R.L. Miller, G.S. Okin, T.H.Painter, G.A. Swayze, M. Turmon, J. Susilouto, and
D.S. Wettergreen. Quantifying Uncertainty for Remote Spectroscopy of Surface Composition. Remote Sensing of
Environment, 2020. doi: https://doi.org/10.1016/j.rse.2020.111898.
sRTMnet emulator:
P.G. Brodrick, D.R. Thompson, J.E. Fahlen, M.L. Eastwood, C.M. Sarture, S.R. Lundeen, W. Olson-Duvall,
N. Carmon, and R.O. Green. Generalized radiative transfer emulation for imaging spectroscopy reflectance
retrievals. Remote Sensing of Environment, 261:112476, 2021.doi: 10.1016/j.rse.2021.112476.
# Add a ray shutdown, just in case this is being re-called
import ray
ray.shutdown()
# Cleanup any previous runs; comment this out if you want to preserve a previous run's output
if Path(paths.working).exists():
shutil.rmtree(paths.working)
apply_oe(
input_radiance = paths.rdn, # Radiance
input_loc = paths.loc, # Location
input_obs = paths.obs, # Observations
working_directory = str(paths.working), # Output directory
sensor = "neon",
surface_path = paths.surface, # Surface priors - often changes
emulator_base = f"{env.srtmnet}/sRTMnet_v120.h5",
surface_category = "multicomponent_surface",
modtran_path = None,
atmosphere_type = "ATM_MIDLAT_SUMMER", # MODTRAN
aerosol_climatology_path = None, # MODTRAN
rdn_factors_path = None, # RCC update used 'on the fly'
model_discrepancy_path = None, # Model discrepancy term - handle things like unknown radiative transfer model effects
channelized_uncertainty_path = None, # Channelized uncertainty - if you have an instrument model
multiple_restarts = False, # Useful if the AOD conditions are really challenging
presolve = True, # Attempts to solve for the right wv range
empirical_line = False, # wavelength-specific local linear interpolation between radiance and reflectance
analytical_line = True, # mathematical representation of OE given that the atmsophere is known
segmentation_size = 10,
num_neighbors = [5],
atm_sigma = [0.5, 0.5],
pressure_elevation = False,
n_cores = os.cpu_count(),
)
INFO:2026-02-24,21:02:27 || apply_oe.py:apply_oe() | Checking input data files...
INFO:2026-02-24,21:02:27 || apply_oe.py:apply_oe() | ...Data file checks complete
INFO:2026-02-24,21:02:27 || apply_oe.py:apply_oe() | Setting up files and directories....
INFO:2026-02-24,21:02:27 || template_construction.py:__init__() | Flightline ID: NIS01_20210403_173647
INFO:2026-02-24,21:02:27 || template_construction.py:__init__() | no noise path found, proceeding without
INFO:2026-02-24,21:02:27 || apply_oe.py:apply_oe() | ...file/directory setup complete
INFO:2026-02-24,21:02:27 || apply_oe.py:apply_oe() | Using inversion windows: [[350.0, 1360.0], [1410, 1800.0], [1970.0, 2500.0]]
INFO:2026-02-24,21:02:27 || template_construction.py:get_wavelengths() | No wavelength file provided. Obtaining wavelength grid from ENVI header of radiance cube.
INFO:2026-02-24,21:02:27 || template_construction.py:get_wavelengths() | Wavelength units of nm inferred...converting to microns
WARNING:2026-02-24,21:02:27 || template_construction.py:check_surface_model() | Center wavelengths provided in surface model file do not match wavelengths in radiance cube. Please consider rebuilding your surface model for optimal performance.
INFO:2026-02-24,21:02:27 || apply_oe.py:apply_oe() | Observation means:
INFO:2026-02-24,21:02:27 || apply_oe.py:apply_oe() | Path (km): 1.0036078691482544
INFO:2026-02-24,21:02:27 || apply_oe.py:apply_oe() | To-sensor azimuth (deg): 153.4481201171875
INFO:2026-02-24,21:02:27 || apply_oe.py:apply_oe() | To-sensor zenith (deg): 1.619314193725586
INFO:2026-02-24,21:02:27 || apply_oe.py:apply_oe() | To-sun azimuth (deg): 145.23248291015625
INFO:2026-02-24,21:02:27 || apply_oe.py:apply_oe() | To-sun zenith (deg): 39.8218994140625
INFO:2026-02-24,21:02:27 || apply_oe.py:apply_oe() | Relative to-sun azimuth (deg): 31.813383102416992
INFO:2026-02-24,21:02:27 || apply_oe.py:apply_oe() | Altitude (km): 2.692207098007202
INFO:2026-02-24,21:02:27 || apply_oe.py:apply_oe() | Segmenting...
2026-02-24 21:02:31,421 INFO worker.py:2013 -- Started a local Ray instance.
/home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.13/site-packages/ray/_private/worker.py:2052: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn(
(segment_chunk pid=3996) INFO:2026-02-24,21:02:33 ||| 0: starting
INFO:2026-02-24,21:02:34 || apply_oe.py:apply_oe() | Extracting /home/runner/work/isofit-tutorials/isofit-tutorials/NEON/output/NIS01_20210403_173647/input/NIS01_20210403_173647_subs_rdn
2026-02-24 21:02:34,033 INFO worker.py:1831 -- Calling ray.init() again after it has already been called.
(segment_chunk pid=3996) INFO:2026-02-24,21:02:34 ||| 0: completing (extract_chunk pid=3996) INFO:2026-02-24,21:02:34 ||| ini does not exist, falling back to defaults: ~/.isofit/isofit.ini
INFO:2026-02-24,21:02:35 || apply_oe.py:apply_oe() | Extracting /home/runner/work/isofit-tutorials/isofit-tutorials/NEON/output/NIS01_20210403_173647/input/NIS01_20210403_173647_subs_obs
2026-02-24 21:02:35,847 INFO worker.py:1831 -- Calling ray.init() again after it has already been called.
INFO:2026-02-24,21:02:35 || apply_oe.py:apply_oe() | Extracting /home/runner/work/isofit-tutorials/isofit-tutorials/NEON/output/NIS01_20210403_173647/input/NIS01_20210403_173647_subs_loc
2026-02-24 21:02:35,873 INFO worker.py:1831 -- Calling ray.init() again after it has already been called.
INFO:2026-02-24,21:02:35 || apply_oe.py:apply_oe() | Skipping None, because is not a path.
INFO:2026-02-24,21:02:35 || apply_oe.py:apply_oe() | Skipping None, because is not a path.
INFO:2026-02-24,21:02:35 || apply_oe.py:apply_oe() | Pre-solve H2O grid: [0.2 0.84 1.49 2.13 2.77 3.42 4.06 4.7 5.35 5.99]
INFO:2026-02-24,21:02:35 || apply_oe.py:apply_oe() | Writing H2O pre-solve configuration file.
INFO:2026-02-24,21:02:35 || ini.py:toTemplate() | Saved converted json to: /home/runner/work/isofit-tutorials/isofit-tutorials/NEON/output/NIS01_20210403_173647/config/NIS01_20210403_173647_h2o.json.tmpl
INFO:2026-02-24,21:02:35 || apply_oe.py:apply_oe() | Run ISOFIT initial guess
WARNING:2026-02-24,21:02:35 || __init__.py:checkNumThreads() | ****************************************************************************************** ! Number of threads is greater than 1 (currently: 4), this may greatly impact performance ! Please set this the environment variables 'MKL_NUM_THREADS' and 'OMP_NUM_THREADS' to '1' ******************************************************************************************
INFO:2026-02-24,21:02:35 || configs.py:create_new_config() | Loading config file: /home/runner/work/isofit-tutorials/isofit-tutorials/NEON/output/NIS01_20210403_173647/config/NIS01_20210403_173647_h2o.json
INFO:2026-02-24,21:02:35 || configs.py:get_config_errors() | Checking config sections for configuration issues
INFO:2026-02-24,21:02:35 || configs.py:get_config_errors() | Configuration file checks complete, no errors found.
/home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.13/site-packages/numpy/_core/fromnumeric.py:3824: RuntimeWarning: Mean of empty slice return _methods._mean(a, axis=axis, dtype=dtype, /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.13/site-packages/numpy/_core/_methods.py:142: RuntimeWarning: invalid value encountered in scalar divide ret = ret.dtype.type(ret / rcount) (extract_chunk pid=3996) INFO:2026-02-24,21:02:35 ||| 0: starting (extract_chunk pid=3996) INFO:2026-02-24,21:02:35 ||| 0: starting (extract_chunk pid=3996) INFO:2026-02-24,21:02:35 ||| 0: starting
2026-02-24 21:02:36,212 INFO worker.py:1831 -- Calling ray.init() again after it has already been called.
INFO:2026-02-24,21:02:36 || isofit.py:run() | Running surfaces: uniform_surface
INFO:2026-02-24,21:02:36 || radiative_transfer_engine.py:__init__() | Loading from wavelength_file: /home/runner/work/isofit-tutorials/isofit-tutorials/NEON/output/NIS01_20210403_173647/data/wavelengths.txt
INFO:2026-02-24,21:02:36 || radiative_transfer_engine.py:__init__() | No LUT store found, beginning initialization and simulations
INFO:2026-02-24,21:02:36 || radiative_transfer_engine.py:__init__() | Initializing LUT file
INFO:2026-02-24,21:02:36 || radiative_transfer_engine.py:runSimulations() | Running any pre-sim functions
INFO:2026-02-24,21:02:36 || sRTMnet.py:preSim() | Creating a simulator configuration
INFO:2026-02-24,21:02:36 || sRTMnet.py:preSim() | Building simulator and executing (6S)
INFO:2026-02-24,21:02:36 || radiative_transfer_engine.py:__init__() | No LUT store found, beginning initialization and simulations
INFO:2026-02-24,21:02:36 || radiative_transfer_engine.py:__init__() | Initializing LUT file
INFO:2026-02-24,21:02:36 || radiative_transfer_engine.py:runSimulations() | Running any pre-sim functions
INFO:2026-02-24,21:02:36 || radiative_transfer_engine.py:runSimulations() | Executing parallel simulations
INFO:2026-02-24,21:02:44 || common.py:__call__() | 20.00% simulations complete (elapsed: 0:00:07.544804, rate: 0:00:00.754480, eta: 0:01:07.903236)
INFO:2026-02-24,21:02:44 || radiative_transfer_engine.py:runSimulations() | Flushing netCDF to disk
INFO:2026-02-24,21:02:44 || common.py:__call__() | 30.00% simulations complete (elapsed: 0:00:07.811227, rate: 0:00:00.781123, eta: 0:00:31.244908)
INFO:2026-02-24,21:02:44 || radiative_transfer_engine.py:runSimulations() | Flushing netCDF to disk
INFO:2026-02-24,21:02:44 || common.py:__call__() | 40.00% simulations complete (elapsed: 0:00:08.069939, rate: 0:00:00.806994, eta: 0:00:18.829858)
INFO:2026-02-24,21:02:44 || radiative_transfer_engine.py:runSimulations() | Flushing netCDF to disk
INFO:2026-02-24,21:02:45 || common.py:__call__() | 50.00% simulations complete (elapsed: 0:00:08.647858, rate: 0:00:00.864786, eta: 0:00:12.971787)
INFO:2026-02-24,21:02:45 || radiative_transfer_engine.py:runSimulations() | Flushing netCDF to disk
INFO:2026-02-24,21:02:48 || common.py:__call__() | 60.00% simulations complete (elapsed: 0:00:11.722653, rate: 0:00:01.172265, eta: 0:00:11.722653)
INFO:2026-02-24,21:02:48 || radiative_transfer_engine.py:runSimulations() | Flushing netCDF to disk
INFO:2026-02-24,21:02:48 || common.py:__call__() | 70.00% simulations complete (elapsed: 0:00:12.137728, rate: 0:00:01.213773, eta: 0:00:08.091819)
INFO:2026-02-24,21:02:48 || radiative_transfer_engine.py:runSimulations() | Flushing netCDF to disk
INFO:2026-02-24,21:02:49 || common.py:__call__() | 80.00% simulations complete (elapsed: 0:00:12.426125, rate: 0:00:01.242612, eta: 0:00:05.325482)
INFO:2026-02-24,21:02:49 || radiative_transfer_engine.py:runSimulations() | Flushing netCDF to disk
INFO:2026-02-24,21:02:49 || common.py:__call__() | 90.00% simulations complete (elapsed: 0:00:12.874530, rate: 0:00:01.287453, eta: 0:00:03.218632)
INFO:2026-02-24,21:02:49 || radiative_transfer_engine.py:runSimulations() | Flushing netCDF to disk
INFO:2026-02-24,21:02:51 || common.py:__call__() | 100.00% simulations complete (elapsed: 0:00:14.592269, rate: 0:00:01.459227, eta: 0:00:01.621363)
INFO:2026-02-24,21:02:51 || radiative_transfer_engine.py:runSimulations() | Flushing netCDF to disk
INFO:2026-02-24,21:02:51 || radiative_transfer_engine.py:runSimulations() | Running any post-sim functions
INFO:2026-02-24,21:02:52 || radiative_transfer_engine.py:runSimulations() | Saving post-sim data to index zero of all dimensions except wl
INFO:2026-02-24,21:02:52 || luts.py:load() | Loading LUT into memory
WARNING:2026-02-24,21:02:52 || luts.py:load() | thermal_upwelling is fully NaN, leaving as-is
WARNING:2026-02-24,21:02:52 || luts.py:load() | thermal_downwelling is fully NaN, leaving as-is
INFO:2026-02-24,21:02:52 || sRTMnet.py:preSim() | Interpolating simulator quantities to emulator size
INFO:2026-02-24,21:02:52 || sRTMnet.py:preSim() | Loading and predicting with emulator on 4 cores
INFO:2026-02-24,21:02:54 || radiative_transfer_engine.py:runSimulations() | Saving pre-sim data to index zero of all dimensions except wl
INFO:2026-02-24,21:02:54 || radiative_transfer_engine.py:runSimulations() | Running any post-sim functions
INFO:2026-02-24,21:02:54 || luts.py:load() | Loading LUT into memory
WARNING:2026-02-24,21:02:54 || luts.py:load() | thermal_upwelling is fully NaN, leaving as-is
WARNING:2026-02-24,21:02:54 || luts.py:load() | thermal_downwelling is fully NaN, leaving as-is
INFO:2026-02-24,21:02:54 || isofit.py:run() | Beginning 420 inversions in 40 chunks using 4 cores
(Worker pid=4295) ERROR:2026-02-24,21:03:00 ||| [x] Data path does not exist (Worker pid=4295) ERROR:2026-02-24,21:03:00 ||| The following path does not exist, please verify your installation environment: /home/runner/.isofit/data/earth_sun_distance.txt (Worker pid=4295) WARNING:2026-02-24,21:03:00 ||| Earth-sun-distance file not found on system. Proceeding without might cause some inaccuracies down the line. (Worker pid=4295) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.13/site-packages/numpy/_core/fromnumeric.py:3824: RuntimeWarning: Mean of empty slice (Worker pid=4295) return _methods._mean(a, axis=axis, dtype=dtype, (Worker pid=4295) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.13/site-packages/numpy/_core/_methods.py:142: RuntimeWarning: invalid value encountered in scalar divide (Worker pid=4295) ret = ret.dtype.type(ret / rcount)
(Worker pid=4295) INFO:2026-02-24,21:03:01 ||| Worker 0 completed 1/~105.0:: 0.95% complete
(Worker pid=4295) INFO:2026-02-24,21:03:07 ||| Worker at start location (42,0) completed 10/11 (Worker pid=4298) ERROR:2026-02-24,21:03:01 ||| [x] Data path does not exist [repeated 3x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (Worker pid=4298) ERROR:2026-02-24,21:03:01 ||| The following path does not exist, please verify your installation environment: /home/runner/.isofit/data/earth_sun_distance.txt [repeated 3x across cluster] (Worker pid=4298) WARNING:2026-02-24,21:03:01 ||| Earth-sun-distance file not found on system. Proceeding without might cause some inaccuracies down the line. [repeated 3x across cluster] (Worker pid=4298) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.13/site-packages/numpy/_core/fromnumeric.py:3824: RuntimeWarning: Mean of empty slice [repeated 3x across cluster] (Worker pid=4298) return _methods._mean(a, axis=axis, dtype=dtype, [repeated 3x across cluster] (Worker pid=4298) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.13/site-packages/numpy/_core/_methods.py:142: RuntimeWarning: invalid value encountered in scalar divide [repeated 3x across cluster] (Worker pid=4298) ret = ret.dtype.type(ret / rcount) [repeated 3x across cluster] (Worker pid=4298) INFO:2026-02-24,21:03:02 ||| Worker 3 completed 1/~105.0:: 0.95% complete [repeated 3x across cluster]
(Worker pid=4295) INFO:2026-02-24,21:03:13 ||| Worker at start location (52,0) completed 9/10 [repeated 4x across cluster] (Worker pid=4296) INFO:2026-02-24,21:03:08 ||| Worker 2 completed 12/~105.0:: 11.43% complete [repeated 4x across cluster]
(Worker pid=4295) INFO:2026-02-24,21:03:19 ||| Worker at start location (95,0) completed 9/10 [repeated 4x across cluster] (Worker pid=4296) INFO:2026-02-24,21:03:15 ||| Worker 2 completed 23/~105.0:: 21.9% complete [repeated 4x across cluster]
(Worker pid=4295) INFO:2026-02-24,21:03:26 ||| Worker at start location (139,0) completed 10/11 [repeated 4x across cluster] (Worker pid=4296) INFO:2026-02-24,21:03:22 ||| Worker 2 completed 34/~105.0:: 32.38% complete [repeated 4x across cluster]
(Worker pid=4295) INFO:2026-02-24,21:03:32 ||| Worker at start location (182,0) completed 10/11 [repeated 4x across cluster] (Worker pid=4298) INFO:2026-02-24,21:03:28 ||| Worker 3 completed 44/~105.0:: 41.9% complete [repeated 4x across cluster]
(Worker pid=4295) INFO:2026-02-24,21:03:39 ||| Worker at start location (225,0) completed 10/11 [repeated 4x across cluster] (Worker pid=4296) INFO:2026-02-24,21:03:35 ||| Worker 2 completed 56/~105.0:: 53.33% complete [repeated 4x across cluster]
(Worker pid=4295) INFO:2026-02-24,21:03:45 ||| Worker at start location (268,0) completed 10/11 [repeated 4x across cluster] (Worker pid=4296) INFO:2026-02-24,21:03:41 ||| Worker 2 completed 67/~105.0:: 63.81% complete [repeated 4x across cluster]
(Worker pid=4295) INFO:2026-02-24,21:03:52 ||| Worker at start location (311,0) completed 10/11 [repeated 4x across cluster] (Worker pid=4296) INFO:2026-02-24,21:03:48 ||| Worker 2 completed 78/~105.0:: 74.29% complete [repeated 4x across cluster]
(Worker pid=4295) INFO:2026-02-24,21:03:59 ||| Worker at start location (354,0) completed 10/11 [repeated 4x across cluster] (Worker pid=4296) INFO:2026-02-24,21:03:55 ||| Worker 2 completed 89/~105.0:: 84.76% complete [repeated 4x across cluster]
(Worker pid=4297) INFO:2026-02-24,21:04:04 ||| Worker at start location (419,0) completed 10/11 [repeated 5x across cluster] (Worker pid=4297) INFO:2026-02-24,21:03:59 ||| Worker 1 completed 97/~105.0:: 92.38% complete [repeated 3x across cluster] INFO:2026-02-24,21:04:04 || isofit.py:run() | Pixel class: uniform_surface inversions complete.
INFO:2026-02-24,21:04:04 || isofit.py:run() | 69.96s total
INFO:2026-02-24,21:04:04 || isofit.py:run() | 6.0031 spectra/s
INFO:2026-02-24,21:04:04 || isofit.py:run() | 1.5008 spectra/s/core
INFO:2026-02-24,21:04:04 || isofit.py:run() | All Inversions complete.
INFO:2026-02-24,21:04:04 || isofit.py:run() | 88.66s total
INFO:2026-02-24,21:04:04 || isofit.py:run() | 4.7373 spectra/s
INFO:2026-02-24,21:04:04 || isofit.py:run() | 1.1843 spectra/s/core
INFO:2026-02-24,21:04:04 || apply_oe.py:apply_oe() | Full (non-aerosol) LUTs:
INFO:2026-02-24,21:04:04 || apply_oe.py:apply_oe() | Elevation: None
INFO:2026-02-24,21:04:04 || apply_oe.py:apply_oe() | To-sensor zenith: [0.9608 2.9675]
INFO:2026-02-24,21:04:04 || apply_oe.py:apply_oe() | To-sun zenith: None
INFO:2026-02-24,21:04:04 || apply_oe.py:apply_oe() | Relative to-sun azimuth: [3.80000e-03 4.12002e+01 8.23965e+01]
INFO:2026-02-24,21:04:04 || apply_oe.py:apply_oe() | H2O Vapor: [0.6033 0.6628]
INFO:2026-02-24,21:04:04 || apply_oe.py:apply_oe() | Writing main configuration file.
INFO:2026-02-24,21:04:04 || template_construction.py:load_climatology() | Loading Climatology
INFO:2026-02-24,21:04:04 || template_construction.py:load_climatology() | Climatology Loaded. Aerosol State Vector:
{'AOT550': {'bounds': [0.04209, 1.0], 'scale': 1, 'init': 0.137881, 'prior_sigma': 10.0, 'prior_mean': 0.137881}}
Aerosol LUT Grid:
{'AOT550': [0.04209, 0.137881, 0.233672, 0.329463, 0.425254, 0.521045, 0.6168359999999999, 0.712627, 0.808418, 0.9042089999999999, 1.0]}
Aerosol model path:/home/runner/work/isofit-tutorials/data/aerosol_model.txt
INFO:2026-02-24,21:04:04 || ini.py:toTemplate() | Saved converted json to: /home/runner/work/isofit-tutorials/isofit-tutorials/NEON/output/NIS01_20210403_173647/config/NIS01_20210403_173647_isofit.json.tmpl
INFO:2026-02-24,21:04:04 || apply_oe.py:apply_oe() | Running ISOFIT with full LUT
WARNING:2026-02-24,21:04:04 || __init__.py:checkNumThreads() | ****************************************************************************************** ! Number of threads is greater than 1 (currently: 4), this may greatly impact performance ! Please set this the environment variables 'MKL_NUM_THREADS' and 'OMP_NUM_THREADS' to '1' ******************************************************************************************
INFO:2026-02-24,21:04:04 || configs.py:create_new_config() | Loading config file: /home/runner/work/isofit-tutorials/isofit-tutorials/NEON/output/NIS01_20210403_173647/config/NIS01_20210403_173647_isofit.json
INFO:2026-02-24,21:04:04 || configs.py:get_config_errors() | Checking config sections for configuration issues
INFO:2026-02-24,21:04:04 || configs.py:get_config_errors() | Configuration file checks complete, no errors found.
/home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.13/site-packages/numpy/_core/fromnumeric.py:3824: RuntimeWarning: Mean of empty slice return _methods._mean(a, axis=axis, dtype=dtype, /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.13/site-packages/numpy/_core/_methods.py:142: RuntimeWarning: invalid value encountered in scalar divide ret = ret.dtype.type(ret / rcount)
2026-02-24 21:04:05,169 INFO worker.py:1831 -- Calling ray.init() again after it has already been called.
INFO:2026-02-24,21:04:05 || isofit.py:run() | Running surfaces: uniform_surface
INFO:2026-02-24,21:04:05 || radiative_transfer_engine.py:__init__() | Loading from wavelength_file: /home/runner/work/isofit-tutorials/isofit-tutorials/NEON/output/NIS01_20210403_173647/data/wavelengths.txt
INFO:2026-02-24,21:04:05 || radiative_transfer_engine.py:__init__() | No LUT store found, beginning initialization and simulations
INFO:2026-02-24,21:04:05 || radiative_transfer_engine.py:__init__() | Initializing LUT file
INFO:2026-02-24,21:04:05 || radiative_transfer_engine.py:runSimulations() | Running any pre-sim functions
INFO:2026-02-24,21:04:05 || sRTMnet.py:preSim() | Creating a simulator configuration
INFO:2026-02-24,21:04:05 || sRTMnet.py:preSim() | Building simulator and executing (6S)
INFO:2026-02-24,21:04:05 || radiative_transfer_engine.py:__init__() | No LUT store found, beginning initialization and simulations
INFO:2026-02-24,21:04:05 || radiative_transfer_engine.py:__init__() | Initializing LUT file
INFO:2026-02-24,21:04:05 || radiative_transfer_engine.py:runSimulations() | Running any pre-sim functions
INFO:2026-02-24,21:04:05 || radiative_transfer_engine.py:runSimulations() | Executing parallel simulations
INFO:2026-02-24,21:04:30 || common.py:__call__() | 10.61% simulations complete (elapsed: 0:00:24.937491, rate: 0:00:00.188920, eta: 0:03:44.437419)
INFO:2026-02-24,21:04:30 || radiative_transfer_engine.py:runSimulations() | Flushing netCDF to disk
INFO:2026-02-24,21:04:46 || common.py:__call__() | 20.45% simulations complete (elapsed: 0:00:40.191390, rate: 0:00:00.304480, eta: 0:02:40.765560)
INFO:2026-02-24,21:04:46 || radiative_transfer_engine.py:runSimulations() | Flushing netCDF to disk
INFO:2026-02-24,21:05:01 || common.py:__call__() | 30.30% simulations complete (elapsed: 0:00:55.769390, rate: 0:00:00.422495, eta: 0:02:10.128577)
INFO:2026-02-24,21:05:01 || radiative_transfer_engine.py:runSimulations() | Flushing netCDF to disk
INFO:2026-02-24,21:05:21 || common.py:__call__() | 40.15% simulations complete (elapsed: 0:01:15.397363, rate: 0:00:00.571192, eta: 0:01:53.096045)
INFO:2026-02-24,21:05:21 || radiative_transfer_engine.py:runSimulations() | Flushing netCDF to disk
INFO:2026-02-24,21:05:37 || common.py:__call__() | 50.00% simulations complete (elapsed: 0:01:31.692184, rate: 0:00:00.694638, eta: 0:01:31.692184)
INFO:2026-02-24,21:05:37 || radiative_transfer_engine.py:runSimulations() | Flushing netCDF to disk
INFO:2026-02-24,21:05:52 || common.py:__call__() | 60.61% simulations complete (elapsed: 0:01:46.380820, rate: 0:00:00.805915, eta: 0:01:10.920547)
INFO:2026-02-24,21:05:52 || radiative_transfer_engine.py:runSimulations() | Flushing netCDF to disk
INFO:2026-02-24,21:06:11 || common.py:__call__() | 70.45% simulations complete (elapsed: 0:02:05.306915, rate: 0:00:00.949295, eta: 0:00:53.702964)
INFO:2026-02-24,21:06:11 || radiative_transfer_engine.py:runSimulations() | Flushing netCDF to disk
INFO:2026-02-24,21:06:26 || common.py:__call__() | 80.30% simulations complete (elapsed: 0:02:20.785525, rate: 0:00:01.066557, eta: 0:00:35.196381)
INFO:2026-02-24,21:06:26 || radiative_transfer_engine.py:runSimulations() | Flushing netCDF to disk
INFO:2026-02-24,21:06:42 || common.py:__call__() | 90.15% simulations complete (elapsed: 0:02:36.902678, rate: 0:00:01.188657, eta: 0:00:17.433631)
INFO:2026-02-24,21:06:42 || radiative_transfer_engine.py:runSimulations() | Flushing netCDF to disk
INFO:2026-02-24,21:06:59 || common.py:__call__() | 100.00% simulations complete (elapsed: 0:02:53.186238, rate: 0:00:01.312017, eta: 0:00:00)
INFO:2026-02-24,21:06:59 || radiative_transfer_engine.py:runSimulations() | Flushing netCDF to disk
INFO:2026-02-24,21:06:59 || radiative_transfer_engine.py:runSimulations() | Running any post-sim functions
INFO:2026-02-24,21:06:59 || radiative_transfer_engine.py:runSimulations() | Saving post-sim data to index zero of all dimensions except wl
INFO:2026-02-24,21:06:59 || luts.py:load() | Loading LUT into memory
WARNING:2026-02-24,21:06:59 || luts.py:load() | thermal_upwelling is fully NaN, leaving as-is
WARNING:2026-02-24,21:06:59 || luts.py:load() | thermal_downwelling is fully NaN, leaving as-is
INFO:2026-02-24,21:06:59 || sRTMnet.py:preSim() | Interpolating simulator quantities to emulator size
INFO:2026-02-24,21:06:59 || sRTMnet.py:preSim() | Loading and predicting with emulator on 4 cores
INFO:2026-02-24,21:07:01 || radiative_transfer_engine.py:runSimulations() | Saving pre-sim data to index zero of all dimensions except wl
INFO:2026-02-24,21:07:02 || radiative_transfer_engine.py:runSimulations() | Running any post-sim functions
INFO:2026-02-24,21:07:02 || luts.py:load() | Loading LUT into memory
WARNING:2026-02-24,21:07:02 || luts.py:load() | thermal_upwelling is fully NaN, leaving as-is
WARNING:2026-02-24,21:07:02 || luts.py:load() | thermal_downwelling is fully NaN, leaving as-is
INFO:2026-02-24,21:07:02 || isofit.py:run() | Beginning 420 inversions in 40 chunks using 4 cores
(Worker pid=5142) ERROR:2026-02-24,21:07:08 ||| [x] Data path does not exist (Worker pid=5142) ERROR:2026-02-24,21:07:08 ||| The following path does not exist, please verify your installation environment: /home/runner/.isofit/data/earth_sun_distance.txt (Worker pid=5142) WARNING:2026-02-24,21:07:08 ||| Earth-sun-distance file not found on system. Proceeding without might cause some inaccuracies down the line. (Worker pid=4295) INFO:2026-02-24,21:04:04 ||| Worker at start location (397,0) completed 10/11 (Worker pid=5142) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.13/site-packages/numpy/_core/fromnumeric.py:3824: RuntimeWarning: Mean of empty slice (Worker pid=5142) return _methods._mean(a, axis=axis, dtype=dtype, (Worker pid=5142) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.13/site-packages/numpy/_core/_methods.py:142: RuntimeWarning: invalid value encountered in scalar divide (Worker pid=5142) ret = ret.dtype.type(ret / rcount)
(Worker pid=5143) INFO:2026-02-24,21:07:13 ||| Worker 3 completed 1/~105.0:: 0.95% complete
(Worker pid=5143) ERROR:2026-02-24,21:07:08 ||| [x] Data path does not exist [repeated 3x across cluster] (Worker pid=5143) ERROR:2026-02-24,21:07:08 ||| The following path does not exist, please verify your installation environment: /home/runner/.isofit/data/earth_sun_distance.txt [repeated 3x across cluster] (Worker pid=5143) WARNING:2026-02-24,21:07:08 ||| Earth-sun-distance file not found on system. Proceeding without might cause some inaccuracies down the line. [repeated 3x across cluster] (Worker pid=5143) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.13/site-packages/numpy/_core/fromnumeric.py:3824: RuntimeWarning: Mean of empty slice [repeated 3x across cluster] (Worker pid=5143) return _methods._mean(a, axis=axis, dtype=dtype, [repeated 3x across cluster] (Worker pid=5143) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.13/site-packages/numpy/_core/_methods.py:142: RuntimeWarning: invalid value encountered in scalar divide [repeated 3x across cluster] (Worker pid=5143) ret = ret.dtype.type(ret / rcount) [repeated 3x across cluster]
(Worker pid=5143) INFO:2026-02-24,21:07:59 ||| Worker at start location (9,0) completed 9/10 (Worker pid=5142) INFO:2026-02-24,21:07:14 ||| Worker 0 completed 1/~105.0:: 0.95% complete [repeated 3x across cluster]
(Worker pid=5143) INFO:2026-02-24,21:08:04 ||| Worker 3 completed 11/~105.0:: 10.48% complete
(Worker pid=5145) INFO:2026-02-24,21:08:06 ||| Worker at start location (31,0) completed 10/11
(Worker pid=5145) INFO:2026-02-24,21:08:12 ||| Worker 1 completed 12/~105.0:: 11.43% complete (Worker pid=5142) INFO:2026-02-24,21:08:09 ||| Worker at start location (42,0) completed 10/11 [repeated 2x across cluster]
(Worker pid=5142) INFO:2026-02-24,21:08:14 ||| Worker 0 completed 12/~105.0:: 11.43% complete [repeated 2x across cluster] (Worker pid=5143) INFO:2026-02-24,21:08:52 ||| Worker at start location (52,0) completed 9/10
(Worker pid=5143) INFO:2026-02-24,21:08:58 ||| Worker 3 completed 21/~105.0:: 20.0% complete
(Worker pid=5144) INFO:2026-02-24,21:09:06 ||| Worker at start location (74,0) completed 10/11
(Worker pid=5144) INFO:2026-02-24,21:09:11 ||| Worker 2 completed 23/~105.0:: 21.9% complete
(Worker pid=5145) INFO:2026-02-24,21:09:09 ||| Worker at start location (63,0) completed 10/11 [repeated 2x across cluster]
(Worker pid=5145) INFO:2026-02-24,21:09:14 ||| Worker 1 completed 23/~105.0:: 21.9% complete [repeated 2x across cluster] (Worker pid=5143) INFO:2026-02-24,21:09:41 ||| Worker at start location (95,0) completed 9/10
(Worker pid=5144) INFO:2026-02-24,21:10:01 ||| Worker at start location (106,0) completed 10/11 (Worker pid=5143) INFO:2026-02-24,21:09:46 ||| Worker 3 completed 31/~105.0:: 29.52% complete
(Worker pid=5144) INFO:2026-02-24,21:10:06 ||| Worker 2 completed 34/~105.0:: 32.38% complete (Worker pid=5142) INFO:2026-02-24,21:10:02 ||| Worker at start location (117,0) completed 10/11
(Worker pid=5145) INFO:2026-02-24,21:10:06 ||| Worker at start location (128,0) completed 10/11
(Worker pid=5145) INFO:2026-02-24,21:10:12 ||| Worker 1 completed 34/~105.0:: 32.38% complete [repeated 2x across cluster]
(Worker pid=5143) INFO:2026-02-24,21:10:40 ||| Worker at start location (139,0) completed 10/11
(Worker pid=5143) INFO:2026-02-24,21:10:43 ||| Worker 3 completed 42/~105.0:: 40.0% complete
(Worker pid=5144) INFO:2026-02-24,21:10:56 ||| Worker at start location (149,0) completed 9/10
(Worker pid=5142) INFO:2026-02-24,21:10:59 ||| Worker at start location (160,0) completed 10/11
(Worker pid=5144) INFO:2026-02-24,21:11:01 ||| Worker 2 completed 44/~105.0:: 41.9% complete
(Worker pid=5145) INFO:2026-02-24,21:11:05 ||| Worker at start location (171,0) completed 10/11
(Worker pid=5145) INFO:2026-02-24,21:11:09 ||| Worker 1 completed 45/~105.0:: 42.86% complete [repeated 2x across cluster]
(Worker pid=5143) INFO:2026-02-24,21:11:34 ||| Worker at start location (182,0) completed 10/11
(Worker pid=5143) INFO:2026-02-24,21:11:40 ||| Worker 3 completed 53/~105.0:: 50.48% complete
(Worker pid=5144) INFO:2026-02-24,21:11:43 ||| Worker at start location (192,0) completed 9/10
(Worker pid=5144) INFO:2026-02-24,21:11:49 ||| Worker 2 completed 54/~105.0:: 51.43% complete
(Worker pid=5145) INFO:2026-02-24,21:11:56 ||| Worker at start location (214,0) completed 10/11
(Worker pid=5145) INFO:2026-02-24,21:12:01 ||| Worker 1 completed 56/~105.0:: 53.33% complete (Worker pid=5142) INFO:2026-02-24,21:11:58 ||| Worker at start location (203,0) completed 10/11
(Worker pid=5143) INFO:2026-02-24,21:12:31 ||| Worker at start location (225,0) completed 10/11 (Worker pid=5142) INFO:2026-02-24,21:12:03 ||| Worker 0 completed 56/~105.0:: 53.33% complete
(Worker pid=5144) INFO:2026-02-24,21:12:36 ||| Worker at start location (235,0) completed 9/10
(Worker pid=5143) INFO:2026-02-24,21:12:37 ||| Worker 3 completed 64/~105.0:: 60.95% complete
(Worker pid=5145) INFO:2026-02-24,21:12:56 ||| Worker at start location (246,0) completed 10/11 (Worker pid=5144) INFO:2026-02-24,21:12:41 ||| Worker 2 completed 64/~105.0:: 60.95% complete
(Worker pid=5145) INFO:2026-02-24,21:13:02 ||| Worker 1 completed 67/~105.0:: 63.81% complete (Worker pid=5142) INFO:2026-02-24,21:12:59 ||| Worker at start location (257,0) completed 10/11
(Worker pid=5143) INFO:2026-02-24,21:13:30 ||| Worker at start location (268,0) completed 10/11 (Worker pid=5142) INFO:2026-02-24,21:13:04 ||| Worker 0 completed 67/~105.0:: 63.81% complete
(Worker pid=5143) INFO:2026-02-24,21:13:35 ||| Worker 3 completed 75/~105.0:: 71.43% complete (Worker pid=5144) INFO:2026-02-24,21:13:34 ||| Worker at start location (279,0) completed 10/11
(Worker pid=5145) INFO:2026-02-24,21:13:49 ||| Worker at start location (289,0) completed 9/10 (Worker pid=5144) INFO:2026-02-24,21:13:40 ||| Worker 2 completed 75/~105.0:: 71.43% complete
(Worker pid=5145) INFO:2026-02-24,21:13:54 ||| Worker 1 completed 77/~105.0:: 73.33% complete
(Worker pid=5142) INFO:2026-02-24,21:13:56 ||| Worker at start location (300,0) completed 10/11
(Worker pid=5142) INFO:2026-02-24,21:14:02 ||| Worker 0 completed 78/~105.0:: 74.29% complete
(Worker pid=5143) INFO:2026-02-24,21:14:28 ||| Worker at start location (311,0) completed 10/11
(Worker pid=5143) INFO:2026-02-24,21:14:33 ||| Worker 3 completed 86/~105.0:: 81.9% complete (Worker pid=5144) INFO:2026-02-24,21:14:29 ||| Worker at start location (322,0) completed 10/11
(Worker pid=5145) INFO:2026-02-24,21:14:41 ||| Worker at start location (332,0) completed 9/10 (Worker pid=5144) INFO:2026-02-24,21:14:34 ||| Worker 2 completed 86/~105.0:: 81.9% complete
(Worker pid=5145) INFO:2026-02-24,21:14:46 ||| Worker 1 completed 87/~105.0:: 82.86% complete
(Worker pid=5142) INFO:2026-02-24,21:14:54 ||| Worker at start location (343,0) completed 10/11
(Worker pid=5142) INFO:2026-02-24,21:14:59 ||| Worker 0 completed 89/~105.0:: 84.76% complete
(Worker pid=5143) INFO:2026-02-24,21:15:25 ||| Worker at start location (354,0) completed 10/11
(Worker pid=5144) INFO:2026-02-24,21:15:27 ||| Worker at start location (365,0) completed 10/11
(Worker pid=5143) INFO:2026-02-24,21:15:29 ||| Worker 3 completed 97/~105.0:: 92.38% complete
(Worker pid=5145) INFO:2026-02-24,21:15:32 ||| Worker at start location (375,0) completed 9/10
(Worker pid=5145) INFO:2026-02-24,21:15:38 ||| Worker 1 completed 97/~105.0:: 92.38% complete [repeated 2x across cluster]
(Worker pid=5142) INFO:2026-02-24,21:15:51 ||| Worker at start location (386,0) completed 10/11
(Worker pid=5143) INFO:2026-02-24,21:16:09 ||| Worker at start location (397,0) completed 10/11
(Worker pid=5144) INFO:2026-02-24,21:16:16 ||| Worker at start location (408,0) completed 10/11
INFO:2026-02-24,21:16:20 || isofit.py:run() | Pixel class: uniform_surface inversions complete.
INFO:2026-02-24,21:16:20 || isofit.py:run() | 558.34s total
INFO:2026-02-24,21:16:20 || isofit.py:run() | 0.7522 spectra/s
INFO:2026-02-24,21:16:20 || isofit.py:run() | 0.1881 spectra/s/core
INFO:2026-02-24,21:16:20 || isofit.py:run() | All Inversions complete.
INFO:2026-02-24,21:16:20 || isofit.py:run() | 735.62s total
INFO:2026-02-24,21:16:20 || isofit.py:run() | 0.5709 spectra/s
INFO:2026-02-24,21:16:20 || isofit.py:run() | 0.1427 spectra/s/core
INFO:2026-02-24,21:16:20 || apply_oe.py:apply_oe() | Analytical line inference
INFO:2026-02-24,21:16:20 || configs.py:create_new_config() | Loading config file: /home/runner/work/isofit-tutorials/isofit-tutorials/NEON/output/NIS01_20210403_173647/config/NIS01_20210403_173647_isofit.json
/home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.13/site-packages/numpy/_core/fromnumeric.py:3824: RuntimeWarning: Mean of empty slice return _methods._mean(a, axis=axis, dtype=dtype, /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.13/site-packages/numpy/_core/_methods.py:142: RuntimeWarning: invalid value encountered in scalar divide ret = ret.dtype.type(ret / rcount)
2026-02-24 21:16:21,062 INFO worker.py:1831 -- Calling ray.init() again after it has already been called.
INFO:2026-02-24,21:16:21 || atm_interpolation.py:atm_interpolation() | Beginning atmospheric interpolation 4 cores
(_run_chunk pid=5398) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.13/site-packages/isofit/utils/atm_interpolation.py:182: LinAlgWarning: An ill-conditioned matrix detected: slice 0 has rcond = 5.806051187360241e-33. (_run_chunk pid=5398) bhat[i, :] = (inv(X.T @ W @ X) @ X.T @ W @ y).T (Worker pid=5145) INFO:2026-02-24,21:16:20 ||| Worker at start location (419,0) completed 10/11
INFO:2026-02-24,21:16:26 || atm_interpolation.py:atm_interpolation() | Parallel atmospheric interpolations complete. 4.94978666305542 s total, 852.7640254690184 spectra/s, 213.1910063672546 spectra/s/core
2026-02-24 21:16:26,042 INFO worker.py:1831 -- Calling ray.init() again after it has already been called.
INFO:2026-02-24,21:16:26 || radiative_transfer_engine.py:__init__() | Loading from wavelength_file: /home/runner/work/isofit-tutorials/isofit-tutorials/NEON/output/NIS01_20210403_173647/data/wavelengths.txt
INFO:2026-02-24,21:16:26 || radiative_transfer_engine.py:__init__() | Prebuilt LUT provided
INFO:2026-02-24,21:16:26 || luts.py:load() | Loading LUT into memory
WARNING:2026-02-24,21:16:26 || luts.py:load() | thermal_upwelling is fully NaN, leaving as-is
WARNING:2026-02-24,21:16:26 || luts.py:load() | thermal_downwelling is fully NaN, leaving as-is
INFO:2026-02-24,21:16:26 || radiative_transfer_engine.py:__init__() | LUT grid loaded from file
(Worker pid=5429) ERROR:2026-02-24,21:16:31 ||| [x] Data path does not exist (Worker pid=5429) ERROR:2026-02-24,21:16:31 ||| The following path does not exist, please verify your installation environment: /home/runner/.isofit/data/earth_sun_distance.txt (Worker pid=5429) WARNING:2026-02-24,21:16:31 ||| Earth-sun-distance file not found on system. Proceeding without might cause some inaccuracies down the line. (Worker pid=5429) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.13/site-packages/numpy/_core/fromnumeric.py:3824: RuntimeWarning: Mean of empty slice (Worker pid=5429) return _methods._mean(a, axis=axis, dtype=dtype, (Worker pid=5429) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.13/site-packages/numpy/_core/_methods.py:142: RuntimeWarning: invalid value encountered in scalar divide (Worker pid=5429) ret = ret.dtype.type(ret / rcount) (_run_chunk pid=5400) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.13/site-packages/isofit/utils/atm_interpolation.py:182: LinAlgWarning: An ill-conditioned matrix detected: slice 0 has rcond = 5.806051187360241e-33. [repeated 3x across cluster] (_run_chunk pid=5400) bhat[i, :] = (inv(X.T @ W @ X) @ X.T @ W @ y).T [repeated 3x across cluster]
(Worker pid=5648) INFO:2026-02-24,21:16:36 ||| Analytical line writing lines: 3 to 4. Surface: uniform_surface (Worker pid=5649) ERROR:2026-02-24,21:16:32 ||| [x] Data path does not exist [repeated 3x across cluster] (Worker pid=5649) ERROR:2026-02-24,21:16:32 ||| The following path does not exist, please verify your installation environment: /home/runner/.isofit/data/earth_sun_distance.txt [repeated 3x across cluster] (Worker pid=5649) WARNING:2026-02-24,21:16:32 ||| Earth-sun-distance file not found on system. Proceeding without might cause some inaccuracies down the line. [repeated 3x across cluster] (Worker pid=5649) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.13/site-packages/numpy/_core/fromnumeric.py:3824: RuntimeWarning: Mean of empty slice [repeated 3x across cluster] (Worker pid=5649) return _methods._mean(a, axis=axis, dtype=dtype, [repeated 3x across cluster] (Worker pid=5649) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.13/site-packages/numpy/_core/_methods.py:142: RuntimeWarning: invalid value encountered in scalar divide [repeated 3x across cluster] (Worker pid=5649) ret = ret.dtype.type(ret / rcount) [repeated 3x across cluster]
(Worker pid=5650) INFO:2026-02-24,21:16:46 ||| Analytical line writing lines: 11 to 12. Surface: uniform_surface [repeated 5x across cluster]
(Worker pid=5648) INFO:2026-02-24,21:16:51 ||| Analytical line writing lines: 16 to 17. Surface: uniform_surface [repeated 4x across cluster]
(Worker pid=5429) INFO:2026-02-24,21:16:59 ||| Analytical line writing lines: 17 to 19. Surface: uniform_surface [repeated 3x across cluster]
(Worker pid=5650) INFO:2026-02-24,21:17:05 ||| Analytical line writing lines: 22 to 24. Surface: uniform_surface [repeated 3x across cluster]
(Worker pid=5648) INFO:2026-02-24,21:17:10 ||| Analytical line writing lines: 32 to 33. Surface: uniform_surface [repeated 4x across cluster]
(Worker pid=5429) INFO:2026-02-24,21:17:18 ||| Analytical line writing lines: 33 to 35. Surface: uniform_surface [repeated 3x across cluster]
(Worker pid=5650) INFO:2026-02-24,21:17:24 ||| Analytical line writing lines: 38 to 40. Surface: uniform_surface [repeated 3x across cluster]
(Worker pid=5429) INFO:2026-02-24,21:17:32 ||| Analytical line writing lines: 50 to 51. Surface: uniform_surface [repeated 4x across cluster]
(Worker pid=5649) INFO:2026-02-24,21:17:39 ||| Analytical line writing lines: 51 to 53. Surface: uniform_surface [repeated 4x across cluster]
(Worker pid=5649) INFO:2026-02-24,21:17:46 ||| Analytical line writing lines: 59 to 61. Surface: uniform_surface [repeated 4x across cluster]
INFO:2026-02-24,21:17:48 || analytical_line.py:analytical_line() | Analytical line inversions complete. 82.1s total, 51.4113 spectra/s, 12.8528 spectra/s/core
INFO:2026-02-24,21:17:48 || apply_oe.py:apply_oe() | Done.
Plotting¶
Below plots the regions of interest defined by a NEON report.
# Load in the ISOFIT reflectance output
ds = envi.open(paths.working / f"output/{neon_str}_rfl.hdr")
rfl = ds.open_memmap(interleave='bip')
rgb = rfl[:, :, [60, 40, 30]].copy()
wl = np.array(ds.metadata['wavelength'], dtype=float)
# Find the bounding box for all regions of interest (RoI)
regions = report[neon_id]
bounds = np.vstack(list(regions.values()))
y = bounds[:, 0].min() - 5 # , bounds[:, 1].max() + 5
x = bounds[:, 2].min() - 5 # , bounds[:, 3].max() + 5
# Plot the RoIs
fig, ax = plt.subplots(figsize=(7, 7))
ax.imshow(rgb / np.max(rgb, axis=(0, 1))) # Dividing brightens the image
ax.set_title(neon_str)
for i, (roi, region) in enumerate(regions.items()):
rect = patches.Rectangle(
(region[2] - x, region[0] - y),
region[3] - region[2],
region[1] - region[0],
linewidth = 1,
edgecolor = f'C{i}',
facecolor = 'none',
label = roi
)
ax.add_patch(rect)
ax.legend(loc='lower right')
<matplotlib.legend.Legend at 0x7f34b627aa50>
fig, axes = plt.subplots(len(regions), sharex=True, figsize=(10, 3*len(regions)))
for i, (roi, region) in enumerate(regions.items()):
ax = axes[i]
in_situ = np.genfromtxt(paths.insitu / f'{roi}01/Data/{roi}01_Refl.dat', skip_header=3)
ax.plot(in_situ[:, 0], in_situ[:, 1], label='In Situ', c='red', ls='-')
mean_rfl = np.mean(
rfl[
region[0] - y : region[1] - y,
region[2] - x : region[3] - x,
],
axis = (0, 1)
)
ax.plot(wl, mean_rfl, label='Isofit', c='black')
ax.set_ylabel('Reflectance')
ax.set_title(roi)
ax.legend()
ax.set_xlabel('Wavelength')
plt.tight_layout()
We can plot out the mapped reflectance (as above), but also the interpolated atmospheric conditions. The windows size is small enough here (and the atmospheric parameters are chosen in such a way) that the map is going to be pretty static...but we can still see it.
dat = envi.open(paths.working / f"output/{neon_str}_atm_interp.hdr")
atm = dat.open_memmap(interleave='bip').copy()
plt.figure(figsize=(7, 7))
plt.title('AOD')
plt.imshow(atm[..., 0])
plt.colorbar()
plt.figure(figsize=(7, 7))
plt.title('Water Vapor')
plt.imshow(atm[..., 1])
plt.colorbar()
<matplotlib.colorbar.Colorbar at 0x7f34b5434190>