NEON In-Situ Comparison¶

Let's put it all together by running ISOFIT for a sensor-specific example and compare ISOFIT derived reflectance to field-collected spectra.

Prerequisites:

• Have ISOFIT installed and sRTMnet configured.
• Download the subset_data.zip and place the unzipped data directory into tutorial's repository root. Alternatively, use isofit download examples --neon to automate this process.

Note: If you downloaded the ISOFIT extra data via isofit download, both sRTMnet and the NEON data files will be installed correctly and available with default settings for this notebook.

In [1]:

# Jupyter magics
%matplotlib inline

# Builtin
import os
import shutil
from pathlib import Path
from types import SimpleNamespace

# External
import matplotlib.pyplot  as plt
import matplotlib.patches as patches
import numpy as np
from spectral.io import envi

# Internal
import isofit
from isofit.data import env
from isofit.utils.apply_oe import apply_oe
from isofit.utils.surface_model import surface_model

# Jupyter magics %matplotlib inline # Builtin import os import shutil from pathlib import Path from types import SimpleNamespace # External import matplotlib.pyplot as plt import matplotlib.patches as patches import numpy as np from spectral.io import envi # Internal import isofit from isofit.data import env from isofit.utils.apply_oe import apply_oe from isofit.utils.surface_model import surface_model

In [2]:

# Below are the default values for the ISOFIT environment. Change these if your environment differs

env.load('~/.isofit/isofit.ini') # Ini file to load
env.changeSection('DEFAULT')     # Section of the ini to use
# env.changeBase('~./isofit')      # Base path for ISOFIT extras (data, examples, etc)
# env.changePath('srtmnet', '/path/to/sRTMnet_v120.h5') # Overwrite the path to sRTMnet - copy this line for other products such as sixs if in non-default locations

print('Using environment paths:')
for key, path in env.items():
    print(f"- {key} = {path}")

# Below are the default values for the ISOFIT environment. Change these if your environment differs env.load('~/.isofit/isofit.ini') # Ini file to load env.changeSection('DEFAULT') # Section of the ini to use # env.changeBase('~./isofit') # Base path for ISOFIT extras (data, examples, etc) # env.changePath('srtmnet', '/path/to/sRTMnet_v120.h5') # Overwrite the path to sRTMnet - copy this line for other products such as sixs if in non-default locations print('Using environment paths:') for key, path in env.items(): print(f"- {key} = {path}")

Using environment paths:
- data = /home/runner/work/isofit-tutorials/data
- examples = /home/runner/work/isofit-tutorials/isofit-tutorials
- srtmnet = /home/runner/work/isofit-tutorials/srtmnet
- sixs = /home/runner/work/isofit-tutorials/sixs
- surface = /home/runner/work/isofit-tutorials/surface
- plots = /home/runner/work/isofit-tutorials/plots
- libradtran = /home/runner/work/isofit-tutorials/libradtran
- windows = /home/runner/work/isofit-tutorials/windows
- srtmnet.file = sRTMnet_v120.h5
- srtmnet.aux = sRTMnet_v120_aux.npz
- libradtran.version = libRadtran-2.0.6

Setup¶

ISOFIT needs at minimum three pieces as input:

1. Radiance measurements (rdn)
2. Observation values    (obs)
3. Location information  (loc)

This sample dataset from NEON has radiance and observation data, but no location values (more recent NEON datasets include the location file). However, we can 'fake' the location file with sufficient accuracy for ISOFIT to run successfully. Note that there are data available for two dates:

Radiance
├── 173647
│   ├── NIS01_20210403_173647_obs_ort
│   ├── NIS01_20210403_173647_obs_ort.hdr
│   ├── NIS01_20210403_173647_rdn_ort
│   └── NIS01_20210403_173647_rdn_ort.hdr
└── 174150
    ├── NIS01_20210403_174150_obs_ort
    ├── NIS01_20210403_174150_obs_ort.hdr
    ├── NIS01_20210403_174150_rdn_ort
    └── NIS01_20210403_174150_rdn_ort.hdr

These files have corresponding in situ data as well, and below we've encoded the locations of each, which we can use to help subset data files.

In [3]:

# Extract the image locations of each point of interest (POI)
# These are defined in the NEON report as pixel locations, so we round here to convert to indices
report = {}
report['173647'] = {     # Upp L Y  | Low R Y  | Upp L X | Low R X
    'WhiteTarp': np.round([2224.9626, 2230.9771, 316.0078, 324.9385,]).astype(int),
    'BlackTarp': np.round([2224.9626, 2231.0032, 328.0086, 333.9731,]).astype(int),
    'Veg'      : np.round([2245.0381, 2258.8103, 343.9006, 346.9423,]).astype(int),
    'RoadEW'   : np.round([2214.9905, 2216.9978, 348.9902, 373.0080,]).astype(int),
    'RoadNS'   : np.round([2205.9580, 2225.9612, 357.9536, 359.9608,]).astype(int)
}
report['174150'] = {     # Upp L Y | Low R Y | Upp L X  | Low R X
    'WhiteTarp': np.round([653.9626, 659.9771, 3143.0078, 3151.9385]).astype(int),
    'BlackTarp': np.round([653.9626, 660.0032, 3155.0086, 3160.9731]).astype(int),
    'Veg'      : np.round([674.0381, 687.8103, 3170.9006, 3173.9423]).astype(int),
    'RoadEW'   : np.round([643.9905, 645.9978, 3175.9902, 3200.0080]).astype(int),
    'RoadNS'   : np.round([634.9580, 654.9612, 3184.9536, 3186.9608]).astype(int)
}

# Which NEON date to process - change this to process a different date
neon_id  = list(report.keys())[0]
neon_str = f"NIS01_20210403_{neon_id}"

# Select the locations from the neon id -- roi == Regions of Interest
roi = report[neon_id]

# Extract the image locations of each point of interest (POI) # These are defined in the NEON report as pixel locations, so we round here to convert to indices report = {} report['173647'] = { # Upp L Y | Low R Y | Upp L X | Low R X 'WhiteTarp': np.round([2224.9626, 2230.9771, 316.0078, 324.9385,]).astype(int), 'BlackTarp': np.round([2224.9626, 2231.0032, 328.0086, 333.9731,]).astype(int), 'Veg' : np.round([2245.0381, 2258.8103, 343.9006, 346.9423,]).astype(int), 'RoadEW' : np.round([2214.9905, 2216.9978, 348.9902, 373.0080,]).astype(int), 'RoadNS' : np.round([2205.9580, 2225.9612, 357.9536, 359.9608,]).astype(int) } report['174150'] = { # Upp L Y | Low R Y | Upp L X | Low R X 'WhiteTarp': np.round([653.9626, 659.9771, 3143.0078, 3151.9385]).astype(int), 'BlackTarp': np.round([653.9626, 660.0032, 3155.0086, 3160.9731]).astype(int), 'Veg' : np.round([674.0381, 687.8103, 3170.9006, 3173.9423]).astype(int), 'RoadEW' : np.round([643.9905, 645.9978, 3175.9902, 3200.0080]).astype(int), 'RoadNS' : np.round([634.9580, 654.9612, 3184.9536, 3186.9608]).astype(int) } # Which NEON date to process - change this to process a different date neon_id = list(report.keys())[0] neon_str = f"NIS01_20210403_{neon_id}" # Select the locations from the neon id -- roi == Regions of Interest roi = report[neon_id]

In [4]:

# Set the paths for this tutorial
base = Path(env.path('examples', 'NEON'))
raws = base / 'data'
data = base / 'neon_subset'

paths = SimpleNamespace(
    rdn = str(data / f'{neon_str}_rdn_ort'),
    loc = str(data / f'{neon_str}_loc_ort'),
    obs = str(data / f'{neon_str}_obs_ort'),
    insitu = raws,
    output = base / 'output',
    working = base / f'output/NIS01_20210403_{neon_id}',
    surface = str(base / 'output/surface.mat'),
    surface_config = env.path('examples', '20171108_Pasadena', 'configs', 'ang20171108t184227_surface.json')
)

paths.output.mkdir(exist_ok=True, parents=True)

# Set the paths for this tutorial base = Path(env.path('examples', 'NEON')) raws = base / 'data' data = base / 'neon_subset' paths = SimpleNamespace( rdn = str(data / f'{neon_str}_rdn_ort'), loc = str(data / f'{neon_str}_loc_ort'), obs = str(data / f'{neon_str}_obs_ort'), insitu = raws, output = base / 'output', working = base / f'output/NIS01_20210403_{neon_id}', surface = str(base / 'output/surface.mat'), surface_config = env.path('examples', '20171108_Pasadena', 'configs', 'ang20171108t184227_surface.json') ) paths.output.mkdir(exist_ok=True, parents=True)

In [5]:

# If you are missing either an OBS file or a LOC file, use these to create faked versions based off the radiance file
# This should not be needed if using the provided data
# Using this may cause the below plots to not generate the same results

# from isotuts import neon_utils

# paths.obs = neon_utils.fakeOBS(
    # f"{paths.rdn}.hdr",
    # sea = 153.4481201171875,
    # sez = 178.3806858062744,
    # soa = 39.8218994140625,
    # soz = 39.8218994140625,
    # slope = 31.813383102416992
# )[:-4] # Remove the .hdr extension from the return
# paths.loc = neon_utils.fakeLOC(
#     rdn = f"{paths.rdn}.hdr",
#     lon = -105.237000,
#     lat = 40.125000,
#     elv = 1689.0
# )[:-4]

# If you are missing either an OBS file or a LOC file, use these to create faked versions based off the radiance file # This should not be needed if using the provided data # Using this may cause the below plots to not generate the same results # from isotuts import neon_utils # paths.obs = neon_utils.fakeOBS( # f"{paths.rdn}.hdr", # sea = 153.4481201171875, # sez = 178.3806858062744, # soa = 39.8218994140625, # soz = 39.8218994140625, # slope = 31.813383102416992 # )[:-4] # Remove the .hdr extension from the return # paths.loc = neon_utils.fakeLOC( # rdn = f"{paths.rdn}.hdr", # lon = -105.237000, # lat = 40.125000, # elv = 1689.0 # )[:-4]

Apply OE¶

The next part walks through running the ISOFIT utility script isofit/utils/apply_oe.py. This is the first step of executing ISOFIT and will generate a default configuration.

In [6]:

# First build a surface model
surface_model(
    config_path = paths.surface_config,
    output_path = paths.surface,
    wavelength_path = f"{paths.rdn}.hdr"
)

# First build a surface model surface_model( config_path = paths.surface_config, output_path = paths.surface, wavelength_path = f"{paths.rdn}.hdr" )

0 ['/home/runner/work/isofit-tutorials/data/reflectance/surface_model_ucsb']

1 ['/home/runner/work/isofit-tutorials/data/reflectance/surface_model_ucsb']

2 ['/home/runner/work/isofit-tutorials/data/reflectance/surface_model_ucsb']

3 ['/home/runner/work/isofit-tutorials/data/reflectance/surface_model_ucsb']

4 ['/home/runner/work/isofit-tutorials/data/reflectance/surface_model_ucsb']
5 ['/home/runner/work/isofit-tutorials/data/reflectance/surface_model_ucsb']
6 ['/home/runner/work/isofit-tutorials/data/reflectance/surface_model_ucsb']

7 ['/home/runner/work/isofit-tutorials/data/reflectance/surface_model_ucsb']

In [7]:

# Add a ray shutdown, just in case this is being re-called
import ray
ray.shutdown()

# Cleanup any previous runs; comment this out if you want to preserve a previous run's output
if Path(paths.working).exists():
    shutil.rmtree(paths.working)

apply_oe(
    input_radiance    = paths.rdn, # Radiance
    input_loc         = paths.loc, # Location
    input_obs         = paths.obs, # Observations
    working_directory = str(paths.working), # Output directory
    sensor            = "neon",
    surface_path      = paths.surface, # Surface priors - often changes
    emulator_base     = f"{env.srtmnet}/sRTMnet_v120.h5",
    surface_category  = "multicomponent_surface",

    modtran_path    = None,
    atmosphere_type = "ATM_MIDLAT_SUMMER", # MODTRAN
    aerosol_climatology_path = None,       # MODTRAN

    rdn_factors_path       = None, # RCC update used 'on the fly'
    model_discrepancy_path = None, # Model discrepancy term - handle things like unknown radiative transfer model effects
    channelized_uncertainty_path = None, # Channelized uncertainty - if you have an instrument model

    multiple_restarts = False, # Useful if the AOD conditions are really challenging

    presolve        = True,  # Attempts to solve for the right wv range
    empirical_line  = False, # wavelength-specific local linear interpolation between radiance and reflectance
    analytical_line = True,  # mathematical representation of OE given that the atmsophere is known

    segmentation_size  = 10,
    num_neighbors      = [5],
    atm_sigma          = [0.5, 0.5],
    pressure_elevation = False,

    n_cores = os.cpu_count(),
)

# Add a ray shutdown, just in case this is being re-called import ray ray.shutdown() # Cleanup any previous runs; comment this out if you want to preserve a previous run's output if Path(paths.working).exists(): shutil.rmtree(paths.working) apply_oe( input_radiance = paths.rdn, # Radiance input_loc = paths.loc, # Location input_obs = paths.obs, # Observations working_directory = str(paths.working), # Output directory sensor = "neon", surface_path = paths.surface, # Surface priors - often changes emulator_base = f"{env.srtmnet}/sRTMnet_v120.h5", surface_category = "multicomponent_surface", modtran_path = None, atmosphere_type = "ATM_MIDLAT_SUMMER", # MODTRAN aerosol_climatology_path = None, # MODTRAN rdn_factors_path = None, # RCC update used 'on the fly' model_discrepancy_path = None, # Model discrepancy term - handle things like unknown radiative transfer model effects channelized_uncertainty_path = None, # Channelized uncertainty - if you have an instrument model multiple_restarts = False, # Useful if the AOD conditions are really challenging presolve = True, # Attempts to solve for the right wv range empirical_line = False, # wavelength-specific local linear interpolation between radiance and reflectance analytical_line = True, # mathematical representation of OE given that the atmsophere is known segmentation_size = 10, num_neighbors = [5], atm_sigma = [0.5, 0.5], pressure_elevation = False, n_cores = os.cpu_count(), )

INFO:2026-07-28,02:15:35 || apply_oe.py:apply_oe() | Checking input data files...

INFO:2026-07-28,02:15:35 || apply_oe.py:apply_oe() | ...Data file checks complete

INFO:2026-07-28,02:15:35 || apply_oe.py:apply_oe() | Setting up files and directories....

INFO:2026-07-28,02:15:35 || template_construction.py:__init__() | Flightline ID: NIS01_20210403_173647

INFO:2026-07-28,02:15:35 || template_construction.py:__init__() | no noise path found, proceeding without

INFO:2026-07-28,02:15:35 || apply_oe.py:apply_oe() | ...file/directory setup complete

INFO:2026-07-28,02:15:35 || apply_oe.py:apply_oe() | Using inversion windows: [[350.0, 1360.0], [1410, 1800.0], [1970.0, 2500.0]]

INFO:2026-07-28,02:15:35 || template_construction.py:get_wavelengths() | No wavelength file provided. Obtaining wavelength grid from ENVI header of radiance cube.

INFO:2026-07-28,02:15:35 || template_construction.py:get_wavelengths() | Wavelength units of nm inferred...converting to microns

WARNING:2026-07-28,02:15:35 || template_construction.py:check_surface_model() | Center wavelengths provided in surface model file do not match wavelengths in radiance cube. Please consider rebuilding your surface model for optimal performance.

INFO:2026-07-28,02:15:35 || apply_oe.py:apply_oe() | Observation means:

INFO:2026-07-28,02:15:35 || apply_oe.py:apply_oe() | Path (km): 1.0036078691482544

INFO:2026-07-28,02:15:35 || apply_oe.py:apply_oe() | To-sensor azimuth (deg): 153.4481201171875

INFO:2026-07-28,02:15:35 || apply_oe.py:apply_oe() | To-sensor zenith (deg): 1.619314193725586

INFO:2026-07-28,02:15:35 || apply_oe.py:apply_oe() | To-sun azimuth (deg): 145.23248291015625

INFO:2026-07-28,02:15:35 || apply_oe.py:apply_oe() | To-sun zenith (deg): 39.8218994140625

INFO:2026-07-28,02:15:35 || apply_oe.py:apply_oe() | Relative to-sun azimuth (deg): 31.813383102416992

INFO:2026-07-28,02:15:35 || apply_oe.py:apply_oe() | Altitude (km): 2.692207098007202

INFO:2026-07-28,02:15:35 || apply_oe.py:apply_oe() | Segmenting...

2026-07-28 02:15:39,486	INFO worker.py:2024 -- Started a local Ray instance.

(segment_chunk pid=6737) INFO:2026-07-28,02:15:42 ||| 0: starting
INFO:2026-07-28,02:15:42 || apply_oe.py:apply_oe() | Extracting /home/runner/work/isofit-tutorials/isofit-tutorials/NEON/output/NIS01_20210403_173647/input/NIS01_20210403_173647_subs_rdn

2026-07-28 02:15:42,420	INFO worker.py:1847 -- Calling ray.init() again after it has already been called.

(segment_chunk pid=6737) INFO:2026-07-28,02:15:42 ||| 0: completing

(extract_chunk pid=6737) INFO:2026-07-28,02:15:44 ||| 0: starting
INFO:2026-07-28,02:15:44 || apply_oe.py:apply_oe() | Extracting /home/runner/work/isofit-tutorials/isofit-tutorials/NEON/output/NIS01_20210403_173647/input/NIS01_20210403_173647_subs_obs

2026-07-28 02:15:44,319	INFO worker.py:1847 -- Calling ray.init() again after it has already been called.

INFO:2026-07-28,02:15:44 || apply_oe.py:apply_oe() | Extracting /home/runner/work/isofit-tutorials/isofit-tutorials/NEON/output/NIS01_20210403_173647/input/NIS01_20210403_173647_subs_loc

2026-07-28 02:15:44,341	INFO worker.py:1847 -- Calling ray.init() again after it has already been called.

INFO:2026-07-28,02:15:44 || apply_oe.py:apply_oe() | Skipping None, because is not a path.

INFO:2026-07-28,02:15:44 || apply_oe.py:apply_oe() | Skipping None, because is not a path.

INFO:2026-07-28,02:15:44 || apply_oe.py:apply_oe() | Pre-solve H2O grid: [0.2  0.49 0.79 1.08 1.37 1.67 1.96 2.25 2.55 2.84]

INFO:2026-07-28,02:15:44 || apply_oe.py:apply_oe() | Writing H2O pre-solve configuration file.

INFO:2026-07-28,02:15:44 || ini.py:toTemplate() | Saved converted json to: /home/runner/work/isofit-tutorials/isofit-tutorials/NEON/output/NIS01_20210403_173647/config/NIS01_20210403_173647_h2o.json.tmpl

INFO:2026-07-28,02:15:44 || apply_oe.py:apply_oe() | Run ISOFIT initial guess

WARNING:2026-07-28,02:15:44 || __init__.py:checkNumThreads() | 
******************************************************************************************
! Number of threads is greater than 1 (currently: 4), this may greatly impact performance
! Please set this the environment variables 'MKL_NUM_THREADS' and 'OMP_NUM_THREADS' to '1'
******************************************************************************************

INFO:2026-07-28,02:15:44 || configs.py:create_new_config() | Loading config file: /home/runner/work/isofit-tutorials/isofit-tutorials/NEON/output/NIS01_20210403_173647/config/NIS01_20210403_173647_h2o.json

INFO:2026-07-28,02:15:44 || configs.py:get_config_errors() | Checking config sections for configuration issues

INFO:2026-07-28,02:15:44 || configs.py:get_config_errors() | Configuration file checks complete, no errors found.

/home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/numpy/_core/fromnumeric.py:3824: RuntimeWarning: Mean of empty slice
  return _methods._mean(a, axis=axis, dtype=dtype,
/home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/numpy/_core/_methods.py:142: RuntimeWarning: invalid value encountered in scalar divide
  ret = ret.dtype.type(ret / rcount)
(extract_chunk pid=6737) INFO:2026-07-28,02:15:44 ||| 0: starting
(extract_chunk pid=6737) INFO:2026-07-28,02:15:44 ||| 0: starting

2026-07-28 02:15:44,586	INFO worker.py:1847 -- Calling ray.init() again after it has already been called.

INFO:2026-07-28,02:15:44 || isofit.py:run() | Running surfaces: uniform_surface

INFO:2026-07-28,02:15:44 || atmosphere.py:__init__() | Loading from wavelength_file: /home/runner/work/isofit-tutorials/isofit-tutorials/NEON/output/NIS01_20210403_173647/data/wavelengths.txt

INFO:2026-07-28,02:15:44 || reader.py:__init__() | No LUT provided, attempting to build it

INFO:2026-07-28,02:15:44 || zarr.py:calc_shards() | Number of points: 4,260

INFO:2026-07-28,02:15:44 || zarr.py:calc_shards() | Target chunks per file: 157532 (6.00gb) (scale: 12)

INFO:2026-07-28,02:15:44 || zarr.py:calc_shards() | Shapes:

INFO:2026-07-28,02:15:44 || zarr.py:calc_shards() |   dimensions: [426  10]

INFO:2026-07-28,02:15:44 || zarr.py:calc_shards() |     chunking: [426   1]

INFO:2026-07-28,02:15:44 || zarr.py:calc_shards() | Recommended

INFO:2026-07-28,02:15:44 || zarr.py:calc_shards() |     sharding: [426  10]

INFO:2026-07-28,02:15:44 || zarr.py:calc_shards() | Produces:

INFO:2026-07-28,02:15:44 || zarr.py:calc_shards() |   Chunks per file: 10 (0.00gb)

INFO:2026-07-28,02:15:44 || zarr.py:calc_shards() |   Number of files: 1

INFO:2026-07-28,02:15:44 || writer.py:runSimulations() | Running any pre-sim functions

INFO:2026-07-28,02:15:44 || sRTMnet.py:preSim() | Building simulator and executing (6S)

INFO:2026-07-28,02:15:44 || reader.py:__init__() | No LUT provided, attempting to build it

INFO:2026-07-28,02:15:44 || zarr.py:calc_shards() | Number of points: 8,610

INFO:2026-07-28,02:15:44 || zarr.py:calc_shards() | Target chunks per file: 77942 (6.00gb) (scale: 12)

INFO:2026-07-28,02:15:44 || zarr.py:calc_shards() | Shapes:

INFO:2026-07-28,02:15:44 || zarr.py:calc_shards() |   dimensions: [861  10]

INFO:2026-07-28,02:15:44 || zarr.py:calc_shards() |     chunking: [861   1]

INFO:2026-07-28,02:15:44 || zarr.py:calc_shards() | Recommended

INFO:2026-07-28,02:15:44 || zarr.py:calc_shards() |     sharding: [861   2]

INFO:2026-07-28,02:15:44 || zarr.py:calc_shards() | Produces:

INFO:2026-07-28,02:15:44 || zarr.py:calc_shards() |   Chunks per file: 2 (0.00gb)

INFO:2026-07-28,02:15:44 || zarr.py:calc_shards() |   Number of files: 5

INFO:2026-07-28,02:15:44 || writer.py:runSimulations() | Running any pre-sim functions

INFO:2026-07-28,02:15:44 || writer.py:runSimulations() | Executing parallel simulations by shards

(shardWriter pid=6737) INFO:2026-07-28,02:15:45 ||| Starting shard (np.int64(0),)

INFO:2026-07-28,02:15:52 || writer.py:parallelize_shards() | Saved chunkless: {'coszen', 'solzen'}

INFO:2026-07-28,02:15:52 || common.py:__call__() |  20.00% shards complete (elapsed: 0:00:07.483608, rate: 0:00:01.496722, eta: 0:12:20.877192)

(shardWriter pid=6737) INFO:2026-07-28,02:15:52 ||| Finished points (np.int64(0),), flushing to disk
(shardWriter pid=6737) INFO:2026-07-28,02:15:52 ||| Finished shard (np.int64(0),)
(shardWriter pid=6737) INFO:2026-07-28,02:15:52 ||| Starting shard (np.int64(4),)

INFO:2026-07-28,02:15:56 || common.py:__call__() |  40.00% shards complete (elapsed: 0:00:11.804774, rate: 0:00:02.360955, eta: 0:09:38.433926)

INFO:2026-07-28,02:15:56 || common.py:__call__() |  60.00% shards complete (elapsed: 0:00:11.868057, rate: 0:00:02.373611, eta: 0:06:23.733843)

INFO:2026-07-28,02:15:56 || common.py:__call__() |  80.00% shards complete (elapsed: 0:00:11.899618, rate: 0:00:02.379924, eta: 0:04:45.590832)

INFO:2026-07-28,02:15:58 || common.py:__call__() | 100.00% shards complete (elapsed: 0:00:13.779303, rate: 0:00:02.755861, eta: 0:04:21.806757)

(shardWriter pid=6737) INFO:2026-07-28,02:15:58 ||| Finished points (np.int64(4),), flushing to disk
(shardWriter pid=6737) INFO:2026-07-28,02:15:58 ||| Finished shard (np.int64(4),)
INFO:2026-07-28,02:15:58 || writer.py:runSimulations() | Running any post-sim functions

INFO:2026-07-28,02:15:59 || writer.py:runSimulations() | Saving post-sim data to index zero of all dimensions except wl

/home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/zarr/api/asynchronous.py:231: ZarrUserWarning: Consolidated metadata is currently not part in the Zarr format 3 specification. It may not be supported by other zarr implementations and may change in the future.
  warnings.warn(
INFO:2026-07-28,02:15:59 || sRTMnet.py:process() | Interpolating simulator quantities to emulator size

INFO:2026-07-28,02:15:59 || sRTMnet.py:process() | Loading and predicting with emulator on 4 cores

INFO:2026-07-28,02:16:00 || common.py:__call__() | 100.00% shards processed (elapsed: 0:00:00.960922, rate: 0:00:00.960922, eta: 0:01:35.131278)

INFO:2026-07-28,02:16:00 || writer.py:runSimulations() | Saving pre-sim data to index zero of all dimensions except wl

INFO:2026-07-28,02:16:01 || writer.py:runSimulations() | Running any post-sim functions

/home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/zarr/api/asynchronous.py:231: ZarrUserWarning: Consolidated metadata is currently not part in the Zarr format 3 specification. It may not be supported by other zarr implementations and may change in the future.
  warnings.warn(
/home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/isofit/atmosphere/atmosphere.py:618: FutureWarning: Deleting a single level of a MultiIndex is deprecated. Previously, this deleted all levels of a MultiIndex. Please also drop the following variables: {'point'} to avoid an error in the future.
  ds[term] = ds[key1] * ds[key2]

INFO:2026-07-28,02:16:01 || isofit.py:run() | Beginning 420 inversions in 40 chunks using 4 cores

(Worker pid=7049) ERROR:2026-07-28,02:16:08 ||| [x] Data path does not exist
(Worker pid=7049) ERROR:2026-07-28,02:16:08 ||| The following path does not exist, please verify your installation environment: /home/runner/.isofit/data/earth_sun_distance.txt
(Worker pid=7049) WARNING:2026-07-28,02:16:08 ||| Earth-sun-distance file not found on system. Proceeding without might cause some inaccuracies down the line.

(Worker pid=7049) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/numpy/_core/fromnumeric.py:3824: RuntimeWarning: Mean of empty slice
(Worker pid=7049)   return _methods._mean(a, axis=axis, dtype=dtype,
(Worker pid=7049) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/numpy/_core/_methods.py:142: RuntimeWarning: invalid value encountered in scalar divide
(Worker pid=7049)   ret = ret.dtype.type(ret / rcount)

(Worker pid=7049) INFO:2026-07-28,02:16:09 ||| Worker 2 completed 1/~105.0:: 0.95% complete

(Worker pid=7049) INFO:2026-07-28,02:16:11 ||| Worker at start location (20,0) completed 10/11

(Worker pid=7051) ERROR:2026-07-28,02:16:09 ||| [x] Data path does not exist [repeated 3x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.)
(Worker pid=7051) ERROR:2026-07-28,02:16:09 ||| The following path does not exist, please verify your installation environment: /home/runner/.isofit/data/earth_sun_distance.txt [repeated 3x across cluster]
(Worker pid=7051) WARNING:2026-07-28,02:16:09 ||| Earth-sun-distance file not found on system. Proceeding without might cause some inaccuracies down the line. [repeated 3x across cluster]
(Worker pid=7051) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/numpy/_core/fromnumeric.py:3824: RuntimeWarning: Mean of empty slice [repeated 3x across cluster]
(Worker pid=7051)   return _methods._mean(a, axis=axis, dtype=dtype, [repeated 3x across cluster]
(Worker pid=7051) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/numpy/_core/_methods.py:142: RuntimeWarning: invalid value encountered in scalar divide [repeated 3x across cluster]
(Worker pid=7051)   ret = ret.dtype.type(ret / rcount) [repeated 3x across cluster]
(Worker pid=7051) INFO:2026-07-28,02:16:12 ||| Worker 1 completed 12/~105.0:: 11.43% complete [repeated 7x across cluster]

(Worker pid=7049) INFO:2026-07-28,02:16:16 ||| Worker at start location (95,0) completed 9/10 [repeated 8x across cluster]

(Worker pid=7051) INFO:2026-07-28,02:16:18 ||| Worker 1 completed 34/~105.0:: 32.38% complete [repeated 8x across cluster]

(Worker pid=7052) INFO:2026-07-28,02:16:22 ||| Worker at start location (192,0) completed 9/10 [repeated 8x across cluster]

(Worker pid=7051) INFO:2026-07-28,02:16:23 ||| Worker 1 completed 56/~105.0:: 53.33% complete [repeated 8x across cluster]

(Worker pid=7049) INFO:2026-07-28,02:16:28 ||| Worker at start location (268,0) completed 10/11 [repeated 8x across cluster]

(Worker pid=7051) INFO:2026-07-28,02:16:29 ||| Worker 1 completed 78/~105.0:: 74.29% complete [repeated 8x across cluster]

(Worker pid=7049) INFO:2026-07-28,02:16:33 ||| Worker at start location (354,0) completed 10/11 [repeated 8x across cluster]

(Worker pid=7052) INFO:2026-07-28,02:16:35 ||| Worker 3 completed 97/~105.0:: 92.38% complete [repeated 7x across cluster]

INFO:2026-07-28,02:16:37 || isofit.py:run() | Pixel class: uniform_surface inversions complete.

INFO:2026-07-28,02:16:37 || isofit.py:run() | Running totals

INFO:2026-07-28,02:16:37 || isofit.py:run() | 35.99s total

INFO:2026-07-28,02:16:37 || isofit.py:run() | 11.6713 spectra/s

INFO:2026-07-28,02:16:37 || isofit.py:run() | 2.9178 spectra/s/core

INFO:2026-07-28,02:16:37 || isofit.py:run() | All Inversions complete.

INFO:2026-07-28,02:16:37 || isofit.py:run() | Final totals

INFO:2026-07-28,02:16:37 || isofit.py:run() | 35.99s total

INFO:2026-07-28,02:16:37 || isofit.py:run() | 11.6713 spectra/s

INFO:2026-07-28,02:16:37 || isofit.py:run() | 2.9178 spectra/s/core

INFO:2026-07-28,02:16:37 || apply_oe.py:apply_oe() | Full (non-aerosol) LUTs:

INFO:2026-07-28,02:16:37 || apply_oe.py:apply_oe() | Elevation: None

INFO:2026-07-28,02:16:37 || apply_oe.py:apply_oe() | To-sensor zenith: [0.9608 2.9675]

INFO:2026-07-28,02:16:37 || apply_oe.py:apply_oe() | To-sun zenith: None

INFO:2026-07-28,02:16:37 || apply_oe.py:apply_oe() | Relative to-sun azimuth: [3.80000e-03 4.12002e+01 8.23965e+01]

INFO:2026-07-28,02:16:37 || apply_oe.py:apply_oe() | H2O Vapor: [0.703  0.7574]

INFO:2026-07-28,02:16:37 || apply_oe.py:apply_oe() | Writing main configuration file.

INFO:2026-07-28,02:16:37 || template_construction.py:load_climatology() | Loading Climatology

INFO:2026-07-28,02:16:37 || template_construction.py:load_climatology() | Climatology Loaded.  Aerosol State Vector:
{'AOT550': {'bounds': [0.04209, 1.0], 'scale': 1, 'init': 0.137881, 'prior_sigma': 0.1, 'prior_mean': 0.137881}}
Aerosol LUT Grid:
{'AOT550': [0.04209, 0.137881, 0.233672, 0.329463, 0.425254, 0.521045, 0.6168359999999999, 0.712627, 0.808418, 0.9042089999999999, 1.0]}
Aerosol model path:/home/runner/work/isofit-tutorials/data/aerosol_model.txt

INFO:2026-07-28,02:16:37 || ini.py:toTemplate() | Saved converted json to: /home/runner/work/isofit-tutorials/isofit-tutorials/NEON/output/NIS01_20210403_173647/config/NIS01_20210403_173647_isofit.json.tmpl

INFO:2026-07-28,02:16:37 || apply_oe.py:apply_oe() | Running ISOFIT with full LUT

WARNING:2026-07-28,02:16:37 || __init__.py:checkNumThreads() | 
******************************************************************************************
! Number of threads is greater than 1 (currently: 4), this may greatly impact performance
! Please set this the environment variables 'MKL_NUM_THREADS' and 'OMP_NUM_THREADS' to '1'
******************************************************************************************

INFO:2026-07-28,02:16:37 || configs.py:create_new_config() | Loading config file: /home/runner/work/isofit-tutorials/isofit-tutorials/NEON/output/NIS01_20210403_173647/config/NIS01_20210403_173647_isofit.json

INFO:2026-07-28,02:16:37 || configs.py:get_config_errors() | Checking config sections for configuration issues

INFO:2026-07-28,02:16:37 || configs.py:get_config_errors() | Configuration file checks complete, no errors found.

/home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/numpy/_core/fromnumeric.py:3824: RuntimeWarning: Mean of empty slice
  return _methods._mean(a, axis=axis, dtype=dtype,
/home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/numpy/_core/_methods.py:142: RuntimeWarning: invalid value encountered in scalar divide
  ret = ret.dtype.type(ret / rcount)

2026-07-28 02:16:37,985	INFO worker.py:1847 -- Calling ray.init() again after it has already been called.

INFO:2026-07-28,02:16:37 || isofit.py:run() | Running surfaces: uniform_surface

INFO:2026-07-28,02:16:37 || atmosphere.py:__init__() | Loading from wavelength_file: /home/runner/work/isofit-tutorials/isofit-tutorials/NEON/output/NIS01_20210403_173647/data/wavelengths.txt

INFO:2026-07-28,02:16:37 || reader.py:__init__() | No LUT provided, attempting to build it

INFO:2026-07-28,02:16:38 || zarr.py:calc_shards() | Number of points: 56,232

INFO:2026-07-28,02:16:38 || zarr.py:calc_shards() | Target chunks per file: 157532 (6.00gb) (scale: 12)

INFO:2026-07-28,02:16:38 || zarr.py:calc_shards() | Shapes:

INFO:2026-07-28,02:16:38 || zarr.py:calc_shards() |   dimensions: [426  11   2   2   3]

INFO:2026-07-28,02:16:38 || zarr.py:calc_shards() |     chunking: [426   1   1   1   1]

INFO:2026-07-28,02:16:38 || zarr.py:calc_shards() | Recommended

INFO:2026-07-28,02:16:38 || zarr.py:calc_shards() |     sharding: [426  11   2   2   3]

INFO:2026-07-28,02:16:38 || zarr.py:calc_shards() | Produces:

INFO:2026-07-28,02:16:38 || zarr.py:calc_shards() |   Chunks per file: 132 (0.01gb)

INFO:2026-07-28,02:16:38 || zarr.py:calc_shards() |   Number of files: 1

INFO:2026-07-28,02:16:38 || writer.py:runSimulations() | Running any pre-sim functions

INFO:2026-07-28,02:16:38 || sRTMnet.py:preSim() | Building simulator and executing (6S)

INFO:2026-07-28,02:16:38 || reader.py:__init__() | No LUT provided, attempting to build it

INFO:2026-07-28,02:16:38 || zarr.py:calc_shards() | Number of points: 113,652

INFO:2026-07-28,02:16:38 || zarr.py:calc_shards() | Target chunks per file: 77942 (6.00gb) (scale: 12)

INFO:2026-07-28,02:16:38 || zarr.py:calc_shards() | Shapes:

INFO:2026-07-28,02:16:38 || zarr.py:calc_shards() |   dimensions: [861  11   2   2   3]

INFO:2026-07-28,02:16:38 || zarr.py:calc_shards() |     chunking: [861   1   1   1   1]

INFO:2026-07-28,02:16:38 || zarr.py:calc_shards() | Recommended

INFO:2026-07-28,02:16:38 || zarr.py:calc_shards() |     sharding: [861  11   1   1   3]

INFO:2026-07-28,02:16:38 || zarr.py:calc_shards() | Produces:

INFO:2026-07-28,02:16:38 || zarr.py:calc_shards() |   Chunks per file: 33 (0.00gb)

INFO:2026-07-28,02:16:38 || zarr.py:calc_shards() |   Number of files: 4

INFO:2026-07-28,02:16:38 || writer.py:runSimulations() | Running any pre-sim functions

INFO:2026-07-28,02:16:38 || writer.py:runSimulations() | Executing parallel simulations by shards

INFO:2026-07-28,02:19:11 || writer.py:parallelize_shards() | Saved chunkless: {'coszen', 'solzen'}

INFO:2026-07-28,02:19:11 || common.py:__call__() |  25.00% shards complete (elapsed: 0:02:33.803466, rate: 0:00:38.450866, eta: 4:13:46.543134)

INFO:2026-07-28,02:19:14 || common.py:__call__() |  50.00% shards complete (elapsed: 0:02:35.880427, rate: 0:00:38.970107, eta: 2:07:18.140923)

INFO:2026-07-28,02:19:14 || common.py:__call__() |  75.00% shards complete (elapsed: 0:02:36.257359, rate: 0:00:39.064340, eta: 1:24:12.321274)

INFO:2026-07-28,02:19:14 || common.py:__call__() | 100.00% shards complete (elapsed: 0:02:36.687312, rate: 0:00:39.171828, eta: 1:02:40.495488)

INFO:2026-07-28,02:19:15 || writer.py:runSimulations() | Running any post-sim functions

INFO:2026-07-28,02:19:15 || writer.py:runSimulations() | Saving post-sim data to index zero of all dimensions except wl

/home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/zarr/api/asynchronous.py:231: ZarrUserWarning: Consolidated metadata is currently not part in the Zarr format 3 specification. It may not be supported by other zarr implementations and may change in the future.
  warnings.warn(
INFO:2026-07-28,02:19:15 || sRTMnet.py:process() | Interpolating simulator quantities to emulator size

INFO:2026-07-28,02:19:15 || sRTMnet.py:process() | Loading and predicting with emulator on 4 cores

INFO:2026-07-28,02:19:17 || common.py:__call__() | 100.00% shards processed (elapsed: 0:00:01.989232, rate: 0:00:01.989232, eta: 0:03:16.933968)

INFO:2026-07-28,02:19:17 || writer.py:runSimulations() | Saving pre-sim data to index zero of all dimensions except wl

INFO:2026-07-28,02:19:18 || writer.py:runSimulations() | Running any post-sim functions

/home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/zarr/api/asynchronous.py:231: ZarrUserWarning: Consolidated metadata is currently not part in the Zarr format 3 specification. It may not be supported by other zarr implementations and may change in the future.
  warnings.warn(
/home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/isofit/atmosphere/atmosphere.py:618: FutureWarning: Deleting a single level of a MultiIndex is deprecated. Previously, this deleted all levels of a MultiIndex. Please also drop the following variables: {'point'} to avoid an error in the future.
  ds[term] = ds[key1] * ds[key2]

INFO:2026-07-28,02:19:19 || isofit.py:run() | Beginning 420 inversions in 40 chunks using 4 cores

(Worker pid=7875) ERROR:2026-07-28,02:19:26 ||| [x] Data path does not exist
(Worker pid=7875) ERROR:2026-07-28,02:19:26 ||| The following path does not exist, please verify your installation environment: /home/runner/.isofit/data/earth_sun_distance.txt
(Worker pid=7875) WARNING:2026-07-28,02:19:26 ||| Earth-sun-distance file not found on system. Proceeding without might cause some inaccuracies down the line.
(Worker pid=7052) INFO:2026-07-28,02:16:37 ||| Worker at start location (419,0) completed 10/11 [repeated 6x across cluster]

(Worker pid=7875) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/numpy/_core/fromnumeric.py:3824: RuntimeWarning: Mean of empty slice
(Worker pid=7875)   return _methods._mean(a, axis=axis, dtype=dtype,
(Worker pid=7875) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/numpy/_core/_methods.py:142: RuntimeWarning: invalid value encountered in scalar divide
(Worker pid=7875)   ret = ret.dtype.type(ret / rcount)

(Worker pid=7877) INFO:2026-07-28,02:19:28 ||| Worker 0 completed 1/~105.0:: 0.95% complete

(Worker pid=7874) ERROR:2026-07-28,02:19:26 ||| [x] Data path does not exist [repeated 3x across cluster]
(Worker pid=7874) ERROR:2026-07-28,02:19:26 ||| The following path does not exist, please verify your installation environment: /home/runner/.isofit/data/earth_sun_distance.txt [repeated 3x across cluster]
(Worker pid=7874) WARNING:2026-07-28,02:19:26 ||| Earth-sun-distance file not found on system. Proceeding without might cause some inaccuracies down the line. [repeated 3x across cluster]
(Worker pid=7877) INFO:2026-07-28,02:19:31 ||| Worker at start location (42,0) completed 10/11 [repeated 2x across cluster]

(Worker pid=7876) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/numpy/_core/fromnumeric.py:3824: RuntimeWarning: Mean of empty slice [repeated 3x across cluster]
(Worker pid=7876)   return _methods._mean(a, axis=axis, dtype=dtype, [repeated 3x across cluster]
(Worker pid=7876) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/numpy/_core/_methods.py:142: RuntimeWarning: invalid value encountered in scalar divide [repeated 3x across cluster]
(Worker pid=7876)   ret = ret.dtype.type(ret / rcount) [repeated 3x across cluster]

(Worker pid=7874) INFO:2026-07-28,02:19:33 ||| Worker 1 completed 12/~105.0:: 11.43% complete [repeated 7x across cluster]

(Worker pid=7875) INFO:2026-07-28,02:19:38 ||| Worker at start location (95,0) completed 9/10 [repeated 7x across cluster]

(Worker pid=7874) INFO:2026-07-28,02:19:40 ||| Worker 1 completed 34/~105.0:: 32.38% complete [repeated 8x across cluster]

(Worker pid=7874) INFO:2026-07-28,02:19:43 ||| Worker at start location (171,0) completed 10/11 [repeated 6x across cluster]

(Worker pid=7876) INFO:2026-07-28,02:19:43 ||| Worker 2 completed 45/~105.0:: 42.86% complete [repeated 4x across cluster]

(Worker pid=7875) INFO:2026-07-28,02:19:49 ||| Worker at start location (235,0) completed 9/10 [repeated 6x across cluster]

(Worker pid=7876) INFO:2026-07-28,02:19:51 ||| Worker 2 completed 67/~105.0:: 63.81% complete [repeated 7x across cluster]

(Worker pid=7876) INFO:2026-07-28,02:19:54 ||| Worker at start location (289,0) completed 9/10 [repeated 6x across cluster]

(Worker pid=7874) INFO:2026-07-28,02:19:55 ||| Worker 1 completed 78/~105.0:: 74.29% complete [repeated 5x across cluster]

(Worker pid=7875) INFO:2026-07-28,02:20:00 ||| Worker at start location (354,0) completed 10/11 [repeated 6x across cluster]

(Worker pid=7877) INFO:2026-07-28,02:20:01 ||| Worker 0 completed 98/~105.0:: 93.33% complete [repeated 7x across cluster]

INFO:2026-07-28,02:20:04 || isofit.py:run() | Pixel class: uniform_surface inversions complete.

INFO:2026-07-28,02:20:04 || isofit.py:run() | Running totals

INFO:2026-07-28,02:20:04 || isofit.py:run() | 45.31s total

INFO:2026-07-28,02:20:04 || isofit.py:run() | 9.2704 spectra/s

INFO:2026-07-28,02:20:04 || isofit.py:run() | 2.3176 spectra/s/core

INFO:2026-07-28,02:20:04 || isofit.py:run() | All Inversions complete.

INFO:2026-07-28,02:20:04 || isofit.py:run() | Final totals

INFO:2026-07-28,02:20:04 || isofit.py:run() | 45.31s total

INFO:2026-07-28,02:20:04 || isofit.py:run() | 9.2704 spectra/s

INFO:2026-07-28,02:20:04 || isofit.py:run() | 2.3176 spectra/s/core

INFO:2026-07-28,02:20:04 || apply_oe.py:apply_oe() | Analytical line inference

INFO:2026-07-28,02:20:04 || configs.py:create_new_config() | Loading config file: /home/runner/work/isofit-tutorials/isofit-tutorials/NEON/output/NIS01_20210403_173647/config/NIS01_20210403_173647_isofit.json

/home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/numpy/_core/fromnumeric.py:3824: RuntimeWarning: Mean of empty slice
  return _methods._mean(a, axis=axis, dtype=dtype,
/home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/numpy/_core/_methods.py:142: RuntimeWarning: invalid value encountered in scalar divide
  ret = ret.dtype.type(ret / rcount)

2026-07-28 02:20:04,785	INFO worker.py:1847 -- Calling ray.init() again after it has already been called.

INFO:2026-07-28,02:20:04 || atm_interpolation.py:atm_interpolation() | Beginning atmospheric interpolation 4 cores

(_run_chunk pid=8001) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/isofit/utils/atm_interpolation.py:182: LinAlgWarning: An ill-conditioned matrix detected: slice 0 has rcond = 3.5544682726881455e-33.
(_run_chunk pid=8001)   bhat[i, :] = (inv(X.T @ W @ X) @ X.T @ W @ y).T
(Worker pid=7877) INFO:2026-07-28,02:20:04 ||| Worker at start location (419,0) completed 10/11 [repeated 6x across cluster]
INFO:2026-07-28,02:20:12 || atm_interpolation.py:atm_interpolation() | Parallel atmospheric interpolations complete.  7.531796932220459 s total, 560.4240313414293 spectra/s, 140.10600783535733 spectra/s/core

2026-07-28 02:20:12,343	INFO worker.py:1847 -- Calling ray.init() again after it has already been called.

INFO:2026-07-28,02:20:12 || atmosphere.py:__init__() | Loading from wavelength_file: /home/runner/work/isofit-tutorials/isofit-tutorials/NEON/output/NIS01_20210403_173647/data/wavelengths.txt

INFO:2026-07-28,02:20:12 || reader.py:__init__() | Prebuilt LUT provided

/home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/isofit/atmosphere/atmosphere.py:618: FutureWarning: Deleting a single level of a MultiIndex is deprecated. Previously, this deleted all levels of a MultiIndex. Please also drop the following variables: {'point'} to avoid an error in the future.
  ds[term] = ds[key1] * ds[key2]

(Worker pid=8249) ERROR:2026-07-28,02:20:19 ||| [x] Data path does not exist
(Worker pid=8249) ERROR:2026-07-28,02:20:19 ||| The following path does not exist, please verify your installation environment: /home/runner/.isofit/data/earth_sun_distance.txt
(Worker pid=8249) WARNING:2026-07-28,02:20:19 ||| Earth-sun-distance file not found on system. Proceeding without might cause some inaccuracies down the line.
(_run_chunk pid=8004) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/isofit/utils/atm_interpolation.py:182: LinAlgWarning: An ill-conditioned matrix detected: slice 0 has rcond = 3.5544682726881455e-33. [repeated 3x across cluster]
(_run_chunk pid=8004)   bhat[i, :] = (inv(X.T @ W @ X) @ X.T @ W @ y).T [repeated 3x across cluster]

(Worker pid=8249) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/numpy/_core/fromnumeric.py:3824: RuntimeWarning: Mean of empty slice
(Worker pid=8249)   return _methods._mean(a, axis=axis, dtype=dtype,
(Worker pid=8249) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/numpy/_core/_methods.py:142: RuntimeWarning: invalid value encountered in scalar divide
(Worker pid=8249)   ret = ret.dtype.type(ret / rcount)

(Worker pid=8249) INFO:2026-07-28,02:20:23 ||| Analytical line writing lines: 0 to 1. Surface: uniform_surface

(Worker pid=8252) ERROR:2026-07-28,02:20:20 ||| [x] Data path does not exist [repeated 3x across cluster]
(Worker pid=8252) ERROR:2026-07-28,02:20:20 ||| The following path does not exist, please verify your installation environment: /home/runner/.isofit/data/earth_sun_distance.txt [repeated 3x across cluster]
(Worker pid=8252) WARNING:2026-07-28,02:20:20 ||| Earth-sun-distance file not found on system. Proceeding without might cause some inaccuracies down the line. [repeated 3x across cluster]
(Worker pid=8252) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/numpy/_core/fromnumeric.py:3824: RuntimeWarning: Mean of empty slice [repeated 3x across cluster]
(Worker pid=8252)   return _methods._mean(a, axis=axis, dtype=dtype, [repeated 3x across cluster]
(Worker pid=8252) /home/runner/work/isofit-tutorials/isofit-tutorials/.venv/lib/python3.12/site-packages/numpy/_core/_methods.py:142: RuntimeWarning: invalid value encountered in scalar divide [repeated 3x across cluster]
(Worker pid=8252)   ret = ret.dtype.type(ret / rcount) [repeated 3x across cluster]

(Worker pid=8249) INFO:2026-07-28,02:20:29 ||| Analytical line writing lines: 6 to 8. Surface: uniform_surface [repeated 5x across cluster]

(Worker pid=8249) INFO:2026-07-28,02:20:35 ||| Analytical line writing lines: 14 to 16. Surface: uniform_surface [repeated 5x across cluster]

(Worker pid=8249) INFO:2026-07-28,02:20:41 ||| Analytical line writing lines: 22 to 24. Surface: uniform_surface [repeated 5x across cluster]

(Worker pid=8251) INFO:2026-07-28,02:20:46 ||| Analytical line writing lines: 32 to 33. Surface: uniform_surface [repeated 4x across cluster]

(Worker pid=8252) INFO:2026-07-28,02:20:52 ||| Analytical line writing lines: 35 to 37. Surface: uniform_surface [repeated 4x across cluster]

(Worker pid=8252) INFO:2026-07-28,02:20:58 ||| Analytical line writing lines: 43 to 45. Surface: uniform_surface [repeated 5x across cluster]

(Worker pid=8252) INFO:2026-07-28,02:21:04 ||| Analytical line writing lines: 51 to 53. Surface: uniform_surface [repeated 5x across cluster]

(Worker pid=8252) INFO:2026-07-28,02:21:09 ||| Analytical line writing lines: 59 to 61. Surface: uniform_surface [repeated 4x across cluster]

INFO:2026-07-28,02:21:11 || analytical_line.py:analytical_line() | Analytical line inversions complete.  58.65s total, 71.9671 spectra/s, 17.9918 spectra/s/core

INFO:2026-07-28,02:21:11 || apply_oe.py:apply_oe() | Done.

Plotting¶

Below plots the regions of interest defined by a NEON report.

In [8]:

# Load in the ISOFIT reflectance output
ds = envi.open(paths.working / f"output/{neon_str}_rfl.hdr")
rfl = ds.open_memmap(interleave='bip')
rgb = rfl[:, :, [60, 40, 30]].copy()
wl = np.array(ds.metadata['wavelength'], dtype=float)

# Find the bounding box for all regions of interest (RoI)
regions = report[neon_id]
bounds = np.vstack(list(regions.values()))
y = bounds[:, 0].min() - 5 # , bounds[:, 1].max() + 5
x = bounds[:, 2].min() - 5 # , bounds[:, 3].max() + 5

# Load in the ISOFIT reflectance output ds = envi.open(paths.working / f"output/{neon_str}_rfl.hdr") rfl = ds.open_memmap(interleave='bip') rgb = rfl[:, :, [60, 40, 30]].copy() wl = np.array(ds.metadata['wavelength'], dtype=float) # Find the bounding box for all regions of interest (RoI) regions = report[neon_id] bounds = np.vstack(list(regions.values())) y = bounds[:, 0].min() - 5 # , bounds[:, 1].max() + 5 x = bounds[:, 2].min() - 5 # , bounds[:, 3].max() + 5

In [9]:

# Plot the RoIs
fig, ax = plt.subplots(figsize=(7, 7))

ax.imshow(rgb / np.max(rgb, axis=(0, 1))) # Dividing brightens the image
ax.set_title(neon_str)

for i, (roi, region) in enumerate(regions.items()):
    rect = patches.Rectangle(
        (region[2] - x, region[0] - y),
        region[3] - region[2],
        region[1] - region[0],
        linewidth = 1,
        edgecolor = f'C{i}',
        facecolor = 'none',
        label = roi
    )
    ax.add_patch(rect)
ax.legend(loc='lower right')

# Plot the RoIs fig, ax = plt.subplots(figsize=(7, 7)) ax.imshow(rgb / np.max(rgb, axis=(0, 1))) # Dividing brightens the image ax.set_title(neon_str) for i, (roi, region) in enumerate(regions.items()): rect = patches.Rectangle( (region[2] - x, region[0] - y), region[3] - region[2], region[1] - region[0], linewidth = 1, edgecolor = f'C{i}', facecolor = 'none', label = roi ) ax.add_patch(rect) ax.legend(loc='lower right')

Out[9]:

<matplotlib.legend.Legend at 0x7ff9f818aed0>

In [10]:

fig, axes = plt.subplots(len(regions), sharex=True, figsize=(10, 3*len(regions)))

for i, (roi, region) in enumerate(regions.items()):
    ax = axes[i]

    in_situ = np.genfromtxt(paths.insitu / f'{roi}01/Data/{roi}01_Refl.dat', skip_header=3)
    ax.plot(in_situ[:, 0], in_situ[:, 1], label='In Situ', c='red', ls='-')

    mean_rfl = np.mean(
        rfl[
            region[0] - y : region[1] - y,
            region[2] - x : region[3] - x,
        ],
        axis = (0, 1)
    )

    ax.plot(wl, mean_rfl, label='Isofit', c='black')

    ax.set_ylabel('Reflectance')
    ax.set_title(roi)
    ax.legend()

ax.set_xlabel('Wavelength')
plt.tight_layout()

fig, axes = plt.subplots(len(regions), sharex=True, figsize=(10, 3*len(regions))) for i, (roi, region) in enumerate(regions.items()): ax = axes[i] in_situ = np.genfromtxt(paths.insitu / f'{roi}01/Data/{roi}01_Refl.dat', skip_header=3) ax.plot(in_situ[:, 0], in_situ[:, 1], label='In Situ', c='red', ls='-') mean_rfl = np.mean( rfl[ region[0] - y : region[1] - y, region[2] - x : region[3] - x, ], axis = (0, 1) ) ax.plot(wl, mean_rfl, label='Isofit', c='black') ax.set_ylabel('Reflectance') ax.set_title(roi) ax.legend() ax.set_xlabel('Wavelength') plt.tight_layout()

We can plot out the mapped reflectance (as above), but also the interpolated atmospheric conditions. The windows size is small enough here (and the atmospheric parameters are chosen in such a way) that the map is going to be pretty static...but we can still see it.

In [11]:

dat = envi.open(paths.working / f"output/{neon_str}_atm_interp.hdr")
atm = dat.open_memmap(interleave='bip').copy()

plt.figure(figsize=(7, 7))
plt.title('AOD')
plt.imshow(atm[..., 0])
plt.colorbar()

plt.figure(figsize=(7, 7))
plt.title('Water Vapor')
plt.imshow(atm[..., 1])
plt.colorbar()

dat = envi.open(paths.working / f"output/{neon_str}_atm_interp.hdr") atm = dat.open_memmap(interleave='bip').copy() plt.figure(figsize=(7, 7)) plt.title('AOD') plt.imshow(atm[..., 0]) plt.colorbar() plt.figure(figsize=(7, 7)) plt.title('Water Vapor') plt.imshow(atm[..., 1]) plt.colorbar()

Out[11]:

<matplotlib.colorbar.Colorbar at 0x7ff9d27ef6b0>

In [ ]: