Skip to content

Writer

source module isofit.luts.writer

Manages the generation of new look-up tables (LUTs)

Classes

  • Writer — Handles writing RTE engine simulation data to an ISOFIT-compatible LUT store

Functions

  • streamSimulation — Run a simulation for a single point and stream the results to a saved lut file.

  • shardWriter — Execute simulations for a set of points and write chunked results to a LUT shard.

source streamSimulation(point: np.array, lut_names: list, simmer: Callable, reader: Callable, max_buffer_time: float = 0.5, rte_configure_and_exit: bool = False)

Run a simulation for a single point and stream the results to a saved lut file.

Parameters

  • point : np.array — conditions to alter in simulation

  • lut_names : list — Dimension names aka lut_names

  • simmer : function — function to run the simulation

  • reader : function — function to read the results of the simulation

  • max_buffer_time : float, optional — description. Defaults to 0.5.

  • rte_configure_and_exit : bool, optional — exit early if not executing simulations

source shardWriter(lut, shard, coord, points, simmer, reader)

Execute simulations for a set of points and write chunked results to a LUT shard.

This function is intended to run as a Ray remote task. For each input point, it runs a simulation, reads the resulting data, separates chunked and non-chunked outputs, and queues the chunked data into a shard buffer. After all points are processed, the buffered data is flushed to disk.

Parameters

  • lut : dict or LUT-like — Either a dictionary of arguments used to initialize a LUT via luts.create or an already-instantiated LUT Create object.

  • shard : int or str — Identifier for the shard being processed. Used for logging.

  • coord : tuple of slice or similar — Coordinate slices defining the region of the LUT corresponding to this shard.

  • points : iterable — Collection of input points to simulate and process. These points are assumed to be in the same shard.

  • simmer : callable — Function to run the simulation.

  • reader : callable — Function to read the results of the simulation.

Returns

  • tuple — A tuple (point, chunkless) where point is the last processed point and chunkless is a dictionary of non-chunked outputs from the last iteration.

source class Writer()

Handles writing RTE engine simulation data to an ISOFIT-compatible LUT store

Methods

  • write — Initialize a LUT and run simulations

  • runSimulations — Executes an engine object to process simulations using the preSim, makeSim, and postSim functions

  • parallelize_shards — Parallelizes simulations into shard groups to handle very large LUTs: VLLUTS

  • parallelize_points — Parallelizes simulations point-wise to stream finished data into the LUT at set completion percentages

  • preSim — This is an optional function that can be defined by a subclass RTE to be called directly before runSim() is executed. A subclass may return a dict containing any single or non-dimensional variables to be saved to the LUT file

  • makeSim — Prepares and executes a radiative transfer engine's simulations

  • readSim — Reads simulation results to standard form

  • postSim — This is an optional function that can be defined by a subclass RTE to be called directly after runSim() is finished. A subclass may return a dict containing any single or non-dimensional variables to be saved to the LUT file

  • point_to_filename — Change a point to a base filename

source method Writer.write(**kwargs)

Initialize a LUT and run simulations

Parameters

  • shard_size : str, default=None — Limit the total size of a single shard eg. "4gb"

  • target_shards : int, default=self.n_cores — Target number of shards in order to maximize system resources with parallelization

Raises

  • AttributeError

source method Writer.runSimulations()

Executes an engine object to process simulations using the preSim, makeSim, and postSim functions

source method Writer.parallelize_shards()

Parallelizes simulations into shard groups to handle very large LUTs: VLLUTS

source method Writer.parallelize_points()

Parallelizes simulations point-wise to stream finished data into the LUT at set completion percentages

source method Writer.preSim()

This is an optional function that can be defined by a subclass RTE to be called directly before runSim() is executed. A subclass may return a dict containing any single or non-dimensional variables to be saved to the LUT file

source method Writer.makeSim(point: np.array, template_only: bool = False)

Prepares and executes a radiative transfer engine's simulations

Parameters

  • point : np.array — conditions to alter in simulation

  • template_only : bool — only write template file and then stop

Raises

  • NotImplemented

source method Writer.readSim(point: np.array)

Reads simulation results to standard form

Parameters

  • point : np.array — conditions to alter in simulation

Raises

  • NotImplemented

source method Writer.postSim()

This is an optional function that can be defined by a subclass RTE to be called directly after runSim() is finished. A subclass may return a dict containing any single or non-dimensional variables to be saved to the LUT file

source method Writer.point_to_filename(point: np.array) → str

Change a point to a base filename

Parameters

  • point : np.array — conditions to alter in simulation

Returns

  • str — basename of the file to use for this point