Resource tracker

source module isofit.debug.resource_tracker

Classes

ResourceTracker — Tracks system resources
FileResources — Subclass of ResourceTracker that writes the polled resources to a json list file

Functions

stream — Generator that yields parsed JSONL objects from a growing json file produced by FileResources

source class ResourceTracker(callback: Callable[[dict], None], interval: float = 2, units: tuple = ('GB', 1024 ** 3), cores: Union[int, Literal['all']] = None, round: Union[bool, int] = 2, summarize: bool = True, allow_unsafe: bool = False)

Tracks system resources

Parameters

callback : Callable[[dict], None] — Function to call on each resource refresh. Signature must accept:

callable(dict) -> None

the first call will contain the non-changing values:
    used_cores : int
        Number of CPU cores in use. See the 'cores' parameter for more
    total_cores : int
        Total number of cores available on the system via os.cpu_count()
    sys_mem_total : float
        Total memory of the system
    mem_unit : str
        Unit label that the memory values are in
    mem_value : float
        The value used to convert the bytes to the mem_unit. This may be
        used to reverse the conversion
    poll_interval : float
        Resource polling interval
    timestamp : float
        The start timestamp of the resource tracker via time.time()

all calls afterwards will consist of:
    pid : int
        Main process ID
    name : str
        Main process name
    mem : float
        Main process memory used
    mem_total : float
        Approximate total memory of the process (actual + shared)
    mem_actual : float
        Approximate private memory in use by the process
    mem_shared : float
        Approximate shared memory
    cpu : float
        Main process CPU percentage over the interval
    sys_cpu_per_core : list[float]
        Per-core usage percentage over the interval
    timestamp : float
        Timestamp of the resource record via time.time()
    status : str
        Main process status, eg. 'running', 'sleeping'
    children : list[dict]
        Information of child processes:
            pid : int
                Child process ID
            name : str
                Child process name
            mem_total : float
                Approximate total memory of the child process (actual + shared)
            mem_actual : float
                Approximate private memory in use by the child process
            mem_shared : float
                Approximate shared memory
            cpu : float
                Child process CPU percentage over the interval
            status : str
                Child process status, eg. 'running', 'sleeping'

if summarize is enabled, these will also be included:
    mem_app_total : float
        Approximate total memory of the main process + children
    mem_app_actual : float
        Approximate total private memory over all processes
    mem_app_shared_avg : float
        Approximate average shared memory over all processes
    mem_used : float
        Memory in use by the system
    mem_avail : float
        Remaining available memory, defined as free + reclaimable
    cpu_avg : float
        Average CPU percentage calculated as: sum(main + children) / cores
    sys_cpu : float
        System-wide CPU percentage over the interval

interval : int | float, default=2 — Interval frequency in seconds to check resources Must be greater than 0. Values less than 0.1 risk high CPU usage and skewing polled results The CPU usage is calculated as the percentage of CPU used over this interval
units : tuple[str, float], default=("GB", 10243) — Units to convert the memory values to. Must be in the form of (str, float) where the float is used to divide the bytes values that psutil returns Some possible conversions: - ('b', 1/8) # Convert to bits (multiply by 8) - ('B', 1) # No conversion, leave as the default bytes - ('KB', 1024) # Kilobytes - ('MB', 10242) # Megabytes - ('GB', 1024**3) # Gigabytes, default
cores : int | 'all', default=1 — Number of cores being used by the source program. This is used for calculating the average CPU percentage. Can be passed 'all' to retrieve the os.cpu_count()
round : int | bool, default=2 — Round the memory variables to this many decimals. Set to False or 0 to disable True will be set to 1
summarize : bool, default=True — Includes summary statistics such as the sum of all children
allow_unsafe : bool, default=False — Bypasses the exception and allows unsafe interval values (less than 0.1) Not recommended

Examples

Basic usage::

import time
from isofit.debug.resource_tracker import ResourceTracker

def myFunction(info: dict):
    print(info)

rt = ResourceTracker(myFunction)
rt.start()
time.sleep(10)
rt.stop()

Methods

start — Starts the _track function in a thread
is_running — Checks if there is a thread running
stop — Sets the stop event to kill any running threads

source method ResourceTracker.start()

Starts the _track function in a thread

source method ResourceTracker.is_running()

Checks if there is a thread running

source method ResourceTracker.stop()

Sets the stop event to kill any running threads

source class FileResources(file: str, /, reset: bool = False, **kwargs)

Bases : ResourceTracker

Subclass of ResourceTracker that writes the polled resources to a json list file

Parameters

file : str — Path to a JSONL file to log resource information to
reset : bool, default=False — If the file exists, reset it

Examples

Basic usage::

import time
from isofit.debug.resource_tracker import FileResources

fr = ResourceTracker("resources.jsonl", reset=True)
fr.start()
time.sleep(10)
fr.stop()