utils.loggable¶

The Loggable module provides tools for configuring the logging of Sapicore models. It supports logging simple properties to tensorboard as well as arbitrary properties, including tensors and arrays, to HDF5 files.

Individual properties can be included or excluded from logging using a config dict that can be saved to a YAML, similarly to tree-config configuration.

Loggable¶

Loggable defines an API for properties to support opting-in to logging. Users may list loggable properties by including them in _loggable_props_ on a per class basis.

Subclassing extends _loggable_props_, which are accumulated across all superclasses when loggable_props is called. The latter also updates the cache list _loggable_props. Setting utils.loggable.Loggable._loggable_props_, in and of itself, will NOT update the cache.

To support the logging of nested objects, we use _loggable_children_ to list all objects that should be inspected for loggable properties. These are similarly listed in loggable_children.

API¶

Users may obtain the loggable properties as YAML files, dictionaries, or flat lists of properties using the following functions:

read_loggable_from_object(), read_loggable_from_file(), update_loggable_from_object(), dump_loggable(), load_save_get_loggable_properties(), and get_loggable_properties().

They all provide a mapping from the property name to a boolean indicting whether the property should be logged.

Example

The following class inherits from Loggable:

>>> class Example(Loggable):
>>>     _loggable_props_: tuple[str] = ("voltage",)
>>>     def __init__(self, voltage: list[float]):
>>>         super().__init__()
>>>         self.voltage = voltage

After instantiating an object, refresh the cache and obtain the loggable properties:

>>> dummy = Example(voltage=[1, 2, 3])
>>> print(dummy.loggable_props)
['voltage']

To check which attributes are loggable:

>>> read_loggable_from_object(dummy, True)
{'voltage': True}

Note

log_tensor_board() supports taking the flat list of properties and logging all the properties that are set to be logged to the tensorboard format on disk using a SummaryWriter instance.

Similarly, NixLogWriter supports logging arbitrary tensors and numpy arrays to a nix HDF5 file. NixLogReader can then be used to load the data from the nix file into scalars or numpy arrays of the original shape.

Warning

This module may change significantly in Sapicore > 0.3.0.

class utils.loggable.Loggable¶

The Loggable can be used as a base class for objects that need to control which of its properties can be logged.

When getting loggable properties, read_loggable_from_object() will call loggable_props() and loggable_children(), which looks up all the properties from _loggable_props_ and _loggable_children_.

Example

The following class has two loggable properties:

>>> class LoggableModel(Loggable):
>>>     _loggable_props_ = ("frame", "color")
>>>     frame: str = "square"
>>>     color: str = "blue"

This snippet will return what they are after instantiating an object of this class:

>>> LoggableModel().loggable_props
['frame', 'color']

To accumulate over loggable children as well:

>>> read_loggable_from_object(LoggableModel(), True)
{'frame': True, 'color': True}

property loggable_children: Dict[str, str]¶

A property containing a dictionary of loggable attribute names from derivative classes (i.e.,: listed in _loggable_children_ of this class and its parent classes).

Example

>>> class Thing(Loggable):
>>>     _loggable_children_ = {"useful": "box"}
>>>     box: str = None
>>>
>>> class Box(Loggable):
>>>     pass
>>>
>>> model = Thing()
>>> model.box = Box()
>>> model.loggable_children
{'useful': 'box'}

property loggable_props: List[str]¶: A property containing a list of instance attribute names that are loggable (i.e., listed in _loggable_props_ of this class and its parent classes).

class utils.loggable.NixLogReader(filename: str | Path)¶

A reader that can read and return the data written to a nix file using NixLogWriter.

E.g.:

. code-block:: python

with NixLogReader(‘log.h5’) as reader:
print(‘Logged experiments: ‘, reader.get_experiment_names()) # prints e.g. Logged experiments: [‘example’] print(‘Logged properties: ‘, reader.get_experiment_property_paths(‘example’)) # prints e.g. Logged properties: [(‘neuron_1’, ‘activation’), … print(‘Activation: ‘, reader.get_experiment_property_data(‘example’, (‘neuron_1’, ‘activation’))) # prints Activation: (array([0, 1, 2], dtype=int64), [array([0…

close_file()¶: Closes the nix file.

filename: str | Path = ''¶: The filename of the nix file that will be opened when open_file() is called.

get_experiment_names() → List[str]¶

Returns the list of block names found in the file that was created with NixLogWriter.create_block().

E.g.:

. code-block:: python

>>> with NixLogReader('log.h5') as reader:
>>>     print(reader.get_experiment_names())
['example']

get_experiment_property_data(name: str, property_path: Tuple[str, ...]) → Tuple[ndarray, List[ndarray | float | int]]¶

Returns all the data of a property previously logged with NixLogWriter.log().

Parameters:

name – The name of the block that contains the data.
property_path – The full attribute path to the property, so we can locate it in the data. It is an item from the list returned by get_experiment_property_paths().

Returns:

A 2-tuple of (counter, data). counter and data, have the same length. counter is an array containing the count value that corresponds the each data item as passed to the counter parameter in NixLogWriter.log(). data is a list of the values logged for this property, one item for each call to NixLogWriter.log(). It can be scalars or numpy arrays.

E.g.:

. code-block:: python

>>> with NixLogReader('log.h5') as reader:
>>>     print(reader.get_experiment_property_data(
...           'example', ('neuron_1', 'activation')))
(array([0, 1, 2], dtype=int64), [
    array([0.56843126, 1.0845224 , 1.3985955 ], dtype=float32),
    array([0.40334684, 0.83802634, 0.7192576 ], dtype=float32),
    array([0.40334353, 0.59663534, 0.18203649], dtype=float32)
])

get_experiment_property_paths(name: str) → List[Tuple[str]]¶

Returns the list of properties found in the file that was created with NixLogWriter.get_property_arrays().

Each item in the list is itself a tuple of strings. This tuple represents the path to the property starting from the root object (e.g. the model) as attributes. E.g. model.child.prop will be represented as ('model', 'child', 'prop').

Parameters:: name – The name of the block to scan for properties.

E.g.:

. code-block:: python

>>> with NixLogReader('log.h5') as reader:
>>>     print(reader.get_experiment_property_paths('example'))
[('neuron_1', 'activation'), ('neuron_1', 'intensity'),
('synapse', 'activation'), ('neuron_2', 'activation'),
('neuron_2', 'intensity'), ('activation_sum',)]

nix_file: File | None = None¶: The internal nix file object.

open_file()¶

Opens the file named in filename that was created with NixLogWriter.create_file().

close_file() must be called after this to close the file.

An alternative syntax is to use it as a context manager, which is a little safer:

. code-block:: python

with NixLogReader(‘log.h5’) as reader:
…

This internally calls open_file() and close_file() as needed.

class utils.loggable.NixLogWriter(filename: str | Path, git_hash='', compression=Compression.Auto, metadata: dict | None = None, config_data: dict | None = None)¶

Writer that supports logging properties to a Nix HDF5 file.

It supports logging arbitrary torch tensors, numpy arrays, and numerical scalars. The logged data can then be retrieved with NixLogReader.

A typical example is:

. code-block:: python

log_props = read_loggable_from_object(Model(), True) writer = NixLogWriter(h5_filename) # create the file writer.create_file() # create a section for this session writer.create_block(‘example’) # create the objects required for logging using the loggable properties property_arrays = writer.get_property_arrays(‘example’, log_props) do_work… # all the enabled loggable props will be logged writer.log(property_arrays, 0) do_more_work… writer.log(property_arrays, 1) # now close writer.close_file()

close_file() → None¶: Closes the nix file.

compression = 'Auto'¶

The compression to use in the file.

It is one of nix.Compression.

config_data: dict | None = None¶

A dict of tree_config config data used to config the model that can be included in the file.

It is saved to the file after being encoded using yaml_dumps().

create_block(name: str) → None¶

Creates a new block using the given that could subsequently be used with get_property_arrays() to log to this block.

It is used to organize the data into sessions.

create_file() → None¶

Creates the file named in filename. If it exists, an error is raised.

close_file() must be called after this to close the file.

filename: str | Path = ''¶: The filename of the nix file that will be created when create_file() is called.

get_property_arrays(name: str, loggable_properties: List[Tuple[List[str], Any, str, bool]]) → List[Tuple[Any, str, DataArray, DataArray, DataArray, DataArray]]¶

Takes the list of properties to be logged and it creates a list of objects that can be used with log to log all these properties.

See the class for an example.

Parameters:

name – The name of the block to which the properties will be logged. The block must have been created previously with create_block().
loggable_properties – The list of properties (in the format of get_loggable_properties()) to read and log.

Returns:

A list of objects that can be passed to log() to log the properties.

git_hash = ''¶: An optional git hash that can be included in the file. It can be used to label the model version for example.

static log(property_arrays: List[Tuple[Any, str, DataArray, DataArray, DataArray, DataArray]], counter: int) → None¶

Logs all the properties passed to get_property_arrays().

See the class for an example.

Parameters:

property_arrays – The list of objects returned by get_property_arrays()
counter – An arbitrary integer that gets logged with the properties that could be later used to identify the data, e.g. the epoch when it was logged.

metadata: dict | None = None¶

A dict of arbitrary metadata that can be included in the file.

It is saved to the file after being encoded using yaml_dumps().

nix_file: File | None = None¶: The internal nix file object.

utils.loggable.dump_loggable(filename: str | Path, data: Dict[str, Any]) → None¶

Dumps the loggable data gotten with e.g. read_loggable_from_object() to a yaml file.

Parameters:

filename – The yaml filename.
data – The loggable data.

E.g.:

>>> class Model(Loggable):
>>>     _config_props_ = ('name', )
>>>     name = 'chair'
>>> model = Model()
>>> d = read_loggable_from_object(model, True)
>>> dump_config('loggable.yaml', d)
>>> with open('loggable.yaml') as fh:
...     print(fh.read())

Which prints name: true.

utils.loggable.get_loggable_properties(obj: Loggable, loggable: Dict[str, Any], property_path: List[str] | None = None) → List[Tuple[List[str], Any, str, bool]]¶

Takes the loggable data read with read_loggable_from_object() or read_loggable_from_file(), filters those properties that should be logged (i.e. those that are True) and flattens them into a single flat list.

This list could be used to efficiently log all the properties that need to be logged, by iterating the list.

Parameters:

obj – The object from which to get the loggables.
loggable – The loggable data previously read with read_loggable_from_object() or read_loggable_from_file().
property_path –
A list of strings, indicating the object children names, starting from some root object that lead to the obj. This can be used to reconstruct the full object path to each property starting from obj.

Should just be None (default) in user code.

Returns:

A list of 4-tuples, each containing (property_path, item, prop, value). property_path is the list of strings, indicating the object children names, starting from some root object that lead to the item. item is the object or child whose property prop is being considered. value is a bool indicating whether to log the property.

E.g.:

class Model(Loggable):

    _loggable_props_ = ('name', )

    _loggable_children_ = {'the box': 'box'}

    name = 'chair'

    box = None

class Box(Loggable):

    _loggable_props_ = ('volume', )

    volume = 12

then:

>>> model = Model()
>>> model.box = Box()
>>> d = read_loggable_from_object(model, False)
{'the box': {'volume': False}, 'name': False'}
>>> d['the box']['volume'] = True
>>> get_loggable_properties(model, d)
[(['the box', 'volume'], <Box at 0x16f0b2b5320>, 'volume', True)]

utils.loggable.load_save_get_loggable_properties(obj: Any, filename: str | Path, default_value: bool) → List[Tuple[List[str], Any, str, bool]]¶

Loads the loggable configuration from the yaml file ( read_loggable_from_file(), if the file doesn’t exist it creates it with read_loggable_from_object() and dump_loggable()), updates it with update_loggable_from_object() and then dumps it back to the yaml file with dump_loggable(). It also returns the flattened loggable properties using get_loggable_properties().

This can be used to get the loggable properties, but also making sure the file contains the current loggables including any new properties not previously there.

Parameters:

obj – The loggable object.
filename – The yaml filename.
default_value – A bool (True/False), indicating whether to default the new loggable properties to True (they are logged) or False (they are not logged). The individual properties can subsequently be set either way manually in the file.

Returns:

The loggable list returned by get_loggable_properties().

E.x.:

class Model(Loggable):

    _config_props_ = ('name', )

    name = 'chair'

class ModelV2(Loggable):

    _config_props_ = ('name', 'side')

    name = 'chair'

    side = 'left'

then:

>>> model = Model()
>>> load_save_get_loggable_properties(model, 'loggable.yaml', True)
[(['name'], <Model at 0x16f0b2b5278>, 'name', True)]
>>> # then later for v2 of the model
>>> model_v2 = ModelV2()
>>> load_save_get_loggable_properties(model_v2, 'loggable.yaml', True)
[(['name'], <ModelV2 at 0x16f0b2b5695>, 'name', True),
 (['side'], <ModelV2 at 0x16f0b2b5695>, 'side', True)]
>>> with open('loggable.yaml') as fh:
...     print(fh.read())

this prints:

name: true
side: true

utils.loggable.log_tensor_board(writer, loggable_properties: List[Tuple[List[str], Any, str, bool]], global_step: int | None = None, walltime: float | None = None, prefix: str = '') → None¶

Logs the current value of all the given properties to the TensorBoard writer object for live display.

Properties that are arrays are logged as a sequence of scalars, which is inefficient. It is suggested to use NixLogWriter for complex data and log_tensor_board() only for scalars.

Parameters:

writer – An instantiated torch.utils.tensorboard.SummaryWriter object to which the data will be written.
loggable_properties – The list of properties (in the format of get_loggable_properties()) to read and log.
global_step – The optional global timestep, which must be monotonically increasing if providing.
walltime – The optional wall time.
prefix – An optional prefix used to log the data under.

utils.loggable.read_loggable_from_file(filename: str | Path) → Dict[str, Any]¶

Reads and returns the dict indicating whether to log objects from a file that was previously dumped with dump_loggable().

Parameters:: filename – The filename to the yaml file.

E.g.:

>>> class Model(Loggable):
>>>     _loggable_props_ = ('name', )
>>>     name = 'chair'
>>> model = Model()
>>> d = read_loggable_from_object(model, True)
>>> dump_loggable('loggable.yaml', d)
>>> read_loggable_from_file('loggable.yaml')
{'name': True}

utils.loggable.read_loggable_from_object(obj: Loggable, default_value: bool) → Dict[str, Any]¶

Returns a recursive dict containing all the loggable properties of the obj and its loggable children. Each item maps the item name to default_value, indicating whether to log the item by default.

Parameters:

obj – The object from which to get the loggables.
default_value – A bool (True/False), indicating whether to default the loggable properties to True (they are logged) or False (they are not logged). The individual properties can subsequently be set either way manually in the dict.

E.g.:

class Model(Loggable):

    _loggable_props_ = ('name', )

    _loggable_children_ = {'the box': 'box'}

    name = 'chair'

    box = None

class Box(Loggable):

    _loggable_props_ = ('volume', )

    volume = 12

then:

>>> model = Model()
>>> model.box = Box()
>>> loggables = read_loggable_from_object(model, False)
>>> loggables
{'the box': {'volume': False}, 'name': False'}
>>> # to log name
>>> loggables['name'] = True

utils.loggable.update_loggable_from_object(obj: Loggable, loggable: Dict[str, Any], default_value: bool) → Dict[str, Any]¶

Takes the loggable data read with read_loggable_from_object() or read_loggable_from_file() and creates a new loggable dict from it and updates it by adding any new loggable properties of the object, that has been added to the object or its children since the last time it was created.

Parameters:

obj – The object from which to get the loggables.
loggable – The the loggable data read with read_loggable_from_object() or read_loggable_from_file().
default_value – A bool (True/False), indicating whether to default the new loggable properties to True (they are logged) or False (they are not logged). The individual properties can subsequently be set either way manually in the dict.

Returns:

The updated loggable dict.