Model API¶

The Plums Model Format main API is in the Model class which allows loading, saving and dynamic creation through a high-level python API. Moreover, it implement python representations of common Model components objects like Training, Producer, CheckpointCollection and Checkpoint.

See also

The Model developer API for more information on the Model API inner workings.

class plums.model.model.Model(producer_name, producer_version_format, producer_version_string, model_name, model_id, configuration, build_parameters)[source]¶

Bases: object

Define a Python representation of a PMF model.

Parameters

producer_name (str) – The name of the package that produced the model
producer_version_format (str) – The version format of the package that produced the model
producer_version_string (str) – The version string representation of the package that produced the model
model_name (str) – The model name
model_id (str) – The model identification string
configuration (Pathlike) – A path to the producer configuration file used to produce the model
build_parameters (dict) – A dictionary containing values necessary to instantiate the model from its weights.

name¶

The model name.

Type: str

id¶

The model unique identifier.

Type: str

build_parameters¶

A dictionary containing values necessary to instantiate the model from its weights.

Type: dict

producer¶

The Model producer.

Type: Producer

training¶

The Model training metadata.

Type: Training

checkpoint_collection¶

The Model attached collection of Checkpoint.

Type: CheckpointCollection

property initialisation¶

The model initialisation network if any, None otherwise.

Type: Model, Checkpoint

property path¶

If the model was loaded from disk, the Path it was loaded from. None otherwise.

Type: PathLike

property checkpoint¶

If the model is a Model initialisation, the checkpoint reference used. None otherwise.

Type: hashable

add_checkpoint(*checkpoints)[source]¶

Add a Checkpoint to the Model checkpoint_collection.

Parameters: *checkpoints (Checkpoint) – One ore several Checkpoint to add to the model CheckpointCollection.

register_training_start(epoch)[source]¶

Register a training start to the training metadata.

Parameters: epoch (int) – The Training staring epoch.

register_epoch(epoch=None)[source]¶

Register a given epoch as being the latest epoch to the training metadata.

Parameters: epoch (int) – Optional. Default to latest_epoch + 1. The epoch to be registered as latest.

register_training_end(success)[source]¶

Register a training end or a training failure to the training metadata.

Parameters: success (bool) – Whether the Training ended normally.

register_initialisation(path, checkpoint_reference=None, name=None)[source]¶

Register a valid PMF model initialisation from a Path to initialisation.

Parameters

path (PathLike) – A valid Path to the model initialisation.
checkpoint_reference (hashable) – If initialising from a PMF model, one needs to provided a reference to the actual model Checkpoint used to construct a valid initialisation.
name (str) – If initialising from a file, one needs to provide a name used to identify the initial network to construct a valid initialisation.

Raises

OSError – If the path provided does not exists on the filesystem or points to neither a file nor a PMF model.
ValueError – If the arguments provided are incompatibles: e.g. using a PMF initialisation without a checkpoint_reference.
SchemaError – If the path provided point to an invalid PMF model.

See also

The initialisation function for more information on the initialisation format.

save(path, force=False, **kwargs)[source]¶

Save a Model to Path.

Parameters

path (PathLike) – The Path where to save.
force (bool) – Optional. Default to False. If path is an existing non-PMF path or a PMF model with the same id, do not raise and carry on saving.

Raises

ValueError – If path points to a file.
OSError – If path points to: * A non-empty directory which does not contains a PMF model and force is False. * A non-empty directory which contains a PMF model with the same id and force is False. * A non-empty directory which contains a PMF model with a different id. * A non-empty directory which contains a PMF model with an invalid metadata file.

classmethod load(path, strict=True, verbose=False, **kwargs)[source]¶

Load a Model from a Path.

Parameters

path (PathLike) – The path to a PMF model to load.
strict (bool) – Whether to perform strict validation (see validate()).
verbose (bool) – Optional. Default to False. Toggle verbose offence reporting.
**kwargs (Any) – Additional arguments are passed to the validate() function.

Returns

A loaded Model instance.

Return type

Model

Raises

PlumsModelTreeValidationError – If the path provided point to an invalid PMF model.
PlumsModelMetadataValidationError – If the path provided point to a PMF model with an invalid metadata.

See also

The validate() function for more information on the PMF validation process.

plums.model.model.initialisation(path, checkpoint_reference=None, name=None)[source]¶

Construct a valid PMF model initialisation from a Path.

The return type depends on the initialisation type:

If the model was initialised from a PMF model, the function will return a Model instance.
If the model was initialised from a single file, the function will return a Checkpoint.

Parameters

path (PathLike) – A valid Path to the model initialisation.
checkpoint_reference (hashable) – If initialising from a PMF model, one needs to provided a reference to the actual model Checkpoint used to construct a valid initialisation.
name (str) – If initialising from a file, one needs to provide a name used to identify the initial network to construct a valid initialisation.

Returns

The retrieved model initialisation.

Return type

Model, Checkpoint

Raises

OSError – If the path provided does not exists on the filesystem or points to neither a file nor a PMF model.
ValueError – If the arguments provided are incompatibles: e.g. using a PMF initialisation without a checkpoint_reference.
PlumsModelTreeValidationError – If the path provided point to an invalid PMF model.
PlumsModelMetadataValidationError – If the path provided point to a PMF model with an invalid metadata.

Components¶

class plums.model.model.Model(producer_name, producer_version_format, producer_version_string, model_name, model_id, configuration, build_parameters)[source]¶

Bases: object

Define a Python representation of a PMF model.

Parameters

producer_name (str) – The name of the package that produced the model
producer_version_format (str) – The version format of the package that produced the model
producer_version_string (str) – The version string representation of the package that produced the model
model_name (str) – The model name
model_id (str) – The model identification string
configuration (Pathlike) – A path to the producer configuration file used to produce the model
build_parameters (dict) – A dictionary containing values necessary to instantiate the model from its weights.

name¶

The model name.

Type: str

id¶

The model unique identifier.

Type: str

build_parameters¶

A dictionary containing values necessary to instantiate the model from its weights.

Type: dict

producer¶

The Model producer.

Type: Producer

training¶

The Model training metadata.

Type: Training

checkpoint_collection¶

The Model attached collection of Checkpoint.

Type: CheckpointCollection

property initialisation¶

The model initialisation network if any, None otherwise.

Type: Model, Checkpoint

property path¶

If the model was loaded from disk, the Path it was loaded from. None otherwise.

Type: PathLike

property checkpoint¶

If the model is a Model initialisation, the checkpoint reference used. None otherwise.

Type: hashable

add_checkpoint(*checkpoints)[source]¶

Add a Checkpoint to the Model checkpoint_collection.

Parameters: *checkpoints (Checkpoint) – One ore several Checkpoint to add to the model CheckpointCollection.

register_training_start(epoch)[source]¶

Register a training start to the training metadata.

Parameters: epoch (int) – The Training staring epoch.

register_epoch(epoch=None)[source]¶

Register a given epoch as being the latest epoch to the training metadata.

Parameters: epoch (int) – Optional. Default to latest_epoch + 1. The epoch to be registered as latest.

register_training_end(success)[source]¶

Register a training end or a training failure to the training metadata.

Parameters: success (bool) – Whether the Training ended normally.

register_initialisation(path, checkpoint_reference=None, name=None)[source]¶

Register a valid PMF model initialisation from a Path to initialisation.

Parameters

path (PathLike) – A valid Path to the model initialisation.
checkpoint_reference (hashable) – If initialising from a PMF model, one needs to provided a reference to the actual model Checkpoint used to construct a valid initialisation.
name (str) – If initialising from a file, one needs to provide a name used to identify the initial network to construct a valid initialisation.

Raises

OSError – If the path provided does not exists on the filesystem or points to neither a file nor a PMF model.
ValueError – If the arguments provided are incompatibles: e.g. using a PMF initialisation without a checkpoint_reference.
SchemaError – If the path provided point to an invalid PMF model.

See also

The initialisation function for more information on the initialisation format.

save(path, force=False, **kwargs)[source]¶

Save a Model to Path.

Parameters

path (PathLike) – The Path where to save.
force (bool) – Optional. Default to False. If path is an existing non-PMF path or a PMF model with the same id, do not raise and carry on saving.

Raises

ValueError – If path points to a file.
OSError – If path points to: * A non-empty directory which does not contains a PMF model and force is False. * A non-empty directory which contains a PMF model with the same id and force is False. * A non-empty directory which contains a PMF model with a different id. * A non-empty directory which contains a PMF model with an invalid metadata file.

classmethod load(path, strict=True, verbose=False, **kwargs)[source]¶

Load a Model from a Path.

Parameters

path (PathLike) – The path to a PMF model to load.
strict (bool) – Whether to perform strict validation (see validate()).
verbose (bool) – Optional. Default to False. Toggle verbose offence reporting.
**kwargs (Any) – Additional arguments are passed to the validate() function.

Returns

A loaded Model instance.

Return type

Model

Raises

PlumsModelTreeValidationError – If the path provided point to an invalid PMF model.
PlumsModelMetadataValidationError – If the path provided point to a PMF model with an invalid metadata.

See also

The validate() function for more information on the PMF validation process.

plums.model.model.initialisation(path, checkpoint_reference=None, name=None)[source]¶

Construct a valid PMF model initialisation from a Path.

The return type depends on the initialisation type:

If the model was initialised from a PMF model, the function will return a Model instance.
If the model was initialised from a single file, the function will return a Checkpoint.

Parameters

path (PathLike) – A valid Path to the model initialisation.
checkpoint_reference (hashable) – If initialising from a PMF model, one needs to provided a reference to the actual model Checkpoint used to construct a valid initialisation.
name (str) – If initialising from a file, one needs to provide a name used to identify the initial network to construct a valid initialisation.

Returns

The retrieved model initialisation.

Return type

Model, Checkpoint

Raises

OSError – If the path provided does not exists on the filesystem or points to neither a file nor a PMF model.
ValueError – If the arguments provided are incompatibles: e.g. using a PMF initialisation without a checkpoint_reference.
PlumsModelTreeValidationError – If the path provided point to an invalid PMF model.
PlumsModelMetadataValidationError – If the path provided point to a PMF model with an invalid metadata.

class plums.model.components.components.Training(start_time=None, start_epoch=None, latest_time=None, latest_epoch=None, end_time=None, end_epoch=None, status='pending')[source]¶

Bases: object

Define a Python representation of a model training state.

Parameters

start_time (float) – The training stating time, or None if non existent
start_epoch (int) – The training stating epoch, or None if non existent
latest_epoch (int) – The training latest epoch, or None if non existent
latest_time (float) – The training latest epoch time, or None if non existent
end_time (float) – The training ending time, or None if non existent
end_epoch (int) – The training ending epoch, or None if non existent
status (str) – The training status, or 'pending' if non existent

property status¶

The training status, i.e. pending, running, failed or finished.

Type: str

property start_epoch¶

The training starting epoch if any, or None otherwise.

Type: int

property start_timestamp¶

The training starting timestamp if any, or None otherwise.

Type: int

property latest_epoch¶

The training latest epoch if any, or None otherwise.

Type: int

property latest_timestamp¶

The training latest timestamp if any, or None otherwise.

Type: int

property end_epoch¶

The training ending epoch if any, or None otherwise.

Type: int

property end_timestamp¶

The training ending timestamp if any, or None otherwise.

Type: int

property is_running¶

Whether the training is running.

Type: bool

property is_pending¶

Whether the training is pending.

Type: bool

property is_finished¶

Whether the training is finished.

Type: bool

property is_failed¶

Whether the training is failed.

Type: bool

start(epoch)[source]¶

Start the training and register the starting epoch and timestamp.

Parameters: epoch (int) – The starting epoch.

interrupt()[source]¶: Interrupt the training in a non-standard way and register the failing epoch and timestamp as the latest.

end()[source]¶: Interrupt the training in a standard way and register the ending epoch and timestamp as the latest.

register_epoch(epoch=None)[source]¶

Register a given epoch as being the latest epoch in the training along with its timestamp.

Parameters: epoch (int) – Optional. Default to latest_epoch + 1. The epoch to be registered as latest.

class plums.model.components.components.CheckpointCollection(*checkpoints)[source]¶

Bases: object

Define a checkpoint collection Python representation.

CheckpointCollection may be added through __setitem__ or through add().

Parameters: checkpoints (Checkpoint) – A Checkpoint to be added to the collection.

property latest¶

A reference to the latest Checkpoint added to the collection.

Type: hashable

property eloc¶

Retrieve a CheckpointCollection from an epoch number.

Type: CheckpointCollection

property iloc¶

Retrieve a Checkpoint from its insertion index.

Type: Checkpoint, (Checkpoint, )

add(checkpoint)[source]¶

Add a Checkpoint to the collection.

Parameters

checkpoint (Checkpoint) – A Checkpoint to add to the collection.

Raises

KeyError – If the Checkpoint name was already registered as a reference.
ValueError – If the Checkpoint epoch is None.

get(reference, default=None)[source]¶

Retrieve a Checkpoint by its unique reference (its name) or a default if reference is not found.

Parameters

reference (hashable) – A Checkpoint unique reference.
default (Any) – The default to return if reference is not a valid Checkpoint reference.

Returns

The corresponding Checkpoint or default.

Return type

Checkpoint, Any

keys()[source]¶

Iterate through the collection’s references as in a dict, in insertion order.

Yields: hashable – A reference of a Checkpoint.

values()[source]¶

Iterate through the collection’s Checkpoint as in a dict, in insertion order.

Yields: Checkpoint – A Checkpoint.

items()[source]¶

Iterate through the collection as in a dict, in insertion order.

Yields: (hashable, Checkpoint) – Reference and Checkpoint.

class plums.model.components.components.Producer(name, version_format, version_string, configuration)[source]¶

Bases: object

Define a Python representation of a PMF model producer with its configuration.

Parameters

name (str) – The name of the package that produced the model.
version_format (str) – The version format of the package that produced the model.
version_string (str) – The version representation string of the package that produced the model.
configuration (Pathlike) – A path to the producer configuration file used to produce the model.

See also

The Version class for more information on the Producer version handling.

name¶

The name of the package that produced the model.

Type: str

version¶

The version of the package that produced the model.

Type: Version

configuration¶

A path to the producer configuration file used to produce the model.

Type: Pathlike

strict_equals(other)[source]¶

Return if two Producer have the same name and version and configuration file.

Parameters: other (Producer) – Another Producer to compare with.
Returns: True if the two Producer are identical in name, Version and configuration.
Return type: bool

class plums.model.components.utils.Checkpoint(name, path=None, epoch=None, hash=None)[source]¶

Bases: object

Define a checkpoint Python representation.

A Checkpoint might be defined by the following parameters:

A name, a path and a hash.
A name and a hash.
A name and a path.

Note that although the epoch is never needed to strictly define a Checkpoint, it is compulsory to inject it into a CheckpointCollection.

Parameters

name (hashable) – The Checkpoint unique identifier.
path (Pathlike) – Optional. default to None. The path to the Checkpoint data file.
epoch (int) – Optional. default to None. The Checkpoint epoch, if known.
hash (str) – Optional. default to None. The Checkpoint data file checksum.

name¶

The Checkpoint unique identifier.

Type: hashable

path¶

The path to the Checkpoint data file.

Type: Pathlike

epoch¶

The Checkpoint epoch, if known.

Type: int

hash¶

The Checkpoint data file checksum.

Type: str

Exceptions¶

exception plums.model.exception.PlumsError[source]¶

Bases: Exception

Base exception for all Plums specific errors.

exception plums.model.exception.PlumsValidationError(autos, errors=None)[source]¶

Bases: schema.SchemaError, plums.model.exception.PlumsError

Base exception for all Plums schema validation specific errors.

It inherits from schema.SchemaError.

exception plums.model.exception.PlumsModelError[source]¶

Bases: plums.model.exception.PlumsError

Base exception for all Plums Model specific errors.

exception plums.model.exception.PlumsModelTreeValidationError(autos, errors=None)[source]¶

Bases: plums.model.exception.PlumsValidationError, plums.model.exception.PlumsModelError

Base exception for all Plums Model validation errors specific to filesystem trees validations.

exception plums.model.exception.PlumsModelMetadataValidationError(autos, errors=None)[source]¶

Bases: plums.model.exception.PlumsValidationError, plums.model.exception.PlumsModelError

Base exception for all Plums Model validation errors specific to filesystem trees validations.