Model developer API¶

The Plums Model Format main API is in the Model class which allows loading, saving and dynamic creation through a high-level python API.

The following classes and functions exposes the inner workings of the main Model API. It is deemed the developer API because it should not be used directly by end-users or model producers/consumers.

Producer version handling¶

plums.model.components.version.version(format, value)[source]¶

Construct a valid Version instance with the correct format.

Parameters

format (str) – A snake case registered format identifier.
value (str) – The Version string representation.

Returns

A Version instance with the correct format.

Return type

Version

Raises

ValueError – If format is not a registered version format.

plums.model.components.version.register(cls, hook=None)[source]¶

Register a Version class to the version function with an optional initialisation hook.

Parameters

cls (Version) – The Version class to register.
hook (Callable) – Optional. Default to the identity. An optional initialisation hook.

class plums.model.components.version.MetaVersion(name, bases, namespace)[source]¶

Bases: abc.ABCMeta

Base meta-class for all version.

It inherits abc.ABCMeta and adds a class property format which give the name of the version format from the class name.

property format¶

The version corresponding format, it is computed from the class name as a snake case.

Type: str

class plums.model.components.version.Version[source]¶

Bases: object

Abstract base class for all Producer version.

Subclasses are expected to override the __init__(), __str__(), __eq__() and __lt__() magic method to make the version format ‘registerable’ to the version factory function.

Parameters: version (str) – A valid version representation string with regard to the version format.

format¶

The version corresponding format, it is computed from the class name as a snake case.

Type: str

property version¶

The version string representation, it is computed with the __str__() method.

Type: str

format = 'version'¶

class plums.model.components.version.PyPA(version)[source]¶

Bases: plums.model.components.version.Version

A Producer version following the standard PyPA versioning scheme.

It follows the rules defined in PEP 440 which are grossly similar to Semantic Versioning with a few notable differences and generalisation.

Parameters: version (str) – A valid PEP 440 version representation string.

format¶

The version corresponding format, it is computed from the class name as a snake case.

Type: str

format = 'py_pa'¶

plums.model.components.version.version(format, value)[source]¶

Construct a valid Version instance with the correct format.

Parameters

format (str) – A snake case registered format identifier.
value (str) – The Version string representation.

Returns

A Version instance with the correct format.

Return type

Version

Raises

ValueError – If format is not a registered version format.

plums.model.components.version.register(cls, hook=None)[source]¶

Register a Version class to the version function with an optional initialisation hook.

Parameters

cls (Version) – The Version class to register.
hook (Callable) – Optional. Default to the identity. An optional initialisation hook.

class plums.model.components.version.MetaVersion(name, bases, namespace)[source]¶

Bases: abc.ABCMeta

Base meta-class for all version.

It inherits abc.ABCMeta and adds a class property format which give the name of the version format from the class name.

property format¶

The version corresponding format, it is computed from the class name as a snake case.

Type: str

class plums.model.components.version.Version[source]¶

Bases: object

Abstract base class for all Producer version.

Subclasses are expected to override the __init__(), __str__(), __eq__() and __lt__() magic method to make the version format ‘registerable’ to the version factory function.

Parameters: version (str) – A valid version representation string with regard to the version format.

format¶

The version corresponding format, it is computed from the class name as a snake case.

Type: str

property version¶

The version string representation, it is computed with the __str__() method.

Type: str

format = 'version'¶

class plums.model.components.version.PyPA(version)[source]¶

Bases: plums.model.components.version.Version

A Producer version following the standard PyPA versioning scheme.

It follows the rules defined in PEP 440 which are grossly similar to Semantic Versioning with a few notable differences and generalisation.

Parameters: version (str) – A valid PEP 440 version representation string.

format¶

The version corresponding format, it is computed from the class name as a snake case.

Type: str

format = 'py_pa'¶

Utils¶

plums.model.components.utils.is_duplicate(file_1, file_2, hash_1=None, hash_2=None, checksum=<function md5_checksum>)[source]¶

Compare two files and return wether their are duplicates.

For efficiency, a first decision is taken based on the two files size.

If it is not enough, their content checksum are used.

Parameters

file_1 (PathLike) – The first file to compare.
file_2 (PathLike) – The second file to compare.
hash_1 (str) – Optional. Default to None. If provided, it is used in place of the first file checksum to compare content.
hash_2 (str) – Optional. Default to None. If provided, it is used in place of the second file checksum to compare content.
checksum (Callable) – A function which computes a checksum from a Path.

Returns

True if both files have the same content.

Return type

bool

plums.model.components.utils.copy(source, destination, lazy=True, src_hash=None, dst_hash=None, checksum=<function md5_checksum>)[source]¶

Copy file from source to destination.

If lazy is set to True and destination exists, the file will be copied if and only if destination is a different file (content-wise) than source.

Parameters

source (PathLike) – The first file to compare.
destination (PathLike) – The second file to compare.
lazy (bool) – Optional. Default to True. If set to True, actual copy performed if destination exists and has a different content than source.
src_hash (str) – Optional. Default to None. If provided, it is used in place of the first file checksum to compare content.
dst_hash (str) – Optional. Default to None. If provided, it is used in place of the second file checksum to compare content.
checksum (Callable) – A function which computes a checksum from a Path.

Raises

OSError – If something went wrong during copy.

plums.model.components.utils.rmtree(path, ignore_errors=False, rm_all=False, black_list=())[source]¶

Partial removal of a filesystem tree from a black list entry point and PMF-trees specific directory entry rules.

For example, to remove a PMF model located at path with a configuration file name configuration, one would use:

rmtree(path, black_list=('metadata', configuration))

Parameters

path (PathLike) – A Path to a PMF Model to erase.
ignore_errors (bool) – Optional. Default to False. If True, any error arising when trying to delete or list elements will be silently swallowed.
rm_all (bool) – Optional. Default to False. If True, the entire filesystem tree descending from path will be deleted, independent on whether it looks PMF related.
black_list (tuple) – A tuple of filename (with or without extensions) to be deleted from path.

Warning

rm_all is to be used with caution, especially coupled with ignore_errors as it may lead to irreversible and indifferenciated data loss.

class plums.model.components.utils.TrainingStatus(status='pending')[source]¶

Bases: object

Helper component class to handle Training status logic.

Parameters: status (str) – Optional. Default to ‘pending’. An optional initial status.

property status¶

The current Training status.

The setter checks that the given value is authorized given the current value.

Raises: ValueError – If the value given to the setter is deemed unauthorized.
Type: str

class plums.model.components.utils.Checkpoint(name, path=None, epoch=None, hash=None)[source]¶

Bases: object

Define a checkpoint Python representation.

A Checkpoint might be defined by the following parameters:

A name, a path and a hash.
A name and a hash.
A name and a path.

Note that although the epoch is never needed to strictly define a Checkpoint, it is compulsory to inject it into a CheckpointCollection.

Parameters

name (hashable) – The Checkpoint unique identifier.
path (Pathlike) – Optional. default to None. The path to the Checkpoint data file.
epoch (int) – Optional. default to None. The Checkpoint epoch, if known.
hash (str) – Optional. default to None. The Checkpoint data file checksum.

name¶

The Checkpoint unique identifier.

Type: hashable

path¶

The path to the Checkpoint data file.

Type: Pathlike

epoch¶

The Checkpoint epoch, if known.

Type: int

hash¶

The Checkpoint data file checksum.

Type: str

class plums.model.components.utils.Mock[source]¶

Bases: object

Mock class which return itself for all attribute access.

__call__(*args, **kwargs)[source]¶: Return an instance of self for all args and kwargs.

plums.model.components.utils.is_duplicate(file_1, file_2, hash_1=None, hash_2=None, checksum=<function md5_checksum>)[source]¶

Compare two files and return wether their are duplicates.

For efficiency, a first decision is taken based on the two files size.

If it is not enough, their content checksum are used.

Parameters

file_1 (PathLike) – The first file to compare.
file_2 (PathLike) – The second file to compare.
hash_1 (str) – Optional. Default to None. If provided, it is used in place of the first file checksum to compare content.
hash_2 (str) – Optional. Default to None. If provided, it is used in place of the second file checksum to compare content.
checksum (Callable) – A function which computes a checksum from a Path.

Returns

True if both files have the same content.

Return type

bool

plums.model.components.utils.copy(source, destination, lazy=True, src_hash=None, dst_hash=None, checksum=<function md5_checksum>)[source]¶

Copy file from source to destination.

If lazy is set to True and destination exists, the file will be copied if and only if destination is a different file (content-wise) than source.

Parameters

source (PathLike) – The first file to compare.
destination (PathLike) – The second file to compare.
lazy (bool) – Optional. Default to True. If set to True, actual copy performed if destination exists and has a different content than source.
src_hash (str) – Optional. Default to None. If provided, it is used in place of the first file checksum to compare content.
dst_hash (str) – Optional. Default to None. If provided, it is used in place of the second file checksum to compare content.
checksum (Callable) – A function which computes a checksum from a Path.

Raises

OSError – If something went wrong during copy.

plums.model.components.utils.rmtree(path, ignore_errors=False, rm_all=False, black_list=())[source]¶

Partial removal of a filesystem tree from a black list entry point and PMF-trees specific directory entry rules.

For example, to remove a PMF model located at path with a configuration file name configuration, one would use:

rmtree(path, black_list=('metadata', configuration))

Parameters

path (PathLike) – A Path to a PMF Model to erase.
ignore_errors (bool) – Optional. Default to False. If True, any error arising when trying to delete or list elements will be silently swallowed.
rm_all (bool) – Optional. Default to False. If True, the entire filesystem tree descending from path will be deleted, independent on whether it looks PMF related.
black_list (tuple) – A tuple of filename (with or without extensions) to be deleted from path.

Warning

rm_all is to be used with caution, especially coupled with ignore_errors as it may lead to irreversible and indifferenciated data loss.

class plums.model.components.utils.TrainingStatus(status='pending')[source]¶

Bases: object

Helper component class to handle Training status logic.

Parameters: status (str) – Optional. Default to ‘pending’. An optional initial status.

property status¶

The current Training status.

The setter checks that the given value is authorized given the current value.

Raises: ValueError – If the value given to the setter is deemed unauthorized.
Type: str

class plums.model.components.utils.Checkpoint(name, path=None, epoch=None, hash=None)[source]¶

Bases: object

Define a checkpoint Python representation.

A Checkpoint might be defined by the following parameters:

A name, a path and a hash.
A name and a hash.
A name and a path.

Note that although the epoch is never needed to strictly define a Checkpoint, it is compulsory to inject it into a CheckpointCollection.

Parameters

name (hashable) – The Checkpoint unique identifier.
path (Pathlike) – Optional. default to None. The path to the Checkpoint data file.
epoch (int) – Optional. default to None. The Checkpoint epoch, if known.
hash (str) – Optional. default to None. The Checkpoint data file checksum.

name¶

The Checkpoint unique identifier.

Type: hashable

path¶

The path to the Checkpoint data file.

Type: Pathlike

epoch¶

The Checkpoint epoch, if known.

Type: int

hash¶

The Checkpoint data file checksum.

Type: str

class plums.model.components.utils.Mock[source]¶

Bases: object

Mock class which return itself for all attribute access.

__call__(*args, **kwargs)[source]¶: Return an instance of self for all args and kwargs.