Model validation API¶

Most Plums Model Format validation is performed with the validate() function, which take in a Path and returns the validated model metadata or raises a PlumsValidationError.

plums.model.validation.validate(path, strict=False, verbose=False, **kwargs)[source]¶

Validate a PMF model location and return offending files or directories.

Parameters

path (Pathlike) – A Path to a model saved in a PMF format.
strict (bool) – Whether to perform strict validation (see Model).
verbose (bool) – Optional. Default to False. Toggle verbose offence reporting.
**kwargs (Any) – Additional arguments are passed to the Model validator.

Returns

A parsed PMF model metadata structure if the model is valid.

Return type

dict

However, if one needs to validate part of a PMF model or needs fine grained access to validation internals for further work, Plums model also exposes a developer validation API.

Developer API¶

The developer API exposes the low-level internal API used by the validate() function. Most of the validation heavy-lifting is made by the excellent Schema library.

Each documented class is a SchemaComponent and defines part of the total validation schema.

PMF data tree validation API¶

You may find here the list of TreeComponent compositing the Tree structure schema.

class plums.model.validation.structure.TreeComponent(strict=False, verbose=False)[source]¶

Bases: plums.model.validation.schema_core.SchemaComponent

A ‘validatable’ component in a filesystem tree schema-like validation.

To alleviate complex nested tree creation and validation, one possible solution is to cut it up into semantically consistent chunks, each defining its innermost tree and then validate it as a python dict schema.

With this, each component validation might be customized (if one needs to check more than the innermost schema data conformity, e.g. for data cross-dependencies or to simplify optional or default value definition) and it is easier to accumulate and report all offending data elements in one shot.

The class TreeComponent defines a simple building block which allows both static and dynamic tree composition by defining its enclosed dict schema respectively as a class attribute or a instance attribute.

See also

The Schema library for more information on the schema validation backend.
The make_dict_structure_from_tree() function to translate a filesystem tree into a nested dictionary structure to validate against a schema.

Parameters

strict (bool) – Optional. Default to True. Toggle strict validation. It is up to subclasses to define what is considered strict validation, however, it usually leads to significantly longer validation time as it might involve costly IO operation.
verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

validate(data)[source]¶

Validate a given data against the enclosed schema.

Parameters

data (Any) – The data to validate.

Returns

The validated data.

Return type

Any

Raises

PlumsModelTreeValidationError – If any offence is detected in the filesystem tree data.
PlumsModelMetadataValidationError – If any offence is detected in the model metadata.

class plums.model.validation.structure.DefaultTree(schema_, default=None, strict=False, verbose=False)[source]¶

Bases: plums.model.validation.schema_core.Default, plums.model.validation.structure.TreeComponent

A ‘validatable’ component in a schema-like tree validation which accepts a default null value.

Parameters

schema (Validatable) – A schema to enclose.
default (Any) – A default null value to accept as a valid data.
strict (bool) – Optional. Default to True. Toggle strict validation. It is up to subclasses to define what is considered strict validation, however, it usually leads to significantly longer validation time as it might involve costly IO operation.
verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

class plums.model.validation.structure.Model(strict=False, verbose=False, checkpoints=True)[source]¶

Bases: plums.model.validation.structure.TreeComponent

A ‘validatable’ component which only accepts valid PMF model tree structure.

Parameters

strict (bool) – Optional. Default to True. Toggle strict validation. When performing strict validation, the configuration file and all registered checkpoints checksum will be matched against the checksum of the corresponding file on the filesystem.
checkpoints (bool) – Whether to check for registered checkpoints existence and validity if strict is True.
verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

validate(data)[source]¶

Validate a given tree against the PMF tree schema.

Parameters: data (Any) – The tree to validate.
Returns: The validated tree.
Return type: Any
Raises: PlumsModelTreeValidationError – If any offence is detected in the data.

class plums.model.validation.structure.Metadata(verbose=False)[source]¶

Bases: plums.model.validation.metadata.Metadata

A ‘validatable’ component which only accepts filename pointing to valid PMF metadata.

Parameters: verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

validate(data)[source]¶

Validate a PMF metadata from its location.

Parameters: data (Any) – The Path to the metadata file to validate.
Returns: The validated metadata.
Return type: Any
Raises: PlumsModelMetadataValidationError – If any offence is detected in the metadata.

class plums.model.validation.structure.InitialisationPath(strict=False, verbose=False)[source]¶

Bases: plums.model.validation.structure.TreeComponent

A ‘validatable’ component which only accepts valid PMF model data ‘initialisation from file’ tree structure.

Parameters

strict (bool) – Optional. Default to True. Toggle strict validation. When performing strict validation, the configuration file and all registered checkpoints checksum will be matched against the checksum of the corresponding file on the filesystem.
verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

class plums.model.validation.structure.InitialisationPMF(strict=False, verbose=False)[source]¶

Bases: plums.model.validation.structure.Model

A ‘validatable’ component which only accepts valid PMF model data ‘initialisation from PMF model’.

Parameters

strict (bool) – Optional. Default to True. Toggle strict validation. When performing strict validation, the configuration file and all registered checkpoints checksum will be matched against the checksum of the corresponding file on the filesystem.
verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

class plums.model.validation.structure.Initialisation(strict=False, verbose=False)[source]¶

Bases: plums.model.validation.structure.DefaultTree

A ‘validatable’ component which only accepts valid PMF model data initialisation tree structure.

Parameters

strict (bool) – Optional. Default to True. Toggle strict validation. When performing strict validation, the configuration file and all registered checkpoints checksum will be matched against the checksum of the corresponding file on the filesystem.
verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

validate(data)[source]¶

Validate a given tree against the default value or one of the pmf or file enclosed tree schema.

Parameters: data (Any) – The tree to validate.
Returns: The validated tree.
Return type: Any
Raises: PlumsModelTreeValidationError – If any offence is detected in the data.

class plums.model.validation.structure.Checkpoint(strict=False, verbose=False)[source]¶

Bases: plums.model.validation.structure.DefaultTree

A ‘validatable’ component which only accepts valid PMF model data checkpoints tree structure.

Parameters

strict (bool) – Optional. Default to True. Toggle strict validation. When performing strict validation, the configuration file and all registered checkpoints checksum will be matched against the checksum of the corresponding file on the filesystem.
verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

class plums.model.validation.structure.Data(strict=False, verbose=False, checkpoints=True)[source]¶

Bases: plums.model.validation.structure.TreeComponent

A ‘validatable’ component which only accepts valid PMF model data tree structure.

Parameters

strict (bool) – Optional. Default to True. Toggle strict validation. When performing strict validation, the configuration file and all registered checkpoints checksum will be matched against the checksum of the corresponding file on the filesystem.
checkpoints (bool) – Whether to check for checkpoints existence.
verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

validate(data)[source]¶

Validate a given tree against the PMF data tree schema.

Parameters: data (Any) – The tree to validate.
Returns: The validated tree.
Return type: Any
Raises: PlumsModelTreeValidationError – If any offence is detected in the data.

Metadata validation API¶

You may find here the list of MetadataComponent compositing the Metadata.yaml schema.

class plums.model.validation.metadata.MetadataComponent(verbose=False)[source]¶

Bases: plums.model.validation.schema_core.SchemaComponent

A ‘validatable’ component in a PMF model metadata schema validation.

To alleviate complex nested schema creation and validation, one possible solution is to cut it up into semantically consistent chunks, each defining its innermost schema.

With this, each component validation might be customized (if one needs to check more than the innermost schema data conformity, e.g. for data cross-dependencies or to simplify optional or default value definition) and it is easier to accumulate and report all offending data elements in one shot.

The class MetadataComponent defines a simple building block which allows both static and dynamic schema composition by defining its enclosed schema respectively as a class attribute or a instance attribute.

See also

The Schema library for more information on the schema validation backend.

Parameters: verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

validate(data)[source]¶

Validate a given data against the enclosed schema.

Parameters: data (Any) – The data to validate.
Returns: The validated data.
Return type: Any
Raises: PlumsModelMetadataValidationError – If any offence is detected in the metadata.

class plums.model.validation.metadata.DefaultMetadata(schema_, default=None, verbose=False)[source]¶

Bases: plums.model.validation.schema_core.Default, plums.model.validation.metadata.MetadataComponent

A ‘validatable’ component in a PMF metadata schema validation which accepts a default null value.

Parameters

schema (Validatable) – A schema to enclose.
default (Any) – A default null value to accept as a valid data.
verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

class plums.model.validation.metadata.Metadata(verbose=False)[source]¶

Bases: plums.model.validation.metadata.MetadataComponent

A ‘validatable’ component which only accepts valid PMF metadata.

Parameters

verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

Schema

format (Format) – The format metadata section.
model (Model) – The model metadata section.

class plums.model.validation.metadata.Configuration(verbose=False)[source]¶

Bases: plums.model.validation.metadata.MetadataComponent

A ‘validatable’ component which only accepts valid PMF model configuration section.

Parameters

verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

Schema

path (Path) – The configuration file path.
hash (MD5Checksum) – The configuration file checksum.

class plums.model.validation.metadata.InitialisationPMF(verbose=False)[source]¶

Bases: plums.model.validation.metadata.MetadataComponent

A ‘validatable’ component which only accepts valid PMF model ‘initialisation from a PMF model’ section.

Parameters

verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

Schema

name (str, None) – The initialisation Model name.
id (str) – The initialisation Model id.
checkpoint (str, int) – The initialisation Model checkpoint reference used to initialise.
path (Path) – The initialisation Model location.

class plums.model.validation.metadata.InitialisationPath(verbose=False)[source]¶

Bases: plums.model.validation.metadata.MetadataComponent

A ‘validatable’ component which only accepts valid PMF model ‘initialisation from a file’ section.

Parameters

verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

Schema

name (str, None) – The initialisation name.
path (Path) – The initialisation file path.
hash (MD5Checksum) – The initialisation file checksum.

class plums.model.validation.metadata.Initialisation(verbose=False)[source]¶

Bases: plums.model.validation.metadata.DefaultMetadata

A ‘validatable’ component which only accepts valid PMF model initialisation section.

Parameters: verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

validate(data)[source]¶

Validate a given data against the default value or one of the pmf or file enclosed schema.

Parameters: data (Any) – The data to validate.
Raises: PlumsModelMetadataValidationError – If any offence is detected in the data.

class plums.model.validation.metadata.Model(verbose=False)[source]¶

Bases: plums.model.validation.metadata.MetadataComponent

A ‘validatable’ component which only accepts valid PMF model section.

Parameters

verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

Schema

name (str, None) – The Model name.
id (str) – The Model name.
initialisation (Initialisation) – The Model initialisation section.
training (Training) – The Model training section.
configuration (Configuration) – The Model configuration section.

class plums.model.validation.metadata.Checkpoint(verbose=False)[source]¶

Bases: plums.model.validation.metadata.DefaultMetadata

A ‘validatable’ component which only accepts valid PMF model training checkpoints section.

Parameters: verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

Validates a mapping between a str or int Checkpoint reference key and the following checkpoint schema:

Schema

epoch (int) – The Checkpoint epoch.
path (Path) – The Checkpoint file path.
hash (MD5Checksum) – The Checkpoint file checksum.

class plums.model.validation.metadata.Training(verbose=False)[source]¶

Bases: plums.model.validation.metadata.MetadataComponent

A ‘validatable’ component which only accepts valid PMF model training section.

Parameters

verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

Schema

status (str) – The Training status (either ‘pending’, ‘running’, ‘finished’ or ‘failed’).
start_time (int, float, None) – The Training starting epoch timestamp.
start_epoch (int, float, None) – The Training starting epoch number.
latest_time (int, float, None) – The Training latest epoch timestamp.
latest_epoch (int, float, None) – The Training latest epoch number.
end_time (int, float, None) – The Training ending epoch timestamp.
end_epoch (int, float, None) – The Training ending epoch number.
latest (str, int) – A reference to the latest registered Checkpoint in the Checkpoint section.
checkpoint (Configuration) – The Training checkpoints section.

validate(data)[source]¶

Validate a given data against the enclosed schema and checks for various cross-dependencies.

Parameters: data (Any) – The data to validate.
Raises: PlumsModelMetadataValidationError – If any offence is detected in the data.

class plums.model.validation.metadata.ProducerVersion(verbose=False)[source]¶

Bases: plums.model.validation.metadata.MetadataComponent

A ‘validatable’ component which only accepts valid PMF format producer version section.

Parameters

verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

Schema

format (str) – The Version format.
value (str) – The Version string representation.

class plums.model.validation.metadata.Producer(verbose=False)[source]¶

Bases: plums.model.validation.metadata.MetadataComponent

A ‘validatable’ component which only accepts valid PMF format producer section.

Parameters

verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

Schema

name (str) – The Producer name.
version (ProducerVersion) – The Producer version section.

class plums.model.validation.metadata.Format(verbose=False)[source]¶

Bases: plums.model.validation.metadata.MetadataComponent

A ‘validatable’ component which only accepts valid PMF format section.

Parameters

verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

Schema

version (str) – The PMF format current version.
version (Producer) – The format producer section.

Core validation API¶

class plums.model.validation.schema_core.SchemaComponent(verbose=False)[source]¶

Bases: object

A ‘validatable’ component in a schema validation.

To alleviate complex nested schema creation and validation, one possible solution is to cut it up into semantically consistent chunks, each defining its innermost schema.

With this, each component validation might be customized (if one needs to check more than the innermost schema data conformity, e.g. for data cross-dependencies or to simplify optional or default value definition) and it is easier to accumulate and report all offending data elements in one shot.

The class SchemaComponent defines a simple building block which allows both static and dynamic schema composition by defining its enclosed schema respectively as a class attribute or a instance attribute.

See also

The Schema library for more information on the schema validation backend.

Parameters: verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

validate(data)[source]¶

Validate a given data against the enclosed schema.

Parameters: data (Any) – The data to validate.
Returns: The validated data.
Return type: Any
Raises: PlumsValidationError – If any offence is detected in the data.

is_valid(data)[source]¶

Return whether a given data is valid with regard to validate.

Parameters: data (Any) – The data to validate.
Returns: True if data is valid.
Return type: bool

class plums.model.validation.schema_core.Default(schema_, default=None, verbose=False)[source]¶

Bases: plums.model.validation.schema_core.SchemaComponent

A ‘validatable’ component in a schema validation which accepts a default null value.

Parameters

schema (Validatable) – A schema to enclose.
default (Any) – A default null value to accept as a valid data.
verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

validate(data)[source]¶

Validate a given data against the enclosed schema or the default value.

Parameters: data (Any) – The data to validate.
Returns: The validated data.
Return type: Any
Raises: PlumsValidationError – If any offence is detected in the data.

class plums.model.validation.schema_core.Path(verbose=False)[source]¶

Bases: plums.model.validation.schema_core.SchemaComponent

A ‘validatable’ component in a schema validation which only accepts syntactically valid path.

Warning

Validation is only performed on syntactical considerations, not on whether the given path points to an existing location.

Parameters: verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

validate(data)[source]¶

Validate a given path.

Parameters: data (Any) – The path to validate.
Raises: PlumsValidationError – If any offence is detected in the data.

class plums.model.validation.schema_core.MD5Checksum(verbose=False)[source]¶

Bases: plums.model.validation.schema_core.SchemaComponent

A ‘validatable’ component in a schema validation which only accepts syntactically valid MD5 hash.

Parameters: verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

class plums.model.validation.metadata.MetadataComponent(verbose=False)[source]¶

Bases: plums.model.validation.schema_core.SchemaComponent

A ‘validatable’ component in a PMF model metadata schema validation.

To alleviate complex nested schema creation and validation, one possible solution is to cut it up into semantically consistent chunks, each defining its innermost schema.

With this, each component validation might be customized (if one needs to check more than the innermost schema data conformity, e.g. for data cross-dependencies or to simplify optional or default value definition) and it is easier to accumulate and report all offending data elements in one shot.

The class MetadataComponent defines a simple building block which allows both static and dynamic schema composition by defining its enclosed schema respectively as a class attribute or a instance attribute.

See also

The Schema library for more information on the schema validation backend.

Parameters: verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

validate(data)[source]¶

Validate a given data against the enclosed schema.

Parameters: data (Any) – The data to validate.
Returns: The validated data.
Return type: Any
Raises: PlumsModelMetadataValidationError – If any offence is detected in the metadata.

class plums.model.validation.metadata.DefaultMetadata(schema_, default=None, verbose=False)[source]¶

Bases: plums.model.validation.schema_core.Default, plums.model.validation.metadata.MetadataComponent

A ‘validatable’ component in a PMF metadata schema validation which accepts a default null value.

Parameters

schema (Validatable) – A schema to enclose.
default (Any) – A default null value to accept as a valid data.
verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

class plums.model.validation.metadata.Metadata(verbose=False)[source]¶

Bases: plums.model.validation.metadata.MetadataComponent

A ‘validatable’ component which only accepts valid PMF metadata.

Parameters

verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

Schema

format (Format) – The format metadata section.
model (Model) – The model metadata section.

class plums.model.validation.metadata.Configuration(verbose=False)[source]¶

Bases: plums.model.validation.metadata.MetadataComponent

A ‘validatable’ component which only accepts valid PMF model configuration section.

Parameters

verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

Schema

path (Path) – The configuration file path.
hash (MD5Checksum) – The configuration file checksum.

class plums.model.validation.metadata.InitialisationPMF(verbose=False)[source]¶

Bases: plums.model.validation.metadata.MetadataComponent

A ‘validatable’ component which only accepts valid PMF model ‘initialisation from a PMF model’ section.

Parameters

verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

Schema

name (str, None) – The initialisation Model name.
id (str) – The initialisation Model id.
checkpoint (str, int) – The initialisation Model checkpoint reference used to initialise.
path (Path) – The initialisation Model location.

class plums.model.validation.metadata.InitialisationPath(verbose=False)[source]¶

Bases: plums.model.validation.metadata.MetadataComponent

A ‘validatable’ component which only accepts valid PMF model ‘initialisation from a file’ section.

Parameters

verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

Schema

name (str, None) – The initialisation name.
path (Path) – The initialisation file path.
hash (MD5Checksum) – The initialisation file checksum.

class plums.model.validation.metadata.Initialisation(verbose=False)[source]¶

Bases: plums.model.validation.metadata.DefaultMetadata

A ‘validatable’ component which only accepts valid PMF model initialisation section.

Parameters: verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

validate(data)[source]¶

Validate a given data against the default value or one of the pmf or file enclosed schema.

Parameters: data (Any) – The data to validate.
Raises: PlumsModelMetadataValidationError – If any offence is detected in the data.

class plums.model.validation.metadata.Model(verbose=False)[source]¶

Bases: plums.model.validation.metadata.MetadataComponent

A ‘validatable’ component which only accepts valid PMF model section.

Parameters

verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

Schema

name (str, None) – The Model name.
id (str) – The Model name.
initialisation (Initialisation) – The Model initialisation section.
training (Training) – The Model training section.
configuration (Configuration) – The Model configuration section.

class plums.model.validation.metadata.Checkpoint(verbose=False)[source]¶

Bases: plums.model.validation.metadata.DefaultMetadata

A ‘validatable’ component which only accepts valid PMF model training checkpoints section.

Parameters: verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

Validates a mapping between a str or int Checkpoint reference key and the following checkpoint schema:

Schema

epoch (int) – The Checkpoint epoch.
path (Path) – The Checkpoint file path.
hash (MD5Checksum) – The Checkpoint file checksum.

class plums.model.validation.metadata.Training(verbose=False)[source]¶

Bases: plums.model.validation.metadata.MetadataComponent

A ‘validatable’ component which only accepts valid PMF model training section.

Parameters

verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

Schema

status (str) – The Training status (either ‘pending’, ‘running’, ‘finished’ or ‘failed’).
start_time (int, float, None) – The Training starting epoch timestamp.
start_epoch (int, float, None) – The Training starting epoch number.
latest_time (int, float, None) – The Training latest epoch timestamp.
latest_epoch (int, float, None) – The Training latest epoch number.
end_time (int, float, None) – The Training ending epoch timestamp.
end_epoch (int, float, None) – The Training ending epoch number.
latest (str, int) – A reference to the latest registered Checkpoint in the Checkpoint section.
checkpoint (Configuration) – The Training checkpoints section.

validate(data)[source]¶

Validate a given data against the enclosed schema and checks for various cross-dependencies.

Parameters: data (Any) – The data to validate.
Raises: PlumsModelMetadataValidationError – If any offence is detected in the data.

class plums.model.validation.metadata.ProducerVersion(verbose=False)[source]¶

Bases: plums.model.validation.metadata.MetadataComponent

A ‘validatable’ component which only accepts valid PMF format producer version section.

Parameters

verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

Schema

format (str) – The Version format.
value (str) – The Version string representation.

class plums.model.validation.metadata.Producer(verbose=False)[source]¶

Bases: plums.model.validation.metadata.MetadataComponent

A ‘validatable’ component which only accepts valid PMF format producer section.

Parameters

verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

Schema

name (str) – The Producer name.
version (ProducerVersion) – The Producer version section.

class plums.model.validation.metadata.Format(verbose=False)[source]¶

Bases: plums.model.validation.metadata.MetadataComponent

A ‘validatable’ component which only accepts valid PMF format section.

Parameters

verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

Schema

version (str) – The PMF format current version.
version (Producer) – The format producer section.

class plums.model.validation.structure.TreeComponent(strict=False, verbose=False)[source]¶

Bases: plums.model.validation.schema_core.SchemaComponent

A ‘validatable’ component in a filesystem tree schema-like validation.

To alleviate complex nested tree creation and validation, one possible solution is to cut it up into semantically consistent chunks, each defining its innermost tree and then validate it as a python dict schema.

With this, each component validation might be customized (if one needs to check more than the innermost schema data conformity, e.g. for data cross-dependencies or to simplify optional or default value definition) and it is easier to accumulate and report all offending data elements in one shot.

The class TreeComponent defines a simple building block which allows both static and dynamic tree composition by defining its enclosed dict schema respectively as a class attribute or a instance attribute.

See also

The Schema library for more information on the schema validation backend.
The make_dict_structure_from_tree() function to translate a filesystem tree into a nested dictionary structure to validate against a schema.

Parameters

strict (bool) – Optional. Default to True. Toggle strict validation. It is up to subclasses to define what is considered strict validation, however, it usually leads to significantly longer validation time as it might involve costly IO operation.
verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

validate(data)[source]¶

Validate a given data against the enclosed schema.

Parameters

data (Any) – The data to validate.

Returns

The validated data.

Return type

Any

Raises

PlumsModelTreeValidationError – If any offence is detected in the filesystem tree data.
PlumsModelMetadataValidationError – If any offence is detected in the model metadata.

class plums.model.validation.structure.DefaultTree(schema_, default=None, strict=False, verbose=False)[source]¶

Bases: plums.model.validation.schema_core.Default, plums.model.validation.structure.TreeComponent

A ‘validatable’ component in a schema-like tree validation which accepts a default null value.

Parameters

schema (Validatable) – A schema to enclose.
default (Any) – A default null value to accept as a valid data.
strict (bool) – Optional. Default to True. Toggle strict validation. It is up to subclasses to define what is considered strict validation, however, it usually leads to significantly longer validation time as it might involve costly IO operation.
verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

class plums.model.validation.structure.Model(strict=False, verbose=False, checkpoints=True)[source]¶

Bases: plums.model.validation.structure.TreeComponent

A ‘validatable’ component which only accepts valid PMF model tree structure.

Parameters

strict (bool) – Optional. Default to True. Toggle strict validation. When performing strict validation, the configuration file and all registered checkpoints checksum will be matched against the checksum of the corresponding file on the filesystem.
checkpoints (bool) – Whether to check for registered checkpoints existence and validity if strict is True.
verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

validate(data)[source]¶

Validate a given tree against the PMF tree schema.

Parameters: data (Any) – The tree to validate.
Returns: The validated tree.
Return type: Any
Raises: PlumsModelTreeValidationError – If any offence is detected in the data.

class plums.model.validation.structure.Metadata(verbose=False)[source]¶

Bases: plums.model.validation.metadata.Metadata

A ‘validatable’ component which only accepts filename pointing to valid PMF metadata.

Parameters: verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

validate(data)[source]¶

Validate a PMF metadata from its location.

Parameters: data (Any) – The Path to the metadata file to validate.
Returns: The validated metadata.
Return type: Any
Raises: PlumsModelMetadataValidationError – If any offence is detected in the metadata.

class plums.model.validation.structure.InitialisationPath(strict=False, verbose=False)[source]¶

Bases: plums.model.validation.structure.TreeComponent

A ‘validatable’ component which only accepts valid PMF model data ‘initialisation from file’ tree structure.

Parameters

strict (bool) – Optional. Default to True. Toggle strict validation. When performing strict validation, the configuration file and all registered checkpoints checksum will be matched against the checksum of the corresponding file on the filesystem.
verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

class plums.model.validation.structure.InitialisationPMF(strict=False, verbose=False)[source]¶

Bases: plums.model.validation.structure.Model

A ‘validatable’ component which only accepts valid PMF model data ‘initialisation from PMF model’.

Parameters

strict (bool) – Optional. Default to True. Toggle strict validation. When performing strict validation, the configuration file and all registered checkpoints checksum will be matched against the checksum of the corresponding file on the filesystem.
verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

class plums.model.validation.structure.Initialisation(strict=False, verbose=False)[source]¶

Bases: plums.model.validation.structure.DefaultTree

A ‘validatable’ component which only accepts valid PMF model data initialisation tree structure.

Parameters

strict (bool) – Optional. Default to True. Toggle strict validation. When performing strict validation, the configuration file and all registered checkpoints checksum will be matched against the checksum of the corresponding file on the filesystem.
verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

validate(data)[source]¶

Validate a given tree against the default value or one of the pmf or file enclosed tree schema.

Parameters: data (Any) – The tree to validate.
Returns: The validated tree.
Return type: Any
Raises: PlumsModelTreeValidationError – If any offence is detected in the data.

class plums.model.validation.structure.Checkpoint(strict=False, verbose=False)[source]¶

Bases: plums.model.validation.structure.DefaultTree

A ‘validatable’ component which only accepts valid PMF model data checkpoints tree structure.

Parameters

strict (bool) – Optional. Default to True. Toggle strict validation. When performing strict validation, the configuration file and all registered checkpoints checksum will be matched against the checksum of the corresponding file on the filesystem.
verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

class plums.model.validation.structure.Data(strict=False, verbose=False, checkpoints=True)[source]¶

Bases: plums.model.validation.structure.TreeComponent

A ‘validatable’ component which only accepts valid PMF model data tree structure.

Parameters

strict (bool) – Optional. Default to True. Toggle strict validation. When performing strict validation, the configuration file and all registered checkpoints checksum will be matched against the checksum of the corresponding file on the filesystem.
checkpoints (bool) – Whether to check for checkpoints existence.
verbose (bool) – Optional. Default to False. Toggle exhaustive schema offence reporting.

validate(data)[source]¶

Validate a given tree against the PMF data tree schema.

Parameters: data (Any) – The tree to validate.
Returns: The validated tree.
Return type: Any
Raises: PlumsModelTreeValidationError – If any offence is detected in the data.

Helper functions¶

plums.model.validation.utils.dict_from_tree.make_dict_structure_from_tree(path)[source]¶

Construct a nested dictionary structure from a filesystem tree.

Parameters: path (Pathlike) – The filesystem tree root.
Returns: A nested dict structure where each element is a key to its Path for files and to another dict for directories.
Return type: dict

plums.model.validation.utils.checksum.md5_checksum(filepath)[source]¶

Compute a MD5 checksum of a file.

Parameters: filepath (Pathlike) – The file location.
Returns: The MD5 checksum of the file.
Return type: str

plums.model.validation.utils.validate_path.ERROR_INVALID_NAME = 123¶: Windows-specific error code indicating an invalid pathname.

See also

Official listing of all such codes.

plums.model.validation.utils.validate_path.is_pathname_valid(pathname)[source]¶

Return whether the passed pathname is a valid pathname for the current OS.

Parameters: pathname (Pathlike) – A path to validate.
Returns: True if the provided path is syntactically correct.
Return type: bool