ror.schemas.common package

Submodules

ror.schemas.common.artifact_schema module

class ror.schemas.common.artifact_schema.ArtifactSchema(source_schema: dataclass)

Bases: object

A simple dataclass schema which is supposed to only contain the perishable fields which where dropped from some source BaseSchema.

get_standard_fields()

source_schema: dataclass

ror.schemas.common.base_schema module

class ror.schemas.common.base_schema.BaseSchema

Bases: object

BaseSchema is extended for any Input our Output dataclass for each stage, which implements a set of functions to get artifacts of the current stage (all fields which are perishable), and to get the carry over to some Output dataclass (fields marked as persistant).

Examples

>>> from dataclasses import dataclass
>>> from pypipeline.schemas import BaseSchema
>>> from pypipeline.schemas.fields import field_perishable, field_persistance

Which will enable you to define input or output dataclasses for your stage

>>> @dataclass
>>> class InputTest(BaseSchema):
>>>     A: str = field_persistance()
>>>     B: str = field_perishable()

>>> @dataclass
>>> class OutputTest(BaseSchema):
>>>     A: str = field_persistance()

Where Output does not contain field B as it is marked perishable in the input, and thus this data is only used in the computation for this stage and not propagated further from the output data of this stage.

get_artifact() → ArtifactSchema

Constructs an ArtfifactSchema instance with the data which was marked as perishable from the instance of this dataclass, and implicitly deletes the perishable fields from this instance.

Returns:

Dataclass containing the perishable data and additional meta-data.

Return type:

ArtifactSchema

get_carry() → dict

Returns a dictionary instance of this dataclass where all the perishable fields are removed from the dictionary.

Returns:

Carry over dictionary without the perishable fields.

Return type:

dict

Module contents