kfp.components.structures package

Definitions for component spec.

class kfp.components.structures.BaseModel[source]

Bases: pydantic.main.BaseModel

class Config[source]

Bases: object

allow_population_by_field_name = True

arbitrary_types_allowed = True

class kfp.components.structures.BasePlaceholder[source]

Bases: kfp.components.structures.BaseModel

Base class for placeholders that could appear in container cmd and args.

class kfp.components.structures.ComponentSpec(*, name: str, description: str = None, inputs: Dict[str, kfp.components.structures.InputSpec] = None, outputs: Dict[str, kfp.components.structures.OutputSpec] = None, implementation: kfp.components.structures.Implementation)[source]

Bases: kfp.components.structures.BaseModel

The definition of a component.

name

The name of the component.

Type: str

description

Optional; the description of the component.

Type: Optional[str]

inputs

Optional; the input definitions of the component.

Type: Optional[Dict[str, kfp.components.structures.InputSpec]]

outputs

Optional; the output definitions of the component.

Type: Optional[Dict[str, kfp.components.structures.OutputSpec]]

implementation

The implementation of the component. Either an executor (container, importer) or a DAG consists of other components.

Type: kfp.components.structures.Implementation

description: Optional[str]

classmethod empty_map(v)[source]

classmethod from_v1_component_spec(v1_component_spec: kfp.components.v1_structures.ComponentSpec) → kfp.components.structures.ComponentSpec[source]

Converts V1 ComponentSpec to V2 ComponentSpec.

Parameters

v1_component_spec – The V1 ComponentSpec.

Returns

Component spec in the form of V2 ComponentSpec.

Raises

ValueError – If implementation is not found.
TypeError – if any argument is neither a str nor Dict.

implementation: kfp.components.structures.Implementation

inputs: Optional[Dict[str, kfp.components.structures.InputSpec]]

classmethod load_from_component_yaml(component_yaml: str) → kfp.components.structures.ComponentSpec[source]

Loads V1 or V2 component yaml into ComponentSpec.

Parameters: component_yaml – the component yaml in string format.
Returns: Component spec in the form of V2 ComponentSpec.

name: str

outputs: Optional[Dict[str, kfp.components.structures.OutputSpec]]

save_to_component_yaml(output_file: str) → None[source]

Saves ComponentSpec into yaml file.

Parameters: output_file – File path to store the component yaml.

to_v1_component_spec() → kfp.components.v1_structures.ComponentSpec[source]

Converts to v1 ComponentSpec.

Returns: Component spec in the form of V1 ComponentSpec.

Needed until downstream accept new ComponentSpec.

classmethod validate_placeholders(values)[source]

class kfp.components.structures.ConcatPlaceholder(*, concat: Sequence[Union[str, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, IfPresentPlaceholder, ConcatPlaceholder]])[source]

Bases: kfp.components.structures.BasePlaceholder

Class that extends basePlaceholders for concatenation.

items

string or ValidCommandArgs for concatenation.

Type: Sequence[Union[str, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, kfp.components.structures.IfPresentPlaceholder, kfp.components.structures.ConcatPlaceholder]]

items: Sequence[Union[str, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, kfp.components.structures.IfPresentPlaceholder, kfp.components.structures.ConcatPlaceholder]]

Bases: kfp.components.structures.BaseModel

Container implementation definition.

image

The container image.

Type: str

command

Optional; the container entrypoint.

Type: Optional[Sequence[Union[str, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, kfp.components.structures.IfPresentPlaceholder, kfp.components.structures.ConcatPlaceholder]]]

args

Optional; the arguments to the container entrypoint.

Type: Optional[Sequence[Union[str, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, kfp.components.structures.IfPresentPlaceholder, kfp.components.structures.ConcatPlaceholder]]]

env

Optional; the environment variables to be passed to the container.

Type: Optional[Mapping[str, Union[str, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, kfp.components.structures.IfPresentPlaceholder, kfp.components.structures.ConcatPlaceholder]]]

resources

Optional; the specification on the resource requirements.

Type: Optional[kfp.components.structures.ResourceSpec]

args: Optional[Sequence[Union[str, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, kfp.components.structures.IfPresentPlaceholder, kfp.components.structures.ConcatPlaceholder]]]

command: Optional[Sequence[Union[str, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, kfp.components.structures.IfPresentPlaceholder, kfp.components.structures.ConcatPlaceholder]]]

classmethod empty_map(v)[source]

classmethod empty_sequence(v)[source]

env: Optional[Mapping[str, Union[str, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, kfp.components.structures.IfPresentPlaceholder, kfp.components.structures.ConcatPlaceholder]]]

image: str

resources: Optional[kfp.components.structures.ResourceSpec]

class kfp.components.structures.DagSpec(*, tasks: Mapping[str, kfp.components.structures.TaskSpec], outputs: Mapping[str, Any])[source]

Bases: kfp.components.structures.BaseModel

DAG(graph) implementation definition.

tasks

The tasks inside the DAG.

Type: Mapping[str, kfp.components.structures.TaskSpec]

outputs

Defines how the outputs of the dag are linked to the sub tasks.

Type: Mapping[str, Any]

outputs: Mapping[str, Any]

tasks: Mapping[str, kfp.components.structures.TaskSpec]

class kfp.components.structures.IfPresentPlaceholder(*, ifPresent: kfp.components.structures.IfPresentPlaceholderStructure)[source]

Bases: kfp.components.structures.BasePlaceholder

Class that extends basePlaceholders for conditional cases.

if_present

holds structure for conditional cases.

Type: ifPresent

if_structure: kfp.components.structures.IfPresentPlaceholderStructure

class kfp.components.structures.IfPresentPlaceholderStructure(*, inputName: str, then: typing.Sequence[typing.Union[str, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, IfPresentPlaceholder, ConcatPlaceholder]], else: typing.Sequence[typing.Union[str, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, IfPresentPlaceholder, ConcatPlaceholder]] = None)[source]

Bases: kfp.components.structures.BaseModel

Class that holds structure for conditional cases.

input_name

name of the input/output.

Type: str

then

If the input/output specified in name is present, the command-line argument will be replaced at run-time by the expanded value of then.

Type: Sequence[Union[str, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, kfp.components.structures.IfPresentPlaceholder, kfp.components.structures.ConcatPlaceholder]]

otherwise

If the input/output specified in name is not present, the command-line argument will be replaced at run-time by the expanded value of otherwise.

Type: Optional[Sequence[Union[str, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, kfp.components.structures.IfPresentPlaceholder, kfp.components.structures.ConcatPlaceholder]]]

classmethod empty_otherwise_sequence(v)[source]

input_name: str

otherwise: Optional[Sequence[Union[str, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, kfp.components.structures.IfPresentPlaceholder, kfp.components.structures.ConcatPlaceholder]]]

then: Sequence[Union[str, kfp.components.structures.InputValuePlaceholder, kfp.components.structures.InputPathPlaceholder, kfp.components.structures.InputUriPlaceholder, kfp.components.structures.OutputPathPlaceholder, kfp.components.structures.OutputUriPlaceholder, kfp.components.structures.IfPresentPlaceholder, kfp.components.structures.ConcatPlaceholder]]

class kfp.components.structures.Implementation(*, container: kfp.components.structures.ContainerSpec = None, graph: kfp.components.structures.DagSpec = None, importer: kfp.components.structures.ImporterSpec = None)[source]

Bases: kfp.components.structures.BaseModel

Implementation definition.

container

container implementation details.

Type: Optional[kfp.components.structures.ContainerSpec]

graph

graph implementation details.

Type: Optional[kfp.components.structures.DagSpec]

importer

importer implementation details.

Type: Optional[kfp.components.structures.ImporterSpec]

container: Optional[kfp.components.structures.ContainerSpec]

graph: Optional[kfp.components.structures.DagSpec]

importer: Optional[kfp.components.structures.ImporterSpec]

class kfp.components.structures.ImporterSpec(*, artifact_uri: str, type_schema: str, reimport: bool, metadata: Mapping[str, Any] = None)[source]

Bases: kfp.components.structures.BaseModel

ImporterSpec definition.

artifact_uri

The URI of the artifact.

Type: str

type_schema

The type of the artifact.

Type: str

reimport

Whether or not import an artifact regardless it has been imported before.

Type: bool

metadata

Optional; the properties of the artifact.

Type: Optional[Mapping[str, Any]]

artifact_uri: str

metadata: Optional[Mapping[str, Any]]

reimport: bool

type_schema: str

class kfp.components.structures.InputPathPlaceholder(*, inputPath: str)[source]

Bases: kfp.components.structures.BasePlaceholder

Class that holds input path for conditional cases.

input_name

name of the input.

Type: str

input_name: str

class kfp.components.structures.InputSpec(*, type: Union[str, dict], default: Any = None, description: str = None)[source]

Bases: kfp.components.structures.BaseModel

Component input definitions.

type

The type of the input.

Type: Union[str, dict]

default

Optional; the default value for the input.

Type: Optional[Any]

description

Optional: the user description of the input.

Type: Optional[str]

optional: Wether the input is optional. An input is optional when it has an explicit default value.

default: Optional[Any]

description: Optional[str]

property optional: bool

type: Union[str, dict]

class kfp.components.structures.InputUriPlaceholder(*, inputUri: str)[source]

Bases: kfp.components.structures.BasePlaceholder

Class that holds input uri for conditional cases.

input_name

name of the input.

Type: str

input_name: str

class kfp.components.structures.InputValuePlaceholder(*, inputValue: str)[source]

Bases: kfp.components.structures.BasePlaceholder

Class that holds input value for conditional cases.

input_name

name of the input.

Type: str

input_name: str

class kfp.components.structures.OutputPathPlaceholder(*, outputPath: str)[source]

Bases: kfp.components.structures.BasePlaceholder

Class that holds output path for conditional cases.

output_name

name of the output.

Type: str

output_name: str

class kfp.components.structures.OutputSpec(*, type: Union[str, dict], description: str = None)[source]

Bases: kfp.components.structures.BaseModel

Component output definitions.

type

The type of the output.

Type: Union[str, dict]

description

Optional: the user description of the output.

Type: Optional[str]

description: Optional[str]

type: Union[str, dict]

class kfp.components.structures.OutputUriPlaceholder(*, outputUri: str)[source]

Bases: kfp.components.structures.BasePlaceholder

Class that holds output uri for conditional cases.

output_name

name of the output.

Type: str

output_name: str

class kfp.components.structures.ResourceSpec(cpu_limit: Optional[float] = None, memory_limit: Optional[float] = None, accelerator_type: Optional[str] = None, accelerator_count: Optional[int] = None)[source]

Bases: object

The resource requirements of a container execution.

cpu_limit

Optional; the limit of the number of vCPU cores.

Type: Optional[float]

memory_limit

Optional; the memory limit in GB.

Type: Optional[float]

accelerator_type

Optional; the type of accelerators attached to the container.

Type: Optional[str]

accelerator_count

Optional; the number of accelerators attached.

Type: Optional[int]

accelerator_count: Optional[int] = None

accelerator_type: Optional[str] = None

cpu_limit: Optional[float] = None

memory_limit: Optional[float] = None

class kfp.components.structures.TaskSpec(*, name: str, inputs: Mapping[str, Any], dependent_tasks: Sequence[str], component_ref: str, trigger_condition: str = None, trigger_strategy: str = None, iterator_items: Any = None, iterator_item_input: str = None, enable_caching: bool = True, display_name: str = None)[source]

Bases: kfp.components.structures.BaseModel

The spec of a pipeline task.

name

The name of the task.

Type: str

inputs

The sources of task inputs. Constant values or PipelineParams.

Type: Mapping[str, Any]

dependent_tasks

The list of upstream tasks.

Type: Sequence[str]

component_ref

The name of a component spec this task is based on.

Type: str

trigger_condition

Optional; an expression which will be evaluated into a boolean value. True to trigger the task to run.

Type: Optional[str]

trigger_strategy

Optional; when the task will be ready to be triggered. Valid values include: “TRIGGER_STRATEGY_UNSPECIFIED”, “ALL_UPSTREAM_TASKS_SUCCEEDED”, and “ALL_UPSTREAM_TASKS_COMPLETED”.

Type: Optional[str]

iterator_items

Optional; the items to iterate on. A constant value or a PipelineParam.

Type: Optional[Any]

iterator_item_input

Optional; the name of the input which has the item from the [items][] collection.

Type: Optional[str]

enable_caching

Optional; whether or not to enable caching for the task. Default is True.

Type: bool

display_name

Optional; the display name of the task. If not specified, the task name will be used as the display name.

Type: Optional[str]

component_ref: str

dependent_tasks: Sequence[str]

display_name: Optional[str]

enable_caching: bool

inputs: Mapping[str, Any]

iterator_item_input: Optional[str]

iterator_items: Optional[Any]

name: str

trigger_condition: Optional[str]

trigger_strategy: Optional[str]

kfp.components.structures.kubernetes package
- kfp.components.structures.kubernetes.v1 module