kfp.compiler package

class kfp.compiler.Compiler[source]

Bases: object

Experimental DSL compiler that targets the PipelineSpec IR.

It compiles pipeline function into PipelineSpec json string. PipelineSpec is the IR protobuf message that defines a pipeline: https://github.com/kubeflow/pipelines/blob/237795539f7b85bac77435e2464367226ee19391/api/v2alpha1/pipeline_spec.proto#L8 In this initial implementation, we only support components authored through Component yaml spec. And we don’t support advanced features like conditions, static and dynamic loops, etc.

Example:

@dsl.pipeline(
  name='name',
  description='description',
)
def my_pipeline(a: int = 1, b: str = "default value"):
    ...

kfp.compiler.Compiler().compile(
    pipeline_func=my_pipeline,
    package_path='path/to/pipeline.json',
)
compile(pipeline_func: Callable[[...], Any], package_path: str, pipeline_name: Optional[str] = None, pipeline_parameters: Optional[Mapping[str, Any]] = None, type_check: bool = True) → None[source]

Compile the given pipeline function into pipeline job json.

Parameters

  • pipeline_func – Pipeline function with @dsl.pipeline decorator.

  • package_path – The output pipeline spec .yaml file path. For example, “~/pipeline_spec.yaml”.

  • pipeline_name – Optional; the name of the pipeline.

  • pipeline_parameters – Optional; the mapping from parameter names to values.

  • type_check – Optional; whether to enable the type check or not. Default is True.