Wait on GCP Resources¶
Wait on the completion of GCP resources spawned from an upstream pipeline component.
Components:
|
Waits for the completion of one or more GCP resources by polling for completion statuses. |
- WaitGcpResourcesOp(gcp_resources: str, gcp_resources: dsl.OutputPath(str))
Waits for the completion of one or more GCP resources by polling for completion statuses.
Currently this component only supports waiting on a DataflowJob resource. To use this component, first create a component that outputs a
gcp_resources
proto as JSON, then pass it to this component’sgcp_resources
parameter. See details on how to create agcp_resources
proto as a component output.dataflow_python_op = gcpc.v1.dataflow.LaunchPythonOp( python_file_path=... ) dataflow_wait_op = WaitGcpResourcesOp( gcp_resources=dataflow_python_op.outputs["gcp_resources"] )