google_cloud_pipeline_components.experimental.bigquery module

Google Cloud Pipeline Bigquery components.

google_cloud_pipeline_components.experimental.bigquery.BigqueryCreateModelJobOp()

bigquery_create_model_job Launch a BigQuery create model job and waits for it to finish.

Args:

project (str):

Required. Project to run BigQuery model creation job.

location (Optional[str]):

Location of the job to create the BigQuery model. If not set, default to US multi-region.

For more details, see https://cloud.google.com/bigquery/docs/locations#specifying_your_location

query (str):

Required. SQL query text to execute. Only standard SQL is supported.

If query are both specified in here and in job_configuration_query, the value in here will override the other one.

query_parameters (Optional[Sequence]):

jobs.query parameters for standard SQL queries.

If query_parameters are both specified in here and in job_configuration_query, the value in here will override the other one.

job_configuration_query (Optional[dict]):

A json formatted string describing the rest of the job configuration.

For more details, see https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationQuery

labels (Optional[dict]):

The labels associated with this job. You can use these to organize and group your jobs. Label keys and values can be no longer than 63 characters, can only containlowercase letters, numeric characters, underscores and dashes. International characters are allowed. Label values are optional. Label keys must start with a letter and each label in the list must have a different key.

Example: { “name”: “wrench”, “mass”: “1.3kg”, “count”: “3” }.

encryption_spec_key_name(Optional[List[str]]):

Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.

If encryption_spec_key_name are both specified in here and in job_configuration_query, the value in here will override the other one.

Returns:

model (google.BQMLModel):

Describes the model which is created.

gcp_resources (str):

Serialized gcp_resources proto tracking the BigQuery job. For more details, see https://github.com/kubeflow/pipelines/blob/master/components/google-cloud/google_cloud_pipeline_components/proto/README.md.

google_cloud_pipeline_components.experimental.bigquery.BigqueryEvaluateModelJobOp()

bigquery_evaluate_model_job Launch a BigQuery evaluate model job and waits for it to finish.

Args:

project (str):

Required. Project to run BigQuery model evaluation job.

location (Optional[str]):

Location to run the BigQuery model evaluation job. If not set, default to US multi-region.

For more details, see https://cloud.google.com/bigquery/docs/locations#specifying_your_location

model_name (str):

Required. BigQuery ML model name for evaluation.

For more details, see https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-predict#predict_model_name

table_name (Optional[str]):

BigQuery table id of the input table that contains the evaluation data.

For more details, see https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-predict#predict_table_name

query_statement (Optional[str]):

Query statement string used to generate the evaluation data.

For more details, see https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-predict#predict_query_statement

threshold (Optional[float]):

A custom threshold for the binary logistic regression model used as the cutoff between two labels. Predictions above the threshold are treated as positive prediction. Predictions below the threshold are negative predictions.

For more details, see https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-predict#threshold

query_parameters (Optional[Sequence]):

jobs.query parameters for standard SQL queries.

If query_parameters are both specified in here and in job_configuration_query, the value in here will override the other one.

job_configuration_query (Optional[dict]):

A json formatted string describing the rest of the job configuration.

For more details, see https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationQuery

labels (Optional[dict]):

The labels associated with this job. You can use these to organize and group your jobs. Label keys and values can be no longer than 63 characters, can only containlowercase letters, numeric characters, underscores and dashes. International characters are allowed. Label values are optional. Label keys must start with a letter and each label in the list must have a different key.

Example: { “name”: “wrench”, “mass”: “1.3kg”, “count”: “3” }.

encryption_spec_key_name(Optional[List[str]]):

Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.

If encryption_spec_key_name are both specified in here and in job_configuration_query, the value in here will override the other one.

Returns:

destination_table (google.BQTable):

Describes the table where the model prediction results should be stored. This property must be set for large results that exceed the maximum response size. For queries that produce anonymous (cached) results, this field will be populated by BigQuery.

gcp_resources (str):

Serialized gcp_resources proto tracking the BigQuery job. For more details, see https://github.com/kubeflow/pipelines/blob/master/components/google-cloud/google_cloud_pipeline_components/proto/README.md.

google_cloud_pipeline_components.experimental.bigquery.BigqueryExportModelJobOp()

bigquery_export_model_job Launch a BigQuery export model job and waits for it to finish.

Args:

project (str):

Required. Project to run BigQuery model export job.

location (Optional[str]):

Location of the job to export the BigQuery model. If not set, default to US multi-region.

For more details, see https://cloud.google.com/bigquery/docs/locations#specifying_your_location

model_name (str):

Required. BigQuery ML model name to export.

model_destination_path(str):

Required. The gcs bucket to export the model to.

job_configuration_extract (Optional[dict]):

A json formatted string describing the rest of the job configuration.

For more details, see https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationQuery

labels (Optional[dict]):

The labels associated with this job. You can use these to organize and group your jobs. Label keys and values can be no longer than 63 characters, can only containlowercase letters, numeric characters, underscores and dashes. International characters are allowed. Label values are optional. Label keys must start with a letter and each label in the list must have a different key.

Example: { “name”: “wrench”, “mass”: “1.3kg”, “count”: “3” }.

Returns:

model_destination_path (str):

Describes the exported model path.

gcp_resources (str):

Serialized gcp_resources proto tracking the BigQuery job. For more details, see https://github.com/kubeflow/pipelines/blob/master/components/google-cloud/google_cloud_pipeline_components/proto/README.md.

google_cloud_pipeline_components.experimental.bigquery.BigqueryPredictModelJobOp()

bigquery_predict_model_job Launch a BigQuery predict model job and waits for it to finish.

Args:

project (str):

Required. Project to run BigQuery model prediction job.

location (Optional[str]):

Location to run the BigQuery model prediction job. If not set, default to US multi-region.

For more details, see https://cloud.google.com/bigquery/docs/locations#specifying_your_location

model_name (str):

Required. BigQuery ML model name for prediction.

For more details, see https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-predict#predict_model_name

table_name (Optional[str]):

BigQuery table id of the input table that contains the prediction data.

For more details, see https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-predict#predict_table_name

query_statement (Optional[str]):

Query statement string used to generate the prediction data.

For more details, see https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-predict#predict_query_statement

threshold (Optional[float]):

A custom threshold for the binary logistic regression model used as the cutoff between two labels. Predictions above the threshold are treated as positive prediction. Predictions below the threshold are negative predictions.

For more details, see https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-predict#threshold

query_parameters (Optional[Sequence]):

jobs.query parameters for standard SQL queries.

If query_parameters are both specified in here and in job_configuration_query, the value in here will override the other one.

job_configuration_query (Optional[dict]):

A json formatted string describing the rest of the job configuration.

For more details, see https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationQuery

labels (Optional[dict]):

The labels associated with this job. You can use these to organize and group your jobs. Label keys and values can be no longer than 63 characters, can only containlowercase letters, numeric characters, underscores and dashes. International characters are allowed. Label values are optional. Label keys must start with a letter and each label in the list must have a different key.

Example: { “name”: “wrench”, “mass”: “1.3kg”, “count”: “3” }.

encryption_spec_key_name(Optional[List[str]]):

Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.

If encryption_spec_key_name are both specified in here and in job_configuration_query, the value in here will override the other one.

Returns:

destination_table (google.BQTable):

Describes the table where the model prediction results should be stored. This property must be set for large results that exceed the maximum response size. For queries that produce anonymous (cached) results, this field will be populated by BigQuery.

gcp_resources (str):

Serialized gcp_resources proto tracking the BigQuery job. For more details, see https://github.com/kubeflow/pipelines/blob/master/components/google-cloud/google_cloud_pipeline_components/proto/README.md.

google_cloud_pipeline_components.experimental.bigquery.BigqueryQueryJobOp()

bigquery_query_job Launch a BigQuery query job and waits for it to finish.

Args:

project (str):

Required. Project to run the BigQuery query job.

location (Optional[str]):

Location for creating the BigQuery job. If not set, default to US multi-region.

For more details, see https://cloud.google.com/bigquery/docs/locations#specifying_your_location

query (str):

Required. SQL query text to execute. Only standard SQL is supported.

If query are both specified in here and in job_configuration_query, the value in here will override the other one.

query_parameters (Optional[Sequence]):

jobs.query parameters for standard SQL queries.

If query_parameters are both specified in here and in job_configuration_query, the value in here will override the other one.

job_configuration_query (Optional[dict]):

A json formatted string describing the rest of the job configuration.

For more details, see https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationQuery

labels (Optional[dict]):

The labels associated with this job. You can use these to organize and group your jobs. Label keys and values can be no longer than 63 characters, can only containlowercase letters, numeric characters, underscores and dashes. International characters are allowed. Label values are optional. Label keys must start with a letter and each label in the list must have a different key.

Example: { “name”: “wrench”, “mass”: “1.3kg”, “count”: “3” }.

encryption_spec_key_name(Optional[List[str]]):

Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.

If encryption_spec_key_name are both specified in here and in job_configuration_query, the value in here will override the other one.

Returns:

destination_table (google.BQTable):

Describes the table where the query results should be stored. This property must be set for large results that exceed the maximum response size. For queries that produce anonymous (cached) results, this field will be populated by BigQuery.

gcp_resources (str):

Serialized gcp_resources proto tracking the BigQuery job. For more details, see https://github.com/kubeflow/pipelines/blob/master/components/google-cloud/google_cloud_pipeline_components/proto/README.md.