Google Professional Data Engineer
Comments
Loading comments...
Ultimate access to all questions.
You are tasked with creating an automated SQL data pipeline using BigQuery. The pipeline must perform an aggregate SQL transformation on a specified BigQuery table every two hours and subsequently append the transformation results to another pre-existing BigQuery table. Additionally, the pipeline needs to be robust, capable of retrying in case of errors. It is crucial that the pipeline is also configured to send an email notification after it encounters three consecutive failures. What steps should you take to achieve this configuration?
A
Use the BigQueryUpsertTableOperator in Cloud Composer, set the retry parameter to three, and set the email_on_failure parameter to true.
B
Use the BigQueryInsertJobOperator in Cloud Composer, set the retry parameter to three, and set the email_on_failure parameter to true.
C
Create a BigQuery scheduled query to run the SQL transformation with schedule options that repeats every two hours, and enable email notifications.
D
Create a BigQuery scheduled query to run the SQL transformation with schedule options that repeats every two hours, and enable notification to Pub/Sub topic. Use Pub/Sub and Cloud Functions to send an email after three failed executions.