You have created three distinct data processing jobs as part of your data pipeline management tasks:

A Cloud Dataflow pipeline that processes data uploaded to Cloud Storage and writes the transformed data to BigQuery.

A data ingestion job that retrieves data from on-premises servers and uploads it to Cloud Storage.

Another Cloud Dataflow pipeline that fetches information from third-party data providers and uploads this information to Cloud Storage.

You need a solution that allows you to schedule and monitor the execution of these three workflows and also gives you the capability to manually trigger them when necessary. What approach should you take?

Exam-Like

Create a Direct Acyclic Graph in Cloud Composer to schedule and monitor the jobs.

76.3%

Use Stackdriver Monitoring and set up an alert with a Webhook notification to trigger the jobs.

10.5%

Develop an App Engine application to schedule and request the status of the jobs using GCP API calls.

5.3%

Set up cron jobs in a Compute Engine instance to schedule and monitor the pipelines using GCP API calls.

7.9%

Google Professional Data Engineer