Ultimate access to all questions.
To facilitate a series of sequential load and transformation jobs, consider the following scenario: Data files are incrementally added to a Cloud Storage bucket through an upstream process, but their arrival times are not predetermined. Upon the arrival of new data, a Dataproc job is triggered to execute initial transformations and then store the processed data in BigQuery. Following this, additional transformation jobs, which vary for each table, must be executed within BigQuery. These subsequent jobs can potentially take several hours to complete. Your task is to identify the most efficient and maintainable workflow that can manage the processing of hundreds of tables while consistently delivering the freshest data to your end users. What strategy should you implement?