Databricks Certified Data Engineer - Associate
Get started today
Ultimate access to all questions.
An engineering manager wants to monitor the performance of a recent project using a Databricks SQL query. For the first week following the project's release, the manager wants the query results to be updated every minute. However, the manager is concerned that the compute resources used for the query will be left running and cost the organization a lot of money beyond the first week of the project's release.
Which approach can the engineering team use to ensure the query does not cost the organization any money beyond the first week of the project's release?
Real Exam
Community
KKeng
A
They can set a limit to the number of DBUs that are consumed by the SQL Endpoint.
B
They can set the query's refresh schedule to end after a certain number of refreshes.
C
They can set the query's refresh schedule to end on a certain date in the query scheduler.
D
They can set a limit to the number of individuals that are able to manage the query's refresh schedule.
Explanation:
Explanation
The correct answer is C because Databricks SQL allows you to set an end date for query refresh schedules. This feature enables you to automatically stop scheduled refreshes after a specific date, which is exactly what the engineering manager needs - to have the query refresh every minute for the first week and then stop automatically.
Why other options are incorrect:
A. Setting a limit to the number of DBUs consumed by the SQL Endpoint:
- This approach controls overall SQL Endpoint consumption but doesn't specifically stop the query from running after the first week
- DBU limits are for cost control but don't automatically stop scheduled queries
- The query could still run and consume resources until the DBU limit is reached
B. Setting the query's refresh schedule to end after a certain number of refreshes:
- While this might seem logical, Databricks SQL doesn't have a built-in feature to stop after a specific number of refreshes
- You would need to calculate the exact number of refreshes for a week (10,080 refreshes for 7 days at 1-minute intervals), which is impractical and error-prone
- There's no guarantee the count would be accurate if there are any failures or skipped refreshes
C. Setting the query's refresh schedule to end on a certain date in the query scheduler: ✓ CORRECT
- Databricks SQL query scheduler allows you to specify both start and end dates for refresh schedules
- You can set the end date to exactly one week after the project release
- The system will automatically stop refreshing the query after that date, preventing any further compute costs
- This is the most precise and reliable method to achieve the requirement
D. Setting a limit to the number of individuals that are able to manage the query's refresh schedule:
- This is a security/access control measure, not a cost control mechanism
- Limiting who can manage the schedule doesn't prevent the query from running if it's already scheduled
- The scheduled query would continue to run regardless of who has permission to modify it
Best Practice Recommendation:
When setting up time-bound queries in Databricks SQL:
- Use the query scheduler's end date feature for precise control
- Consider adding alerts or notifications when the schedule is about to end
- Monitor query costs during the active period to ensure they align with expectations
- Document the schedule end date clearly for team awareness
Comments
Loading comments...