Databricks Certified Data Engineer - Associate
Get started today
Ultimate access to all questions.
A data engineering team has been using a Databricks SQL query to monitor the performance of an ELT job. The ELT job is triggered by a specific number of input records being ready to process. The Databricks SQL query returns the number of minutes since the job’s most recent runtime.
Which of the following approaches can enable the data engineering team to be notified if the ELT job has not been run in an hour?
Exam-Like
Community
KKeng
A
They can set up an Alert for the accompanying dashboard to notify them if the returned value is greater than 60.
B
They can set up an Alert for the query to notify when the ELT job fails.
C
They can set up an Alert for the accompanying dashboard to notify when it has not refreshed in 60 minutes.
D
They can set up an Alert for the query to notify them if the returned value is greater than 60.
E
This type of alerting is not possible in Databricks.
Explanation:
Explanation
The correct answer is D because:
- The query returns the number of minutes since the job's most recent runtime - This is a numeric value that can be monitored.
- Databricks SQL Alerts can be configured on query results - You can set up alerts that trigger when query results meet specific conditions.
- Alert condition: value > 60 - Since the query returns minutes since last run, a value greater than 60 indicates the job hasn't run in over an hour.
- Query-based alerts vs dashboard alerts - Option D correctly suggests setting up an alert on the query itself, not the dashboard.
Why other options are incorrect:
- A: Dashboard alerts monitor dashboard refresh status, not query result values.
- B: This would only alert on job failure, not on the time since last run.
- C: This monitors dashboard refresh frequency, not the actual job runtime status.
- E: This type of alerting is absolutely possible in Databricks using SQL Alerts.
How to implement this in Databricks:
- Create a SQL query that calculates minutes since last ELT job run
- Set up a SQL Alert on this query
- Configure the alert to trigger when the result value exceeds 60
- Set up notification channels (email, Slack, etc.) for the alert
Comments
Loading comments...