Databricks Certified Data Engineer - Associate

Explanation

To identify why a notebook is running slowly in a Databricks Job, the correct approach is:

Option C: They can navigate to the Runs tab in the Jobs UI and click on the active run to review the processing notebook.

Here's why:

1. Runs Tab vs Tasks Tab: In Databricks Jobs UI:

   • The Runs tab shows all job runs (past and current)
   • The Tasks tab shows the job configuration and task definitions, not the active runs

2. Correct Workflow:

   • Navigate to the Jobs UI
   • Go to the Runs tab (not Tasks tab)
   • Find and click on the active run (current running job)
   • This will show you the detailed run information including:
     • Task execution status
     • Logs for each task
     • Performance metrics
     • Spark UI links for detailed Spark execution analysis

3. Why other options are incorrect:

   • Option A: Incorrect because you need to click on the active run, not just navigate to the Runs tab
   • Option B: Incorrect because the Tasks tab shows configuration, not active runs
   • Option D: Incorrect because the Tasks tab doesn't show active runs and you need to click on the specific run

4. Additional Investigation Steps: Once you're viewing the active run, you can:

   • Click on the specific task (notebook) that's running slowly
   • View the Spark UI to analyze:
     • Stage execution times
     • Task distribution
     • Shuffle operations
     • Memory usage
   • Check logs for errors or warnings
   • Review input/output data sizes

This approach allows the tech lead to diagnose performance issues by examining the actual execution details of the current job run.