How can you parameterize a query to filter data based on a batch date that changes with each run, without manually altering the code each time?

Real Exam

Store the batch date in the Spark configuration and use a Spark DataFrame to filter the data based on the Spark configuration.

11.9%

Create a notebook parameter for batch date, assign its value to a Python variable, and use a Spark DataFrame to filter the data based on this variable.

70.4%

Manually edit the code every time to change the batch date.

0.3%

Create a dynamic view that automatically calculates the batch date and use this view to query the data.

16.8%

There is no way to combine a Python variable and Spark code for filtering.

0.6%

Databricks Certified Data Engineer - Associate

Get started today

Comments

How can you parameterize a query to filter data based on a batch date that changes with each run, without manually altering the code each time?