Databricks Certified Data Engineer - Associate
Get started today
Ultimate access to all questions.
Ultimate access to all questions.
A data engineer needs to ingest heart rate recordings data from medical devices in JSON format using Auto Loader and then create a temporary view. The column 'time' is incorrectly inferred as a 'string' instead of a 'timestamp' data type. Which of the following code blocks correctly enforces the schema information for this column?
Explanation:
Auto Loader schema inference works by sampling the first 50 GB or 1000 files, whichever comes first, to infer the schema. For formats like JSON and CSV, it infers all columns as strings by default to avoid schema evolution issues. However, you can override this behavior using schema hints to enforce specific data types. The correct option uses cloudFiles.schemaHints to specify that the 'time' column should be treated as a TIMESTAMP, ensuring the data is correctly typed when ingested.