Databricks Certified Data Engineer - Associate
Get started today
Ultimate access to all questions.
You are working on a data engineering project that requires processing a large volume of Avro files stored in a directory named 'avro_data'. These files are named in the pattern 'data_YYYYMMDD.avro', where YYYYMMDD represents the date. Your task is to create a Spark DataFrame query that efficiently extracts data from these files and creates a temporary view named 'avro_view'. The solution must consider the following constraints: 1) The query should be optimized for performance given the large volume of data. 2) It must correctly specify the Avro file format to ensure accurate data parsing. 3) The temporary view should be accessible for subsequent SQL queries. Choose the best option from the following:
You are working on a data engineering project that requires processing a large volume of Avro files stored in a directory named 'avro_data'. These files are named in the pattern 'data_YYYYMMDD.avro', where YYYYMMDD represents the date. Your task is to create a Spark DataFrame query that efficiently extracts data from these files and creates a temporary view named 'avro_view'. The solution must consider the following constraints: 1) The query should be optimized for performance given the large volume of data. 2) It must correctly specify the Avro file format to ensure accurate data parsing. 3) The temporary view should be accessible for subsequent SQL queries. Choose the best option from the following:
Simulated