Answer-first summary for fast verification
Answer: The table was set up as an external table.
In Spark SQL, dropping a managed table results in the deletion of both the table's metadata and its data files. However, if the table is an external table, only the metadata is removed upon dropping, leaving the data files intact. This design prevents accidental loss of data that might be utilized by other tables or applications. The persistence of data files after the `DROP TABLE` command indicates that `my_table` was an external table, hence only its metadata was deleted. Managed tables, conversely, are fully under Spark's control, meaning both data and metadata are deleted when the table is dropped.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
A data engineer attempts to delete a Spark SQL table named my_table, intending to remove both its metadata and data. They use the command: DROP TABLE IF EXISTS my_table;. After execution, my_table disappears from the table list, but the data files remain. Why do the data files persist despite the metadata deletion?
A
The table was a managed table.
B
The table was set up as an external table.
C
The table's data exceeded 10 GB.
D
The table's data was under 10 GB.
E
The table was specified without a location.
No comments yet.