
Answer-first summary for fast verification
Answer: The table was external
## Explanation In Databricks/Spark SQL, there are two types of tables: 1. **Managed Tables (Internal Tables)**: - When you drop a managed table using `DROP TABLE`, both the metadata (table definition) AND the underlying data files are deleted. - The data files are stored in a location managed by Spark/Databricks. 2. **External Tables**: - When you drop an external table using `DROP TABLE`, only the metadata (table definition) is deleted. - The underlying data files remain intact in their original location. - This is because external tables reference data stored outside of Spark/Databricks control. In this scenario: - The table no longer appears in `SHOW TABLES` → metadata was deleted ✓ - The data files still exist → only metadata was deleted, not data files ✓ This behavior matches exactly what happens with **external tables**. The `DROP TABLE` command for external tables only removes the table definition from the metastore, leaving the actual data files untouched. **Why the other options are incorrect**: - **A & B**: The size of the data (10 GB threshold) is irrelevant to whether data files are deleted when dropping a table. - **D**: If a table didn't have a location, it would be a managed table, and dropping it would delete both metadata and data (if any existed). - **E**: If the table was managed, dropping it would delete both metadata AND data files, which is the opposite of what happened. **Key takeaway**: Always be aware of whether you're working with managed or external tables, as dropping them has different consequences for data persistence.
Author: Keng Suppaseth
Ultimate access to all questions.
A data engineer is attempting to drop a Spark SQL table my_table. The data engineer wants to delete all table metadata and data. They run the following command:
DROP TABLE IF EXISTS my_table - While the object no longer appears when they run SHOW TABLES, the data files still exist. Which of the following describes why the data files still exist and the metadata files were deleted?
A
The table's data was larger than 10 GB
B
The table's data was smaller than 10 GB
C
The table was external
D
The table did not have a location
E
The table was managed
No comments yet.