
Answer-first summary for fast verification
Answer: The table was managed
## Explanation In Databricks/Spark SQL, there are two types of tables: 1. **Managed Tables** (also called internal tables): - The data files are stored in the default warehouse directory managed by Spark/Databricks - When you drop a managed table using `DROP TABLE`, both the metadata (table definition) AND the underlying data files are deleted - Spark/Databricks has full control over the data lifecycle 2. **External Tables** (also called unmanaged tables): - The data files are stored in an external location (like S3, ADLS, etc.) that you specify - When you drop an external table using `DROP TABLE`, only the metadata (table definition) is deleted - The underlying data files remain intact in the external location In this scenario, since both the metadata files AND data files were deleted from the file system after running `DROP TABLE IF EXISTS my_table;`, this indicates that `my_table` was a **managed table**. **Key points:** - Option A is correct because managed tables have their data lifecycle controlled by Spark/Databricks - Option D (external table) is incorrect because dropping an external table would only delete metadata, not the data files - Options B and C are incorrect because table size doesn't affect whether data gets deleted when dropping a table - Option E is incorrect because tables without a specified location are typically managed tables, but the key distinction is whether it's managed vs external, not just the presence of a location specification **Additional context:** - Managed tables are created without a `LOCATION` clause or with data stored in the default warehouse directory - External tables are created with a `LOCATION` clause pointing to external storage - The `IF EXISTS` clause simply prevents an error if the table doesn't exist, but doesn't affect whether data gets deleted
Author: Keng Suppaseth
Ultimate access to all questions.
No comments yet.
A data engineer is attempting to drop a Spark SQL table my_table and runs the following command:
DROP TABLE IF EXISTS my_table;
DROP TABLE IF EXISTS my_table;
After running this command, the engineer notices that the data files and metadata files have been deleted from the file system.
Which of the following describes why all of these files were deleted?
A
The table was managed
B
The table's data was smaller than 10 GB
C
The table's data was larger than 10 GB
D
The table was external
E
The table did not have a location