
Answer-first summary for fast verification
Answer: The table was managed
## Explanation In Databricks/Spark SQL, there are two types of tables: 1. **Managed Tables** (also called internal tables): - The data is managed by Spark/Databricks - When you drop a managed table using `DROP TABLE`, both the metadata (table definition) AND the actual data files are deleted - Spark controls the lifecycle of the data - The data is stored in the Spark SQL warehouse directory 2. **External Tables**: - The data is stored in an external location (like ADLS, S3, etc.) - When you drop an external table using `DROP TABLE`, only the metadata (table definition) is deleted - The actual data files remain intact in the external location - You specify the location using `LOCATION` clause when creating the table In this scenario, since both the data files AND metadata files were deleted after running `DROP TABLE`, this indicates that `my_table` was a **managed table**. **Key points**: - Option A is correct because managed tables have their data deleted when dropped - Option B is incorrect - the size of data doesn't affect deletion behavior - Option C is incorrect - all tables have a location (managed tables have default locations) - Option D is incorrect - external tables only delete metadata, not data files **Additional context**: - To create a managed table: `CREATE TABLE my_table ...` (without LOCATION clause) - To create an external table: `CREATE TABLE my_table ... LOCATION 'path/to/data'` - To check if a table is managed or external: `DESCRIBE EXTENDED my_table` (look for 'Type' field)
Author: Keng Suppaseth
Ultimate access to all questions.
A data engineer is attempting to drop a Spark SQL table my_table and runs the following command:
DROP TABLE IF EXISTS my_table;
After running this command, the engineer notices that the data files and metadata files have been deleted from the file system.
What is the reason behind the deletion of all these files?
A
The table was managed
B
The table's data was smaller than 10 GB
C
The table did not have a location
D
The table was external
No comments yet.