Databricks Certified Data Engineer - Associate
Get started today
Ultimate access to all questions.
A data engineer is attempting to drop a Spark SQL table my_table. The data engineer wants to delete all table metadata and data. They run the following command:
DROP TABLE IF EXISTS my_table - While the object no longer appears when they run SHOW TABLES, the data files still exist. Which of the following describes why the data files still exist and the metadata files were deleted?
Real Exam
Community
KKeng
A
The table's data was larger than 10 GB
B
The table's data was smaller than 10 GB
C
The table was external
D
The table did not have a location
E
The table was managed
Explanation:
Explanation
In Databricks/Spark SQL, there are two types of tables:
-
Managed Tables (Internal Tables):
- When you drop a managed table using
DROP TABLE, both the metadata (table definition) AND the underlying data files are deleted. - The data files are stored in a location managed by Spark/Databricks.
- When you drop a managed table using
-
External Tables:
- When you drop an external table using
DROP TABLE, only the metadata (table definition) is deleted. - The underlying data files remain intact in their original location.
- This is because external tables reference data stored outside of Spark/Databricks control.
- When you drop an external table using
In this scenario:
- The table no longer appears in
SHOW TABLES→ metadata was deleted ✓ - The data files still exist → only metadata was deleted, not data files ✓
This behavior matches exactly what happens with external tables. The DROP TABLE command for external tables only removes the table definition from the metastore, leaving the actual data files untouched.
Why the other options are incorrect:
- A & B: The size of the data (10 GB threshold) is irrelevant to whether data files are deleted when dropping a table.
- D: If a table didn't have a location, it would be a managed table, and dropping it would delete both metadata and data (if any existed).
- E: If the table was managed, dropping it would delete both metadata AND data files, which is the opposite of what happened.
Key takeaway: Always be aware of whether you're working with managed or external tables, as dropping them has different consequences for data persistence.
Comments
Loading comments...