Databricks Certified Data Engineer - Associate
Get started today
Ultimate access to all questions.
A data engineer is attempting to drop a Spark SQL table my_table. The data engineer wants to delete all table metadata and data. They run the following command: DROP TABLE IF EXISTS my_table - While the object no longer appears when they run SHOW TABLES, the data files still exist. Which of the following describes why the data files still exist and the metadata files were deleted?
Real Exam
Community
KKeng
A
The table's data was larger than 10 GB
B
The table's data was smaller than 10 GB
C
The table was external
D
The table did not have a location
E
The table was managed
Explanation:
Explanation
In Databricks/Spark SQL, there are two types of tables:
-
Managed Tables: When you drop a managed table, both the metadata (table definition) AND the underlying data files are deleted. The data is stored in the default location managed by Spark/Databricks.
-
External Tables: When you drop an external table, only the metadata (table definition) is deleted. The underlying data files remain intact in their original location. This is because external tables reference data stored outside of Spark's control.
In this scenario:
- The table no longer appears in
SHOW TABLES→ metadata was deleted ✓ - The data files still exist → table was external ✓
Why other options are incorrect:
- A & B: The size of the data (10 GB threshold) doesn't affect whether data files are deleted when dropping a table.
- D: If a table didn't have a location, it wouldn't have data files to begin with.
- E: If the table was managed, dropping it would delete both metadata AND data files.
Additional Note: To completely remove data from an external table, you need to:
- First drop the table:
DROP TABLE IF EXISTS my_table - Then manually delete the data files from the external location (e.g., using
dbutils.fs.rm()or directly from the storage system).
Comments
Loading comments...