Databricks Certified Data Engineer - Associate

Explanation

In Databricks/Spark SQL, there are two types of tables:

1. Managed Tables: These are tables where Spark manages both the metadata (in the metastore) and the data files. When you drop a managed table using DROP TABLE, Spark deletes both the metadata from the metastore and the underlying data files from the storage.

2. External Tables: These are tables where Spark only manages the metadata in the metastore, while the data files are stored in an external location (like ADLS, S3, etc.) that you specify. When you drop an external table, Spark only removes the metadata from the metastore, but does not delete the underlying data files.

In this scenario:

• The engineer ran DROP TABLE IF EXISTS my_table;
• Both data files and metadata files were deleted
• This behavior indicates that my_table was a managed table

Key points:

• Option A is correct because managed tables have their data deleted when dropped
• Option D is incorrect because external tables preserve data files when dropped
• Options B and C are irrelevant to the deletion behavior
• The IF EXISTS clause just prevents errors if the table doesn't exist, but doesn't affect the deletion behavior

Best Practice: When working with important data that should persist beyond table operations, use external tables with explicit location specifications.