Databricks Certified Data Engineer - Associate

Explanation

In Databricks/Spark SQL, there are two types of tables:

1. Managed Tables (also called internal tables):

   • The data files are stored in the default warehouse directory managed by Spark/Databricks
   • When you drop a managed table using DROP TABLE, both the metadata (table definition) AND the underlying data files are deleted
   • Spark/Databricks has full control over the data lifecycle

2. External Tables (also called unmanaged tables):

   • The data files are stored in an external location (like S3, ADLS, etc.) that you specify
   • When you drop an external table using DROP TABLE, only the metadata (table definition) is deleted
   • The underlying data files remain intact in the external location

In this scenario, since both the metadata files AND data files were deleted from the file system after running DROP TABLE IF EXISTS my_table;, this indicates that my_table was a managed table.

Key points:

• Option A is correct because managed tables have their data lifecycle controlled by Spark/Databricks
• Option D (external table) is incorrect because dropping an external table would only delete metadata, not the data files
• Options B and C are incorrect because table size doesn't affect whether data gets deleted when dropping a table
• Option E is incorrect because tables without a specified location are typically managed tables, but the key distinction is whether it's managed vs external, not just the presence of a location specification

Additional context:

• Managed tables are created without a LOCATION clause or with data stored in the default warehouse directory
• External tables are created with a LOCATION clause pointing to external storage
• The IF EXISTS clause simply prevents an error if the table doesn't exist, but doesn't affect whether data gets deleted