Databricks Certified Data Engineer - Associate
Get started today
Ultimate access to all questions.
Ultimate access to all questions.
In a data engineering project, you are tasked with validating email addresses stored in a 'user_email' column of a dataset to ensure none contain the '@' symbol, as part of a data quality check. The dataset is large and stored in a Delta Lake table. Considering the need for efficiency and accuracy in Spark SQL, which of the following queries would you use to correctly identify all rows where the 'user_email' column does not contain the '@' symbol? Choose the best option.
Explanation:
Option A is the correct answer because it efficiently uses the NOT LIKE operator to filter out rows where the 'user_email' column contains the '@' symbol, which is the standard and most efficient way to perform such a check in Spark SQL. Option B is incorrect because the CONTAINS operator is not valid syntax in Spark SQL for this purpose. Option C is incorrect because the EXCEPT clause is not the right approach for filtering rows based on a condition. Option D is incorrect because the POSITION function with the IN operator is not the correct syntax for checking the presence of a character in a string in Spark SQL.