Databricks Certified Data Engineer - Professional
Get started today
Ultimate access to all questions.
Comments
Loading comments...
Ultimate access to all questions.
When creating a Spark SQL UDF in Python for complex string manipulation, which approach best ensures effective error handling and keeps the pipeline running smoothly even with unexpected input formats?
A
Encapsulating the UDF logic in an external service and calling it via REST API from Spark.
B
Using try-except blocks within the UDF to catch and log errors, returning a default value in case of exceptions.
C
Avoiding UDFs altogether in favor of built-in Spark SQL functions for reliability.
D
Implementing UDFs exclusively in Scala for better error handling capabilities.