Answer-first summary for fast verification
Answer: Scikit-learn
In Spark, a User-Defined Function (UDF) enables the integration of custom functions into a Spark DataFrame transformation pipeline. While libraries such as Spark MLLib are optimized for Spark and do not require a UDF for distributed model inference, others like Scikit-learn, which are not inherently designed for distributed computing, will necessitate a UDF to distribute the model inference process.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
No comments yet.
A data science team is restructuring their machine learning projects to facilitate model inference distribution. They are categorizing projects based on the modeling library used to identify which ones will need a User-Defined Function (UDF) for distributing the inference process. Which of the following modeling libraries would require a UDF for this purpose?
A
Scikit-learn
B
All of the options
C
MLLib
D
None of the options