Databricks Certified Machine Learning - Associate
Get started today
Ultimate access to all questions.
Comments
Loading comments...
Ultimate access to all questions.
In the context of hyperparameter tuning using Spark ML, explain the process of selecting the best hyperparameters for a machine learning model. Provide a code snippet demonstrating the use of Spark ML's TrainValidationSplit or CrossValidator for hyperparameter tuning and explain the key considerations to keep in mind during this process.
A
Use the TrainValidationSplit class from the pyspark.ml.tuning module to perform hyperparameter tuning by splitting the data into training and validation sets, and selecting the best hyperparameters based on the model's performance on the validation set.
B
Use the CrossValidator class from the pyspark.ml.tuning module to perform hyperparameter tuning by performing k-fold cross-validation, and selecting the best hyperparameters based on the average performance across all folds.
C
Use the RandomForestRegressor class from the pyspark.ml.regression module with default hyperparameters and train the model without performing hyperparameter tuning.
D
Use the LogisticRegression class from the pyspark.ml.classification module with default hyperparameters and train the model without performing hyperparameter tuning.