
Ultimate access to all questions.
Consider a scenario where you have a dataset with 10 features and you want to train a machine learning model using Spark ML. Describe the steps involved in training and evaluating the model, and explain the role of Spark ML estimator and Spark ML transformer in this process. Provide a code snippet demonstrating the training and evaluation of a model using Spark ML.
A
Use the fit method of the Spark ML estimator to train the model on the training data, and then use the transform method of the Spark ML transformer to make predictions on the test data.
B
Use the fit method of the Spark ML estimator to train the model on the entire dataset, and then use the predict method of the Spark ML model to make predictions on the test data.
C
Use the fit method of the Spark ML estimator to train the model on the training data, and then use the evaluate method of the Spark ML model to evaluate the model's performance on the test data.
D
Use the fitAndTransform method of the Spark ML estimator to train the model and make predictions on the entire dataset, and then use the evaluate method of the Spark ML model to evaluate the model's performance.