Ultimate access to all questions.
You are a Machine Learning Engineer at a company that deployed a predictive model into production a year ago. To monitor the model's performance, a subset of raw requests sent to the model prediction service is evaluated monthly by a human labeling service. Over the past year, it has been observed that the model's performance can significantly drop after just one month or may take several months to decline. Given the high cost associated with the human labeling service, the company is looking for a cost-effective strategy to maintain the model's high performance. Which of the following strategies would BEST address this need by determining the optimal frequency for model retraining? (Choose two options if option E is available, otherwise choose one.)
A
Analyze temporal patterns in your model’s performance over the previous year. Utilize these patterns to create a schedule for sending serving data to the labeling service for the next year.
B
Compare the cost of the labeling service with the revenue lost due to model performance degradation over the past year. Adjust the frequency of model retraining based on whether the lost revenue exceeds the labeling cost.
C
Train an anomaly detection model on the training dataset, and use it to analyze all incoming requests. Send the most recent serving data to the labeling service if an anomaly is detected.
D
Execute training-serving skew detection batch jobs every few days to compare the aggregate statistics of the features in the training dataset with recent serving data. If skew is detected, send the most recent serving data to the labeling service.
E
Implement a combination of analyzing temporal patterns in model performance and executing training-serving skew detection batch jobs to dynamically adjust the frequency of sending data to the labeling service.