Answer-first summary for fast verification
Answer: ROC-AUC, Log Loss
The ROC-AUC (Receiver Operating Characteristic - Area Under the Curve) is the optimal choice for evaluating the model's ability to distinguish between the two classes because it is both scale invariant and classification threshold invariant. It provides a comprehensive view of the model's performance across all classification thresholds by plotting the True Positive Rate (TPR) against the False Positive Rate (FPR). A higher AUC value indicates better model performance. Log Loss is also a viable option as it measures the uncertainty of the predictions based on how much they vary from the actual labels, making it useful for probabilistic models like logistic regression. However, it is more sensitive to the classification threshold than ROC-AUC. Mean Absolute Error (A) and Mean Square Error (E) are incorrect because they are loss functions primarily used in regression tasks, not classification. One-hot encoding (C) is a feature engineering technique, not an evaluation metric.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
No comments yet.
As a junior Data Scientist at a rapidly growing e-commerce company, you're tasked with developing a logistic regression model to categorize customer support text messages into 'urgent/important' and 'non-urgent/unimportant'. The model's performance is critical for prioritizing customer issues efficiently. You need to select an evaluation metric that not only assesses the model's ability to distinguish between these two classes but also adheres to the following requirements: it must be scale invariant and classification threshold invariant. Additionally, the company emphasizes the importance of minimizing false positives to avoid misallocating resources. Given these constraints, which of the following methodologies should you choose? (Choose two correct options if E is available, otherwise choose one.)
A
Mean Absolute Error
B
ROC-AUC
C
One-hot encoding
D
Log Loss
E
Mean Square Error