A company is using Amazon SageMaker to deploy a machine learning model that predicts equipment failure. The model is a binary classifier that outputs a probability. The company wants to set a threshold such that the model correctly identifies 95% of actual failures (recall = 0.95). The model's precision at the current threshold of 0.5 is 0.7. The data scientist evaluates the model on a test set and obtains the following confusion matrix at threshold 0.5: TP=95, FN=5, FP=40, TN=860. The total actual positives are 100. Which threshold adjustment should the data scientist make to achieve the recall goal?

Question

A company is using Amazon SageMaker to deploy a machine learning model that predicts equipment failure. The model is a binary classifier that outputs a probability. The company wants to set a threshold such that the model correctly identifies 95% of actual failures (recall >= 0.95). The model's precision at the current threshold of 0.5 is 0.7. The data scientist evaluates the model on a test set and obtains the following confusion matrix at threshold 0.5: TP=95, FN=5, FP=40, TN=860. The total actual positives are 100. Which threshold adjustment should the data scientist make to achieve the recall goal?

Accepted Answer

Keep the threshold at 0.5. At a threshold of 0.5, recall is TP/(TP+FN) = 95/(95+5) = 0.95, which already meets the goal of recall ≥ 0.95. No adjustment to the threshold is necessary. Decreasing the threshold (e.g., to 0.3 or 0.1) would increase recall but also increase false positives, which is not required. Increasing the threshold (e.g., to 0.7) would reduce recall, missing the goal. Therefore, the correct choice is to keep the threshold at 0.5 (Option C).

Answer

Decrease the threshold to 0.1

Answer

Increase the threshold to 0.7

Answer

Decrease the threshold to 0.3

MLS-C01 Modeling Practice Question

Why each option matters

Option-by-option breakdown

Related to this question