MLS-C01 Modeling Practice Question
A data scientist is building a recommender system using Amazon SageMaker. The dataset contains user-item interactions with implicit feedback (clicks). Which THREE evaluation metrics are appropriate for this use case?
⚠ Common exam trap
Many exam-takers confuse regression metrics (RMSE) or binary classification metrics (AUC-ROC) as applicable to implicit feedback, not realizing that recommender systems with implicit feedback require ranking-based metrics that handle only positive observations and no explicit negative labels.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Precision@k
For implicit feedback (e.g., clicks) in recommender systems, evaluation metrics should focus on ranking quality without requiring explicit negative labels. Precision@k measures the proportion of relevant items among top-k recommendations, Recall@k measures the proportion of relevant items retrieved in top-k, and Mean Average Precision (MAP) averages precision across multiple recall levels, all suitable for top-k recommendation tasks. RMSE is for explicit rating prediction and requires ground truth ratings, not applicable to implicit feedback. AUC-ROC is for binary classification with both positive and negative samples, but implicit feedback lacks explicit negatives, making it unsuitable.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Root Mean Squared Error (RMSE)
Why it's wrong here
RMSE is for explicit ratings, not implicit feedback.
- ✓
Precision@k
Why this is correct
Precision@k measures relevance of top-k recommendations.
- ✓
Mean Average Precision (MAP)
Why this is correct
MAP summarizes precision across different recall levels.
- ✓
Recall@k
Why this is correct
Recall@k measures coverage of relevant items in top-k.
- ✗
Area Under the ROC Curve (AUC-ROC)
Why it's wrong here
AUC-ROC is for binary classification, not ranking.
Go deeper
Related to this question
About these practice questions
One of 1,672 original MLS-C01 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.