AI-900 Practice Test 10 — 10 Questions

Question 1

A data scientist trains a machine learning model to predict house prices based on features like square footage, number of bedrooms, and location. The model achieves a very low error on the training data but performs poorly on a held-out test set. Which term best describes this situation?

Accepted Answer

Overfitting. The model performs exceptionally well on training data but poorly on test data, which is the classic symptom of overfitting. Overfitting occurs when the model learns noise and specific patterns in the training set rather than generalizing to unseen data. In Azure Machine Learning, this can be detected by monitoring the gap between training and validation metrics, and mitigated using techniques like regularization or early stopping.

Answer

Underfitting

Answer

High bias

Answer

High variance

Question 2

A data scientist is building a classification model to detect fraudulent transactions. The dataset has 1,000,000 legitimate transactions and only 1,000 fraudulent ones. The model achieves 99.9% accuracy on the test set, but it fails to catch most fraudulent cases. Which metric should the data scientist prioritize to better evaluate the model's performance on this imbalanced dataset?

Accepted Answer

Recall. Recall measures the proportion of actual positive cases (fraudulent transactions) correctly identified by the model. With only 1,000 fraud cases out of 1,001,000 total transactions, a model that predicts 'legitimate' for every transaction would achieve 99.9% accuracy but 0% recall, making recall the critical metric for imbalanced fraud detection.

Answer

Accuracy

Answer

Mean Squared Error

Answer

R-squared

Question 3

A data scientist trains a binary classification model to detect a rare disease. The dataset contains 99% negative cases and only 1% positive cases. The model predicts all cases as negative, achieving an accuracy of 99% on the test set. However, the business requires the model to identify as many positive cases as possible. Which metric should the data scientist examine to best reveal that the model is failing to identify any positive cases?

Accepted Answer

Recall. Recall (sensitivity) measures the proportion of actual positive cases correctly identified by the model. With all predictions as negative, recall is 0%, directly revealing the model's failure to detect any positive cases despite the high accuracy.

Answer

Precision

Answer

F1 score

Answer

AUC-ROC

Question 4

A data scientist is evaluating a binary classification model that predicts whether a transaction is fraudulent. The test set contains 1,000 transactions: 990 legitimate and 10 fraudulent. The model's predictions are shown in the confusion matrix below. Confusion matrix: Predicted Legitimate Predicted Fraudulent Actual Legitimate 942 48 Actual Fraudulent 2 8 Which metric should the data scientist prioritize if the business goal is to minimize the number of fraudulent transactions that are missed (false negatives)?

Accepted Answer

Recall. Recall (sensitivity) measures the proportion of actual positives correctly identified, calculated as TP/(TP+FN). With 2 false negatives (missed fraudulent transactions), recall is 8/(8+2)=0.80. Minimizing missed fraud directly corresponds to maximizing recall, making it the correct priority for this business goal.

Answer

Precision

Answer

Accuracy

Answer

Specificity

Question 5

A data scientist trains a linear regression model to predict house prices. The model's training error is very high, and its test error is nearly as high. Which term best describes this situation?

Accepted Answer

Underfitting. Underfitting occurs when a model is too simple to capture the underlying patterns in the data, resulting in high training error and similarly high test error. In this linear regression scenario, the model fails to learn the relationship between features and house prices, leading to poor performance on both training and test sets.

Answer

Overfitting

Answer

High bias

Answer

High variance

Question 6

A data scientist trains a classification model to predict whether an email is 'phishing' or 'legitimate'. The model achieves 99% accuracy on the training data but only 68% accuracy on the test data. Which action is most likely to help improve the model's generalization performance?

Accepted Answer

Apply regularization techniques such as L1 or L2 regularization.. The model's high training accuracy (99%) paired with much lower test accuracy (68%) is a classic sign of overfitting, where the model has memorized the training data rather than learning generalizable patterns. Regularization techniques like L1 (Lasso) or L2 (Ridge) add a penalty to the loss function that discourages overly complex models by shrinking the weights of less important features, directly reducing overfitting and improving generalization on unseen data.

Answer

Increase the number of training epochs significantly.

Answer

Remove some of the training data to make the dataset smaller.

Answer

Add more layers and neurons to the neural network.

Question 7

A data scientist trains a regression model to predict daily electricity consumption (in kWh) for a commercial building. The business team needs a metric that heavily penalizes large prediction errors (outliers) more than small errors. Which metric should the data scientist report to best meet this requirement?

Accepted Answer

Root Mean Squared Error (RMSE). Root Mean Squared Error (RMSE) is the correct metric because it squares the residuals before averaging, which disproportionately amplifies the impact of large errors (outliers) compared to small errors. This aligns directly with the business requirement to heavily penalize large prediction errors in the regression model for daily electricity consumption.

Answer

Mean Absolute Error (MAE)

Answer

R-squared

Answer

Mean Absolute Percentage Error (MAPE)

Question 8

A data scientist is training a credit risk model and wants to use Azure Machine Learning's Responsible AI dashboard to identify if the model is biased against a certain demographic group. Which component of the dashboard should they use to evaluate this?

Accepted Answer

Model Fairness Assessment. The Model Fairness Assessment component of Azure Machine Learning's Responsible AI dashboard is specifically designed to evaluate and mitigate bias in machine learning models. It allows data scientists to assess disparities in model performance across demographic groups defined by sensitive features (e.g., race, gender) using metrics like demographic parity, equal opportunity, and disparate impact. This directly addresses the question of identifying bias against a certain demographic group.

Answer

Model Interpretability

Answer

Error Analysis

Answer

Data Balance Analysis

Question 9

A data scientist trains a binary classification model to detect fraudulent transactions. The dataset contains only 1% fraudulent cases. The model predicts 'not fraudulent' for all transactions and achieves 99% accuracy. Which metric would best reveal the model's poor performance on fraud detection?

Accepted Answer

Recall. Recall (sensitivity) measures the proportion of actual positive cases (fraudulent transactions) correctly identified by the model. With 1% fraud, a model that predicts 'not fraudulent' for all transactions will have a recall of 0% because it fails to catch any true positives, despite 99% accuracy. This makes recall the best metric to reveal the model's inability to detect fraud.

Answer

Precision

Answer

F1 score

Answer

Accuracy

Question 10

A data scientist trains a classification model on a dataset of 10,000 labeled emails to distinguish spam from non-spam. The model achieves 99% accuracy on the training data but only 70% accuracy on a held-out test set. Which term best describes this situation?

Accepted Answer

B) Overfitting. The model performs exceptionally well on training data (99% accuracy) but poorly on unseen test data (70% accuracy), which is the classic symptom of overfitting. Overfitting occurs when the model learns noise and specific patterns in the training set rather than generalizing to new data, often due to excessive model complexity or insufficient regularization.

Answer

A) Underfitting

Answer

C) Bias-variance tradeoff

Answer

D) Regularization