AI-900 Describe fundamental principles of machine learning on Azure Practice Test 6 — 15 Questions

Question 1

A data scientist evaluates a regression model that predicts house prices. On the test set, the Mean Absolute Error (MAE) is $8,000 and the Root Mean Squared Error (RMSE) is $25,000. What does the large difference between MAE and RMSE indicate about the model's errors?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The model has some predictions with very large errors. The large difference between MAE ($8,000) and RMSE ($25,000) indicates that the model has some predictions with very large errors. RMSE squares the errors before averaging, which heavily penalizes large deviations, so a significantly higher RMSE relative to MAE suggests the presence of outliers or extreme prediction errors in the test set.

Answer

The model is overfitting the training data

Answer

The model predictions are consistently biased high

Answer

The model has high variance due to outliers in training data

Question 2

A data scientist trains a model on historical data and achieves high accuracy on both the training set and a held-out test set. However, when the model is deployed in production, it performs poorly on new, unseen data. Which issue is most likely the cause?

Accepted Answer

Data leakage. Data leakage occurs when information from outside the training dataset is inadvertently used to train the model, causing it to learn patterns that do not generalize to new data. In this scenario, the high accuracy on both training and test sets but poor production performance indicates that the test set was contaminated with information from the future or from the target variable, making the model appear accurate during validation but fail in real-world deployment.

Answer

Overfitting

Answer

Underfitting

Answer

Concept drift

Question 3

A data scientist trains a binary classification model to detect fraudulent transactions. The dataset contains 99% legitimate transactions (negative class) and 1% fraudulent transactions (positive class). The model predicts 'legitimate' for every transaction in the test set and achieves 99% accuracy. Which metric would best reveal that the model is failing to identify any fraudulent transactions?

Accepted Answer

Recall. Recall (also known as sensitivity or true positive rate) measures the proportion of actual positive cases (fraudulent transactions) that the model correctly identifies. With 99% accuracy but zero true positives, the recall is 0%, which immediately reveals the model's complete failure to detect fraud. In Azure Machine Learning, the classification metrics pane would show recall = 0.0 for the positive class, highlighting this issue despite high accuracy.

Answer

Accuracy

Answer

Precision

Answer

F1-score

Question 4

A data scientist trains a regression model to predict house prices. The model achieves very low error on the training data but significantly higher error on a held-out test set. Which problem does this scenario best describe?

Accepted Answer

Overfitting. The scenario describes overfitting, where the model learns the training data too well, including noise and outliers, resulting in very low training error but poor generalization to new data. In Azure Machine Learning, this is often detected by comparing training and validation metrics; a large gap indicates overfitting. The correct answer is B.

Answer

Underfitting

Answer

High bias

Answer

High variance

Question 5

A data scientist has a dataset with 100 features and 10,000 samples. They want to reduce the number of features while retaining as much variance as possible, to improve model training speed and reduce overfitting. Which technique should they use?

Accepted Answer

Principal Component Analysis (PCA). Principal Component Analysis (PCA) is an unsupervised dimensionality reduction technique that transforms the original features into a new set of orthogonal components, ordered by the amount of variance they capture. By selecting only the top principal components, the data scientist can significantly reduce the feature count (e.g., from 100 to 20) while retaining the majority of the dataset's variance, which directly improves model training speed and reduces overfitting.

Answer

Feature scaling

Answer

Regularization

Answer

Cross-validation

Question 6

A data scientist trains a regression model to predict housing prices. The model uses polynomial features up to degree 5. It achieves an R-squared of 0.95 on the training set but only 0.60 on the test set. Which problem is the model most likely experiencing?

Accepted Answer

Overfitting. The model performs exceptionally well on the training data (R-squared 0.95) but poorly on the test data (R-squared 0.60), which is the classic symptom of overfitting. Using polynomial features up to degree 5 introduces high model complexity, causing the model to learn noise and specific patterns in the training set that do not generalize to unseen data.

Answer

Underfitting

Answer

Data leakage

Answer

Multicollinearity

Question 7

A data scientist is training a regression model to predict energy consumption. The dataset includes features like temperature, humidity, time of day, and day of week. After training, the model performs well on the training set but poorly on new data. Which approach would most likely help reduce this problem?

Accepted Answer

Use a simpler model with fewer parameters.. The model performs well on the training set but poorly on new data, which is classic overfitting. Using a simpler model with fewer parameters reduces the model's capacity to memorize noise and irrelevant patterns, forcing it to learn the underlying generalizable relationships. This directly addresses the variance problem without requiring additional data or computational resources.

Answer

Add more features to the model.

Answer

Increase the number of training epochs.

Answer

Use a more complex model to capture more patterns.

Question 8

A data scientist trains a binary classification model to detect spam emails. The dataset contains 95% legitimate emails (negative class) and 5% spam (positive class). The model predicts all emails as legitimate. The accuracy is 95%, but the model is useless. Which metric would best indicate the model's failure?

Accepted Answer

Recall. Recall (sensitivity) measures the proportion of actual positive cases correctly identified. With 5% spam and the model predicting all as legitimate, recall is 0% because no spam emails are detected. This directly exposes the model's failure to identify the positive class despite high accuracy.

Answer

Precision

Answer

F1 score

Answer

Specificity

Question 9

A data scientist trains a binary classification model to predict loan defaults. The dataset contains 98% non-default cases and only 2% default cases. The model predicts 'non-default' for every instance, achieving 98% accuracy on the test set. Which metric would best reveal that the model fails to identify any actual defaults?

Accepted Answer

Recall for the default class. Recall for the default class measures the proportion of actual default cases that the model correctly identifies. With the model predicting 'non-default' for every instance, recall for the default class is 0%, because it fails to capture any true positives. This directly reveals the model's inability to detect any actual defaults, despite the high overall accuracy.

Answer

Precision for the default class

Answer

F1 score for the default class

Answer

Accuracy

Question 10

A data scientist trains a multiclass classification model to identify different species of flowers (Iris setosa, Iris virginica, Iris versicolor). The overall accuracy is 94%, but the accuracy for the Iris virginica class is only 60%. Which additional metric should the data scientist examine to better understand the model's performance on the minority class?

Accepted Answer

F1-score. The F1-score is the harmonic mean of precision and recall, providing a single metric that balances both false positives and false negatives. Since the model has high overall accuracy but poor performance on the minority class (Iris virginica), the F1-score is ideal for evaluating the model's effectiveness on that class, as it accounts for class imbalance better than accuracy alone.

Answer

Precision

Answer

Recall

Answer

Mean Absolute Error (MAE)

Question 11

A data scientist has trained a binary classification model to detect fraudulent credit card transactions. The dataset contains 99.9% legitimate transactions and only 0.1% fraudulent ones. The model predicts all transactions as legitimate, achieving 99.9% accuracy on the test set. However, the business requires the model to actually catch as many fraudulent transactions as possible. Which metric would best reveal the model's failure to identify fraud?

Accepted Answer

Recall. Recall (also known as sensitivity) measures the proportion of actual positive cases (fraudulent transactions) that were correctly identified by the model. In this scenario, the model predicts all transactions as legitimate, so it correctly identifies 0 out of the 0.1% fraudulent transactions, yielding a recall of 0%. This directly reveals the model's complete failure to catch fraud, despite the high accuracy.

Answer

Accuracy

Answer

Precision

Answer

F1 score

Question 12

A data scientist trains a model to predict house prices using features like number of bedrooms, square footage, and location. The model achieves a mean absolute error (MAE) of $5,000 on the training data but $25,000 on the test data. Which problem is the model most likely experiencing?

Accepted Answer

Overfitting. The model performs well on training data (MAE $5,000) but poorly on test data (MAE $25,000), which is the classic symptom of overfitting. Overfitting occurs when the model learns noise and specific patterns in the training data too well, failing to generalize to unseen data. In Azure Machine Learning, this can be detected by comparing training vs. validation metrics and is often mitigated using regularization techniques or simplifying the model.

Answer

Underfitting

Answer

Multicollinearity

Answer

Class imbalance

Question 13

A data scientist trains a binary classification model to detect fraudulent credit card transactions. The dataset contains 99.5% legitimate transactions and 0.5% fraudulent transactions. The model predicts every transaction as legitimate and achieves 99.5% accuracy on the test set. Which metric would best reveal that the model is failing to identify any fraudulent transactions?

Accepted Answer

Recall. Recall (also known as sensitivity) measures the proportion of actual positive cases correctly identified by the model. In this scenario, the model predicts all transactions as legitimate, so it correctly identifies zero fraudulent transactions, giving a recall of 0%. Accuracy alone is misleading because the dataset is highly imbalanced (99.5% legitimate, 0.5% fraudulent), and a 99.5% accuracy can be achieved by simply predicting the majority class. Recall directly reveals the model's failure to detect any fraud.

Answer

Precision

Answer

F1 score

Answer

Mean Absolute Error (MAE)

Question 14

A data science team trains a regression model to predict house prices. They evaluate the model using Mean Absolute Error (MAE). After deployment, they notice that the model occasionally produces large errors (e.g., underpredicting a luxury home by $500,000) while most predictions are within $20,000. The business is more concerned about the impact of these large errors than the average small error. Which additional metric should the team use to better capture the penalty for large errors?

Accepted Answer

Root Mean Squared Error (RMSE). Root Mean Squared Error (RMSE) is the correct additional metric because it squares the residuals before averaging, which heavily penalizes large errors like the $500,000 underprediction. Unlike MAE, which treats all errors equally, RMSE amplifies the impact of outliers, making it a better fit for a business that cares more about catastrophic failures than typical small errors. This aligns with the need to capture the penalty for large deviations in regression model evaluation.

Answer

R-squared

Answer

Mean Absolute Percentage Error (MAPE)

Answer

F1 score

Question 15

A bank uses a machine learning model to predict credit card fraud. The model's output is a probability score. The business wants to minimize the number of false positives (legitimate transactions incorrectly flagged as fraud) because these cause customer dissatisfaction. At the same time, they must also catch most fraudulent transactions. Which metric should the bank optimize to balance these two goals?

Accepted Answer

D: F1 score. The F1 score is the harmonic mean of precision and recall, making it the ideal metric when a balance between minimizing false positives (precision) and catching most fraudulent transactions (recall) is required. In this credit card fraud detection scenario, optimizing F1 ensures the model reduces customer dissatisfaction from false positives while still maintaining high detection of actual fraud.

Answer

A: Accuracy

Answer

B: Precision

Answer

C: Recall