AI-900 Describe fundamental principles of machine learning on Azure Practice Test 6 — 15 Questions

Question 1

A data scientist trains a regression model to predict housing prices. The model uses polynomial features up to degree 5. It achieves an R-squared of 0.95 on the training set but only 0.60 on the test set. Which problem is the model most likely experiencing?

Accepted Answer

Overfitting. The model performs exceptionally well on the training data (R-squared 0.95) but poorly on the test data (R-squared 0.60), which is the classic symptom of overfitting. Using polynomial features up to degree 5 introduces high model complexity, causing the model to learn noise and specific patterns in the training set that do not generalize to unseen data.

Answer

Underfitting

Answer

Data leakage

Answer

Multicollinearity

Question 2

A data scientist is training a regression model to predict energy consumption. The dataset includes features like temperature, humidity, time of day, and day of week. After training, the model performs well on the training set but poorly on new data. Which approach would most likely help reduce this problem?

Accepted Answer

Use a simpler model with fewer parameters.. The model performs well on the training set but poorly on new data, which is classic overfitting. Using a simpler model with fewer parameters reduces the model's capacity to memorize noise and irrelevant patterns, forcing it to learn the underlying generalizable relationships. This directly addresses the variance problem without requiring additional data or computational resources.

Answer

Add more features to the model.

Answer

Increase the number of training epochs.

Answer

Use a more complex model to capture more patterns.

Question 3

A data scientist trains a binary classification model to detect spam emails. The dataset contains 95% legitimate emails (negative class) and 5% spam (positive class). The model predicts all emails as legitimate. The accuracy is 95%, but the model is useless. Which metric would best indicate the model's failure?

Accepted Answer

Recall. Recall (sensitivity) measures the proportion of actual positive cases correctly identified. With 5% spam and the model predicting all as legitimate, recall is 0% because no spam emails are detected. This directly exposes the model's failure to identify the positive class despite high accuracy.

Answer

Precision

Answer

F1 score

Answer

Specificity

Question 4

A data scientist trains a binary classification model to predict loan defaults. The dataset contains 98% non-default cases and only 2% default cases. The model predicts 'non-default' for every instance, achieving 98% accuracy on the test set. Which metric would best reveal that the model fails to identify any actual defaults?

Accepted Answer

Recall for the default class. Recall for the default class measures the proportion of actual default cases that the model correctly identifies. With the model predicting 'non-default' for every instance, recall for the default class is 0%, because it fails to capture any true positives. This directly reveals the model's inability to detect any actual defaults, despite the high overall accuracy.

Answer

Precision for the default class

Answer

F1 score for the default class

Answer

Accuracy

Question 5

A data scientist trains a multiclass classification model to identify different species of flowers (Iris setosa, Iris virginica, Iris versicolor). The overall accuracy is 94%, but the accuracy for the Iris virginica class is only 60%. Which additional metric should the data scientist examine to better understand the model's performance on the minority class?

Accepted Answer

F1-score. The F1-score is the harmonic mean of precision and recall, providing a single metric that balances both false positives and false negatives. Since the model has high overall accuracy but poor performance on the minority class (Iris virginica), the F1-score is ideal for evaluating the model's effectiveness on that class, as it accounts for class imbalance better than accuracy alone.

Answer

Precision

Answer

Recall

Answer

Mean Absolute Error (MAE)

Question 6

A data scientist trains a multiclass classification model to categorize customer support tickets into three types: 'Billing', 'Technical', and 'General'. The dataset contains 80% 'General', 15% 'Billing', and only 5% 'Technical' tickets. Overall accuracy on a test set is 85%, but the model misclassifies most 'Technical' tickets as 'General'. Which metric would best help the data scientist understand the model's poor performance on the 'Technical' class?

Accepted Answer

F1-score for the 'Technical' class. The F1-score for the 'Technical' class is the best metric because it combines precision and recall into a single harmonic mean, directly capturing the model's inability to correctly identify the minority class. Since the dataset is heavily imbalanced (only 5% 'Technical'), overall accuracy (85%) is misleadingly high, as the model can achieve it by simply predicting the majority class 'General'. The F1-score penalizes both false positives and false negatives, making it the standard metric for evaluating classifier performance on imbalanced classes.

Answer

Overall accuracy

Answer

Confusion matrix

Answer

Precision for the 'General' class

Question 7

A data scientist has trained a binary classification model to detect fraudulent credit card transactions. The dataset contains 99.9% legitimate transactions and only 0.1% fraudulent ones. The model predicts all transactions as legitimate, achieving 99.9% accuracy on the test set. However, the business requires the model to actually catch as many fraudulent transactions as possible. Which metric would best reveal the model's failure to identify fraud?

Accepted Answer

Recall. Recall (also known as sensitivity) measures the proportion of actual positive cases (fraudulent transactions) that were correctly identified by the model. In this scenario, the model predicts all transactions as legitimate, so it correctly identifies 0 out of the 0.1% fraudulent transactions, yielding a recall of 0%. This directly reveals the model's complete failure to catch fraud, despite the high accuracy.

Answer

Accuracy

Answer

Precision

Answer

F1 score

Question 8

A data scientist trains a model to predict house prices using features like number of bedrooms, square footage, and location. The model achieves a mean absolute error (MAE) of $5,000 on the training data but $25,000 on the test data. Which problem is the model most likely experiencing?

Accepted Answer

Overfitting. The model performs well on training data (MAE $5,000) but poorly on test data (MAE $25,000), which is the classic symptom of overfitting. Overfitting occurs when the model learns noise and specific patterns in the training data too well, failing to generalize to unseen data. In Azure Machine Learning, this can be detected by comparing training vs. validation metrics and is often mitigated using regularization techniques or simplifying the model.

Answer

Underfitting

Answer

Multicollinearity

Answer

Class imbalance

Question 9

A data scientist trains a binary classification model to detect fraudulent credit card transactions. The dataset contains 99.5% legitimate transactions and 0.5% fraudulent transactions. The model predicts every transaction as legitimate and achieves 99.5% accuracy on the test set. Which metric would best reveal that the model is failing to identify any fraudulent transactions?

Accepted Answer

Recall. Recall (also known as sensitivity) measures the proportion of actual positive cases correctly identified by the model. In this scenario, the model predicts all transactions as legitimate, so it correctly identifies zero fraudulent transactions, giving a recall of 0%. Accuracy alone is misleading because the dataset is highly imbalanced (99.5% legitimate, 0.5% fraudulent), and a 99.5% accuracy can be achieved by simply predicting the majority class. Recall directly reveals the model's failure to detect any fraud.

Answer

Precision

Answer

F1 score

Answer

Mean Absolute Error (MAE)

Question 10

A data science team trains a regression model to predict house prices. They evaluate the model using Mean Absolute Error (MAE). After deployment, they notice that the model occasionally produces large errors (e.g., underpredicting a luxury home by $500,000) while most predictions are within $20,000. The business is more concerned about the impact of these large errors than the average small error. Which additional metric should the team use to better capture the penalty for large errors?

Accepted Answer

Root Mean Squared Error (RMSE). Root Mean Squared Error (RMSE) is the correct additional metric because it squares the residuals before averaging, which heavily penalizes large errors like the $500,000 underprediction. Unlike MAE, which treats all errors equally, RMSE amplifies the impact of outliers, making it a better fit for a business that cares more about catastrophic failures than typical small errors. This aligns with the need to capture the penalty for large deviations in regression model evaluation.

Answer

R-squared

Answer

Mean Absolute Percentage Error (MAPE)

Answer

F1 score

Question 11

A bank uses a machine learning model to predict credit card fraud. The model's output is a probability score. The business wants to minimize the number of false positives (legitimate transactions incorrectly flagged as fraud) because these cause customer dissatisfaction. At the same time, they must also catch most fraudulent transactions. Which metric should the bank optimize to balance these two goals?

Accepted Answer

D: F1 score. The F1 score is the harmonic mean of precision and recall, making it the ideal metric when a balance between minimizing false positives (precision) and catching most fraudulent transactions (recall) is required. In this credit card fraud detection scenario, optimizing F1 ensures the model reduces customer dissatisfaction from false positives while still maintaining high detection of actual fraud.

Answer

A: Accuracy

Answer

B: Precision

Answer

C: Recall

Question 12

A data scientist trains a regression model to predict energy consumption for a smart building. The model achieves very low error on the training data but performs significantly worse on a held-out validation set. Which technique would most directly address this problem?

Accepted Answer

Regularization. The model's low training error but high validation error indicates overfitting, where the model has memorized the training data rather than learning generalizable patterns. Regularization (e.g., L1 or L2) directly penalizes large coefficients, reducing model complexity and improving generalization to unseen data.

Answer

Feature engineering

Answer

Cross-validation

Answer

Hyperparameter tuning

Question 13

A data scientist trains a binary classification model to detect a rare disease. The dataset contains 99% negative cases and only 1% positive cases. The model predicts all cases as negative, achieving an accuracy of 99% on the test set. However, the business requires the model to identify as many positive cases as possible. Which metric should the data scientist examine to best reveal that the model is failing to identify any positive cases?

Accepted Answer

Recall. Recall (sensitivity) measures the proportion of actual positive cases correctly identified by the model. With all predictions as negative, recall is 0%, directly revealing the model's failure to detect any positive cases despite the high accuracy.

Answer

Precision

Answer

F1 score

Answer

AUC-ROC

Question 14

An e-commerce company has a dataset of customer purchase histories with no predefined categories. The data analyst wants to identify natural groupings of customers based on their purchasing behavior to target marketing campaigns. Which type of machine learning should the analyst use?

Accepted Answer

Clustering. Clustering is the correct choice because it is an unsupervised learning technique used to discover inherent groupings in data without predefined labels. In this scenario, the analyst wants to identify natural customer segments based on purchase behavior, which aligns perfectly with clustering algorithms like K-Means or DBSCAN that partition data into clusters of similar patterns.

Answer

Regression

Answer

Classification

Answer

Reinforcement learning

Question 15

A data scientist trains a regression model to predict house prices using features like bedrooms, square footage, and location. The model achieves an R-squared of 0.95 on the test set. However, when deployed to predict prices in a new city with different property characteristics, the predictions are very inaccurate. Which concept best explains this poor performance?

Accepted Answer

Overfitting. The model achieved an R-squared of 0.95 on the test set, indicating excellent performance on data from the same distribution. However, when deployed to a new city with different property characteristics, the predictions were very inaccurate. This is a classic symptom of overfitting, where the model has learned noise and patterns specific to the training data (e.g., city-specific price trends) that do not generalize to unseen data from a different distribution.

Answer

Underfitting

Answer

High bias

Answer

Data drift