AI-900 Describe fundamental principles of machine learning on Azure Practice Test 5 — 15 Questions

Question 1

A data scientist trains a machine learning model to predict house prices based on features like square footage, number of bedrooms, and location. The model achieves a very low error on the training data but performs poorly on a held-out test set. Which term best describes this situation?

Accepted Answer

Overfitting. The model performs exceptionally well on training data but poorly on test data, which is the classic symptom of overfitting. Overfitting occurs when the model learns noise and specific patterns in the training set rather than generalizing to unseen data. In Azure Machine Learning, this can be detected by monitoring the gap between training and validation metrics, and mitigated using techniques like regularization or early stopping.

Answer

Underfitting

Answer

High bias

Answer

High variance

Question 2

A retail company has historical data about customers, including age, purchase history, and whether they have churned (yes/no). They want to train a model that predicts if a new customer will churn. Which type of machine learning should they use?

Accepted Answer

Supervised classification. The goal is to predict a categorical outcome (churn: yes/no) from historical labeled data. Supervised classification algorithms, such as logistic regression or decision trees, learn from input features (age, purchase history) and the target label (churn status) to assign new customers to one of the discrete classes. This directly matches the requirement for a binary classification model.

Answer

Supervised regression

Answer

Unsupervised clustering

Answer

Reinforcement learning

Question 3

A medical research team trains a model to detect a rare disease from lab results. The disease occurs in only 1% of patients. The model predicts 'no disease' for every patient and achieves 99% accuracy. Which metric best reveals that the model is failing to identify actual disease cases?

Accepted Answer

Recall. Recall (sensitivity) measures the proportion of actual positive cases correctly identified by the model. With a 99% accuracy but zero true positives (since the model always predicts 'no disease'), recall is 0%, which directly reveals the model's failure to detect any actual disease cases. In Azure Machine Learning, recall is a key metric for imbalanced classification tasks, especially when missing a positive case has severe consequences.

Answer

Accuracy

Answer

Precision

Answer

F1 score

Question 4

A manufacturer trains a model to detect defective parts on an assembly line. Only 2% of parts are defective. The model predicts 'non-defective' for all parts and achieves 98% accuracy. Which metric best reveals the model's inability to identify defective parts?

Accepted Answer

Recall. Recall (sensitivity) measures the proportion of actual defective parts correctly identified by the model. With 98% accuracy but zero true positives (since the model labels everything as non-defective), recall is 0%, which directly exposes the model's failure to detect any defective parts despite high accuracy.

Answer

Accuracy

Answer

Precision

Answer

F1 Score

Question 5

A data scientist trains a classification model on a dataset of 10,000 labeled emails to distinguish spam from non-spam. The model achieves 99% accuracy on the training data but only 70% accuracy on a held-out test set. Which term best describes this situation?

Accepted Answer

B) Overfitting. The model performs exceptionally well on training data (99% accuracy) but poorly on unseen test data (70% accuracy), which is the classic symptom of overfitting. Overfitting occurs when the model learns noise and specific patterns in the training set rather than generalizing to new data, often due to excessive model complexity or insufficient regularization.

Answer

A) Underfitting

Answer

C) Bias-variance tradeoff

Answer

D) Regularization

Question 6

A real estate company has a dataset containing square footage, number of bedrooms, and location for 10,000 houses, along with their sale prices. They want to train a model that predicts the sale price of a new house based on these features. Which type of machine learning should they use?

Accepted Answer

Supervised regression. The goal is to predict a continuous numeric value (sale price) from input features (square footage, bedrooms, location). This is a classic supervised regression problem because the training data includes labeled target values (prices) and the output is a real number, not a category.

Answer

Supervised classification

Answer

Unsupervised clustering

Answer

Reinforcement learning

Question 7

A data scientist is evaluating a binary classification model that predicts whether a transaction is fraudulent. The test set contains 1,000 transactions: 990 legitimate and 10 fraudulent. The model's predictions are shown in the confusion matrix below. Confusion matrix: Predicted Legitimate Predicted Fraudulent Actual Legitimate 942 48 Actual Fraudulent 2 8 Which metric should the data scientist prioritize if the business goal is to minimize the number of fraudulent transactions that are missed (false negatives)?

Accepted Answer

Recall. Recall (sensitivity) measures the proportion of actual positives correctly identified, calculated as TP/(TP+FN). With 2 false negatives (missed fraudulent transactions), recall is 8/(8+2)=0.80. Minimizing missed fraud directly corresponds to maximizing recall, making it the correct priority for this business goal.

Answer

Precision

Answer

Accuracy

Answer

Specificity

Question 8

A data scientist trains a binary classification model to predict whether a loan applicant will default (positive class) or not (negative class). The training data contains 5% default cases. The model predicts 'no default' for every applicant in the test set and achieves 95% accuracy. Which evaluation metric best reveals that the model is failing to identify any default cases?

Accepted Answer

B. Recall for the default class. Recall for the default class (positive class) measures the proportion of actual default cases that the model correctly identifies. With a model that predicts 'no default' for every applicant, recall for the default class is 0% because it fails to identify any true positive cases. This metric directly reveals the model's inability to detect defaults, despite the high overall accuracy of 95%.

Answer

A. Precision for the default class

Answer

C. F1-score for the default class

Answer

D. Overall accuracy

Question 9

A retail company wants to analyze customer purchase histories to identify natural groups of customers with similar buying patterns. They do not have predefined categories. Which type of machine learning should they use?

Accepted Answer

Unsupervised clustering. Unsupervised clustering is the correct approach because the company wants to discover natural groupings in customer purchase histories without predefined labels. Clustering algorithms, such as K-Means or DBSCAN, partition data into clusters based on feature similarity, enabling the identification of customer segments with similar buying patterns without any prior training on labeled examples.

Answer

Reinforcement learning

Answer

Supervised classification

Answer

Supervised regression

Question 10

A real estate company trains a model to predict house prices. They evaluate it on a test set of 100 houses. The model predictions have a mean absolute error (MAE) of $5,000 and a root mean squared error (RMSE) of $20,000. What does the large difference between MAE and RMSE indicate about the model's errors?

Accepted Answer

The model has many small errors and a few large errors.. The mean absolute error (MAE) of $5,000 and root mean squared error (RMSE) of $20,000 show a large discrepancy because RMSE squares errors before averaging, which heavily penalizes large deviations. Since RMSE is four times larger than MAE, this indicates that while most predictions are close (small errors), there are a few predictions with very large errors that inflate the RMSE. This pattern is classic for a model that performs well on most houses but fails badly on a few outliers.

Answer

The model consistently overestimates prices.

Answer

The model has a high bias and low variance.

Answer

The model is perfectly accurate.

Question 11

A data scientist evaluates a regression model that predicts house prices. On the test set, the Mean Absolute Error (MAE) is $8,000 and the Root Mean Squared Error (RMSE) is $25,000. What does the large difference between MAE and RMSE indicate about the model's errors?

Accepted Answer

The model has some predictions with very large errors. The large difference between MAE ($8,000) and RMSE ($25,000) indicates that the model has some predictions with very large errors. RMSE squares the errors before averaging, which heavily penalizes large deviations, so a significantly higher RMSE relative to MAE suggests the presence of outliers or extreme prediction errors in the test set.

Answer

The model is overfitting the training data

Answer

The model predictions are consistently biased high

Answer

The model has high variance due to outliers in training data

Question 12

A data scientist trains a model on historical data and achieves high accuracy on both the training set and a held-out test set. However, when the model is deployed in production, it performs poorly on new, unseen data. Which issue is most likely the cause?

Accepted Answer

Data leakage. Data leakage occurs when information from outside the training dataset is inadvertently used to train the model, causing it to learn patterns that do not generalize to new data. In this scenario, the high accuracy on both training and test sets but poor production performance indicates that the test set was contaminated with information from the future or from the target variable, making the model appear accurate during validation but fail in real-world deployment.

Answer

Overfitting

Answer

Underfitting

Answer

Concept drift

Question 13

A data scientist trains a binary classification model to detect fraudulent transactions. The dataset contains 99% legitimate transactions (negative class) and 1% fraudulent transactions (positive class). The model predicts 'legitimate' for every transaction in the test set and achieves 99% accuracy. Which metric would best reveal that the model is failing to identify any fraudulent transactions?

Accepted Answer

Recall. Recall (also known as sensitivity or true positive rate) measures the proportion of actual positive cases (fraudulent transactions) that the model correctly identifies. With 99% accuracy but zero true positives, the recall is 0%, which immediately reveals the model's complete failure to detect fraud. In Azure Machine Learning, the classification metrics pane would show recall = 0.0 for the positive class, highlighting this issue despite high accuracy.

Answer

Accuracy

Answer

Precision

Answer

F1-score

Question 14

A data scientist trains a regression model to predict house prices. The model achieves very low error on the training data but significantly higher error on a held-out test set. Which problem does this scenario best describe?

Accepted Answer

Overfitting. The scenario describes overfitting, where the model learns the training data too well, including noise and outliers, resulting in very low training error but poor generalization to new data. In Azure Machine Learning, this is often detected by comparing training and validation metrics; a large gap indicates overfitting. The correct answer is B.

Answer

Underfitting

Answer

High bias

Answer

High variance

Question 15

A data scientist has a dataset with 100 features and 10,000 samples. They want to reduce the number of features while retaining as much variance as possible, to improve model training speed and reduce overfitting. Which technique should they use?

Accepted Answer

Principal Component Analysis (PCA). Principal Component Analysis (PCA) is an unsupervised dimensionality reduction technique that transforms the original features into a new set of orthogonal components, ordered by the amount of variance they capture. By selecting only the top principal components, the data scientist can significantly reduce the feature count (e.g., from 100 to 20) while retaining the majority of the dataset's variance, which directly improves model training speed and reduces overfitting.

Answer

Feature scaling

Answer

Regularization

Answer

Cross-validation