AI-900 Describe fundamental principles of machine learning on Azure Practice Test 2 — 15 Questions

Question 1

A data scientist trains a machine learning model to predict housing prices. On the training data, the model achieves an R-squared value of 0.99, but on a separate validation dataset it achieves an R-squared of only 0.65. What is the most likely issue with this model?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Overfitting. The model performs exceptionally well on the training data (R² = 0.99) but poorly on the validation data (R² = 0.65), which is a classic symptom of overfitting. Overfitting occurs when the model learns noise and specific patterns in the training set that do not generalize to unseen data, often due to excessive complexity (e.g., too many features or deep decision trees). In Azure Machine Learning, this can be detected by comparing training and validation metrics in automated ML runs or by using regularization techniques like L1/L2 penalties.

Answer

Underfitting

Answer

High bias

Answer

Insufficient training data

Question 2

A data scientist trains a machine learning model to predict housing prices. On the training data, the model achieves an R-squared value of 0.99, but on a separate validation dataset it achieves an R-squared of only 0.65. What is the most likely issue with this model?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Overfitting. The model's high accuracy on training data (98%) but significantly lower accuracy on test data (72%) is a classic symptom of overfitting, where the model learns noise and specific patterns in the training set rather than generalizing to new, unseen data. In Azure Machine Learning, this often occurs when the model is too complex (e.g., deep decision trees or high-degree polynomial features) relative to the amount of training data, and regularization techniques like L1/L2 regularization or early stopping are not applied.

Answer

Underfitting

Answer

Data leakage

Answer

Class imbalance

Question 3

A retail company wants to predict the exact number of units of a product that will be sold next month. They have historical sales data and information about promotions and holidays. The target variable is the number of units sold, which is a continuous value. Which type of machine learning task should they perform?

Accepted Answer

Regression. Regression is the correct choice because the target variable—number of units sold—is a continuous numeric value. Regression algorithms, such as linear regression or decision forest regression, are designed to predict a numeric quantity from historical features like sales data, promotions, and holidays. In Azure Machine Learning, regression models output a real number, making them ideal for this forecasting scenario.

Answer

Binary classification

Answer

Multiclass classification

Answer

Clustering

Question 4

A data scientist trains a regression model on a dataset with 100 features and 10,000 samples. The model achieves a low training error but a much higher error on a held-out test set. Which approach is most likely to improve the model's generalization performance?

Accepted Answer

Reduce the number of features or apply regularization. The model exhibits high variance (overfitting), as indicated by low training error but high test error. Reducing the number of features or applying regularization (e.g., L1/L2) directly constrains model complexity, forcing it to learn more general patterns rather than memorizing noise. This is the standard approach to improve generalization in regression models.

Answer

Increase the complexity of the model by adding more layers or parameters

Answer

Add more training data

Answer

Use a different train-test split ratio like 80-20 instead of 70-30

Question 5

A data scientist is training a binary classification model to detect fraudulent transactions. The dataset contains only 1% fraudulent transactions. The model achieves 99% accuracy on the test set, but when deployed, it fails to detect most actual fraud cases. Which metric would best reveal this issue?

Accepted Answer

Recall. Recall (sensitivity) measures the proportion of actual positive cases correctly identified. In this highly imbalanced dataset (1% fraud), a model can achieve 99% accuracy by simply predicting 'non-fraud' for every transaction, which yields zero true positives. Recall reveals this failure because it focuses solely on how many fraudulent transactions were caught, ignoring the vast majority of non-fraud cases.

Answer

Accuracy

Answer

Precision

Answer

F1 score

Question 6

A data scientist is building a classification model to predict customer churn. The dataset has only 5% churn cases. The model achieves 95% accuracy on the test set, but upon investigation, the data scientist finds the model predicts 'not churn' for nearly every customer. Which metric should the data scientist primarily use to evaluate the model's performance on this imbalanced dataset?

Accepted Answer

F1 score. In an imbalanced dataset with only 5% churn, a model that predicts 'not churn' for every case achieves 95% accuracy by always guessing the majority class. This accuracy is misleading because it fails to identify any churn cases. The F1 score (option B) is the harmonic mean of precision and recall, making it the primary metric for evaluating classification performance on imbalanced data, as it penalizes both false positives and false negatives and is not skewed by class imbalance.

Answer

Accuracy

Answer

Mean Absolute Error (MAE)

Answer

R-squared

Question 7

A bike-sharing company wants to predict the number of rentals per hour. Their model's predictions are usually close but occasionally have large errors due to unexpected events like sudden rain. They want a metric that heavily penalizes these large errors to ensure the model is not overly confident. Which evaluation metric should they primarily use?

Accepted Answer

Mean Squared Error (MSE). Mean Squared Error (MSE) is the correct choice because it squares the residuals, which heavily penalizes large errors. Since the bike-sharing company wants to discourage occasional large prediction errors (e.g., due to sudden rain), MSE’s quadratic penalty ensures that models with even a few large outliers receive a much worse score, forcing the model to avoid overconfidence.

Answer

Mean Absolute Error (MAE)

Answer

Classification Accuracy

Answer

R-squared

Question 8

A data scientist trains a regression model to predict house prices. The model has a mean absolute error (MAE) of $5,000 on the test set. Which statement best interprets this metric?

Accepted Answer

On average, the model's predictions are $5,000 away from the actual prices.. Mean Absolute Error (MAE) measures the average absolute difference between predicted and actual values. An MAE of $5,000 means that, on average, each prediction deviates from the true house price by $5,000. This is a standard interpretation of MAE in regression metrics.

Answer

The model is accurate 95% of the time.

Answer

The model's predictions are within $5,000 of the actual prices for 50% of the houses.

Answer

The square root of the average squared error is $5,000.

Question 9

A company builds a machine learning model to predict whether a customer will purchase a product. They use a training dataset with 50% purchasers and 50% non-purchasers. The model achieves 90% accuracy on the test set. However, when deployed, the model performs poorly because the actual customer base has only 5% purchasers. What is the most likely cause of this poor performance?

Accepted Answer

The training and deployment data have different distributions.. The model was trained on a balanced dataset (50% purchasers, 50% non-purchasers) but deployed on a real-world dataset with only 5% purchasers. This mismatch in class distribution between training and deployment data causes the model to fail, as it learned decision boundaries optimized for balanced classes. This is a classic case of distribution shift, specifically prior probability shift, which invalidates the model's assumptions about the target variable's base rate.

Answer

The model is overfitted to the training data.

Answer

The model is underfitted and fails to capture key patterns.

Answer

Data leakage caused inflated accuracy during testing.

Question 10

A data scientist trains a classification model to distinguish between images of cats and dogs. The model achieves 99% accuracy on the training set but only 75% accuracy on a validation set. Which concept best describes this situation?

Accepted Answer

Overfitting. The model performs exceptionally well on the training data (99% accuracy) but significantly worse on unseen validation data (75% accuracy). This gap indicates the model has memorized noise and specific patterns in the training set rather than learning generalizable features, which is the classic definition of overfitting.

Answer

Underfitting

Answer

Model bias

Answer

Data leakage

Question 11

A data scientist is building a model to predict the exact temperature in degrees Celsius based on humidity and atmospheric pressure. The model will output a single numeric value for each input. Which type of machine learning task is this?

Accepted Answer

Regression. This is a regression task because the goal is to predict a continuous numeric value (temperature in degrees Celsius) from input features (humidity and atmospheric pressure). Regression models output a real number, unlike classification which predicts discrete categories. In Azure Machine Learning, regression algorithms like Linear Regression or Decision Forest Regression are used for such tasks.

Answer

Classification

Answer

Clustering

Answer

Object detection

Question 12

A media company wants to automatically organize a large collection of news articles into several topic-based categories (e.g., politics, sports, technology) without using any predefined labels. They plan to use Azure Machine Learning. Which type of machine learning task should they use?

Accepted Answer

Clustering. Clustering is the correct choice because the media company wants to group unlabeled news articles into topic-based categories based on inherent similarities in the data, without using predefined labels. Azure Machine Learning provides clustering algorithms like K-Means that automatically partition the dataset into distinct clusters, making it ideal for unsupervised learning tasks where the goal is to discover natural groupings.

Answer

Regression

Answer

Classification

Answer

Anomaly detection

Question 13

A retail company wants to segment its customers into different groups based on purchasing behavior, without using predefined categories. Which type of machine learning task should they use?

Accepted Answer

Clustering. Clustering is the correct choice because it is an unsupervised learning technique that groups data points based on inherent similarities without requiring predefined labels. In this scenario, the retail company wants to discover natural segments in customer purchasing behavior, such as high-frequency buyers or discount seekers, without providing any existing categories. Azure Machine Learning offers clustering algorithms like K-Means, which iteratively assigns customers to clusters by minimizing within-cluster variance based on features like purchase frequency and average order value.

Answer

Classification

Answer

Regression

Answer

Reinforcement learning

Question 14

A data scientist trains a deep neural network on a small dataset. The model achieves 100% accuracy on the training data but only 60% accuracy on a validation set. Which technique is most appropriate to address this issue?

Accepted Answer

Apply regularization. The model's perfect training accuracy (100%) paired with poor validation accuracy (60%) is a classic sign of overfitting, where the model has memorized the training data rather than learning generalizable patterns. Regularization techniques (e.g., L1/L2 regularization, dropout) penalize large weights or randomly drop neurons during training, which forces the network to learn simpler, more robust features and reduces overfitting on small datasets.

Answer

Increase the number of training epochs

Answer

Add more hidden layers

Answer

Increase the learning rate

Question 15

An online retailer wants to build a recommendation system that learns from user interactions. The system suggests a product, and if the user clicks it, it receives a positive reward; if ignored, a negative reward. Over time, the system learns to make better suggestions. Which type of machine learning best describes this approach?

Accepted Answer

Reinforcement learning. Reinforcement learning is correct because the system learns by interacting with its environment (user clicks) and receiving rewards (positive for clicks, negative for ignores) to maximize cumulative reward over time. This trial-and-error feedback loop, without explicit labeled data, is the hallmark of reinforcement learning.

Answer

Supervised learning

Answer

Unsupervised learning

Answer

Semi-supervised learning