AI-900 Describe fundamental principles of machine learning on Azure Practice Test 2 — 15 Questions

Question 1

A data scientist trains a regression model on a dataset with 100 features and 10,000 samples. The model achieves a low training error but a much higher error on a held-out test set. Which approach is most likely to improve the model's generalization performance?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Reduce the number of features or apply regularization. The model exhibits high variance (overfitting), as indicated by low training error but high test error. Reducing the number of features or applying regularization (e.g., L1/L2) directly constrains model complexity, forcing it to learn more general patterns rather than memorizing noise. This is the standard approach to improve generalization in regression models.

Answer

Increase the complexity of the model by adding more layers or parameters

Answer

Add more training data

Answer

Use a different train-test split ratio like 80-20 instead of 70-30

Question 2

A data scientist is training a binary classification model to detect fraudulent transactions. The dataset contains only 1% fraudulent transactions. The model achieves 99% accuracy on the test set, but when deployed, it fails to detect most actual fraud cases. Which metric would best reveal this issue?

Accepted Answer

Recall. Recall (sensitivity) measures the proportion of actual positive cases correctly identified. In this highly imbalanced dataset (1% fraud), a model can achieve 99% accuracy by simply predicting 'non-fraud' for every transaction, which yields zero true positives. Recall reveals this failure because it focuses solely on how many fraudulent transactions were caught, ignoring the vast majority of non-fraud cases.

Answer

Accuracy

Answer

Precision

Answer

F1 score

Question 3

A data scientist is building a classification model to predict customer churn. The dataset has only 5% churn cases. The model achieves 95% accuracy on the test set, but upon investigation, the data scientist finds the model predicts 'not churn' for nearly every customer. Which metric should the data scientist primarily use to evaluate the model's performance on this imbalanced dataset?

Accepted Answer

F1 score. In an imbalanced dataset with only 5% churn, a model that predicts 'not churn' for every case achieves 95% accuracy by always guessing the majority class. This accuracy is misleading because it fails to identify any churn cases. The F1 score (option B) is the harmonic mean of precision and recall, making it the primary metric for evaluating classification performance on imbalanced data, as it penalizes both false positives and false negatives and is not skewed by class imbalance.

Answer

Accuracy

Answer

Mean Absolute Error (MAE)

Answer

R-squared

Question 4

A bike-sharing company wants to predict the number of rentals per hour. Their model's predictions are usually close but occasionally have large errors due to unexpected events like sudden rain. They want a metric that heavily penalizes these large errors to ensure the model is not overly confident. Which evaluation metric should they primarily use?

Accepted Answer

Mean Squared Error (MSE). Mean Squared Error (MSE) is the correct choice because it squares the residuals, which heavily penalizes large errors. Since the bike-sharing company wants to discourage occasional large prediction errors (e.g., due to sudden rain), MSE’s quadratic penalty ensures that models with even a few large outliers receive a much worse score, forcing the model to avoid overconfidence.

Answer

Mean Absolute Error (MAE)

Answer

Classification Accuracy

Answer

R-squared

Question 5

A data scientist trains a regression model to predict house prices. The model has a mean absolute error (MAE) of $5,000 on the test set. Which statement best interprets this metric?

Accepted Answer

On average, the model's predictions are $5,000 away from the actual prices.. Mean Absolute Error (MAE) measures the average absolute difference between predicted and actual values. An MAE of $5,000 means that, on average, each prediction deviates from the true house price by $5,000. This is a standard interpretation of MAE in regression metrics.

Answer

The model is accurate 95% of the time.

Answer

The model's predictions are within $5,000 of the actual prices for 50% of the houses.

Answer

The square root of the average squared error is $5,000.

Question 6

A company builds a machine learning model to predict whether a customer will purchase a product. They use a training dataset with 50% purchasers and 50% non-purchasers. The model achieves 90% accuracy on the test set. However, when deployed, the model performs poorly because the actual customer base has only 5% purchasers. What is the most likely cause of this poor performance?

Accepted Answer

The training and deployment data have different distributions.. The model was trained on a balanced dataset (50% purchasers, 50% non-purchasers) but deployed on a real-world dataset with only 5% purchasers. This mismatch in class distribution between training and deployment data causes the model to fail, as it learned decision boundaries optimized for balanced classes. This is a classic case of distribution shift, specifically prior probability shift, which invalidates the model's assumptions about the target variable's base rate.

Answer

The model is overfitted to the training data.

Answer

The model is underfitted and fails to capture key patterns.

Answer

Data leakage caused inflated accuracy during testing.

Question 7

A data scientist trains a classification model to distinguish between images of cats and dogs. The model achieves 99% accuracy on the training set but only 75% accuracy on a validation set. Which concept best describes this situation?

Accepted Answer

Overfitting. The model performs exceptionally well on the training data (99% accuracy) but significantly worse on unseen validation data (75% accuracy). This gap indicates the model has memorized noise and specific patterns in the training set rather than learning generalizable features, which is the classic definition of overfitting.

Answer

Underfitting

Answer

Model bias

Answer

Data leakage

Question 8

A data scientist is building a model to predict the exact temperature in degrees Celsius based on humidity and atmospheric pressure. The model will output a single numeric value for each input. Which type of machine learning task is this?

Accepted Answer

Regression. This is a regression task because the goal is to predict a continuous numeric value (temperature in degrees Celsius) from input features (humidity and atmospheric pressure). Regression models output a real number, unlike classification which predicts discrete categories. In Azure Machine Learning, regression algorithms like Linear Regression or Decision Forest Regression are used for such tasks.

Answer

Classification

Answer

Clustering

Answer

Object detection

Question 9

A media company wants to automatically organize a large collection of news articles into several topic-based categories (e.g., politics, sports, technology) without using any predefined labels. They plan to use Azure Machine Learning. Which type of machine learning task should they use?

Accepted Answer

Clustering. Clustering is the correct choice because the media company wants to group unlabeled news articles into topic-based categories based on inherent similarities in the data, without using predefined labels. Azure Machine Learning provides clustering algorithms like K-Means that automatically partition the dataset into distinct clusters, making it ideal for unsupervised learning tasks where the goal is to discover natural groupings.

Answer

Regression

Answer

Classification

Answer

Anomaly detection

Question 10

A retail company wants to segment its customers into different groups based on purchasing behavior, without using predefined categories. Which type of machine learning task should they use?

Accepted Answer

Clustering. Clustering is the correct choice because it is an unsupervised learning technique that groups data points based on inherent similarities without requiring predefined labels. In this scenario, the retail company wants to discover natural segments in customer purchasing behavior, such as high-frequency buyers or discount seekers, without providing any existing categories. Azure Machine Learning offers clustering algorithms like K-Means, which iteratively assigns customers to clusters by minimizing within-cluster variance based on features like purchase frequency and average order value.

Answer

Classification

Answer

Regression

Answer

Reinforcement learning

Question 11

A data scientist trains a deep neural network on a small dataset. The model achieves 100% accuracy on the training data but only 60% accuracy on a validation set. Which technique is most appropriate to address this issue?

Accepted Answer

Apply regularization. The model's perfect training accuracy (100%) paired with poor validation accuracy (60%) is a classic sign of overfitting, where the model has memorized the training data rather than learning generalizable patterns. Regularization techniques (e.g., L1/L2 regularization, dropout) penalize large weights or randomly drop neurons during training, which forces the network to learn simpler, more robust features and reduces overfitting on small datasets.

Answer

Increase the number of training epochs

Answer

Add more hidden layers

Answer

Increase the learning rate

Question 12

An online retailer wants to build a recommendation system that learns from user interactions. The system suggests a product, and if the user clicks it, it receives a positive reward; if ignored, a negative reward. Over time, the system learns to make better suggestions. Which type of machine learning best describes this approach?

Accepted Answer

Reinforcement learning. Reinforcement learning is correct because the system learns by interacting with its environment (user clicks) and receiving rewards (positive for clicks, negative for ignores) to maximize cumulative reward over time. This trial-and-error feedback loop, without explicit labeled data, is the hallmark of reinforcement learning.

Answer

Supervised learning

Answer

Unsupervised learning

Answer

Semi-supervised learning

Question 13

A manufacturing team wants to predict product defects based on sensor readings from the production line. They have 10,000 historical samples, each labeled as 'defective' or 'non-defective'. Which type of machine learning should they use in Azure Machine Learning?

Accepted Answer

Supervised learning. This is a supervised learning problem because the dataset contains labeled historical samples (defective or non-defective), and the goal is to predict a categorical outcome based on sensor readings. In Azure Machine Learning, supervised learning algorithms such as two-class logistic regression or boosted decision trees are used to train a model that maps input features to known labels.

Answer

Unsupervised learning

Answer

Reinforcement learning

Answer

Semi-supervised learning

Question 14

A data scientist trains a classification model to predict whether an email is 'phishing' or 'legitimate'. The model achieves 99% accuracy on the training data but only 68% accuracy on the test data. Which action is most likely to help improve the model's generalization performance?

Accepted Answer

Apply regularization techniques such as L1 or L2 regularization.. The model's high training accuracy (99%) paired with much lower test accuracy (68%) is a classic sign of overfitting, where the model has memorized the training data rather than learning generalizable patterns. Regularization techniques like L1 (Lasso) or L2 (Ridge) add a penalty to the loss function that discourages overly complex models by shrinking the weights of less important features, directly reducing overfitting and improving generalization on unseen data.

Answer

Increase the number of training epochs significantly.

Answer

Remove some of the training data to make the dataset smaller.

Answer

Add more layers and neurons to the neural network.

Question 15

A data scientist is building a machine learning model to predict whether a credit card transaction is fraudulent or legitimate. The dataset contains 100,000 historical transactions, each labeled as 'fraudulent' or 'legitimate'. Which type of machine learning task should the data scientist use in Azure Machine Learning?

Accepted Answer

Binary classification. Binary classification is the correct choice because the prediction task involves distinguishing between exactly two mutually exclusive classes: 'fraudulent' and 'legitimate'. In Azure Machine Learning, binary classification algorithms (e.g., Two-Class Logistic Regression, Two-Class Boosted Decision Tree) are designed to output a probability score for one of two labels, making them ideal for this fraud detection scenario.

Answer

Regression

Answer

Multi-class classification

Answer

Clustering