AI-900 Describe fundamental principles of machine learning on Azure — Timed 25 Questions

Question 1

A data scientist is training a regression model to predict house prices. The model performs near perfectly on the training data but poorly on a held-out test set. The scientist suspects the model is memorizing the training data instead of learning general patterns. Which technique is most appropriate to directly address this issue?

25 minute time limit — choose an answer to begin.

Accepted Answer

Apply L2 regularization to the model. L2 regularization (also known as Ridge regularization) directly addresses overfitting by adding a penalty term proportional to the square of the model weights to the loss function. This discourages the model from assigning excessively large coefficients to features, forcing it to learn simpler, more general patterns rather than memorizing noise in the training data.

Answer

Increase the size of the training dataset

Answer

Increase the complexity of the model (e.g., add more features)

Answer

Switch to a different regression algorithm

Question 2

A data scientist trains a regression model to predict house prices using features like square footage, number of bedrooms, and location. The model achieves very high accuracy on the training data but performs poorly on a held-out test set. Which technique should the data scientist apply to reduce overfitting?

Accepted Answer

Use regularization. Regularization (Option C) is the correct technique to reduce overfitting because it adds a penalty term to the loss function (e.g., L1 or L2 regularization), which discourages the model from learning overly complex patterns that fit noise in the training data. This helps the model generalize better to unseen data, such as the held-out test set, by constraining the magnitude of feature weights.

Answer

Increase the number of features

Answer

Decrease the training data size

Answer

Increase the number of training epochs

Question 3

A botanist uses Azure Automated Machine Learning to train a model that classifies iris flowers into three species: setosa, versicolor, and virginica. The dataset contains exactly 50 examples of each species, making it perfectly balanced. The botanist wants the primary metric to give equal importance to the classification performance of each species, regardless of their frequency. Which primary metric should the botanist select in Azure AutoML?

Accepted Answer

Normalized macro recall. Azure AutoML does not support Macro F1 as a primary classification metric. The supported primary metric that gives equal importance to each class is normalized macro recall (norm_macro_recall). It averages recall across classes after normalizing by class frequency, so each species contributes equally even if class sizes differ.

Answer

Accuracy

Answer

Weighted F1

Answer

Micro F1

Question 4

A data scientist trains a model to predict customer churn. The dataset includes features like age, income, and number of support calls. The model performs well on historical data but poorly on new data from a different customer segment. Which technique is most likely to help improve generalization?

Accepted Answer

Feature engineering. The model overfits to the historical customer segment. Cross-validation (B) merely validates on subsets of the same data and cannot improve generalization to a different distribution. Feature engineering (A) is more appropriate because it allows creation of features that capture generalizable patterns (e.g., normalized income or segment-specific interactions) that help the model perform on new segments. Increasing model complexity (C) or larger learning rate (D) would worsen overfitting and generalization.

Answer

Cross-validation

Answer

Increasing model complexity

Answer

Using a larger learning rate

Question 5

A data scientist trains a binary classification model to predict whether a loan applicant will default (positive class) or not (negative class). The training data contains 5% default cases. The model predicts 'no default' for every applicant in the test set and achieves 95% accuracy. Which evaluation metric best reveals that the model is failing to identify any default cases?

Accepted Answer

B. Recall for the default class. Recall for the default class (positive class) measures the proportion of actual default cases that the model correctly identifies. With a model that predicts 'no default' for every applicant, recall for the default class is 0% because it fails to identify any true positive cases. This metric directly reveals the model's inability to detect defaults, despite the high overall accuracy of 95%.

Answer

A. Precision for the default class

Answer

C. F1-score for the default class

Answer

D. Overall accuracy

Question 6

A data scientist trains a multiclass classification model to categorize customer support tickets into three types: 'Billing', 'Technical', and 'General'. The dataset contains 80% 'General', 15% 'Billing', and only 5% 'Technical' tickets. Overall accuracy on a test set is 85%, but the model misclassifies most 'Technical' tickets as 'General'. Which metric would best help the data scientist understand the model's poor performance on the 'Technical' class?

Accepted Answer

F1-score for the 'Technical' class. The F1-score for the 'Technical' class is the best metric because it combines precision and recall into a single harmonic mean, directly capturing the model's inability to correctly identify the minority class. Overall accuracy is misleading on imbalanced datasets; for example, a model that always predicted 'General' would achieve 80% accuracy, and while 85% is slightly higher, it still masks the model's poor performance on the minority 'Technical' class. The F1-score penalizes both false positives and false negatives, making it a standard metric for evaluating classifier performance on imbalanced classes.

Answer

Overall accuracy

Answer

Confusion matrix

Answer

Precision for the 'General' class

Question 7

What is the difference between 'supervised' and 'unsupervised' learning?

Accepted Answer

Supervised learning trains on labelled data; unsupervised discovers patterns in unlabelled data. Supervised learning requires a labeled dataset where each training example has an input-output pair, allowing the model to learn a mapping from inputs to known outputs. In contrast, unsupervised learning works with unlabeled data, and the model must find inherent patterns, groupings, or structures (e.g., clustering or dimensionality reduction) without any predefined labels. This distinction is fundamental to choosing the right machine learning approach in Azure, such as using Azure Machine Learning for supervised tasks like regression/classification and unsupervised tasks like clustering.

Answer

Supervised learning requires a human to watch the training process; unsupervised runs automatically

Answer

Supervised learning uses neural networks; unsupervised uses decision trees only

Answer

Unsupervised learning always produces better results than supervised learning

Question 8

A data scientist wants to train a machine learning model to predict the exact market price of a house based on features such as square footage, number of bedrooms, and location. Which type of machine learning task should be used?

Accepted Answer

Regression. Predicting the exact market price of a house is a regression task because the target variable (price) is a continuous numeric value. Regression algorithms, such as linear regression or decision tree regression, learn the relationship between input features (e.g., square footage, bedrooms, location) and a continuous output. In Azure Machine Learning, you would select a regression model from the designer or AutoML to solve this problem.

Answer

Classification

Answer

Clustering

Answer

Anomaly Detection

Question 9

A data scientist has trained a binary classification model to predict whether an email is spam (positive) or not spam (negative). On a test set, the model correctly identifies 90 out of 100 actual spam emails and 80 out of 100 actual non-spam emails. Which metric shows the proportion of actual spam emails that the model correctly predicted?

Accepted Answer

B. Recall. Recall (also known as sensitivity or true positive rate) measures the proportion of actual positive cases that were correctly predicted by the model. In this scenario, the model correctly identified 90 out of 100 actual spam emails, so the recall is 90/100 = 0.9 (90%). This metric directly answers the question about how well the model captures actual spam emails.

Answer

A. Precision

Answer

C. F1 Score

Answer

D. Accuracy

Question 10

A retail company wants to predict which customers are likely to stop using their service. They have a dataset with many customer attributes including age, income, purchase history, website activity, and support interactions. They suspect some features are redundant. Which technique should they use to reduce the number of features while preserving as much information as possible?

Accepted Answer

Principal Component Analysis (PCA). Principal Component Analysis (PCA) is an unsupervised dimensionality reduction technique that transforms the original correlated features into a smaller set of uncorrelated principal components, ordered by the variance they capture. By retaining only the top components, PCA reduces the number of features while preserving as much of the total variance (information) as possible, making it ideal for handling redundant features in customer datasets.

Answer

Normalization

Answer

One-hot encoding

Answer

Regression analysis

Question 11

A retail company wants to automatically group its customers into distinct segments based on their purchasing patterns, without having pre-defined categories. The goal is to discover natural groupings in the customer data to tailor marketing campaigns. Which type of machine learning task should the company use?

Accepted Answer

Unsupervised learning - Clustering. The company wants to discover natural groupings in customer data without pre-defined categories, which is the definition of unsupervised learning. Clustering algorithms (e.g., K-Means, DBSCAN) automatically partition data into segments based on similarity in purchasing patterns, making it the correct choice for this scenario.

Answer

Supervised learning - Classification

Answer

Reinforcement learning

Answer

Supervised learning - Regression

Question 12

A hospital has a dataset with historical patient records, each labeled as either 'readmitted within 30 days' or 'not readmitted'. The hospital wants to train a model to predict which current patients are likely to be readmitted. Which type of machine learning task is this?

Accepted Answer

Supervised classification. This is a supervised classification task because the dataset contains labeled historical patient records (readmitted or not readmitted), and the goal is to predict a discrete category (binary outcome) for new patients. In Azure Machine Learning, this would use a classification algorithm like logistic regression or decision tree to assign each patient to one of the two classes.

Answer

Supervised regression

Answer

Unsupervised clustering

Answer

Reinforcement learning

Question 13

A data scientist trains a machine learning model to predict housing prices. On the training data, the model achieves an R-squared value of 0.99, but on a separate validation dataset it achieves an R-squared of only 0.65. What is the most likely issue with this model?

Accepted Answer

Overfitting. The model performs exceptionally well on the training data (R² = 0.99) but poorly on the validation data (R² = 0.65), which is a classic symptom of overfitting. Overfitting occurs when the model learns noise and specific patterns in the training set that do not generalize to unseen data, often due to excessive complexity (e.g., too many features or deep decision trees). In Azure Machine Learning, this can be detected by comparing training and validation metrics in automated ML runs or by using regularization techniques like L1/L2 penalties.

Answer

Underfitting

Answer

High bias

Answer

Insufficient training data

Question 14

A data scientist trains a machine learning model on a dataset of housing prices. The model achieves 98% accuracy on the training data but only 72% accuracy on a separate test set. What is the most likely problem with this model?

Accepted Answer

Overfitting. The model's high accuracy on training data (98%) but significantly lower accuracy on test data (72%) is a classic symptom of overfitting, where the model learns noise and specific patterns in the training set rather than generalizing to new, unseen data. In Azure Machine Learning, this often occurs when the model is too complex (e.g., deep decision trees or high-degree polynomial features) relative to the amount of training data, and regularization techniques like L1/L2 regularization or early stopping are not applied.

Answer

Underfitting

Answer

Data leakage

Answer

Class imbalance

Question 15

A retail company wants to predict the exact number of units of a product that will be sold next month. They have historical sales data and information about promotions and holidays. The target variable is the number of units sold, which is a continuous value. Which type of machine learning task should they perform?

Accepted Answer

Regression. Regression is the correct choice because the target variable—number of units sold—is a continuous numeric value. Regression algorithms, such as linear regression or decision forest regression, are designed to predict a numeric quantity from historical features like sales data, promotions, and holidays. In Azure Machine Learning, regression models output a real number, making them ideal for this forecasting scenario.

Answer

Binary classification

Answer

Multiclass classification

Answer

Clustering

Question 16

A data scientist trains a regression model on a dataset with 100 features and 10,000 samples. The model achieves a low training error but a much higher error on a held-out test set. Which approach is most likely to improve the model's generalization performance?

Accepted Answer

Reduce the number of features or apply regularization. The model exhibits high variance (overfitting), as indicated by low training error but high test error. Reducing the number of features or applying regularization (e.g., L1/L2) directly constrains model complexity, forcing it to learn more general patterns rather than memorizing noise. This is the standard approach to improve generalization in regression models.

Answer

Increase the complexity of the model by adding more layers or parameters

Answer

Add more training data

Answer

Use a different train-test split ratio like 80-20 instead of 70-30

Question 17

A data scientist is training a binary classification model to detect fraudulent transactions. The dataset contains only 1% fraudulent transactions. The model achieves 99% accuracy on the test set, but when deployed, it fails to detect most actual fraud cases. Which metric would best reveal this issue?

Accepted Answer

Recall. Recall (sensitivity) measures the proportion of actual positive cases correctly identified. In this highly imbalanced dataset (1% fraud), a model can achieve 99% accuracy by simply predicting 'non-fraud' for every transaction, which yields zero true positives. Recall reveals this failure because it focuses solely on how many fraudulent transactions were caught, ignoring the vast majority of non-fraud cases.

Answer

Accuracy

Answer

Precision

Answer

F1 score

Question 18

A data scientist is building a classification model to predict customer churn. The dataset has only 5% churn cases. The model achieves 95% accuracy on the test set, but upon investigation, the data scientist finds the model predicts 'not churn' for nearly every customer. Which metric should the data scientist primarily use to evaluate the model's performance on this imbalanced dataset?

Accepted Answer

F1 score. In an imbalanced dataset with only 5% churn, a model that predicts 'not churn' for every case achieves 95% accuracy by always guessing the majority class. This accuracy is misleading because it fails to identify any churn cases. The F1 score (option B) is the harmonic mean of precision and recall, making it the primary metric for evaluating classification performance on imbalanced data, as it penalizes both false positives and false negatives and is not skewed by class imbalance.

Answer

Accuracy

Answer

Mean Absolute Error (MAE)

Answer

R-squared

Question 19

A bike-sharing company wants to predict the number of rentals per hour. Their model's predictions are usually close but occasionally have large errors due to unexpected events like sudden rain. They want a metric that heavily penalizes these large errors to ensure the model is not overly confident. Which evaluation metric should they primarily use?

Accepted Answer

Mean Squared Error (MSE). Mean Squared Error (MSE) is the correct choice because it squares the residuals, which heavily penalizes large errors. Since the bike-sharing company wants to discourage occasional large prediction errors (e.g., due to sudden rain), MSE’s quadratic penalty ensures that models with even a few large outliers receive a much worse score, forcing the model to avoid overconfidence.

Answer

Mean Absolute Error (MAE)

Answer

Classification Accuracy

Answer

R-squared

Question 20

A data scientist trains a regression model to predict house prices. The model has a mean absolute error (MAE) of $5,000 on the test set. Which statement best interprets this metric?

Accepted Answer

On average, the model's predictions are $5,000 away from the actual prices.. Mean Absolute Error (MAE) measures the average absolute difference between predicted and actual values. An MAE of $5,000 means that, on average, each prediction deviates from the true house price by $5,000. This is a standard interpretation of MAE in regression metrics.

Answer

The model is accurate 95% of the time.

Answer

The model's predictions are within $5,000 of the actual prices for 50% of the houses.

Answer

The square root of the average squared error is $5,000.