AI-900 Practice Test 46 — 15 Questions

Question 1

A data scientist trains a binary classification model to predict loan defaults. The dataset contains 98% non-default cases and only 2% default cases. The model predicts 'non-default' for every instance, achieving 98% accuracy on the test set. Which metric would best reveal that the model fails to identify any actual defaults?

Accepted Answer

Recall for the default class. Recall for the default class measures the proportion of actual default cases that the model correctly identifies. With the model predicting 'non-default' for every instance, recall for the default class is 0%, because it fails to capture any true positives. This directly reveals the model's inability to detect any actual defaults, despite the high overall accuracy.

Answer

Precision for the default class

Answer

F1 score for the default class

Answer

Accuracy

Question 2

A game development company uses Azure OpenAI Service to automatically generate in-game dialog for non-player characters (NPCs) based on character profiles. They need to ensure the generated text does not contain offensive language or harmful suggestions. Which Azure OpenAI Service feature should they configure to prevent this?

Accepted Answer

Content filters. Content filters in Azure OpenAI Service allow you to define categories of harmful content (e.g., hate, violence, self-harm) and set severity thresholds. When generating NPC dialog, the service automatically evaluates each output against these filters and blocks or flags any text that violates the configured policies, ensuring offensive language or harmful suggestions are prevented.

Answer

Model deployment

Answer

Token limit

Answer

Prompt engineering

Question 3

A data scientist trains a machine learning model on a dataset of housing prices. The model achieves 98% accuracy on the training data but only 72% accuracy on a separate test set. What is the most likely problem with this model?

Accepted Answer

Overfitting. The model's high accuracy on training data (98%) but significantly lower accuracy on test data (72%) is a classic symptom of overfitting, where the model learns noise and specific patterns in the training set rather than generalizing to new, unseen data. In Azure Machine Learning, this often occurs when the model is too complex (e.g., deep decision trees or high-degree polynomial features) relative to the amount of training data, and regularization techniques like L1/L2 regularization or early stopping are not applied.

Answer

Underfitting

Answer

Data leakage

Answer

Class imbalance

Question 4

A data scientist is using Azure Automated Machine Learning to build a binary classification model for a highly imbalanced dataset (95% negative, 5% positive). The data scientist wants AutoML to select the best model based on a metric that is robust to class imbalance. Which primary metric should the data scientist configure in the AutoML settings?

Accepted Answer

AUC_weighted. AUC_weighted is the correct primary metric for imbalanced binary classification because it computes the area under the ROC curve for each class and averages them with weight proportional to the class support. This weighting ensures that the metric reflects performance on both the majority (95% negative) and minority (5% positive) classes, making it robust to severe imbalance. Azure Automated Machine Learning uses AUC_weighted as a recommended metric when the dataset is skewed, as it penalizes models that ignore the minority class.

Answer

Accuracy

Answer

F1_score

Answer

Log_loss

Question 5

A data scientist trains a decision tree model to predict customer churn. The model achieves 99% accuracy on the training data but only 80% on the test data. Which concept best explains this performance difference?

Accepted Answer

Overfitting. The model's high accuracy on training data (99%) but significantly lower accuracy on test data (80%) indicates that it has memorized the training data rather than learning generalizable patterns. This is the classic symptom of overfitting, where the decision tree captures noise and outliers in the training set, leading to poor performance on unseen data.

Answer

Underfitting

Answer

Bias-variance tradeoff

Answer

Cross-validation

Question 6

A hospital collects patient experience feedback in free-text form. They need to automatically (1) extract specific mentions of symptoms (e.g., 'headache', 'fever', 'fatigue') from the text, and (2) determine the overall emotional tone of each feedback (e.g., positive, negative, neutral). Which combination of Azure AI Language features should they use?

Accepted Answer

A. Entity recognition and sentiment analysis. Option A is correct because the hospital needs two distinct NLP capabilities: extracting specific symptom mentions (entity recognition) and determining emotional tone (sentiment analysis). Azure AI Language's entity recognition identifies named entities like symptoms, while sentiment analysis evaluates text for positive, negative, or neutral sentiment. Together, they directly address both requirements without extraneous features.

Answer

B. Key phrase extraction and question answering

Answer

C. Language detection and text classification

Answer

D. Summarization and translation

Question 7

A data scientist trains a regression model to predict house prices using features like square footage, number of bedrooms, and location. The model achieves very high accuracy on the training data but performs poorly on a held-out test set. Which technique should the data scientist apply to reduce overfitting?

Accepted Answer

Use regularization. Regularization (Option C) is the correct technique to reduce overfitting because it adds a penalty term to the loss function (e.g., L1 or L2 regularization), which discourages the model from learning overly complex patterns that fit noise in the training data. This helps the model generalize better to unseen data, such as the held-out test set, by constraining the magnitude of feature weights.

Answer

Increase the number of features

Answer

Decrease the training data size

Answer

Increase the number of training epochs

Question 8

A data scientist trains a deep neural network on a small dataset. The model achieves 100% accuracy on the training data but only 60% accuracy on a validation set. Which technique is most appropriate to address this issue?

Accepted Answer

Apply regularization. The model's perfect training accuracy (100%) paired with poor validation accuracy (60%) is a classic sign of overfitting, where the model has memorized the training data rather than learning generalizable patterns. Regularization techniques (e.g., L1/L2 regularization, dropout) penalize large weights or randomly drop neurons during training, which forces the network to learn simpler, more robust features and reduces overfitting on small datasets.

Answer

Increase the number of training epochs

Answer

Add more hidden layers

Answer

Increase the learning rate

Question 9

A data scientist trains a model to predict customer churn. The dataset includes features like age, income, and number of support calls. The model performs well on historical data but poorly on new data from a different customer segment. Which technique is most likely to help improve generalization?

Accepted Answer

Cross-validation. Cross-validation (Option B) is the most effective technique to improve generalization because it evaluates the model on multiple subsets of the training data, reducing overfitting to a specific segment. By partitioning the data into folds and training/validating iteratively, cross-validation ensures the model learns patterns that are consistent across different data distributions, not just the historical segment. This directly addresses the problem of poor performance on new customer segments by providing a more robust estimate of model performance on unseen data.

Answer

Feature engineering

Answer

Increasing model complexity

Answer

Using a larger learning rate

Question 10

A developer uses Azure OpenAI Service to generate product descriptions. Each description must be concise and not exceed 50 words. Which parameter should the developer set in the API request to control the output length?

Accepted Answer

max_tokens. The `max_tokens` parameter in the Azure OpenAI API directly controls the maximum number of tokens (words or subwords) in the generated output. By setting `max_tokens` to a value that corresponds to 50 words, the developer ensures the model stops generating once the limit is reached, producing concise descriptions.

Answer

Temperature

Answer

top_p

Answer

frequency_penalty

Question 11

What is 'chain of thought' prompting in generative AI?

Accepted Answer

A prompting technique that elicits step-by-step reasoning to improve accuracy on complex tasks. Chain of thought prompting is a technique where the model is asked to produce intermediate reasoning steps before arriving at a final answer, which significantly improves performance on multi-step arithmetic, logic, and common-sense reasoning tasks. Unlike a simple answer request, it forces the model to externalize its reasoning process, reducing errors from shortcut or pattern-matching behaviors. This is a prompting strategy, not a model architecture change, and is particularly effective in large language models like GPT-4 or Azure OpenAI's GPT-3.5 Turbo.

Answer

Connecting multiple AI models in a processing pipeline

Answer

Linking multiple conversation turns to maintain context

Answer

Training a model using sequential text data only

Question 12

A law firm needs to automatically process incoming legal documents. They have two specific requirements: (1) extract the names of all parties involved, the court name, and the filing date; (2) categorize each document as a 'complaint', 'motion', or 'subpoena'. Which two Azure AI Language features should they use? (Choose two.)

Accepted Answer

Custom text classification. Custom text classification (C) is correct because it allows the law firm to train a model to categorize legal documents into custom classes like 'complaint', 'motion', or 'subpoena' based on labeled examples. This feature is designed for domain-specific classification tasks where predefined categories are insufficient.

Answer

Sentiment analysis

Answer

Key phrase extraction

Question 13

A developer wants to use Azure OpenAI to build a customer service chatbot that can answer questions about a company's return policy. They create a set of example question-answer pairs in the prompt without retraining the model. Which technique is being used?

Accepted Answer

Few-shot learning. Few-shot learning is the correct technique because the developer provides a small set of example question-answer pairs directly in the prompt to guide the model's responses, without retraining or updating the model's weights. This leverages the model's pre-existing knowledge to generalize from the examples, which is a hallmark of few-shot prompting in Azure OpenAI.

Answer

Fine-tuning

Answer

Reinforcement learning

Answer

Transfer learning

Question 14

A data scientist trains a binary classification model to distinguish between images of cats and dogs. On the test set, the model achieves 98% accuracy, but a deeper inspection reveals that the test set contains 95% cats and 5% dogs, and the model predicts 'cat' for every single image. Which metric should the data scientist prioritize to get a more realistic evaluation of the model's performance on this imbalanced dataset?

Accepted Answer

F1-score. The F1-score is the harmonic mean of precision and recall, providing a single metric that balances both when classes are imbalanced. In this scenario, accuracy is misleadingly high (98%) because the model always predicts the majority class (cat), achieving high accuracy without actually learning to distinguish cats from dogs. The F1-score penalizes the model for its poor recall on the minority class (dogs), giving a more realistic evaluation of its performance.

Answer

Precision

Answer

Recall

Answer

Accuracy

Question 15

A logistics company wants to use AI to automatically sort packages based on their destination address printed on the label. Which AI workload combination is needed?

Accepted Answer

OCR to read the printed address and classification to determine routing. Option B is correct because the scenario requires two distinct AI workloads: OCR (Optical Character Recognition) to extract the printed destination address from the package label, and classification to map that address to the correct routing category (e.g., zip code, region, or delivery route). This combination directly solves the problem of reading unstructured text and assigning it to a predefined output class.

Answer

Reinforcement learning to optimize package sorting speed

Answer

Facial recognition to identify the delivery person

Answer

Sentiment analysis to assess package condition