AI0-001 Practice Exam 4 — 20 Questions

Question 1

A team trains a neural network for image classification. During training, the loss decreases on the training set but increases on the validation set after a few epochs. What is the most likely cause?

Accepted Answer

Overfitting. Overfitting occurs when the model learns the training data too well, including noise and irrelevant patterns, causing it to memorize rather than generalize. This is evidenced by the loss decreasing on the training set while increasing on the validation set after a few epochs, as the model's performance on unseen data degrades.

Answer

Vanishing gradients

Answer

Incorrect learning rate scheduling

Answer

Underfitting

Question 2

An organisation is developing an AI policy. According to the NIST AI RMF, which function involves establishing policies and procedures to ensure the organisation governs AI responsibly?

Accepted Answer

Govern. The Govern function of the NIST AI RMF establishes policies, procedures, and organizational structures to ensure AI is developed and used responsibly. It is the foundational function that cultivates a risk-management culture and assigns accountability across the AI lifecycle.

Answer

Manage

Answer

Measure

Answer

Map

Question 3

Which machine learning paradigm involves training an agent to make decisions by interacting with an environment and receiving rewards or penalties based on its actions?

Accepted Answer

Reinforcement learning. Reinforcement learning (RL) is the correct paradigm because it explicitly involves an agent learning a policy through trial-and-error interactions with an environment, receiving scalar reward signals (positive or negative) to maximize cumulative reward. This matches the question's description of making decisions based on rewards or penalties, which is the defining characteristic of RL, as opposed to learning from labeled data or discovering hidden patterns without feedback.

Answer

Unsupervised learning

Answer

Supervised learning

Answer

Self-supervised learning

Question 4

A company uses an LLM API to generate customer support responses. They want to prevent the LLM from generating harmful content, even when users attempt jailbreaking. Which defense is MOST effective at the application layer?

Accepted Answer

Output filtering and content moderation. Output filtering and content moderation is the most effective defense at the application layer because it directly inspects the LLM's generated response before it reaches the user. This approach can catch and block harmful content that results from successful jailbreaking attempts, which input validation alone cannot prevent since the model may still produce undesirable outputs even with sanitized inputs.

Answer

Input validation and sanitization

Answer

Robust training techniques

Answer

Rate limiting

Question 5

A company wants to roll out a new recommendation model to production. They decide to run an A/B test where 10% of users see the new model and 90% see the old model. After one week, the new model shows a 5% improvement in click-through rate. What is the next best action?

Accepted Answer

Increase the testing percentage gradually while monitoring performance metrics and guardrails. A 5% improvement observed over only one week with a 10% traffic split is insufficient to confirm statistical significance or rule out novelty effects, data drift, or seasonal bias. The recommended best practice in AI deployment is to gradually increase the testing percentage (e.g., 10% → 25% → 50% → 100%) while continuously monitoring performance metrics and guardrails (e.g., click-through rate, conversion rate, latency, and error rates) to ensure the new model generalizes safely across the full user population.

Answer

Immediately roll out the new model to 100% of users

Answer

Revert to the old model because the improvement is minimal

Answer

Run the test for another month to ensure statistical significance

Question 6

A financial institution deploys an AI credit scoring model. After six months, the model's performance drops significantly. Analysis shows that the relationship between features and labels has changed. Which term describes this phenomenon?

Accepted Answer

Concept drift. Concept drift occurs when the statistical relationship between input features and the target label changes over time, which is exactly what happened when the credit scoring model's performance dropped due to a shift in the feature-label relationship. This is distinct from data drift, which only involves changes in the input data distribution without affecting the label mapping.

Answer

Model decay

Answer

Overfitting

Answer

Data drift

Question 7

A company deploys an LLM-based chatbot that retrieves data from external databases. An attacker embeds malicious instructions in a database record. When the chatbot retrieves that record, it executes the instructions, overriding its system prompt. Which type of attack is this?

Accepted Answer

Indirect prompt injection. This is an indirect prompt injection attack because the malicious instructions are embedded in a third-party data source (the database record) rather than being sent directly by the user. When the LLM retrieves and processes that record, the injected instructions override the system prompt, causing the chatbot to behave contrary to its intended design.

Answer

Model inversion attack

Answer

Direct prompt injection

Answer

Membership inference attack

Question 8

In the AI lifecycle, which phase involves splitting data into training, validation, and test sets?

Accepted Answer

Data preprocessing. Data preprocessing is the phase where raw data is cleaned, transformed, and prepared for modeling. Splitting the dataset into training, validation, and test sets is a critical step during this phase to ensure unbiased evaluation and prevent data leakage. This split occurs before any model training begins, making it part of preprocessing rather than training or evaluation.

Answer

Model training

Answer

Data collection

Answer

Model evaluation

Question 9

A security team is evaluating the risk of adversarial examples against their image classification system. Which of the following BEST describes an adversarial example?

Accepted Answer

An input crafted with small, intentional perturbations that cause the model to output an incorrect prediction. An adversarial example is specifically an input that has been deliberately modified with small, often imperceptible perturbations to cause a machine learning model to misclassify it. This exploits the model's sensitivity to high-dimensional input spaces, where tiny changes in pixel values can shift the decision boundary without altering human perception of the image.

Answer

A technique that reconstructs training data from the model's outputs

Answer

An attack that injects malicious data into the training set to corrupt the model

Answer

A method to determine if a specific data point was used in the training set

Question 10

A security team is conducting a red team exercise on a new LLM-powered customer support system. Which activity is part of red teaming?

Accepted Answer

Attempting to jailbreak the LLM to bypass safety guardrails. Red teaming in the context of an LLM-powered system involves actively probing for security vulnerabilities, such as attempting to bypass safety guardrails through jailbreak prompts. Option A directly describes this adversarial testing, which is the core activity of a red team exercise to identify weaknesses before malicious actors can exploit them.

Answer

Reviewing the model's training data for bias

Answer

Implementing access controls on the model API

Answer

Monitoring system performance metrics

Question 11

A company is building a document intelligence system that extracts key fields from scanned invoices. They have a labeled dataset of 10,000 invoices but need to decide between a traditional OCR+rule-based pipeline and an AI-based model. Which use case characteristic STRONGLY favors the AI-based approach?

Accepted Answer

Invoice layouts vary significantly between different vendors and often change. An AI-based approach strongly favors scenarios where invoice layouts vary significantly between vendors and change over time, because AI models (especially deep learning) can generalize and adapt to variations without manual rule updates. Traditional OCR+rule-based pipelines struggle with such variability.

Answer

The system must process invoices in real time with sub-second latency

Answer

The team has limited access to labeled training data

Answer

Invoices have a fixed, standardized layout across all vendors

Question 12

A company wants to build a conversational agent that can handle complex multi-step tasks such as booking a flight, reserving a hotel, and scheduling a car rental in a single session. The agent must be able to break down the user's request into sub-tasks, call external APIs, and reason about the results. Which design pattern is BEST suited for this requirement?

Accepted Answer

An agentic workflow implementing the ReAct pattern with tool use. The ReAct (Reasoning + Acting) pattern interleaves chain-of-thought reasoning with tool/API calls, allowing the agent to decompose a multi-step request, invoke external services (flight, hotel, car APIs), observe results, and iterate. This is precisely what's needed for orchestrating dependent sub-tasks across multiple systems. Agentic workflows with tool use are the industry-standard design for multi-step task automation with LLMs.

Answer

Retrieval-Augmented Generation (RAG) with a vector store

Answer

A single large language model prompt with all instructions

Answer

Fine-tuning a model on a dataset of flight, hotel, and rental conversations

Question 13

Refer to the exhibit. An AI auditor reviews the fairness configuration. What is the purpose of this policy?

Accepted Answer

Ensure equal positive prediction rates across groups. The policy sets a fairness constraint that requires the model's positive prediction rate (the fraction of instances predicted as the positive class) to be equal across all defined groups. This is a standard demographic parity requirement, which is implemented by adjusting the decision threshold or reweighting training data to ensure that each group receives the same proportion of positive predictions, regardless of the actual outcome distribution.

Answer

Ensure equal error rates across groups

Answer

Ensure equal accuracy across groups

Answer

Ensure model interpretability

Question 14

A developer is using Hugging Face Transformers to fine-tune a BERT model for sentiment analysis. They want to track experiments, log metrics, and compare runs. Which MLOps tool should they integrate?

Accepted Answer

MLflow. MLflow is the correct choice because it is purpose-built for experiment tracking, metric logging, and run comparison in machine learning workflows. It provides an API to log parameters, metrics, and artifacts, and its UI allows easy comparison of different fine-tuning runs, which directly matches the developer's need to track experiments and compare runs for a BERT sentiment analysis model.

Answer

Apache Airflow

Answer

Docker

Answer

Kubeflow

Question 15

A company deploys an AI model to predict equipment failure. The model performs well on historical data but fails to generalize to new data from a different factory. Which concept best describes this issue?

Accepted Answer

Overfitting. (Overfitting) is correct because the model learned patterns specific to the historical data from the original factory, including noise and factory-specific nuances, rather than generalizable features. When applied to new data from a different factory, those learned patterns do not hold, causing poor performance. This is the classic symptom of overfitting: high accuracy on training data but low accuracy on unseen data.

Answer

Transfer learning

Answer

Underfitting

Answer

Bias-variance tradeoff

Question 16

A recommendation system for an e-commerce site is producing stale suggestions that do not reflect recent user behavior. The system is updated offline every 24 hours. Which change would MOST directly address this issue?

Accepted Answer

Implement online learning to update the model incrementally in real time. Implementing online learning to update the model incrementally in real time directly addresses the staleness issue by allowing the model to incorporate recent user behavior as it happens. Online learning updates model parameters continuously or at short intervals, so recommendations reflect the latest interactions. This is the most direct solution to the problem of a 24-hour offline update cycle.

Answer

Increase the number of features used in the model

Answer

Add more training data from the past year

Answer

Use a deeper neural network architecture

Question 17

A company is building a recommendation system that uses user embeddings stored in a vector database. The system must retrieve the top 10 most similar items for a given user query. Which vector database feature is MOST critical for this task?

Accepted Answer

Approximate nearest neighbor (ANN) search. Approximate nearest neighbor (ANN) search is the most critical feature because it enables the vector database to efficiently find the top-10 most similar items to a user query embedding without scanning the entire dataset. Unlike exact nearest neighbor search, ANN algorithms (e.g., HNSW, IVF) trade a small amount of accuracy for massive performance gains, which is essential for real-time recommendation systems handling millions of high-dimensional vectors.

Answer

Built-in data versioning

Answer

ACID transaction support

Answer

SQL query interface

Question 18

A team is deploying an AI model for credit approval. Which TWO ethical considerations must be addressed?

Accepted Answer

Model interpretability. Model interpretability (B) is essential for credit approval because financial decisions must be explainable to regulators and customers under laws like GDPR or ECOA. A black-box model that cannot justify why a loan was denied violates compliance requirements, making interpretability a core ethical and legal necessity.

Answer

Training speed

Answer

Model accuracy

Answer

Model size

Question 19

A data analyst wants to predict housing prices based on square footage, number of bedrooms, and location. Which machine learning approach is most suitable?

Accepted Answer

Linear regression. Linear regression is the most suitable approach because the problem involves predicting a continuous numeric target (housing prices) from multiple independent variables (square footage, bedrooms, location). Linear regression models the linear relationship between the features and the target, providing interpretable coefficients and efficient training for this type of regression task.

Answer

K-means clustering

Answer

Decision tree regression

Answer

Association rule mining

Question 20

A retail analytics team is preparing a dataset for a demand forecasting model. The dataset contains a 'store_id' column with several thousand unique values, a 'product_category' column with about twenty values, and a 'day_of_week' column. The team wants to encode these categorical variables so a tree-based model can use them effectively without creating an enormous number of columns. Which TWO encoding approaches are most appropriate? (Choose two.)

Accepted Answer

Use target encoding for 'store_id', replacing each store with a statistic such as the mean of the target computed from training folds.. High-cardinality identifiers like store_id need a compact numeric representation, and out-of-fold target encoding provides one column that captures store-level signal without exploding dimensionality. Low-cardinality nominal features like product_category and day_of_week are best handled with one-hot encoding, which avoids implying an order and keeps the feature space small. Together these choices match encoding strategy to cardinality.

Answer

Use one-hot encoding for 'store_id' so every store gets its own binary column.

Answer

Use binary encoding for 'day_of_week' so each day is represented by a binary code across multiple columns.

Answer

Use label encoding for 'product_category' so categories become arbitrary integers based on alphabetical order.