AI0-001 Practice Test 15 — 15 Questions

Question 1

A team wants to deploy a large language model on edge devices with limited memory and compute. They need to reduce model size by at least 50% while preserving accuracy. Which combination of techniques is most effective?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Apply INT8 quantization and weight pruning. INT8 quantization reduces the precision of weights and activations from 32-bit to 8-bit, cutting memory usage by approximately 75% for those tensors, while weight pruning removes redundant connections, often achieving over 50% size reduction with minimal accuracy loss when combined. Together, they directly address the constraints of edge devices by shrinking the model footprint and computational requirements without requiring a complete architecture redesign.

Answer

Distill the model into a smaller architecture without quantization or pruning

Answer

Use FP32 precision and increase batch size

Answer

Use FP16 quantization and add more layers

Question 2

A data scientist is evaluating a binary classification model. The model achieves 95% accuracy on the test set, but the precision is 0.60 and recall is 0.55. The dataset has 90% negative class samples. Which metric should the team focus on to improve the model?

Accepted Answer

F1 score. With 90% negative class samples, accuracy is misleading because a trivial majority-class classifier would score 90%. The low precision (0.60) and recall (0.55) indicate the model struggles on the minority positive class. The F1 score, being the harmonic mean of precision and recall, is the right metric to optimise because it balances both concerns on imbalanced data.

Answer

Perplexity

Answer

BLEU score

Answer

Accuracy

Question 3

Which chunking strategy for RAG is MOST appropriate when documents have a natural hierarchical structure (e.g., sections, subsections)?

Accepted Answer

Hierarchical chunking that preserves document structure. Hierarchical chunking is designed to respect the document's inherent structure—such as sections, subsections, and paragraphs—by creating chunks that align with these boundaries. This preserves the logical flow and context, which is crucial for retrieval-augmented generation (RAG) because the retriever can fetch coherent units that match the query's intent. By maintaining the hierarchy, the system can also leverage parent-child relationships to improve retrieval accuracy and generation quality.

Answer

Fixed-size chunking with no overlap

Answer

Semantic chunking based on sentence boundaries

Answer

Random chunking with varying sizes

Question 4

A data scientist is deploying a machine learning model to production. The model was trained on an imbalanced dataset. Which technique should be used during deployment to mitigate bias without retraining the model?

Accepted Answer

Apply post-processing calibration to adjust decision thresholds. Post-processing calibration adjusts the decision threshold of the model to account for the class imbalance present in the training data. This technique modifies the output probabilities or classification boundary without requiring access to the original training data or retraining the model, making it suitable for deployment scenarios where the model is already fixed.

Answer

Use an ensemble of models trained on balanced subsets

Answer

Rebalance the dataset using SMOTE before inference

Answer

Remove sensitive features from the input data

Question 5

A company uses a third-party AI model for sentiment analysis. They want to create a software bill of materials (SBOM) for this AI system. What is the PRIMARY purpose of an SBOM in this context?

Accepted Answer

To list all software components and dependencies used in the AI system. The primary purpose of an SBOM for an AI system is to provide a complete inventory of all software components, libraries, and dependencies that make up the system. This is critical for vulnerability management, license compliance, and supply chain risk assessment, especially when third-party AI models are integrated. It does not track performance metrics, training details, or user instructions.

Answer

To record the model's accuracy on benchmark datasets

Answer

To document the model's training hyperparameters

Answer

To provide a user manual for the AI model

Question 6

During a red-team exercise on an AI model, testers successfully extracted training data. Which vulnerability is this?

Accepted Answer

Model inversion. Model inversion attacks allow an adversary to reconstruct training data by exploiting the model's learned representations. In this scenario, the testers successfully extracted training data, which is the hallmark of a model inversion attack, not just inferring membership.

Answer

Membership inference

Answer

Adversarial example

Answer

Data poisoning

Question 7

A security team is evaluating the risk of adversarial examples against their image classification model. Which characteristic best describes an adversarial example?

Accepted Answer

An input modified by small, intentional perturbations designed to cause misclassification. An adversarial example is specifically crafted by adding small, often imperceptible perturbations to a legitimate input. These perturbations are designed to exploit the model's decision boundaries, causing it to output an incorrect classification with high confidence. This is a fundamental concept in AI security, highlighting the vulnerability of deep learning models to input manipulation.

Answer

A naturally occurring image that the model misclassifies due to poor training data

Answer

An image that has been resized incorrectly and appears distorted to the model

Answer

A corrupted image with missing pixels that the model cannot process

Question 8

An organization implements AI governance following the NIST AI Risk Management Framework. They need to ensure that all model decisions are logged with sufficient detail for later audit. Which logging requirement is most critical for traceability?

Accepted Answer

Timestamp, input data, output, and model version. Timestamp, input data, output, and model version together provide full traceability for audit. Option A is wrong because logging only input data and model name misses outputs, timestamp, and version, which are essential for traceability. Option B is wrong because source code and training dataset hash are not part of the inference audit trail; they are more relevant to model development. Option C is wrong because logging only model outputs and confidence scores misses inputs and model version, making it impossible to fully trace decisions.

Answer

Input data and model name only

Answer

Source code and training dataset hash

Answer

Model outputs and confidence scores only

Question 9

A company is deploying a text generation model for customer service emails. They want to ensure the model's responses are factual and based on internal knowledge bases. Which technique is most effective?

Accepted Answer

Use Retrieval-Augmented Generation (RAG). Retrieval-Augmented Generation (RAG) is the most effective technique because it combines a pre-trained language model with a retrieval system that fetches relevant documents from an internal knowledge base at inference time. This ensures the model's responses are grounded in factual, up-to-date information from the knowledge base, reducing hallucinations.

Answer

Fine-tune the model on historical customer service emails

Answer

Write a detailed system prompt

Answer

Set the temperature to 0

Question 10

During testing of a customer service chatbot, the team notices that the model sometimes generates plausible-sounding but factually incorrect answers about company policies. Which evaluation approach is BEST to systematically detect and quantify this issue?

Accepted Answer

Evaluation framework with faithfulness and answer relevancy metrics on a held-out test set. An evaluation framework with faithfulness and answer relevancy metrics on a held-out test set is the best approach because it directly measures whether the model's output is grounded in the provided source (faithfulness) and whether it actually addresses the user's question (answer relevancy). These are the standard RAG/LLM evaluation metrics designed to detect hallucinations and off-topic answers systematically. A held-out test set ensures the measurement is repeatable and quantifiable across model versions.

Answer

Regression testing comparing old and new model outputs

Answer

Unit tests on the data pipeline

Answer

Integration tests for API calls

Question 11

An AI model's performance drops significantly in production compared to testing. The data shows distribution shift. What is the best first step?

Accepted Answer

Retrain model with new data. When distribution shift occurs, the model's assumptions about the data distribution no longer hold, so the best first step is to retrain the model with new data that reflects the current distribution. This directly addresses the root cause by updating the model to learn the new patterns, rather than applying superficial fixes like adding features or changing algorithms.

Answer

Add more features

Answer

Use a different algorithm

Answer

Reduce model complexity

Question 12

A company deployed a machine learning model on a cloud inference service. Users report high latency during peak hours. The model is deployed on a single instance. Which action should the team take to reduce latency without significant architectural changes?

Accepted Answer

Enable autoscaling for the inference instances. Enabling autoscaling for the inference instances directly addresses the root cause: a single instance cannot handle peak-hour traffic, causing queuing and high latency. Autoscaling horizontally adds more instances to distribute the load, reducing per-request response time without re-architecting the system. This is a standard cloud-native pattern for stateless inference services and requires minimal configuration change.

Answer

Increase the model size to improve accuracy

Answer

Switch to a batch inference pipeline

Answer

Add an API gateway to route requests

Question 13

A company deploys an LLM chatbot that has access to a database of customer orders. They want to prevent the LLM from revealing order details unless the user is authenticated as the owner. Which security control should be implemented?

Accepted Answer

Access controls on the model and API. Access controls on the model and API (Option D) are the correct security control because they enforce authentication and authorization at the API gateway or model endpoint level, ensuring that only the authenticated owner can query their own order details. This prevents unauthorized users from invoking the LLM to retrieve sensitive data, regardless of the prompt content. Without such access controls, the LLM would have no inherent mechanism to verify user identity before processing requests.

Answer

Output filtering

Answer

Rate limiting

Answer

Input validation and sanitization

Question 14

An AI developer observes that the training accuracy of a neural network is high, but the test accuracy is low. The model uses a ReLU activation function and Adam optimizer. Which approach is most likely to improve test accuracy?

Accepted Answer

Add L2 regularization to the loss function. The scenario describes overfitting: high training accuracy but low test accuracy. Adding L2 regularization (weight decay) penalizes large weights, which reduces model complexity and improves generalization to unseen data. This directly addresses the overfitting problem without altering the model architecture or optimization algorithm.

Answer

Increase the learning rate

Answer

Switch to a stochastic gradient descent optimizer

Answer

Increase the number of epochs

Question 15

A team deploys a machine learning model as a REST API. They want to monitor model drift. Which metric is MOST appropriate for detecting drift in the input data distribution?

Accepted Answer

Population stability index (PSI) comparing training and recent data.. Population stability index (PSI) is the most appropriate metric for detecting drift in input data distribution because it directly measures the shift between the training data distribution and the recent production data distribution. PSI is calculated by binning both distributions and computing the sum of (proportion in bin of recent data minus proportion in bin of training data) times the natural log of their ratio, making it sensitive to changes in feature distributions without requiring ground truth labels.

Answer

Model accuracy on a recent holdout set.

Answer

F1 score on the training data.

Answer

Root mean squared error (RMSE) on test data.