AI0-001 Practice Test 18 — 15 Questions

Question 1

A machine learning engineer is deploying a production model that requires strict monitoring. Which TWO monitoring strategies should be implemented to detect data drift and model degradation? (Choose TWO.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Monitoring the distribution of model predictions. Tracking prediction distribution shifts and monitoring feature distribution over time help detect drift and degradation.

Answer

Logging all input data for manual review

Answer

Monitoring prediction confidence scores over time

Answer

Retraining the model weekly as a routine

Question 2

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Accepted Answer

Use Retrieval-Augmented Generation (RAG) with the policy documents indexed in a vector store. RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

Answer

Fine-tune a base LLM on the policy documents monthly

Answer

Train a custom model from scratch on the policy documents each month

Answer

Use a larger foundation model with a longer context window and paste all documents into each prompt

Question 3

A team is building a recommendation system for an e-commerce platform. They want to use collaborative filtering but have a cold-start problem for new users. Which hybrid approach BEST addresses cold start while leveraging collaborative signals?

Accepted Answer

Implement a hybrid model that combines content-based features with collaborative filtering via a weighted ensemble. Content-based filtering uses user/item features to handle cold start, then combines with collaborative filtering for accuracy. Clustering or matrix factorization alone do not solve cold start.

Answer

Apply user clustering based on demographic data and then use collaborative filtering within clusters

Answer

Use only content-based filtering for all users

Answer

Use matrix factorization with implicit feedback only

Question 4

An AI agent is designed to book flights by calling an external API. The agent must decide which tool to call based on user input, then generate the correct API parameters. Which pattern is MOST appropriate for this workflow?

Accepted Answer

ReAct pattern with tool descriptions and function calling. The ReAct (Reasoning + Acting) pattern interleaves reasoning steps with tool calls, allowing the agent to decide when to call a function and what arguments to use.

Answer

Chain-of-thought prompting only

Answer

Zero-shot prompting with JSON mode

Answer

Simple prompt with no tool descriptions

Question 5

A team is fine-tuning a large language model using LoRA. They have limited GPU memory. Which technique can further reduce memory consumption while maintaining similar fine-tuning quality?

Accepted Answer

Use QLoRA with 4-bit quantization of the base model. QLoRA combines 4-bit quantization of the base model with LoRA adapters, drastically reducing memory usage while preserving fine-tuning performance.

Answer

Fine-tune all layers instead of using LoRA

Answer

Increase the rank of LoRA adapters

Answer

Use a larger batch size

Question 6

A data scientist is preparing a dataset for training a customer churn prediction model. To prevent train/test leakage, which TWO practices should be followed? (Select TWO)

Accepted Answer

Split the data chronologically (e.g., use data before a certain date for training, after for testing). To prevent leakage, time-based splitting respects temporal order (no future data in training). Not normalizing before splitting avoids information from the test set influencing training. The other options either cause leakage or are unrelated.

Answer

Remove duplicate records only from the test set to ensure uniqueness

Answer

Shuffle the entire dataset randomly before splitting into train and test sets

Answer

Normalize numerical features using statistics computed on the entire dataset before splitting

Question 7

A data science team is fine-tuning a large language model for a domain-specific task using LoRA. They have a limited GPU budget and want to minimize memory usage during training. Which technique should they use?

Accepted Answer

Use QLoRA (Quantized LoRA) with a 4-bit quantized base model. QLoRA (Quantized LoRA) quantizes the base model to 4-bit, drastically reducing memory usage while still applying LoRA adapters. Standard LoRA uses full precision. PEFT is a category, not a specific technique. Full fine-tuning uses the most memory.

Answer

Use LoRA (Low-Rank Adaptation) with the base model in full precision

Answer

Use PEFT (Parameter-Efficient Fine-Tuning) without specifying a specific method

Answer

Full fine-tuning of the entire model

Question 8

A company wants to fine-tune a 70B-parameter LLM for a specialized domain but has limited GPU memory (e.g., 24 GB VRAM). Which technique allows fine-tuning with minimal memory footprint?

Accepted Answer

QLoRA (Quantized Low-Rank Adaptation). QLoRA (Quantized Low-Rank Adaptation) uses 4-bit quantization and low-rank adapters, reducing memory requirements dramatically while preserving fine-tuning quality.

Answer

Instruction tuning with a smaller dataset

Answer

LoRA (Low-Rank Adaptation) on the base model

Answer

Full fine-tuning with gradient checkpointing

Question 9

During testing of a customer service chatbot, the team notices that the model sometimes generates plausible-sounding but factually incorrect answers about company policies. Which evaluation approach is BEST to systematically detect and quantify this issue?

Accepted Answer

Evaluation framework with faithfulness and answer relevancy metrics on a held-out test set. A comprehensive LLM output evaluation framework using a ground-truth dataset of question-answer pairs and metrics like faithfulness and answer relevancy can detect hallucination systematically.

Answer

Regression testing comparing old and new model outputs

Answer

Unit tests on the data pipeline

Answer

Integration tests for API calls

Question 10

A team is setting up a test suite for an AI system that includes a data pipeline, an LLM API call, and an output evaluation step. Which TWO types of tests should they prioritize to ensure the system's reliability?

Accepted Answer

Unit tests for data pipeline components. Unit tests for data pipelines catch data transformation errors early, and integration tests for API calls verify that the LLM endpoint responds correctly. E2E tests and regression tests are also important but not the first priority for reliability.

Answer

End-to-end tests simulating full user workflows

Answer

Regression tests comparing output to previous versions

Answer

Load tests to measure performance under high traffic

Question 11

A recommendation system for an e-commerce platform is experiencing a high false positive rate in its anomaly detection module, causing legitimate transactions to be flagged as fraudulent. The team wants to reduce false positives without significantly increasing false negatives. Which action is MOST effective?

Accepted Answer

Increase the anomaly detection threshold. Adjusting the classification threshold to be more conservative (requiring higher anomaly score) will reduce false positives at the cost of some increase in false negatives, but the goal is to minimize false positives while maintaining acceptable recall.

Answer

Decrease the anomaly detection threshold

Answer

Use a different anomaly detection algorithm

Answer

Increase the size of the training dataset

Question 12

A company wants to build a conversational agent that can handle complex multi-step tasks such as booking a flight, reserving a hotel, and scheduling a car rental in a single session. The agent must be able to break down the user's request into sub-tasks, call external APIs, and reason about the results. Which design pattern is BEST suited for this requirement?

Accepted Answer

An agentic workflow implementing the ReAct pattern with tool use. Agentic workflows, particularly the ReAct pattern, combine reasoning and acting (tool calls) allowing the agent to iteratively decompose tasks, use APIs, and adapt based on results.

Answer

Retrieval-Augmented Generation (RAG) with a vector store

Answer

A single large language model prompt with all instructions

Answer

Fine-tuning a model on a dataset of flight, hotel, and rental conversations

Question 13

A team is implementing a RAG system. They are designing the document loading and chunking strategy. Which TWO techniques are commonly used for chunking documents? (Select two.)

Accepted Answer

Fixed-size chunking with a token limit. Fixed-size chunking (based on token count) and semantic chunking (based on natural boundaries) are both standard approaches. Hierarchical chunking is less common, and character-level is rarely used. Overlap is a parameter, not a chunking strategy.

Answer

Frequency-based chunking by term occurrence

Answer

Character-level chunking with no overlap

Answer

Hierarchical chunking using document structure

Question 14

A data scientist is training a binary classifier and observes that the training accuracy is 99% but the test accuracy is only 70%. Which of the following is the MOST likely cause?

Accepted Answer

The model is overfitting the training data. A large gap between training and test accuracy indicates overfitting: the model memorized training data but fails to generalize.

Answer

The model is underfitting the training data

Answer

The learning rate is too high

Answer

The test set contains data leakage from the training set

Question 15

Which THREE are common machine learning algorithms used for regression?

Accepted Answer

Linear regression. Linear regression is a fundamental supervised learning algorithm used for regression tasks, where the goal is to predict a continuous numeric output based on one or more input features. It models the relationship between the dependent and independent variables by fitting a linear equation to the observed data, making it a core algorithm for regression problems.

Answer

Logistic regression

Answer

K-means