AI0-001 Implementing AI Solutions Practice Test 7 — 15 Questions

Question 1

A company wants to build a conversational agent that can handle complex multi-step tasks such as booking a flight, reserving a hotel, and scheduling a car rental in a single session. The agent must be able to break down the user's request into sub-tasks, call external APIs, and reason about the results. Which design pattern is BEST suited for this requirement?

Accepted Answer

An agentic workflow implementing the ReAct pattern with tool use. Agentic workflows, particularly the ReAct pattern, combine reasoning and acting (tool calls) allowing the agent to iteratively decompose tasks, use APIs, and adapt based on results.

Answer

Retrieval-Augmented Generation (RAG) with a vector store

Answer

A single large language model prompt with all instructions

Answer

Fine-tuning a model on a dataset of flight, hotel, and rental conversations

Question 2

During the data preparation phase of an AI project, a data scientist discovers that the target variable in a binary classification dataset is heavily imbalanced: 95% negative class and 5% positive class. Which technique should be applied to improve model performance on the minority class?

Accepted Answer

Apply oversampling of the minority class using techniques like SMOTE. Oversampling the minority class (e.g., SMOTE) or undersampling the majority class are standard techniques to handle imbalanced datasets and improve recall on the minority class.

Answer

Remove all samples from the majority class to balance the dataset

Answer

Normalize all numerical features to have zero mean and unit variance

Answer

Use a train-test split of 80-20 without any modification

Question 3

A developer is implementing a RAG system for legal document review. The documents are long (50-100 pages) with dense sections. They need to chunk the documents in a way that preserves semantic coherence while keeping chunks small enough for effective retrieval. Which chunking strategy is MOST appropriate?

Accepted Answer

Semantic chunking based on paragraph and section boundaries. Semantic chunking splits text at natural boundaries (e.g., paragraphs, sections) while ensuring each chunk is coherent, which is crucial for legal documents where meaning can span multiple sentences.

Answer

Hierarchical chunking with parent-child relationships

Answer

Fixed-size chunking with 512 tokens and no overlap

Answer

Chunking by a fixed number of sentences without considering content

Question 4

An AI team is deploying a large language model for a customer-facing application. They need to ensure that the model's output is always in valid JSON format for downstream processing. Which prompt engineering technique should they use?

Accepted Answer

Enable JSON mode in the model's API parameters. Structured output via JSON mode (available in many LLM APIs) constrains the model to output only valid JSON, which is critical for programmatic consumption.

Answer

Use few-shot examples of JSON outputs in the prompt

Answer

Post-process the output with a JSON validator and reject invalid responses

Answer

Add a system prompt that says 'You are a helpful assistant.'

Question 5

A data science team is training an image classification model for a medical imaging application. To prevent data leakage, they must partition the dataset correctly. Which approach ensures that no patient images appear in both training and test sets?

Accepted Answer

Split by patient ID so that all images of a patient go to one set only. Data leakage occurs when information from the test set leaks into training. Splitting by patient ID ensures that all images from the same patient are kept together in one partition.

Answer

Shuffle the dataset and take the first 80% for training and last 20% for testing

Answer

Use k-fold cross-validation without grouping

Answer

Randomly split all images into training and test sets

Question 6

A company is fine-tuning a large language model using PEFT (Parameter-Efficient Fine-Tuning) to reduce GPU memory usage. They have limited hardware and need to fine-tune a 70B parameter model on a single GPU with 24 GB VRAM. Which technique is MOST suitable?

Accepted Answer

QLoRA (Quantization-aware LoRA) with 4-bit quantization. QLoRA combines quantization (4-bit) and LoRA to fine-tune very large models on limited hardware, achieving significant memory reduction while maintaining performance.

Answer

Full fine-tuning with gradient checkpointing

Answer

Instruction tuning with a smaller 7B model

Answer

LoRA (Low-Rank Adaptation) alone

Question 7

In the AI project lifecycle, after a model is trained and evaluated, it is deployed to a production environment. What is the NEXT critical step to ensure the model continues to perform well over time?

Accepted Answer

Monitoring the model's performance and data drift. Monitoring tracks model performance, data drift, and concept drift in production, triggering retraining or alerts when degradation is detected.

Answer

Collect more training data

Answer

Archive the model and start a new project

Answer

Re-train the model from scratch

Question 8

A developer is building an AI microservice that processes document intelligence requests asynchronously. Users upload PDFs, and the service extracts text and analyzes it with an LLM. The processing time per document can be up to 5 minutes. Which integration pattern is MOST appropriate?

Accepted Answer

Async processing with a message queue and separate worker service. Async processing queues decouple the frontend from long-running tasks, allowing the user to receive results later without blocking.

Answer

Synchronous REST API call that waits for the LLM response

Answer

WebSocket connection for real-time streaming

Answer

Serverless function triggered by HTTP request

Question 9

A team is evaluating an LLM-based code generation assistant. They want to measure the quality of generated code for correctness, security, and efficiency. Which evaluation framework is BEST suited for this task?

Accepted Answer

Pass@k metric using unit tests from benchmarks like HumanEval. HumanEval and similar frameworks (e.g., MBPP) use unit tests to automatically assess functional correctness of generated code, which is the most objective measure for code generation tasks.

Answer

Human evaluation by a panel of experienced developers

Answer

Perplexity of the model on a code corpus

Answer

BLEU score comparing generated code to reference code

Question 10

When implementing a vector store for a RAG system, which similarity search metric is MOST commonly used to find the most relevant document chunks for a given query embedding?

Accepted Answer

Cosine similarity. Cosine similarity is the most common metric for comparing embedding vectors in RAG because it measures the angle between vectors, which works well for high-dimensional semantic embeddings.

Answer

Manhattan distance

Answer

Euclidean distance

Answer

Dot product

Question 11

A company is deciding between fine-tuning and RAG for a domain-specific legal assistant that must provide accurate answers based on a changing set of statutes and regulations. The statutes are updated quarterly. Which approach is PREFERRED and why?

Accepted Answer

RAG, because it can retrieve the latest documents without retraining the model. RAG is preferred when the knowledge base changes frequently, as it retrieves the latest documents at inference time without requiring model retraining.

Answer

Fine-tuning, because it allows the model to internalize the statutes for faster inference

Answer

Fine-tuning with LoRA, because it is parameter-efficient and can be updated frequently

Answer

RAG with fine-tuning on the initial statutes, then update the index quarterly

Question 12

A recommendation system for an e-commerce platform is experiencing a high false positive rate in its anomaly detection module, causing legitimate transactions to be flagged as fraudulent. The team wants to reduce false positives without significantly increasing false negatives. Which action is MOST effective?

Accepted Answer

Increase the anomaly detection threshold. Adjusting the classification threshold to be more conservative (requiring higher anomaly score) will reduce false positives at the cost of some increase in false negatives, but the goal is to minimize false positives while maintaining acceptable recall.

Answer

Decrease the anomaly detection threshold

Answer

Use a different anomaly detection algorithm

Answer

Increase the size of the training dataset

Question 13

A team is setting up a test suite for an AI system that includes a data pipeline, an LLM API call, and an output evaluation step. Which TWO types of tests should they prioritize to ensure the system's reliability?

Accepted Answer

Unit tests for data pipeline components. Unit tests for data pipelines catch data transformation errors early, and integration tests for API calls verify that the LLM endpoint responds correctly. E2E tests and regression tests are also important but not the first priority for reliability.

Answer

End-to-end tests simulating full user workflows

Answer

Regression tests comparing output to previous versions

Answer

Load tests to measure performance under high traffic

Question 14

A data scientist is preparing a dataset for a text classification model. To prevent train/test leakage, which THREE practices should they follow?

Accepted Answer

Use time-based splitting for temporal data. Option B is correct because time-based splitting preserves the temporal order of data, which is critical for time-series or temporal text data to prevent the model from learning from future information that would not be available at inference time. This avoids train/test leakage where future data leaks into the training set, artificially inflating model performance.

Answer

Shuffle the entire dataset before splitting to ensure randomness

Answer

Apply feature scaling to the entire dataset before splitting

Question 15

A developer is building an AI agent that needs to call external tools (e.g., weather API, database) and reason about the results to answer user queries. Which THREE components are essential for implementing this agentic workflow?

Accepted Answer

Planning capability (e.g., step-by-step decomposition). Option A is correct because planning capability enables the agent to decompose complex user queries into manageable sub-tasks, such as retrieving weather data before making a recommendation. This step-by-step reasoning is critical for multi-step workflows where the order of tool calls affects the final answer. Without planning, the agent would lack the structured approach needed to handle dependencies between external tool outputs.

Answer

Fine-tuned domain-specific model

Answer

A vector store for long-term memory