AI0-001 Implementing AI Solutions Practice Test 1 — 15 Questions

Question 1

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use Retrieval-Augmented Generation (RAG) with the policy documents indexed in a vector store. Retrieval-Augmented Generation (RAG) is the most appropriate approach because it allows the chatbot to answer questions based on the latest policy documents without retraining the model. By indexing the documents in a vector store, the system retrieves relevant chunks for each query and passes them to a pre-trained LLM for generation, ensuring up-to-date responses with minimal maintenance overhead.

Answer

Train a custom model from scratch on the policy documents each month

Answer

Fine-tune a base LLM on the policy documents monthly

Answer

Use a larger foundation model with a longer context window and paste all documents into each prompt

Question 2

A team is evaluating a fine-tuned LLM for a code generation task. They notice the model rarely generates correct syntax but often produces plausible-looking code. Which evaluation metric is MOST appropriate to quantify this issue?

Accepted Answer

Pass@k (execution success rate). Pass@k measures the probability that at least one of k generated code samples passes a set of unit tests, directly quantifying execution correctness. Since the model produces plausible-looking but syntactically incorrect code, execution success rate (Pass@k) is the most appropriate metric to capture whether the code actually runs correctly, unlike surface-level similarity metrics.

Answer

BLEU score

Answer

Perplexity

Answer

Exact match accuracy

Question 3

A company is fine-tuning an LLM for a domain-specific task using LoRA. They have limited GPU memory and need to reduce memory footprint without sacrificing fine-tuning quality. Which approach should they consider?

Accepted Answer

Use QLoRA with 4-bit quantized base model. QLoRA combines 4-bit NormalFloat quantization of the base model with LoRA adapters, drastically reducing GPU memory usage while preserving fine-tuning quality through techniques like double quantization and paged optimizers. This directly addresses the constraint of limited GPU memory without sacrificing the model's ability to learn domain-specific tasks effectively.

Answer

Use a larger batch size to speed up training

Answer

Fine-tune all layers of the base model

Answer

Increase the rank of LoRA adapters

Question 4

A team is developing an AI agent to assist users with multi-step tasks such as booking a flight, reserving a hotel, and scheduling a car rental. The agent needs to reason about the order of steps and handle dependencies. Which pattern is BEST suited?

Accepted Answer

ReAct pattern (Reasoning and Acting). The ReAct pattern (Reasoning and Acting) is best suited because it interleaves reasoning traces with tool calls, allowing the agent to dynamically plan and adjust steps based on intermediate results. For multi-step tasks with dependencies (e.g., booking a flight before a hotel), ReAct enables the agent to reason about order, handle failures, and call external APIs step-by-step, which is essential for robust task completion.

Answer

Simple tool use without reasoning

Answer

Using a single prompt with all instructions

Answer

Fine-tuning a model to output all steps at once

Question 5

A team is implementing a RAG system for legal document retrieval. The documents are long and cover multiple topics. Which chunking strategy is MOST appropriate to ensure each chunk contains coherent information?

Accepted Answer

Semantic chunking based on topic boundaries. Semantic chunking based on topic boundaries is the most appropriate strategy because legal documents are long and cover multiple topics. By splitting at natural topic shifts (e.g., clauses, sections, or argument transitions), each chunk preserves coherent meaning, which is critical for accurate retrieval and generation in a RAG system. This approach avoids mixing unrelated content within a single chunk, which would degrade the quality of retrieved context.

Answer

Hierarchical chunking with overlapping windows

Answer

Fixed-size chunking with 512 tokens

Answer

Character-level chunking with no overlap

Question 6

A data science team is building a binary classifier to detect fraudulent transactions. The dataset has only 2% fraud cases. Which data preparation technique is MOST critical to address this imbalance?

Accepted Answer

Apply synthetic minority oversampling (SMOTE) to the training set. With only 2% fraud cases, the dataset is severely imbalanced, which can cause the classifier to be biased toward the majority class (non-fraud) and achieve high accuracy without learning to detect fraud. SMOTE (Synthetic Minority Oversampling Technique) addresses this by generating synthetic examples of the minority class (fraud) in the training set, balancing the class distribution and improving the model's ability to generalize to fraud cases. This is the most critical technique among the options because it directly tackles the class imbalance problem, which is the primary challenge in this scenario.

Answer

Use one-hot encoding on categorical features

Answer

Remove outliers from the transaction amounts

Answer

Normalize all numerical features to have zero mean and unit variance

Question 7

An AI engineer needs to select a similarity measure for comparing dense embedding vectors in a vector store for document retrieval. Which two measures are commonly used?

Accepted Answer

Dot product and cosine similarity. Dot product and cosine similarity are the two most commonly used measures for comparing dense embedding vectors in vector stores. Cosine similarity computes the cosine of the angle between vectors, making it length-invariant, while dot product is efficient and directly related to cosine similarity when vectors are normalized. Both are widely supported in vector databases like FAISS and Pinecone for document retrieval tasks.

Answer

Pearson correlation and Spearman rank

Answer

Cosine similarity and Jaccard similarity

Answer

Euclidean distance and Manhattan distance

Question 8

A data science team is preparing a dataset for a binary classification model. The dataset has 95% negative class and 5% positive class. Which technique should they apply to avoid biased model predictions?

Accepted Answer

Apply resampling techniques such as SMOTE or random undersampling. Handling imbalanced data (e.g., oversampling the minority class or undersampling the majority class) is necessary to prevent the model from always predicting the majority class.

Answer

Normalise all numerical features to a [0,1] range

Answer

Shuffle the dataset randomly before splitting into train and test sets

Answer

Remove all rows with missing values

Question 9

In the AI project lifecycle, which phase involves partitioning the dataset into training, validation, and test sets?

Accepted Answer

Data preparation. Data preparation includes splitting the data to evaluate model performance and prevent leakage.

Answer

Data acquisition

Answer

Model selection

Answer

Problem definition

Question 10

A developer is implementing a RAG system and needs to chunk large legal documents. The documents contain nested clauses and cross-references that should not be split across chunks. Which chunking strategy is MOST suitable?

Accepted Answer

Semantic chunking based on sentence and paragraph boundaries. Semantic chunking uses sentence boundaries or natural breakpoints, preserving logical units.

Answer

Random chunking with varying sizes

Answer

Hierarchical chunking combining small and large chunks

Answer

Fixed-size chunking with 256 tokens

Question 11

An AI system uses a pre-trained image classification model to detect defects in manufacturing. The team wants to deploy the model in an edge device with limited GPU memory. Which technique should they consider first?

Accepted Answer

Apply quantization to reduce model size. Quantization reduces the precision of the model's weights and activations (e.g., from 32-bit floating point to 8-bit integer), which significantly shrinks the model size and memory footprint while often maintaining acceptable accuracy. This is the most direct and effective first step for deploying a pre-trained model on an edge device with limited GPU memory, as it requires no retraining and immediately addresses the memory constraint.

Answer

Train the model from scratch using a smaller dataset

Answer

Use a larger model with more parameters for higher accuracy

Answer

Increase the batch size to improve throughput

Question 12

A team is deploying a multi-modal AI model that processes both text and images. They need to ensure that inference requests are handled quickly even during traffic spikes. Which integration pattern is BEST suited for this use case?

Accepted Answer

Use an event-driven architecture with a queue and worker instances. Asynchronous processing with a message queue allows requests to be buffered and processed without blocking the user, scaling out workers as needed.

Answer

Deploy a synchronous REST API with auto-scaling

Answer

Stream responses directly from the model to the client

Answer

Pre-compute all possible outputs and cache them

Question 13

In prompt engineering, which technique involves providing a few correct input-output examples in the prompt to guide the model's response?

Accepted Answer

Few-shot prompting. Few-shot learning includes examples in the prompt to demonstrate the desired output format and reasoning pattern.

Answer

System prompt engineering

Answer

Chain-of-thought prompting

Answer

Zero-shot prompting

Question 14

A recommendation system for an e-commerce site is producing stale suggestions that do not reflect recent user behavior. The system is updated offline every 24 hours. Which change would MOST directly address this issue?

Accepted Answer

Implement online learning to update the model incrementally in real time. Shorter update cycles or online learning can incorporate recent interactions more quickly, improving freshness.

Answer

Increase the number of features used in the model

Answer

Add more training data from the past year

Answer

Use a deeper neural network architecture

Question 15

During testing a chatbot, the QA team observes that the bot sometimes responds with harmful content when given adversarial prompts. Which type of testing should be prioritised to catch these edge cases?

Accepted Answer

Red-teaming and adversarial testing. Red-teaming and adversarial testing are specifically designed to probe an AI system for vulnerabilities, including generating harmful or unsafe outputs from adversarial prompts. This approach simulates real-world attacks to uncover edge cases that standard functional tests miss, making it the correct priority for catching harmful content in a chatbot.

Answer

Unit tests for data pipeline functions

Answer

Regression testing on previously fixed bugs

Answer

Integration tests for API connectivity