AI0-001 Implementing AI Solutions Practice Test 2 — 15 Questions

Question 1

A company wants to build a code generation tool that helps developers write Python functions. The tool must generate syntactically correct code. Which prompt engineering technique is MOST effective?

Accepted Answer

Few-shot prompting with examples of valid Python functions. Few-shot examples showing valid Python function syntax help the model understand the expected output format and generate correct code.

Answer

Chain-of-thought prompting with step-by-step reasoning

Answer

System prompt instructing the model to output JSON

Answer

Instruction fine-tuning on a large Python corpus

Question 2

An AI system performs anomaly detection on sensor data in a manufacturing plant. The model is deployed and running well. After two months, the plant installs new sensors that produce data with a different distribution. The anomaly detection starts failing with many false positives. Which action should the team take?

Accepted Answer

Retrain the model using data from the new sensors and re-deploy. The new sensors produce data with a different distribution, which is a classic case of data drift (covariate shift). The model's learned decision boundary no longer matches the input space, causing false positives. Retraining on data that includes the new sensor distribution realigns the model with the current environment and restores accuracy. This is the standard MLOps remediation for concept/data drift.

Answer

Roll back the model to the previous version and ignore the new sensors

Answer

Increase the anomaly threshold to reduce false positives

Answer

Use an ensemble of the old and a new model without retraining

Question 3

A team is implementing a RAG system. They are designing the document loading and chunking strategy. Which TWO techniques are commonly used for chunking documents? (Select two.)

Accepted Answer

Fixed-size chunking with a token limit. Option A, fixed-size chunking with a token limit, is correct because it is a standard RAG technique that splits documents into uniform chunks based on a maximum number of tokens (or characters), often with an overlap parameter to preserve context across boundaries. Option D, semantic chunking using sentence boundaries, is correct because it groups text into coherent units by detecting natural linguistic boundaries such as sentences or paragraphs, which helps preserve meaning and improves retrieval quality. Option B is not a standard chunking technique; term frequency is used for scoring or keyword extraction, not for defining chunk boundaries. Option C is not typically used as described because character-level chunking with no overlap destroys semantic coherence and is generally inferior for RAG. Option E is not among the commonly cited chunking methods in this context; while document structure can inform chunking, hierarchical chunking is not one of the two standard techniques referenced here.

Answer

Frequency-based chunking by term occurrence

Answer

Character-level chunking with no overlap

Answer

Hierarchical chunking using document structure

Question 4

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Accepted Answer

Use Retrieval-Augmented Generation (RAG) with the policy documents indexed in a vector store. RAG allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

Answer

Fine-tune a base LLM on the policy documents monthly

Answer

Train a custom model from scratch on the policy documents each month

Answer

Use a larger foundation model with a longer context window and paste all documents into each prompt

Question 5

A data science team is building a model to detect fraudulent transactions. They have a dataset of 1 million normal transactions and 1,000 fraudulent ones. What is the MOST effective data preparation step to handle this imbalance?

Accepted Answer

Apply SMOTE to generate synthetic fraudulent transactions and randomly undersample normal transactions. SMOTE (Synthetic Minority Over-sampling Technique) generates new synthetic fraudulent samples by interpolating between existing minority-class instances, while random undersampling of the majority class reduces the 1M normal transactions to a manageable size. Combining both balances the class distribution without discarding all majority data or naively duplicating minority samples, which is the most effective preparation step for a 1000:1 imbalance. This approach improves the model's ability to learn fraud patterns without overfitting to duplicated records.

Answer

Delete all normal transactions until the dataset is balanced

Answer

Duplicate the fraudulent transactions 1,000 times

Answer

Train the model on the original dataset; class imbalance does not affect model performance

Question 6

A developer is implementing a RAG system for legal document review. The documents are long (50-100 pages) with dense sections. They need to chunk the documents in a way that preserves semantic coherence while keeping chunks small enough for effective retrieval. Which chunking strategy is MOST appropriate?

Accepted Answer

Semantic chunking based on paragraph and section boundaries. Semantic chunking based on paragraph and section boundaries is correct because legal documents have dense, logically structured content where meaning is carried by complete paragraphs and sections. Splitting at natural semantic boundaries preserves the context needed for accurate retrieval, ensuring each chunk is a coherent unit that a retriever can match against a legal query. This balances semantic coherence with chunk sizes small enough for effective embedding and retrieval.

Answer

Hierarchical chunking with parent-child relationships

Answer

Fixed-size chunking with 512 tokens and no overlap

Answer

Chunking by a fixed number of sentences without considering content

Question 7

An AI team is deploying a large language model for a customer-facing application. They need to ensure that the model's output is always in valid JSON format for downstream processing. Which prompt engineering technique should they use?

Accepted Answer

Enable JSON mode in the model's API parameters. Enabling JSON mode in the model's API parameters forces the model to output valid JSON, ensuring downstream processing can reliably parse the response. This is a built-in feature of many LLM APIs (e.g., OpenAI's response_format parameter) that constrains the model's output to valid JSON. It is the most direct and reliable technique for guaranteeing JSON format.

Answer

Use few-shot examples of JSON outputs in the prompt

Answer

Post-process the output with a JSON validator and reject invalid responses

Answer

Add a system prompt that says 'You are a helpful assistant.'

Question 8

A team is evaluating an LLM-based code generation assistant. They want to measure the quality of generated code for correctness, security, and efficiency. Which evaluation framework is BEST suited for this task?

Accepted Answer

Pass@k metric using unit tests from benchmarks like HumanEval. The Pass@k metric using unit tests from benchmarks like HumanEval is specifically designed to evaluate code generation models. It measures the probability that at least one of k generated code samples passes all unit tests, directly assessing correctness. Security and efficiency can be evaluated by incorporating additional tests or static analysis, but Pass@k is the standard for functional correctness.

Answer

Human evaluation by a panel of experienced developers

Answer

Perplexity of the model on a code corpus

Answer

BLEU score comparing generated code to reference code

Question 9

A company is deciding between fine-tuning and RAG for a domain-specific legal assistant that must provide accurate answers based on a changing set of statutes and regulations. The statutes are updated quarterly. Which approach is PREFERRED and why?

Accepted Answer

RAG, because it can retrieve the latest documents without retraining the model. RAG is preferred because it retrieves the latest documents from an external knowledge base at inference time, allowing the model to access updated statutes without retraining. This is crucial for a changing set of statutes updated quarterly. Fine-tuning would require retraining the model each time the statutes change, which is costly and time-consuming.

Answer

Fine-tuning, because it allows the model to internalize the statutes for faster inference

Answer

Fine-tuning with LoRA, because it is parameter-efficient and can be updated frequently

Answer

RAG with fine-tuning on the initial statutes, then update the index quarterly

Question 10

A team is setting up a test suite for an AI system that includes a data pipeline, an LLM API call, and an output evaluation step. Which TWO types of tests should they prioritize to ensure the system's reliability?

Accepted Answer

Unit tests for data pipeline components. Option B is correct because unit tests for data pipeline components verify each transformation, parsing, and validation function in isolation, catching logic errors early before they propagate into the LLM prompt or evaluation stage. Option E is correct because integration tests for API calls to the LLM validate the contract between the system and the external model endpoint, including request formatting, authentication, timeouts, retries, and response parsing, which are common failure points in AI pipelines. Together, these two test types address reliability at the component and interface levels, which is the priority for a pipeline feeding an LLM and an evaluation step. Option A is not marked correct because end-to-end tests are valuable but slower and more brittle, so they are not the first priority for ensuring reliability of individual pipeline and API interactions. Option C is not marked correct because regression tests comparing outputs to previous versions are useful for detecting drift but depend on having stable baselines and do not directly validate the pipeline or API contract. Option D is not marked correct because load tests measure performance under high traffic rather than functional reliability of the data pipeline or LLM API integration.

Answer

End-to-end tests simulating full user workflows

Answer

Regression tests comparing output to previous versions

Answer

Load tests to measure performance under high traffic

Question 11

An AI engineer needs to select a similarity measure for comparing dense embedding vectors in a vector store for document retrieval. Which two measures are commonly used?

Accepted Answer

Dot product and cosine similarity. Dot product and cosine similarity are the two most commonly used measures for comparing dense embedding vectors in vector stores. Cosine similarity computes the cosine of the angle between vectors, making it length-invariant, while dot product is efficient and directly related to cosine similarity when vectors are normalized. Both are widely supported in vector databases like FAISS and Pinecone for document retrieval tasks.

Answer

Pearson correlation and Spearman rank

Answer

Cosine similarity and Jaccard similarity

Answer

Euclidean distance and Manhattan distance

Question 12

A company wants to fine-tune a 70B-parameter LLM for a specialized domain but has limited GPU memory (e.g., 24 GB VRAM). Which technique allows fine-tuning with minimal memory footprint?

Accepted Answer

QLoRA (Quantized Low-Rank Adaptation). QLoRA (Quantized Low-Rank Adaptation) enables fine-tuning of very large models like a 70B-parameter LLM on limited GPU memory by quantizing the base model to 4-bit precision and training only small low-rank adapter matrices. This dramatically reduces memory footprint — a 70B model in 4-bit is roughly 35 GB, and with QLoRA's paged optimizers and adapter-only training, it can fit on a single 24 GB GPU with careful configuration. It is the standard technique for memory-constrained fine-tuning of large models.

Answer

Instruction tuning with a smaller dataset

Answer

LoRA (Low-Rank Adaptation) on the base model

Answer

Full fine-tuning with gradient checkpointing

Question 13

A team is building a RAG system with a large repository of technical manuals. They want to ensure that each retrieved chunk is semantically coherent and that related concepts are grouped together. Which chunking strategy is BEST?

Accepted Answer

Semantic chunking using a sentence splitter with topic boundaries. Semantic chunking using a sentence splitter with topic boundaries ensures that each chunk is a self-contained, semantically coherent unit by detecting natural topic shifts (e.g., via embedding similarity or discourse markers). This directly supports the requirement for semantically coherent chunks and grouping of related concepts, unlike methods that ignore content meaning.

Answer

Chunking by page number

Answer

Fixed-size chunking with 512 tokens

Answer

Hierarchical chunking with parent-child relationships

Question 14

A data scientist notices that a model's performance on the training set is excellent, but validation accuracy is poor. The team used the same dataset for feature engineering and model selection. What is the MOST likely cause?

Accepted Answer

Train/test leakage caused by using the same data for feature engineering and model selection. Using the same dataset for feature engineering, model selection, and validation causes train/test leakage, where information from the validation set influences the model, leading to overly optimistic training performance and poor validation accuracy. This is the most likely cause when the team reused data across these stages. The correct answer identifies this leakage explicitly.

Answer

The dataset is too small for the model complexity

Answer

The learning rate is too high

Answer

The model is overfitting due to high variance

Question 15

An organization is deploying an AI agent that uses the ReAct pattern to answer customer queries by calling external APIs. Which THREE components are essential in this agentic workflow?

Accepted Answer

A planning algorithm to decompose tasks. The ReAct (Reason + Act) pattern is defined by interleaving reasoning traces with actions, so option C (multi-step reasoning with a reasoning trace) is essential — the agent must generate explicit thought steps that guide each subsequent action. Option A (a planning algorithm to decompose tasks) is correct because ReAct agents must break a complex customer query into sub-goals and decide the next step to take toward the answer. Option D (tool use / function calling) is correct because the 'Act' half of ReAct requires invoking external APIs to retrieve data or perform actions the LLM cannot do alone. Option B (fine-tuning the LLM on domain-specific data) is not required — ReAct works with a general-purpose LLM via prompting, and fine-tuning is an optional optimization, not an essential component. Option E (a vector store for long-term memory) is also not essential — ReAct's reasoning trace provides short-term working context, and while retrieval can augment an agent, it is not a defining requirement of the pattern.

Answer

Fine-tuning the LLM on domain-specific data

Answer

A vector store for long-term memory