AI0-001 Implementing AI Solutions Practice Test 5 — 15 Questions

Question 1

A company is building a RAG-based Q&A system for a large collection of technical manuals. They need to choose an embedding model and a similarity search method. Which TWO choices are most appropriate for this scenario? (Select TWO)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use a domain-specific embedding model fine-tuned on technical documentation. Cosine similarity is the standard metric for comparing embeddings (normalized vectors). Using a domain-specific embedding model (e.g., fine-tuned on technical text) yields better retrieval accuracy. Dot product can be used but is less common; Euclidean distance is not ideal for high-dimensional embeddings; a general-purpose model may perform poorly on domain-specific language.

Answer

Use a general-purpose embedding model like text-embedding-ada-002

Answer

Use dot product as the similarity metric for non-normalized embeddings

Answer

Use Euclidean distance as the similarity metric for vector search

Question 2

A team is deploying a fine-tuned LLM for generating code snippets. They want to test the system thoroughly before production. Which THREE testing types should they include in their test plan? (Select THREE)

Accepted Answer

Integration tests for API calls between the application and the LLM endpoint. Unit tests (E) validate data pipelines for training and inference data quality. Integration tests (A) verify that API calls between the application and the LLM endpoint function correctly. An evaluation framework (D) assesses LLM output quality, e.g., correctness or syntax using metrics like BLEU. Load testing (C) is for performance and scalability, not for testing correctness; regression testing (B) is important but typically performed after model updates, not as a primary testing type. Therefore, the three core testing types are integration tests, evaluation framework, and unit tests (A, D, E).

Answer

Regression testing only after model updates

Answer

Load testing to determine maximum concurrent users

Question 3

A data scientist is preparing a dataset for training a customer churn prediction model. To prevent train/test leakage, which TWO practices should be followed? (Select TWO)

Accepted Answer

Split the data chronologically (e.g., use data before a certain date for training, after for testing). To prevent leakage, time-based splitting respects temporal order (no future data in training). Not normalizing before splitting avoids information from the test set influencing training. The other options either cause leakage or are unrelated.

Answer

Remove duplicate records only from the test set to ensure uniqueness

Answer

Shuffle the entire dataset randomly before splitting into train and test sets

Answer

Normalize numerical features using statistics computed on the entire dataset before splitting

Question 4

A data scientist is preparing a dataset for a binary classification model. The dataset has 95% majority class and 5% minority class. Which data preparation technique is BEST to address the class imbalance?

Accepted Answer

SMOTE oversampling of the minority class. SMOTE (Synthetic Minority Oversampling TEchnique) generates synthetic samples for the minority class, balancing the dataset without simply duplicating existing minority instances.

Answer

Min-max normalization of all features

Answer

Random undersampling of the majority class

Answer

Removing all minority class samples

Question 5

A team is building a recommendation system for an e-commerce platform. They need to update recommendations in real-time as users browse. Which integration pattern is MOST suitable?

Accepted Answer

AI microservices with a REST API. AI microservices deployed behind a REST API allow each recommendation request to be handled independently, scaling horizontally to meet real-time demands.

Answer

Streaming responses from a single monolithic model

Answer

Batch processing with nightly updates

Answer

Async processing queue with delayed responses

Question 6

During testing of a customer service chatbot, the team notices that the model sometimes generates plausible-sounding but factually incorrect answers about company policies. Which evaluation approach is BEST to systematically detect and quantify this issue?

Accepted Answer

Evaluation framework with faithfulness and answer relevancy metrics on a held-out test set. A comprehensive LLM output evaluation framework using a ground-truth dataset of question-answer pairs and metrics like faithfulness and answer relevancy can detect hallucination systematically.

Answer

Regression testing comparing old and new model outputs

Answer

Unit tests on the data pipeline

Answer

Integration tests for API calls

Question 7

An AI practitioner is fine-tuning a large language model for a domain-specific task using a small labeled dataset (500 examples). They have limited GPU memory. Which technique is MOST suitable?

Accepted Answer

QLoRA (Quantized Low-Rank Adaptation). QLoRA (Quantized Low-Rank Adaptation) is the most suitable technique because it combines 4-bit quantization of the base model with low-rank adapter modules, drastically reducing GPU memory usage while still allowing fine-tuning on a small dataset. This approach preserves the model's pre-trained knowledge and avoids catastrophic forgetting, which is critical when only 500 labeled examples are available.

Answer

Full fine-tuning of all model parameters

Answer

Instruction tuning with the full dataset

Answer

Retrieval-Augmented Generation (RAG) without fine-tuning

Question 8

A developer is building an AI agent that needs to call external APIs (e.g., get weather, send email) based on user requests. Which pattern is BEST for enabling the agent to autonomously decide when to call these APIs?

Accepted Answer

Implement function calling in the LLM to generate structured API calls. Function calling allows the LLM to output structured commands that invoke specific API functions, enabling autonomous tool use in a controlled manner.

Answer

Hard-code the API calls in the agent's logic

Answer

Use a chain-of-thought prompt to reason about the steps

Answer

Use a planning agent with a predefined workflow

Question 9

A team is training a image classification model. They split the dataset into training, validation, and test sets. After training, the model achieves 98% accuracy on the training set but only 72% on the test set. Which step in the AI project lifecycle should the team focus on?

Accepted Answer

Model selection – use regularization or reduce model complexity. The large gap indicates overfitting, which is a model selection/regularization issue. They need to apply techniques like dropout, data augmentation, or reduce model complexity.

Answer

Data acquisition – collect more data

Answer

Deployment – re-deploy with a different serving framework

Answer

Data preparation – check for train/test leakage

Question 10

Which similarity search metric is BEST for comparing dense vector embeddings when the magnitude of the vectors is not important, only the direction?

Accepted Answer

Cosine similarity. Cosine similarity measures the cosine of the angle between vectors, focusing on direction and ignoring magnitude, which is ideal when only direction matters.

Answer

Euclidean distance

Answer

Dot product

Answer

Manhattan distance

Question 11

A company wants to build a code generation tool that helps developers write Python functions. The tool must generate syntactically correct code. Which prompt engineering technique is MOST effective?

Accepted Answer

Few-shot prompting with examples of valid Python functions. Few-shot examples showing valid Python function syntax help the model understand the expected output format and generate correct code.

Answer

Chain-of-thought prompting with step-by-step reasoning

Answer

System prompt instructing the model to output JSON

Answer

Instruction fine-tuning on a large Python corpus

Question 12

An AI system performs anomaly detection on sensor data in a manufacturing plant. The model is deployed and running well. After two months, the plant installs new sensors that produce data with a different distribution. The anomaly detection starts failing with many false positives. Which action should the team take?

Accepted Answer

Retrain the model using data from the new sensors and re-deploy. The model needs monitoring and retraining on data from the new sensors. This falls under model monitoring and lifecycle management in deployment.

Answer

Roll back the model to the previous version and ignore the new sensors

Answer

Increase the anomaly threshold to reduce false positives

Answer

Use an ensemble of the old and a new model without retraining

Question 13

A team is building a document intelligence application that extracts key fields from invoices. They have 10,000 labeled invoices. What is the first step in the AI project lifecycle?

Accepted Answer

Problem definition – specify which fields to extract and accuracy targets. Problem definition must come first to scope the project, define success criteria, and decide what fields to extract before any data work.

Answer

Model selection – choose a pre-trained vision transformer

Answer

Data preparation – clean and normalize the invoice images

Answer

Data acquisition – collect additional invoices from public sources

Question 14

A team is considering whether to fine-tune a base LLM or use RAG for a question-answering system over a large, static corpus of scientific papers. The answer must be highly accurate and grounded in the papers. Which approach is BEST and why?

Accepted Answer

RAG because it retrieves and grounds answers in the source documents. RAG retrieves exact passages and grounds answers in the source, which is critical for accuracy and citation in scientific domains. Fine-tuning may memorize but can hallucinate.

Answer

Fine-tuning because it adapts the model to the scientific domain

Answer

Fine-tuning because it is faster at inference time

Answer

RAG because it does not require any labeled data

Question 15

A team is implementing a RAG system. They are designing the document loading and chunking strategy. Which TWO techniques are commonly used for chunking documents? (Select two.)

Accepted Answer

Fixed-size chunking with a token limit. Fixed-size chunking (based on token count) and semantic chunking (based on natural boundaries) are both standard approaches. Hierarchical chunking is less common, and character-level is rarely used. Overlap is a parameter, not a chunking strategy.

Answer

Frequency-based chunking by term occurrence

Answer

Character-level chunking with no overlap

Answer

Hierarchical chunking using document structure