AI0-001 Implementing AI Solutions Practice Test 2 — 15 Questions

Question 1

A company is building an AI-powered document intelligence system to extract key fields from scanned invoices. The data contains 95% of invoices from one vendor and 5% from others. During model training, the F1 score is 0.95 on the overall test set, but the performance on the minority vendor invoices is very poor. What is the MOST likely cause?

Accepted Answer

The dataset is imbalanced, and the model is biased toward the majority class. The imbalanced dataset causes the model to learn mostly from the majority class, leading to poor performance on the minority class. The other options are less likely given the high overall F1 score.

Answer

The model is overfitting on the minority class

Answer

The data has a train/test leakage problem

Answer

The feature extraction is incorrect for the minority vendor invoices

Question 2

A data scientist is preparing a dataset for a classification model. The dataset has missing values in several features and features with very different scales. Which two data preparation steps should be applied?

Accepted Answer

Cleaning and normalization. Cleaning handles missing values (e.g., imputation), and normalization scales features to a similar range, which is important for many ML algorithms.

Answer

Outlier removal and binning

Answer

Feature selection and dimensionality reduction

Answer

Data augmentation and one-hot encoding

Question 3

A team is implementing a RAG system for legal document retrieval. The documents are long (50-100 pages) with clear section headings. They want to ensure that retrieved chunks are semantically coherent and respect document structure. Which chunking strategy is MOST appropriate?

Accepted Answer

Hierarchical chunking: first split by sections, then further split each section into fixed-size chunks with overlap. Hierarchical chunking preserves document structure by first splitting into sections, then further into chunks, maintaining semantic coherence.

Answer

Semantic chunking based on sentence embeddings

Answer

Fixed-size chunking with 256 tokens and no overlap

Answer

Recursive character text splitting with chunk size 1000 and chunk overlap 200

Question 4

A developer is building an AI agent that needs to call external APIs to complete user requests. The agent must decide which API to call based on the user's natural language input. Which technique should the developer use to enable the agent to invoke APIs?

Accepted Answer

Function calling. Function calling is a technique where the LLM outputs a structured request to call a predefined function/API, enabling tool use.

Answer

Fine-tuning the LLM on API documentation

Answer

Chain-of-thought reasoning

Answer

Few-shot prompting with examples of API calls

Question 5

A machine learning engineer is deploying a real-time anomaly detection system for manufacturing sensor data. The system must process thousands of readings per second with minimal latency. Which deployment architecture is BEST suited?

Accepted Answer

AI microservices with an async processing queue and streaming responses. AI microservices with async processing queues and streaming responses can handle high throughput and low latency for real-time data.

Answer

Batch processing using Apache Spark jobs triggered hourly

Answer

Serverless functions deployed on a CDN

Answer

A monolithic web application with a relational database

Question 6

A data scientist is building a binary classification model to predict customer churn. The dataset has 90% non-churn and 10% churn. After training, the model achieves 90% accuracy, but the recall for the churn class is only 20%. Which metric should the team primarily focus on to evaluate the model's effectiveness?

Accepted Answer

Recall for the churn class. When classes are imbalanced, accuracy is misleading. Recall (or F1) for the minority class is more informative.

Answer

Accuracy

Answer

Area Under the ROC Curve (AUC-ROC)

Answer

Precision for the non-churn class

Question 7

A developer is fine-tuning a large language model for a code generation task. The available GPU has only 8GB of VRAM, and the base model is 7B parameters. Which fine-tuning technique is MOST feasible?

Accepted Answer

QLoRA (Quantized Low-Rank Adaptation). QLoRA (Quantized Low-Rank Adaptation) combines quantization and LoRA to fine-tune large models on limited VRAM.

Answer

LoRA (Low-Rank Adaptation)

Answer

Instruction tuning with a smaller model

Answer

Full fine-tuning of all parameters

Question 8

A company has an existing AI chatbot that uses a fine-tuned LLM to answer customer queries. They want to add the ability to retrieve real-time order status from their database. Which integration pattern should they use?

Accepted Answer

Implement function calling so the model can trigger a database query and receive the result. Function calling allows the LLM to request database queries, and the application can execute them and return results.

Answer

Prompt the user to check the order status manually

Answer

Use RAG to retrieve order status from a vector store

Answer

Embed the database query results directly into the model's training data

Question 9

A data science team is preparing a dataset for a supervised learning task. They split the data into training and test sets. The team then normalizes the features using the mean and standard deviation calculated from the entire dataset before splitting. What issue does this introduce?

Accepted Answer

It introduces train/test leakage. Using statistics from the entire dataset before splitting causes test data information to influence the training process, leading to train/test leakage and overly optimistic performance estimates.

Answer

It improves model generalization

Answer

It causes the model to overfit the training data

Answer

It reduces the variance of the features

Question 10

A developer is building a RAG system and needs to choose a similarity metric for retrieving document chunks. The embedding model they use produces normalized vectors (unit vectors). Which similarity metric is equivalent to cosine similarity in this case?

Accepted Answer

Dot product. For normalized vectors, cosine similarity and dot product are equivalent because the dot product equals the cosine of the angle times the product of magnitudes (which are 1).

Answer

Jaccard similarity

Answer

Euclidean distance

Answer

Manhattan distance

Question 11

An organization wants to implement an AI system to automatically categorize support tickets into predefined categories. They have a labeled dataset of 10,000 tickets. Which approach is MOST appropriate?

Accepted Answer

Fine-tune a pre-trained text classification model. Fine-tuning a pre-trained text classification model is a standard and effective approach for supervised classification when labeled data is available.

Answer

Use a rule-based system with keyword matching

Answer

Use a prompt-based LLM with few-shot examples

Answer

Train a custom neural network from scratch

Question 12

A team is developing an AI agent to assist users with multi-step tasks such as booking a flight, reserving a hotel, and scheduling a car rental. The agent needs to reason about the order of steps and handle dependencies. Which pattern is BEST suited?

Accepted Answer

ReAct pattern (Reasoning and Acting). The ReAct pattern (Reasoning and Acting) is best suited because it interleaves reasoning traces with tool calls, allowing the agent to dynamically plan and adjust steps based on intermediate results. For multi-step tasks with dependencies (e.g., booking a flight before a hotel), ReAct enables the agent to reason about order, handle failures, and call external APIs step-by-step, which is essential for robust task completion.

Answer

Simple tool use without reasoning

Answer

Using a single prompt with all instructions

Answer

Fine-tuning a model to output all steps at once

Question 13

A data scientist is preparing a dataset for a regression model. The dataset contains 100 features, some of which are highly correlated. To improve model performance and reduce overfitting, which TWO techniques should the data scientist apply? (Select TWO)

Accepted Answer

Feature selection. Feature selection reduces the number of features, and dimensionality reduction (e.g., PCA) handles multicollinearity, both helping to reduce overfitting.

Answer

Data augmentation

Answer

Adding more hidden layers to the neural network

Answer

Increasing the learning rate

Question 14

A company is deploying a generative AI application that produces structured JSON output for downstream processing. They want to ensure the output is consistently valid JSON and matches a specific schema. Which THREE techniques should they use? (Select THREE)

Accepted Answer

Provide few-shot examples of the desired output. Structured output (JSON mode) forces valid JSON, system prompts instruct the model, and few-shot examples demonstrate the required schema.

Answer

Fine-tune the model on a dataset of JSON outputs

Answer

Increase the temperature parameter to 1.5

Question 15

A team is building an AI-powered recommendation system for an e-commerce platform. They want to test the system before deployment. Which TWO types of testing are MOST relevant for this AI system? (Select TWO)

Accepted Answer

Evaluation frameworks for model output quality. Unit tests for data pipelines ensure data integrity, and evaluation frameworks for LLM output quality (or model evaluation) test recommendation relevance.

Answer

Load testing the web server

Answer

Integration tests for API calls

Answer

Regression testing on the UI