AI0-001 Practice Test 17 — 15 Questions

Question 1

A team is implementing a RAG system for a large legal document repository. They need to chunk the documents for efficient retrieval. The documents contain long sections with subsections, and the team wants to preserve the hierarchical structure. Which chunking strategy is MOST appropriate?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Hierarchical chunking that preserves section and subsection boundaries. Hierarchical chunking preserves the document structure by maintaining parent-child relationships, which is crucial for legal documents where context from headings matters. Fixed-size may break logical sections; semantic chunking splits by topic but loses hierarchy; overlapping chunks help continuity but don't preserve structure.

Answer

Overlapping chunking with a 10% token overlap

Answer

Semantic chunking based on topic segmentation

Answer

Fixed-size chunking with 512 tokens per chunk

Question 2

A data scientist is preparing a dataset for a classification model. The dataset has missing values in several features and features with very different scales. Which two data preparation steps should be applied?

Accepted Answer

Cleaning and normalization. Cleaning handles missing values (e.g., imputation), and normalization scales features to a similar range, which is important for many ML algorithms.

Answer

Outlier removal and binning

Answer

Feature selection and dimensionality reduction

Answer

Data augmentation and one-hot encoding

Question 3

A data scientist is building a binary classification model to predict customer churn. The dataset has 90% non-churn and 10% churn. After training, the model achieves 90% accuracy, but the recall for the churn class is only 20%. Which metric should the team primarily focus on to evaluate the model's effectiveness?

Accepted Answer

Recall for the churn class. When classes are imbalanced, accuracy is misleading. Recall (or F1) for the minority class is more informative.

Answer

Accuracy

Answer

Area Under the ROC Curve (AUC-ROC)

Answer

Precision for the non-churn class

Question 4

A team is designing an AI agent that needs to interact with external APIs, search the web, and perform multi-step reasoning. Which TWO architectural components are essential for this agentic workflow? (Choose TWO.)

Accepted Answer

ReAct pattern (Reasoning + Acting). Tool use allows the agent to call external APIs, and the ReAct pattern (Reasoning + Acting) enables iterative reasoning and action steps.

Answer

Fine-tuning the base model

Answer

Single-turn response generation

Answer

Static prompt with no iterations

Question 5

An organization is deploying an image classification model to detect defects on a production line. Which TWO steps are essential during the model monitoring phase of the AI project lifecycle?

Accepted Answer

Track model performance metrics such as precision and recall on a held-out test set over time. Monitoring should track data drift (changes in input distribution) and model drift (performance degradation). Retraining is an action after monitoring, not a step of monitoring itself. Data preparation is completed before deployment.

Answer

Perform data cleaning and normalization on the production data before inference

Answer

Periodically retrain the model with newly labeled data

Answer

Automatically roll back to the previous model version if a metric drops below a threshold

Question 6

A recommendation system for an e-commerce site is producing stale suggestions that do not reflect recent user behavior. The system is updated offline every 24 hours. Which change would MOST directly address this issue?

Accepted Answer

Implement online learning to update the model incrementally in real time. Shorter update cycles or online learning can incorporate recent interactions more quickly, improving freshness.

Answer

Increase the number of features used in the model

Answer

Add more training data from the past year

Answer

Use a deeper neural network architecture

Question 7

A data science team is preparing a dataset for a binary classification model. The dataset has 95% negative class and 5% positive class. Which technique should they apply to avoid biased model predictions?

Accepted Answer

Apply resampling techniques such as SMOTE or random undersampling. Handling imbalanced data (e.g., oversampling the minority class or undersampling the majority class) is necessary to prevent the model from always predicting the majority class.

Answer

Normalise all numerical features to a [0,1] range

Answer

Shuffle the dataset randomly before splitting into train and test sets

Answer

Remove all rows with missing values

Question 8

In prompt engineering, which technique involves providing a few correct input-output examples in the prompt to guide the model's response?

Accepted Answer

Few-shot prompting. Few-shot learning includes examples in the prompt to demonstrate the desired output format and reasoning pattern.

Answer

System prompt engineering

Answer

Chain-of-thought prompting

Answer

Zero-shot prompting

Question 9

A company is deploying a code generation AI assistant for internal developers. They want to ensure the assistant does not generate code with security vulnerabilities. Which testing approach is MOST critical?

Accepted Answer

Integration tests that send security-focused prompts and validate the generated code against a static analysis tool. Integration tests that call the model with security-related prompts and scan outputs for vulnerabilities directly assess this requirement. Unit tests on data pipelines are for data correctness, not security of generated code.

Answer

Unit tests for the data pipeline that preprocesses prompts

Answer

Evaluation framework that measures BLEU score on a held-out set of code samples

Answer

Regression tests that compare outputs of new model versions against a golden dataset

Question 10

A company has an existing AI chatbot that uses a fine-tuned LLM to answer customer queries. They want to add the ability to retrieve real-time order status from their database. Which integration pattern should they use?

Accepted Answer

Implement function calling so the model can trigger a database query and receive the result. Function calling allows the LLM to request database queries, and the application can execute them and return results.

Answer

Prompt the user to check the order status manually

Answer

Use RAG to retrieve order status from a vector store

Answer

Embed the database query results directly into the model's training data

Question 11

A data scientist is preparing a dataset for a binary classification model to detect fraudulent transactions. The dataset has 1% fraud cases (minority class) and 99% non-fraud cases. Which data preparation technique is MOST appropriate to address the class imbalance before training?

Accepted Answer

Apply SMOTE (Synthetic Minority Over-sampling Technique) to the minority class. Synthetic Minority Over-sampling Technique (SMOTE) generates synthetic samples for the minority class, balancing the dataset without losing data. Undersampling would discard valuable majority samples, and oversampling by duplication can cause overfitting. Normalization does not fix class imbalance.

Answer

Duplicate the minority class samples until the class ratio is 50:50

Answer

Normalize all features to a range of 0 to 1

Answer

Random undersample the majority class to match the minority class size

Question 12

Which similarity measure is commonly used in vector search to find the angle between vectors, making it well-suited for high-dimensional embeddings?

Accepted Answer

Cosine similarity. Cosine similarity measures the cosine of the angle between two vectors, ranging from -1 to 1, and is robust to magnitude differences.

Answer

Manhattan distance

Answer

Euclidean distance

Answer

Dot product

Question 13

A team is building an AI-powered recommendation system for an e-commerce platform. They want to test the system before deployment. Which TWO types of testing are MOST relevant for this AI system? (Select TWO)

Accepted Answer

Evaluation frameworks for model output quality. Unit tests for data pipelines ensure data integrity, and evaluation frameworks for LLM output quality (or model evaluation) test recommendation relevance.

Answer

Load testing the web server

Answer

Integration tests for API calls

Answer

Regression testing on the UI

Question 14

A developer is building a RAG system and needs to choose a similarity metric for retrieving document chunks. The embedding model they use produces normalized vectors (unit vectors). Which similarity metric is equivalent to cosine similarity in this case?

Accepted Answer

Dot product. For normalized vectors, cosine similarity and dot product are equivalent because the dot product equals the cosine of the angle times the product of magnitudes (which are 1).

Answer

Jaccard similarity

Answer

Euclidean distance

Answer

Manhattan distance

Question 15

Which component in a RAG system is responsible for converting document chunks into numerical representations that enable similarity search?

Accepted Answer

Embedding model. An embedding model (or encoder) transforms text into dense vectors. The vector store indexes these vectors, and the LLM generates answers. Chunking is the splitting step.

Answer

Vector store index

Answer

Document chunker

Answer

Large language model (LLM)