AI0-001 Practice Test 38 — 15 Questions

Question 1

During testing of an AI system that classifies support tickets into categories, the team notices the model frequently misclassifies tickets about a new product feature that was introduced after the model was trained. Which type of testing should the team prioritize to catch this issue?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Regression testing with a test set that includes examples of the new feature. The model's misclassification of the new product feature is a classic case of data drift, where the production data distribution differs from the training data. Regression testing with a test set that includes examples of the new feature directly validates whether the model still performs correctly on this unseen category. This is the most targeted approach to catch the regression in classification accuracy caused by the new feature.

Answer

Unit tests for the data pipeline

Answer

Integration tests for API calls

Answer

Evaluation framework for LLM output quality

Question 2

A data scientist needs to explain why a black-box model denied a loan application. Which explainability technique generates local feature importance values using a simpler interpretable model around the prediction?

Accepted Answer

LIME. LIME (Local Interpretable Model-agnostic Explanations) is the correct technique because it generates local feature importance values by fitting a simpler, interpretable model (e.g., linear regression or decision tree) around the prediction of the black-box model. This allows the data scientist to explain why a specific loan application was denied by identifying which features (e.g., income, credit score) most influenced that particular decision. Unlike global methods, LIME focuses on the local neighborhood of the instance, making it ideal for explaining individual predictions.

Answer

Model card

Answer

Attention visualisation

Answer

SHAP values

Question 3

A company uses the above policy to control AI model access. A data scientist tries to run inference with model "llama-3-70b" at 150 requests in 30 minutes. What will happen?

Accepted Answer

The first 100 requests are allowed; the remaining 50 are denied. The policy allows up to 100 requests per 30 minutes for models in the allowed list, and 'llama-3-70b' is in that list. The rate limit is applied per 30-minute window, not per minute, so the first 100 requests are allowed, and the remaining 50 exceed the limit and are denied.

Answer

All requests are allowed because the model is in the allowed list

Answer

All requests are denied because the rate limit is per minute and 150 exceeds the limit

Answer

All requests are denied because the second rule blocks all models

Question 4

In the AI project lifecycle, which phase involves partitioning the dataset into training, validation, and test sets?

Accepted Answer

Data preparation. Data preparation includes splitting the data to evaluate model performance and prevent leakage.

Answer

Data acquisition

Answer

Model selection

Answer

Problem definition

Question 5

A team is deploying a multi-modal AI model that processes both text and images. They need to ensure that inference requests are handled quickly even during traffic spikes. Which integration pattern is BEST suited for this use case?

Accepted Answer

Use an event-driven architecture with a queue and worker instances. Asynchronous processing with a message queue allows requests to be buffered and processed without blocking the user, scaling out workers as needed.

Answer

Deploy a synchronous REST API with auto-scaling

Answer

Stream responses directly from the model to the client

Answer

Pre-compute all possible outputs and cache them

Question 6

A batch inference pipeline fails intermittently with out-of-memory errors when processing large datasets. The pipeline uses pandas DataFrames and feeds a pre-trained model. Which change would most effectively reduce memory consumption?

Accepted Answer

Split the data into smaller chunks and process sequentially. Splitting a large dataset into smaller chunks and processing them sequentially directly addresses the root cause of the out-of-memory error: the entire dataset is loaded into memory at once via pandas DataFrames. By processing data in batches, each chunk fits within the available RAM, preventing memory exhaustion while still allowing the pipeline to complete the full inference workload.

Answer

Increase the instance size of the compute node

Answer

Use a database instead of CSV files

Answer

Convert the model to use half-precision

Question 7

A team is implementing a RAG system for legal document retrieval. The documents are long (50-100 pages) with clear section headings. They want to ensure that retrieved chunks are semantically coherent and respect document structure. Which chunking strategy is MOST appropriate?

Accepted Answer

Hierarchical chunking: first split by sections, then further split each section into fixed-size chunks with overlap. Hierarchical chunking preserves document structure by first splitting into sections, then further into chunks, maintaining semantic coherence.

Answer

Semantic chunking based on sentence embeddings

Answer

Fixed-size chunking with 256 tokens and no overlap

Answer

Recursive character text splitting with chunk size 1000 and chunk overlap 200

Question 8

A developer is fine-tuning a large language model for a code generation task. The available GPU has only 8GB of VRAM, and the base model is 7B parameters. Which fine-tuning technique is MOST feasible?

Accepted Answer

QLoRA (Quantized Low-Rank Adaptation). QLoRA (Quantized Low-Rank Adaptation) combines quantization and LoRA to fine-tune large models on limited VRAM.

Answer

LoRA (Low-Rank Adaptation)

Answer

Instruction tuning with a smaller model

Answer

Full fine-tuning of all parameters

Question 9

A team is implementing a document intelligence solution to extract key-value pairs from invoices. They plan to use a pre-trained vision-language model with a RAG pipeline that indexes invoice images. Which chunking strategy is BEST suited for invoice documents that have a consistent layout but vary in length?

Accepted Answer

Hierarchical chunking that groups lines into logical sections (header, line items, totals). Invoices typically have sections (header, line items, totals). Hierarchical chunking preserves this structure, enabling retrieval at the section level. Fixed-size may split important fields, semantic chunking is less predictable on structured documents.

Answer

Semantic chunking based on sentence boundaries

Answer

Fixed-size chunking with 512 tokens per chunk

Answer

No chunking; pass the entire invoice as one document per query

Question 10

A chatbot application uses a system prompt to set the assistant's behavior. The developer wants the LLM to output structured JSON for downstream processing. Which technique BEST ensures the output is valid JSON?

Accepted Answer

Use the LLM's built-in JSON mode (e.g., response_format='json_object'). Many LLMs support a JSON mode that constrains output to valid JSON. System prompts can request JSON but may be ignored. Few-shot examples help but are not foolproof. Chain-of-thought is for reasoning, not formatting.

Answer

Set the temperature to 0 to make output deterministic

Answer

Include a few-shot example showing a JSON output in the prompt

Answer

Add a chain-of-thought reasoning step before the output

Question 11

A machine learning engineer is building a recommendation system for an e-commerce platform. The system should suggest products based on user purchase history and browsing behavior. Which model selection is BEST suited for this task?

Accepted Answer

Collaborative filtering model (e.g., matrix factorization). Collaborative filtering models (e.g., matrix factorization) are effective for recommendation tasks using user-item interaction data. Linear regression is for regression, not recommendation. Image classification is unrelated. Random forests can be used but are less common for collaborative filtering.

Answer

Image classification model (e.g., CNN)

Answer

Linear regression

Answer

Random forest classifier

Question 12

A company is deploying an LLM-based chatbot that must output responses in a structured JSON format for downstream processing. Which THREE prompt engineering techniques should the team use to ensure the output is valid and correctly structured? (Select three.)

Accepted Answer

Include few-shot examples of correct JSON outputs. A system prompt defining JSON structure, few-shot examples of valid JSON, and JSON mode in the model API all help produce valid structured output. Chain-of-thought and temperature adjustment do not directly enforce JSON format.

Answer

Set temperature to 0 to increase determinism

Answer

Use chain-of-thought prompting to reason before output

Question 13

A company is building a code generation assistant for internal developers. They want the assistant to generate code snippets consistent with the company's coding style and use private libraries. They have a few thousand examples of internal code. Which THREE considerations are critical when deciding between fine-tuning a base LLM and using RAG?

Accepted Answer

RAG eliminates the need for any model updates when private libraries change, because it retrieves the latest documentation at inference time.. Fine-tuning can embed coding style and internal library knowledge into model weights, but requires regular updates. RAG is easier to update but may miss stylistic nuances. The volume of examples (a few thousand) is moderate; fine-tuning may still be feasible. Security and latency/availability are relevant for deployment.

Answer

Fine-tuning a few thousand examples is insufficient; millions are required for any meaningful adaptation.

Answer

RAG requires the model to have a high context window size to accommodate retrieved code snippets.

Question 14

An MLOps team uses a CI/CD pipeline to automate model retraining. The pipeline triggers on new labeled data, runs feature engineering, retrains the model, evaluates against a holdout set, and deploys if metrics exceed thresholds. Recently, a retrained model passed validation but caused a 5% accuracy drop in production. Which improvement best prevents this?

Accepted Answer

Implement canary deployment with shadow scoring to compare with current model. Canary deployment with shadow scoring allows the new model to serve predictions to a small subset of traffic while comparing its outputs against the current production model in real time, without affecting all users. This catches subtle data drift or concept drift that a static holdout set may miss, preventing the 5% accuracy drop from reaching full production.

Answer

Require manual approval before deployment

Answer

Use the entire production dataset for validation instead of a holdout set

Answer

Increase the amount of training data used in each retraining cycle

Question 15

A company is implementing an AI ethics board. Which TWO responsibilities should the board typically have?

Accepted Answer

Approving or rejecting high-risk AI initiatives. An AI ethics board should oversee AI projects for ethical compliance and approve high-risk AI initiatives. Implementing technical models and auditing data privacy are operational tasks not typically under the board's direct purview.

Answer

Writing code for fairness algorithms

Answer

Developing marketing strategies for AI products

Answer

Conducting daily data privacy audits