AI0-001 Implementing AI Solutions Practice Test 9 — 15 Questions

Question 1

A data science team is developing a churn prediction model. Which TWO data preparation best practices are MOST important to prevent overfitting and ensure generalization?

Accepted Answer

Split data into training and test sets before any preprocessing. Option A is correct because splitting into training and test sets before any preprocessing prevents data leakage: statistics such as means, standard deviations, or encodings must be learned only from the training set and then applied to the test set, otherwise the model indirectly sees test data and overfitting goes undetected. Option C is correct because cross-validation (e.g., k-fold) tunes hyperparameters on multiple training/validation splits, giving a more reliable estimate of generalization performance and reducing the chance of selecting hyperparameters that overfit a single validation split. Option B is not appropriate because normalizing with statistics computed from the entire dataset leaks test-set information into training; normalization statistics should come from the training fold only. Option D is not appropriate because removing outliers based on the full dataset distribution also leaks test information and can distort the true data distribution; outlier handling should be based on training data. Option E is not appropriate because target encoding on the full dataset leaks the target variable into the features, a classic cause of overfitting; target encoding must be fit within training folds, ideally with smoothing or out-of-fold encoding.

Answer

Normalize all features using the entire dataset's statistics

Answer

Remove outliers based on the full dataset distribution

Answer

Encode categorical variables with target encoding on the full dataset

Question 2

A company is choosing between fine-tuning and RAG for a legal document assistant. Which TWO factors would MOST strongly favor RAG over fine-tuning?

Accepted Answer

The legal documents are updated frequently (weekly). Option A is correct because RAG retrieves from an external knowledge base at query time, so weekly-updated legal documents can be re-indexed without retraining the model, whereas fine-tuning would require repeated, costly retraining to keep pace. Option D is correct because RAG naturally returns the retrieved passages that grounded the answer, enabling precise source citations, while a fine-tuned model's parametric knowledge offers no traceable provenance. Option B does not favor RAG, since understanding complex legal terminology is largely a function of the base model's pretraining and can be addressed by either approach. Option C does not favor RAG either, as deep multi-document reasoning is a model capability rather than a retrieval benefit, and RAG alone does not guarantee it. Option E is not a strong RAG advantage, because building and operating a vector store and retriever also incurs infrastructure and compute costs, so a limited training budget does not decisively favor RAG.

Answer

The model needs to understand complex legal terminology

Answer

The queries require deep reasoning across multiple documents

Answer

The company has limited compute budget for training

Question 3

An AI engineer is selecting a PEFT technique to fine-tune a large language model. Which TWO are examples of PEFT (Parameter-Efficient Fine-Tuning)?

Accepted Answer

LoRA. LoRA (Low-Rank Adaptation) is correct because it freezes the pretrained model weights and injects trainable low-rank decomposition matrices into the transformer layers, updating only a tiny fraction of parameters. QLoRA is correct because it extends LoRA by quantizing the base model to 4-bit (NF4) and adding trainable low-rank adapters, further reducing memory while remaining a parameter-efficient method. Instruction tuning (A) is a training objective/paradigm that typically updates all or many parameters, not a PEFT technique itself. Gradient checkpointing (D) is a memory-saving trick that recomputes activations during backpropagation and does not reduce the number of trainable parameters. Full fine-tuning (E) updates every model parameter, which is the opposite of parameter-efficient fine-tuning.

Answer

Instruction tuning on a large dataset

Answer

Gradient checkpointing

Answer

Full fine-tuning of all parameters

Question 4

A data science team is preparing a dataset for a binary classification model to detect fraudulent transactions. The dataset has 99% legitimate and 1% fraudulent examples. Which TWO techniques should the team apply to improve model performance on the minority class?

Accepted Answer

Use class weights in the loss function. Option A is correct because assigning class weights in the loss function (e.g., class_weight='balanced' in scikit-learn or a weighted binary cross-entropy) penalizes misclassification of the 1% fraudulent class more heavily, directly countering the 99:1 imbalance during training. Option B is correct because SMOTE (Synthetic Minority Over-sampling Technique) generates synthetic minority-class samples by interpolating between existing fraudulent examples and their k-nearest neighbors, increasing minority representation and helping the model learn the fraudulent decision boundary. Option C is not selected because random undersampling discards potentially useful majority-class data and can increase variance, making it a less preferred technique here. Option D is not selected because z-score normalization only rescales feature distributions and does nothing to address class imbalance. Option E is not selected because shuffling prevents ordering bias and train/test leakage but has no effect on the skewed class ratio.

Answer

Undersample the majority class randomly

Answer

Apply data normalisation (z-score) to all features

Answer

Randomly shuffle the dataset to prevent train/test leakage

Question 5

An AI team is deploying a real-time document intelligence service that extracts key-value pairs from invoices. The pipeline includes an LLM that calls a function to parse structured output. Which TWO testing strategies are essential before production deployment?

Accepted Answer

An evaluation framework that compares extracted fields against ground truth for a test set of invoices. Option A is correct because an evaluation framework that compares extracted key-value pairs against a ground-truth labeled set of invoices is the only way to quantitatively measure field-level accuracy, precision, and recall of the extraction LLM before production, which is essential for a document intelligence service. Option D is correct because integration tests that invoke the actual LLM API with sample invoices and validate the returned JSON structure verify that the function-calling contract, schema conformance, and end-to-end wiring between the pipeline and the model work as expected, catching serialization or tool-call failures that unit tests would miss. Option B is not essential here because load testing addresses throughput and latency at peak volume, which is a performance concern rather than a correctness concern for the extraction quality being validated. Option C is not essential because regression tests on the base LLM training pipeline are the model provider's responsibility and do not validate this team's deployed inference service. Option E is not essential because unit tests for image cleaning and normalization cover only a preprocessing component, not the LLM extraction behavior or its structured output contract that this deployment hinges on.

Answer

Load testing to simulate peak invoice volume (e.g., end of month)

Answer

Regression tests on the model training pipeline to ensure the base LLM hasn't changed

Answer

Unit tests for the data pipeline that cleans and normalises invoice images

Question 6

A team is implementing a RAG system for a legal document Q&A. They need to chunk documents effectively. Which THREE chunking strategies should they consider to improve retrieval accuracy for legal texts that contain hierarchical sections (clauses, sub-clauses, definitions)?

Accepted Answer

Hierarchical chunking that indexes chunks at clause and sub-clause levels with parent relationships. Option A is correct because hierarchical chunking indexes chunks at clause and sub-clause levels while preserving parent relationships, which matches the legal document's nested structure and lets retrieval return a precise sub-clause while still offering the enclosing clause for context. Option D is correct because chunking based on the document's table of contents and section hierarchy aligns chunk boundaries with the document's own logical divisions, so each chunk corresponds to a meaningful legal section rather than an arbitrary token span. Option E is correct because semantic chunking splits at natural boundaries such as section headings and paragraph breaks, keeping definitions and clauses intact and avoiding mid-sentence or mid-clause cuts that would degrade embedding quality and retrieval accuracy. Option B is not among the correct answers because a fixed 10% overlap is a generic heuristic that does not respect legal hierarchy and can duplicate or fragment clause text. Option C is not among the correct answers because fixed-size 512-token windows with no overlap ignore section structure and will frequently split clauses, sub-clauses, and definitions across chunk boundaries.

Answer

Overlapping chunks with a 10% overlap between consecutive chunks

Answer

Fixed-size chunking with a 512-token window and no overlap

Question 7

A company wants to use AI to automatically detect anomalies in server log data. The data is time-series and labeled with 'normal' and 'anomaly' for the past year. Which TWO techniques are appropriate for this use case?

Accepted Answer

Use a time-series anomaly detection model (e.g., Isolation Forest with sliding windows). Option B is correct because the data is time-series log data, and techniques like Isolation Forest applied over sliding windows (or similar time-series anomaly detectors) are designed to capture temporal patterns and flag deviations from normal behavior without requiring the labels, which suits anomaly detection on sequential log streams. Option C is correct because the dataset is labeled with 'normal' and 'anomaly' for a full year, so a supervised classifier such as XGBoost can be trained on extracted features (e.g., counts, rates, error codes, latency statistics) to directly learn the mapping from features to the anomaly label. Option A is not appropriate because converting logs to graph screenshots and using a CNN image classifier discards the underlying time-series structure and numeric log semantics, making it an indirect and lossy approach. Option D is wrong because code generation models fix code rather than detect anomalies in log data, which is the stated goal. Option E is wrong because a recommendation system based on user activity logs addresses personalization, not anomaly detection in server logs.

Answer

Train an image classification model (CNN) on screenshots of log graphs

Answer

Use a code generation model to fix the anomalies automatically

Answer

Build a recommendation system based on user activity logs

Question 8

A retail bank deploys a machine learning model that scores loan applications. Compliance requires that the bank be able to explain to regulators why any individual applicant was denied, in terms of the applicant's own feature values. The model is a gradient-boosted tree ensemble trained on 200 features. Which approach BEST satisfies this requirement?

Accepted Answer

Apply SHAP (SHapley Additive exPlanations) values to produce per-applicant feature attributions for each decision.. The requirement is individualized, feature-value-based justification for each denial on a complex ensemble. SHAP attributions decompose a single prediction into additive feature contributions, which is exactly what an adverse-action explanation needs. Global importance, coefficient tables, and raw input logging all describe population behavior or stored data rather than the reason for one applicant's specific outcome, so they cannot satisfy the regulator's question.

Answer

Log the model's predicted probability alongside the applicant's raw input record for each decision.

Answer

Report the global feature importance ranking produced by the tree ensemble's gain-based split scores.

Answer

Retrain the model as a logistic regression and present the learned coefficients as the explanation.

Question 9

A hospital's radiology AI triage model was validated at 94% sensitivity on a curated research dataset. After six months in production, clinicians report that it misses many positive cases on images from a newly installed scanner. The data science team confirms the model has not been retrained. Which action should the team take FIRST to diagnose and correct the problem?

Accepted Answer

Compare the statistical distribution of production images from the new scanner against the training data to detect covariate shift.. Performance collapse on images from new hardware, with no retraining, points to a change in the input distribution rather than a change in the label relationship. Detecting and quantifying that shift through distribution comparison establishes the root cause before any intervention. Threshold changes, targeted fine-tuning, and full retraining are all remedies that presuppose a cause and can waste effort or introduce new bias if applied blindly.

Answer

Increase the model's classification threshold so that more cases are flagged as positive.

Answer

Fine-tune the model on the most recent production images that clinicians have labeled as missed positives.

Answer

Replace the triage model with a larger architecture trained from scratch on a combined dataset of all available scanners.

Question 10

A media company is deploying a generative AI assistant to summarize customer support calls. The assistant must produce concise summaries in English, but the call transcripts are in Spanish. The team wants to use a single model that can handle both translation and summarization. Which approach is MOST appropriate?

Accepted Answer

Use a multilingual large language model and prompt it to translate and summarize in one step.. A multilingual LLM can directly process Spanish transcripts and generate English summaries in one step, satisfying the need for a single model that handles both translation and summarization. This reduces pipeline complexity and avoids error propagation from separate components, making it the most efficient and effective solution for the described scenario.

Answer

Use a Spanish-only language model and prompt it to output English summaries.

Answer

Fine-tune a monolingual English summarization model on translated Spanish transcripts.

Answer

Deploy a dedicated machine translation model first, then pass the English output to a separate summarization model.

Question 11

A media company is deploying a generative AI assistant that drafts marketing copy. Legal requires that every generated draft be attributable to source material and that the system must not reproduce copyrighted passages verbatim. The team wants to enforce this at generation time rather than only reviewing outputs afterward. Which implementation approach BEST meets these requirements?

Accepted Answer

Retrieve grounding passages from an approved internal corpus, pass them to the model as context, and attach source citations to the generated draft.. Grounding generation in an approved internal corpus with retrieval and attaching citations satisfies both legal requirements simultaneously: attributability and reduced verbatim reproduction. The other approaches either act after generation, rely on sampling randomness, or use fine-tuning, none of which guarantees source attribution or constrains output to licensed material at inference time.

Answer

Add a post-generation plagiarism check that blocks drafts containing long matching n-grams against a public web index.

Answer

Fine-tune the base model on the company's existing marketing copy and rely on the fine-tuned weights to avoid verbatim reproduction.

Answer

Increase the model's temperature setting so the assistant paraphrases source material instead of reproducing it.

Question 12

A financial services company is deploying a credit-scoring model built with the AI+ toolkit. The model must produce an explanation for each decision that regulators can review, showing which input features most influenced the score. The data science team has already trained a gradient-boosted tree ensemble. Which approach should the team use to satisfy the regulatory requirement?

Accepted Answer

Apply SHAP (SHapley Additive exPlanations) values to the trained ensemble to generate per-instance feature attributions.. Per-decision explanations require local, instance-level attribution methods. SHAP values quantify how much each feature pushed a single prediction above or below the baseline, and TreeSHAP makes this tractable for tree ensembles. Global importance, model substitution, and raw input-output disclosure all fail to explain why a specific applicant received a specific score.

Answer

Publish the model's raw prediction probabilities alongside the input feature values for each applicant.

Answer

Replace the ensemble with a single decision tree limited to a depth of three so that the decision path is human-readable.

Answer

Compute global feature importance by averaging the ensemble's split gains across the entire training set.

Question 13

A retail bank is deploying a customer-facing AI assistant that must never disclose internal policy text. The team has a system prompt with instructions, but red-team testing shows users can extract the policy by asking the model to 'repeat everything above this line.' Which implementation change most directly mitigates this prompt-injection extraction risk in production?

Accepted Answer

Add an input guardrail that detects and blocks prompt-extraction patterns before the request reaches the model.. The extraction attack succeeds because untrusted user input is concatenated with trusted instructions and the model follows the most recent instruction. Placing a guardrail in front of the model detects and blocks known extraction phrasing, which is the most direct runtime mitigation. Temperature, fine-tuning, and context size do not intercept the malicious instruction before it is processed.

Answer

Reduce the context window size so there is less room for the model to repeat the system prompt.

Answer

Move the policy text from the system prompt into the fine-tuning dataset so the model learns it as weights.

Answer

Increase the model temperature so responses become less deterministic and harder to reverse engineer.

Question 14

A hospital is deploying a vision model that flags possible pneumonia on chest radiographs. Radiologists report that the model performs well overall but frequently flags images from a newly installed portable X-ray unit. The images are technically adequate. The team must diagnose the cause before changing the model. Which action should the team take FIRST?

Accepted Answer

Compare the statistical distribution of pixel intensities, resolution, and metadata between images from the new unit and the original training set.. When a model degrades on inputs from a new device, the first step is to characterize the input distribution and compare it with training data. That comparison can reveal distribution shift caused by differences in resolution, exposure, or preprocessing, which must be understood before retraining or adjusting thresholds. Only after the cause is identified can the team choose an appropriate remediation.

Answer

Retrain the model on a larger dataset that includes images from the new portable unit.

Answer

Add a rule that discards any image whose DICOM metadata indicates the portable unit was used.

Answer

Lower the model's classification threshold so fewer images are flagged as positive.

Question 15

A hospital is deploying an AI triage assistant that suggests priority levels for emergency room patients. Clinicians will review every suggestion before acting. The compliance team requires that the system log who reviewed each suggestion, what the clinician decided, and whether they overrode the AI. Which implementation practice best satisfies this requirement?

Accepted Answer

Record a human-in-the-loop audit trail that captures the AI suggestion, the clinician's decision, and any override reason.. Human oversight in high-stakes AI requires evidence that a qualified person reviewed each suggestion and retained authority to disagree. An audit trail that links the suggestion, the human decision, and the override rationale provides that evidence and supports continuous monitoring. Auto-acceptance, model-only logging, and delayed batch feedback all fail to document meaningful human involvement.

Answer

Store clinician feedback in a separate quality-improvement database that is refreshed monthly.

Answer

Enable automatic acceptance of AI suggestions when the model's confidence score exceeds 0.95.

Answer

Log only the model's input features and output priority so that predictions can be reproduced later.