AI0-001 Practice Test 5 — 10 Questions

Question 1

A model trained on a dataset has high bias and low variance. What does this indicate?

Accepted Answer

Underfitting. High bias and low variance indicate that the model is too simple to capture the underlying patterns in the data, leading to systematic errors on both training and test sets. This is the classic signature of underfitting, where the model fails to learn the training data adequately.

Answer

Good fit

Answer

Data leakage

Answer

Overfitting

Question 2

A machine learning team is splitting a dataset for a binary classification problem. They want to ensure robust evaluation and avoid data leakage. Which TWO practices should they follow? (Choose 2)

Accepted Answer

Split into training, validation, and test sets. Option B is correct because splitting the dataset into separate training, validation, and test sets allows the model to be tuned on the validation set while the test set remains untouched for an unbiased final evaluation, which is essential for robust assessment of a binary classifier. Option D is correct because shuffling the data before splitting ensures that the training, validation, and test sets are representative of the overall distribution and prevents ordering bias or temporal artifacts from skewing the split, which helps avoid leakage when the data is not inherently ordered. Option A is not correct because normalising the entire dataset before splitting leaks information from the validation and test sets into the training process, since the scaling parameters would be computed using all data. Option C is not correct because including validation data in the training set removes the independent validation signal needed for hyperparameter tuning and model selection, undermining robust evaluation. Option E is not correct because reusing the same split for all experiments can lead to overfitting to that particular split and does not provide the variability needed to assess generalization reliably.

Answer

Normalise the entire dataset before splitting

Answer

Include validation data in the training set for more data

Answer

Use the same split for all experiments

Question 3

A model trained on customer reviews achieves 98% accuracy on the test set. However, when deployed, it performs poorly on real-world data. The data scientist suspects distribution shift. Which action is MOST important to address this?

Accepted Answer

Implement a monitoring system to detect data drift and retrain with fresh data. Distribution shift (data drift) causes the model's training distribution to differ from the real-world distribution, degrading performance despite high test accuracy. Implementing a monitoring system to detect drift and retraining with fresh data directly addresses this by ensuring the model adapts to the current data distribution, which is the most critical action for maintaining performance in production.

Answer

Reduce the learning rate during training

Answer

Add more features to the model

Answer

Increase the number of cross-validation folds

Question 4

A data scientist needs to store large volumes of unstructured log data for future AI model training. They also need to run SQL-based analytics on the data. Which THREE services are appropriate for this requirement? (Choose 3)

Accepted Answer

Snowflake. Snowflake is correct because it is a cloud-native data warehouse that supports both structured and semi-structured data (like JSON, Avro, Parquet) via its VARIANT data type, enabling SQL-based analytics on unstructured log data. It also integrates with cloud storage (e.g., Amazon S3) for storing large volumes of raw logs, making it suitable for AI model training pipelines.

Answer

Pinecone

Answer

pgvector

Question 5

An ML engineering team has a retraining pipeline that triggers automatically when model accuracy drops below a threshold. Recently, the model's accuracy has been fluctuating, causing frequent retraining and high compute costs. The team suspects the data distribution is changing slowly. Which approach should the team implement to reduce unnecessary retraining while maintaining model performance?

Accepted Answer

Implement a statistical drift detection method on input features. Implementing a statistical drift detection method (e.g., using KL divergence, PSI, or ADWIN) on input features allows the team to identify when the data distribution has genuinely changed, rather than reacting to random accuracy fluctuations. This reduces unnecessary retraining by triggering the pipeline only when statistically significant drift is detected, maintaining model performance without the high compute costs of frequent retraining.

Answer

Use a simpler model to reduce variability

Answer

Increase the frequency of model retraining

Answer

Reduce the batch size for inference

Question 6

Which THREE techniques can help reduce overfitting in neural networks?

Accepted Answer

Increasing training data size. Increasing the training data size helps reduce overfitting by providing the model with more examples to learn from, which reduces the variance and improves generalization. With more data, the model is less likely to memorize noise and instead learns the underlying patterns, making it more robust on unseen data.

Answer

Using a larger learning rate

Answer

Increasing number of layers

Question 7

A team is deploying a generative AI model for a real-time customer-facing application. They need to balance cost and latency. Which deployment strategy is MOST suitable?

Accepted Answer

AI microservices with streaming responses and async processing queues. AI microservices with streaming responses and async processing queues decouple inference from the request lifecycle, allowing the system to handle variable loads efficiently while maintaining low latency for real-time interactions. This architecture balances cost by scaling only the necessary components (e.g., GPU-backed inference services) and uses streaming (e.g., Server-Sent Events or WebSockets) to deliver partial results, reducing perceived latency for the customer.

Answer

Monolithic API with serverless functions

Answer

Edge deployment on user devices

Answer

Batch processing with synchronous requests

Question 8

An organization has a dataset with categorical features having high cardinality (e.g., ZIP codes). They plan to use a tree-based model. Which encoding method is most appropriate?

Accepted Answer

Target encoding (mean encoding). Target encoding (mean encoding) replaces each category with the mean of the target variable for that category, which works well with tree-based models on high-cardinality features because it captures the predictive signal without exploding the feature space. This method avoids the dimensionality explosion of one-hot encoding and the arbitrary ordering of label encoding, making it the most appropriate choice for high-cardinality categorical features in tree-based models.

Answer

Label encoding

Answer

One-hot encoding

Answer

Frequency encoding

Question 9

A company is deploying a generative AI system that produces text content. To comply with emerging transparency obligations, which THREE measures should they implement?

Accepted Answer

Watermark AI-generated content. Watermarking AI-generated content (Option A) is correct because it embeds an imperceptible, machine-detectable signal into the output, enabling provenance verification. This directly addresses transparency obligations by allowing downstream systems to identify synthetic text, which is a key requirement in emerging AI regulations such as the EU AI Act.

Answer

Encrypt all training data

Answer

Limit model access to internal employees only

Question 10

An MLOps team uses a CI/CD pipeline to automate model retraining. The pipeline triggers on new labeled data, runs feature engineering, retrains the model, evaluates against a holdout set, and deploys if metrics exceed thresholds. Recently, a retrained model passed validation but caused a 5% accuracy drop in production. Which improvement best prevents this?

Accepted Answer

Implement canary deployment with shadow scoring to compare with current model. Canary deployment with shadow scoring allows the new model to serve predictions to a small subset of traffic while comparing its outputs against the current production model in real time, without affecting all users. This catches subtle data drift or concept drift that a static holdout set may miss, preventing the 5% accuracy drop from reaching full production.

Answer

Require manual approval before deployment

Answer

Use the entire production dataset for validation instead of a holdout set

Answer

Increase the amount of training data used in each retraining cycle