AI0-001 Practice Test 9 — 10 Questions

Question 1

A company wants to roll out a new recommendation model to production. They decide to run an A/B test where 10% of users see the new model and 90% see the old model. After one week, the new model shows a 5% improvement in click-through rate. What is the next best action?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Increase the testing percentage gradually while monitoring performance metrics and guardrails. A 5% improvement observed over only one week with a 10% traffic split is insufficient to confirm statistical significance or rule out novelty effects, data drift, or seasonal bias. The recommended best practice in AI deployment is to gradually increase the testing percentage (e.g., 10% → 25% → 50% → 100%) while continuously monitoring performance metrics and guardrails (e.g., click-through rate, conversion rate, latency, and error rates) to ensure the new model generalizes safely across the full user population.

Answer

Immediately roll out the new model to 100% of users

Answer

Revert to the old model because the improvement is minimal

Answer

Run the test for another month to ensure statistical significance

Question 2

A financial institution deploys an AI credit scoring model. After six months, the model's performance drops significantly. Analysis shows that the relationship between features and labels has changed. Which term describes this phenomenon?

Accepted Answer

Concept drift. Concept drift occurs when the statistical relationship between input features and the target label changes over time, which is exactly what happened when the credit scoring model's performance dropped due to a shift in the feature-label relationship. This is distinct from data drift, which only involves changes in the input data distribution without affecting the label mapping.

Answer

Model decay

Answer

Overfitting

Answer

Data drift

Question 3

Refer to the exhibit. A machine learning pipeline configuration is shown. During a deployment, the model evaluation passes with accuracy 0.86 and precision 0.79. However, the pipeline proceeds to deploy. What is the most likely reason for this behavior?

Accepted Answer

The deployment only checks the accuracy threshold for rollback condition. The pipeline configuration shows a rollback condition that only checks the accuracy metric (accuracy < 0.85). Since the model achieved accuracy 0.86, which is above the threshold, the condition is not triggered, and the pipeline proceeds to deploy regardless of the precision value. The precision metric is not part of the rollback evaluation logic in this configuration.

Answer

The precision metric is not included in the evaluation script

Answer

The deployment target is set to staging instead of production

Answer

The operator manually overrode the threshold

Question 4

A company streams sensor data from IoT devices. The data arrives as JSON messages at high velocity. Which data pipeline architecture is BEST suited to handle this streaming data for near-real-time analytics?

Accepted Answer

Stream processing using Apache Kafka and Spark Streaming.. Apache Kafka acts as a distributed, fault-tolerant ingestion layer that can handle high-velocity JSON messages, while Spark Streaming processes the data in micro-batches for near-real-time analytics. This combination provides the low-latency, scalable pipeline required for streaming IoT sensor data, unlike batch or single-node approaches.

Answer

Batch processing using Hadoop MapReduce every 24 hours.

Answer

Batch processing using nightly ETL jobs.

Answer

Single-node database with periodic inserts.

Question 5

A data engineer needs to combine two datasets, each with unique customer_id, to include all records from both datasets. Which join type should be used?

Accepted Answer

FULL OUTER JOIN. A FULL OUTER JOIN returns all records from both datasets, matching rows where the customer_id is present in both and filling in NULLs for missing matches. This is the only join type that guarantees every unique customer_id from either dataset appears in the result, which is exactly what the requirement specifies.

Answer

RIGHT JOIN

Answer

LEFT JOIN

Answer

INNER JOIN

Question 6

A team is building a regression model to predict house prices. Which data transformation is most appropriate if the target variable exhibits right skewness?

Accepted Answer

Log transformation. Log transformation is the most appropriate technique for right-skewed target variables because it compresses the long tail, making the distribution more symmetric and closer to Gaussian. This stabilizes variance and often improves the performance of regression models that assume normally distributed errors, such as linear regression.

Answer

Principal component analysis (PCA)

Answer

Standardization (Z-score)

Answer

One-hot encoding

Question 7

Refer to the exhibit. A stream processor ingests events. One event arrives with missing "user_id". What will happen?

Accepted Answer

The event will be rejected because user_id is required.. In stream processing systems like Apache Kafka or AWS Kinesis, if a required field such as 'user_id' is missing and the schema (e.g., Avro, JSON Schema) defines it as required, the event will be rejected at ingestion. The stream processor typically validates the event against the schema; if validation fails, the event is not accepted into the stream, preventing downstream processing errors.

Answer

The event will be stored in a dead-letter queue automatically.

Answer

The event will be accepted but user_id will be set to null.

Answer

The event will be accepted with a default user_id of 0.

Question 8

During feature engineering, a data scientist creates a new feature that is a linear combination of two existing features. What risk does this pose to the model?

Accepted Answer

Multicollinearity. Creating a new feature as a linear combination of two existing features introduces perfect multicollinearity, where the new feature is an exact linear function of the original ones. This violates the assumption of no perfect multicollinearity in linear models, causing the design matrix to become singular and making coefficient estimates unstable or impossible to compute. Even in non-linear models, high multicollinearity can inflate variance and reduce interpretability.

Answer

Data leakage

Answer

Overfitting

Answer

Underfitting

Question 9

A healthcare provider wants to use AI to predict patient readmission risk. They have structured data (age, diagnosis, lab results) and unstructured clinical notes. Which approach is most appropriate?

Accepted Answer

Multimodal model combining structured and text embeddings. The scenario involves both structured data (age, diagnosis, lab results) and unstructured clinical notes. A multimodal model can process both types by combining embeddings from text (e.g., via a transformer or RNN) with structured features, enabling the model to learn cross-modal patterns that improve readmission risk prediction. This approach leverages the complementary strengths of structured and unstructured data, which is essential for capturing the full clinical picture.

Answer

Convolutional neural network (CNN) on clinical notes

Answer

Recurrent neural network (RNN) on structured data

Answer

Logistic regression on structured data only

Question 10

A computer vision team is building an image classifier for rare wildlife species. The dataset has only 500 images per class, and the model overfits. Which THREE data augmentation techniques are most likely to reduce overfitting? (Choose three.)

Accepted Answer

Horizontal flip. Horizontal flip is a simple and effective data augmentation technique that doubles the training data by mirroring images, which helps the model generalize better to variations in orientation. This is particularly useful for wildlife images where the animal may appear facing left or right, reducing overfitting by exposing the model to more diverse examples without collecting new data.

Answer

Adding Gaussian noise

Answer

Random cropping