AI0-001 AI Implementation and Operations Practice Test 3 — 15 Questions

Question 1

A company deployed a machine learning model on a cloud inference service. Users report high latency during peak hours. The model is deployed on a single instance. Which action should the team take to reduce latency without significant architectural changes?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Enable autoscaling for the inference instances. Enabling autoscaling allows the inference service to automatically add instances during high demand, distributing the load and reducing latency. Increasing the model size would worsen latency. Switching to a batch inference pipeline would increase latency for real-time requests. Adding an API gateway does not address compute capacity.

Answer

Increase the model size to improve accuracy

Answer

Switch to a batch inference pipeline

Answer

Add an API gateway to route requests

Question 2

A data science team uses Git for version control of model code and DVC for data versioning. They want to implement a model registry to track trained models, their hyperparameters, and performance metrics. Which tool is specifically designed for this purpose and integrates with the existing workflow?

Accepted Answer

MLflow Model Registry. MLflow Model Registry is specifically designed for managing model versions, tracking metadata, and integrating with Git and DVC. Apache Airflow is for workflow orchestration, not model registry. Kubernetes is for container orchestration. Docker is for containerization.

Answer

Apache Airflow

Answer

Docker

Answer

Kubernetes

Question 3

An ML team monitors a production model using a dashboard that shows daily performance metrics. Over the past month, the model's accuracy has dropped from 92% to 87%, while the data distribution of input features has remained stable according to statistical tests. Which type of model drift is most likely occurring?

Accepted Answer

Concept drift. Concept drift occurs when the relationship between input features and the target variable changes, even if the input data distribution remains stable. In this scenario, the model's accuracy declines from 92% to 87% while input feature distributions are unchanged, indicating that the underlying mapping from features to labels has shifted—a classic sign of concept drift.

Answer

Data drift (covariate shift)

Answer

Model decay

Answer

Overfitting

Question 4

A company must deploy a new model version with zero downtime. The current model is served via a REST API on a Kubernetes cluster. Which deployment strategy should the team use to gradually shift traffic to the new version while monitoring for errors?

Accepted Answer

Canary deployment. A canary deployment gradually shifts a small percentage of traffic to the new model version while the majority continues to hit the stable version. This allows the team to monitor for errors and roll back quickly if issues arise, achieving zero downtime. It is the ideal strategy for validating a new model in production with minimal risk.

Answer

Blue-green deployment

Answer

Recreate deployment

Answer

Rolling update

Question 5

An operations team monitors a classification model in production. The confusion matrix for the model shows the following values: TP=1500, FN=500, FP=600, TN=2400. Which metric should the team calculate to assess the model's ability to avoid false positives?

Accepted Answer

Precision. Precision (TP / (TP + FP)) directly measures the proportion of positive identifications that were actually correct, making it the ideal metric to assess the model's ability to avoid false positives. With TP=1500 and FP=600, precision is 1500/(1500+600)=0.714, indicating that 71.4% of predicted positives are true positives, while 28.6% are false positives. The question explicitly asks about avoiding false positives, which is the inverse of precision's focus on the correctness of positive predictions.

Answer

Accuracy

Answer

F1-score

Answer

Recall

Question 6

An ML engineering team has a retraining pipeline that triggers automatically when model accuracy drops below a threshold. Recently, the model's accuracy has been fluctuating, causing frequent retraining and high compute costs. The team suspects the data distribution is changing slowly. Which approach should the team implement to reduce unnecessary retraining while maintaining model performance?

Accepted Answer

Implement a statistical drift detection method on input features. Implementing a statistical drift detection method (e.g., using KL divergence, PSI, or ADWIN) on input features allows the team to identify when the data distribution has genuinely changed, rather than reacting to random accuracy fluctuations. This reduces unnecessary retraining by triggering the pipeline only when statistically significant drift is detected, maintaining model performance without the high compute costs of frequent retraining.

Answer

Use a simpler model to reduce variability

Answer

Increase the frequency of model retraining

Answer

Reduce the batch size for inference

Question 7

A company deploys an AI model via a REST API that handles sensitive customer data. To secure the endpoint, the security team requires that only authenticated and authorized applications can invoke the API. Which mechanism should be implemented?

Accepted Answer

API key or bearer token in the HTTP header. API keys or bearer tokens (e.g., OAuth 2.0 access tokens) are the standard mechanism for authenticating and authorizing client applications when invoking a REST API. These tokens are passed in the HTTP Authorization header, allowing the server to verify the client's identity and permissions before processing requests containing sensitive customer data.

Answer

TLS encryption for the connection

Answer

Input sanitization to prevent injection

Answer

IP whitelisting

Question 8

A data scientist submits a model training job to a cloud ML platform. The job fails with an error: "Out of memory: Killed process." The training code is proven to work on the developer's local machine with 16GB RAM. The cloud instance has 32GB RAM. What is the most likely cause?

Accepted Answer

The training code is loading the entire dataset into memory without batch processing. The error 'Out of memory: Killed process' occurs when the operating system's OOM killer terminates a process that exceeds available RAM. Even though the cloud instance has 32GB RAM, if the training code loads the entire dataset into memory without batch processing (e.g., using pandas.read_csv() without chunking), it can consume far more memory than the dataset size due to Python object overhead, temporary copies, and intermediate tensors. The local machine with 16GB RAM may have worked due to a smaller dataset or different memory pressure, but the cloud instance's 32GB is insufficient if the code is not memory-efficient.

Answer

The cloud instance does not have enough memory

Answer

The training model is too large for the GPU memory

Answer

The cloud instance ran out of disk space

Question 9

A company serves a large language model (LLM) on a Kubernetes cluster. The inference latency is acceptable but the cost is high due to GPU usage. The model is 7 billion parameters and requires 16GB GPU memory. The team wants to reduce cost without increasing latency. Which strategy should they implement?

Accepted Answer

Use model quantization to reduce precision. Model quantization reduces the precision of the model's weights (e.g., from FP32 to INT8), which decreases the GPU memory footprint from 16GB to approximately 4GB for a 7B parameter model. This directly lowers GPU cost per inference while maintaining acceptable latency, as the model can run on fewer or cheaper GPUs without increasing inference time.

Answer

Increase the batch size for inference

Answer

Add more GPU nodes to distribute the load

Answer

Switch to CPU-based inference

Question 10

A team monitors a production model for bias. They measure the selection rate for two demographic groups and find a significant difference. Which TWO actions should the team take to mitigate bias? (Choose two.)

Accepted Answer

Retrain the model with a balanced training dataset. Retraining with a balanced training dataset (Option C) directly addresses the root cause of bias by ensuring the model learns from equal representation of both demographic groups, which reduces skewed selection rates. This is a standard data-level mitigation technique in AI fairness, as it prevents the model from overfitting to majority patterns.

Answer

Increase the complexity of the model to capture more patterns

Answer

Add more training data from both groups

Answer

Remove the protected attribute from the model input

Question 11

An ML operations team needs to monitor a deployed model's performance. Which TWO metrics are most useful for detecting concept drift in a regression model? (Choose two.)

Accepted Answer

Distribution of residuals between predictions and actuals. Monitoring the distribution of residuals (predicted vs. actual values) directly reveals when the relationship between inputs and outputs has shifted, which is the essence of concept drift. In a regression model, if the residuals become systematically biased or their variance changes over time, it indicates that the underlying data-generating process has changed, even if input feature distributions remain stable.

Answer

Distribution of input features

Answer

Classification accuracy

Answer

Model inference latency

Question 12

An organization wants to implement a robust MLOps pipeline. Which THREE components are essential for a complete MLOps framework? (Choose three.)

Accepted Answer

Continuous integration and continuous deployment (CI/CD) pipeline. A is correct because a CI/CD pipeline automates the integration of code changes and deployment of models into production, ensuring consistent and reliable releases. In MLOps, this includes building, testing, and deploying both application code and ML model artifacts, which is fundamental for operationalizing machine learning at scale.

Answer

Automated code review and approval gates

Answer

Real-time model monitoring dashboard

Question 13

A model serving pod is failing with OOMKilled. What is the most likely cause?

Accepted Answer

The model requires more memory than the 2Gi limit. An OOMKilled error in Kubernetes indicates that a container exceeded its memory limit and was terminated by the Out Of Memory (OOM) killer. The most common cause is that the model's inference or training workload requires more memory than the configured resource limit (e.g., 2Gi), forcing the kernel to kill the process. This is a direct result of the container's memory request/limit mismatch with the actual consumption.

Answer

The container image is corrupted

Answer

The model version is outdated

Answer

The Kubernetes cluster has run out of disk space

Question 14

An ML team uses the model registry above. After deploying version 3 to production, they discover it has a critical bug. What is the fastest way to roll back to a stable version without re-deploying from scratch?

Accepted Answer

Redeploy version 1 by updating the production deployment to use its artifact. It directly updates the production deployment to reference the artifact of version 1, which is a stable, previously validated version. This avoids the overhead of re-deploying from scratch by simply pointing the existing deployment to a known good artifact in the model registry, leveraging the registry's artifact storage and deployment integration.

Answer

Promote version 2 from Staging to Production

Answer

Retrain the model with corrected data

Answer

Delete version 3 from the registry

Question 15

A data scientist is deploying a machine learning model to production. The model was trained on an imbalanced dataset. Which technique should be used during deployment to mitigate bias without retraining the model?

Accepted Answer

Apply post-processing calibration to adjust decision thresholds. Post-processing calibration adjusts the decision threshold of the model to account for the class imbalance present in the training data. This technique modifies the output probabilities or classification boundary without requiring access to the original training data or retraining the model, making it suitable for deployment scenarios where the model is already fixed.

Answer

Use an ensemble of models trained on balanced subsets

Answer

Rebalance the dataset using SMOTE before inference

Answer

Remove sensitive features from the input data