PMLE Scaling prototypes into ML models — Timed 20 Questions

Question 1

A startup has developed a prototype ML model using scikit-learn on a single machine. They now need to scale it to handle larger datasets and deploy it for real-time predictions. The team is small and wants minimal operational overhead. Which Google Cloud service should they use?

Accepted Answer

Vertex AI. Vertex AI (option B) is the correct choice because it provides a unified, fully managed MLOps platform that integrates model training, deployment, and scaling with minimal operational overhead. It supports scikit-learn models natively, offers auto-scaling for real-time predictions, and eliminates the need to manage infrastructure, making it ideal for a small team transitioning from a prototype.

Answer

AI Platform Prediction

Answer

Cloud Functions

Answer

Compute Engine with TensorFlow Serving

Question 2

A data science team has trained a TensorFlow model on-premises using a large dataset. When they try to deploy the model to Vertex AI for online predictions, the deployed model fails to start with a ‘MemoryError’. The model artifact is 2 GB, and the machine type is n1-standard-4 (15 GB RAM). What is the most likely cause?

Accepted Answer

The model is too large for the machine's memory, causing an out-of-memory (OOM) error during loading.. Option C is correct because the model artifact is 2 GB, and loading it into memory on an n1-standard-4 machine (15 GB RAM) can still cause a MemoryError. TensorFlow models often require additional memory for graph construction, intermediate tensors, and framework overhead, which can easily exceed the available RAM, especially when the model is loaded entirely into memory before serving.

Answer

The model is stored in a regional bucket and the Vertex AI endpoint is in a different region.

Answer

The machine type does not support TensorFlow models larger than 1 GB.

Answer

The model file is corrupted or missing dependencies, causing a crash.

Question 3

A company has a prototype ML model that works well on historical data, but when deployed to production, the model performance degrades over time. The data distribution shifts gradually. Which strategy should they implement to maintain model accuracy?

Accepted Answer

Implement a retraining pipeline that periodically retrains the model on recent data.. Option C is correct because gradual data distribution shifts (concept drift) require the model to adapt to new patterns over time. A retraining pipeline that periodically retrains on recent data ensures the model remains aligned with the current production distribution, directly addressing the degradation caused by drift without relying on static historical data.

Answer

Increase the regularization strength to prevent overfitting.

Answer

Increase the amount of training data by using more historical records.

Answer

Switch to a more complex model architecture to better capture patterns.

Question 4

An ML engineer is scaling a prototype to production using Vertex AI Pipelines. The pipeline includes data validation, preprocessing, training, and deployment steps. They want to ensure that the pipeline can be reproduced and audited. What is the best practice?

Accepted Answer

Define the pipeline using Kubeflow Pipelines SDK and run it on Vertex AI Pipelines.. Using a fully managed pipeline service like Vertex AI Pipelines automatically tracks artifacts, parameters, and lineage, ensuring reproducibility and auditability. Option A is not a service; Option B is about environment consistency but does not provide built-in tracking. Option D is about dependencies but not the pipeline orchestration.

Answer

Use a Docker container with fixed tags and manually record runs.

Answer

Store all data and models in a single Cloud Storage bucket with no versioning.

Answer

Pin all library versions in a requirements.txt file.

Question 5

A team has trained a sentiment analysis model using PyTorch on Vertex AI Training. They now want to deploy it for online predictions with low latency. Which TWO actions should they take? (Choose 2)

Accepted Answer

Use a machine type with a GPU for faster inference.. Option B is correct because GPU-accelerated inference significantly reduces latency for deep learning models like sentiment analysis, especially when using PyTorch, which has native CUDA support. Vertex AI Prediction supports GPU machine types (e.g., n1-standard-4 with NVIDIA T4) that can process batched requests faster than CPUs, directly addressing the low-latency requirement.

Answer

Create multiple model versions for A/B testing.

Answer

Enable batch prediction instead of online prediction.

Answer

Convert the model to TensorFlow SavedModel format.

Question 6

A company has a prototype ML model that predicts equipment failure. They want to deploy it to production using Vertex AI. The model must be retrained weekly with new data. They also need to monitor for data drift and model performance. Which THREE components should they include in their MLOps pipeline? (Choose 3)

Accepted Answer

A scheduled training pipeline that retrains the model weekly.. Option A is correct because the requirement specifies weekly retraining, which is best implemented as a scheduled training pipeline in Vertex AI using Cloud Scheduler or a recurring AI Platform Pipeline run. This automates the retraining process without manual intervention, ensuring the model stays current with new data.

Answer

A manual QA step where data scientists approve each deployment.

Answer

A manual review of new data before it is used for training.

Question 7

An ML engineer is trying to upload a TensorFlow model to Vertex AI using the gcloud command shown. The model was trained using TensorFlow 2.11 and saved with model.save('model/'). The engineer sees the error. What is the most likely cause?

Accepted Answer

The model was saved in a format other than SavedModel (e.g., HDF5) or the artifact path does not contain the expected directory structure.. Option D is correct because the error indicates that Vertex AI cannot find the expected SavedModel artifacts (saved_model.pb and variables/ directory) at the specified path. When using model.save('model/') with TensorFlow 2.11, the default format is the SavedModel format, but the artifact path must point to the directory containing the saved_model.pb file, not a parent directory or a model saved in HDF5 format. The gcloud command likely references a path that does not contain the required SavedModel structure, causing the upload to fail.

Answer

The container port should be 8080 instead of 8501.

Answer

The service account does not have permission to access the bucket.

Answer

The container image is for TensorFlow 2.11 but the model was saved with an older version.

Question 8

You are an ML engineer at a fintech company. You have a prototype credit risk model built using XGBoost that achieves high accuracy on historical data. The model is trained on a dataset with 500,000 rows and 50 features. The company wants to deploy this model to production to score loan applications in real-time. The production environment must handle a peak load of 100 requests per second with a latency under 200ms. You have decided to use Vertex AI for deployment. After deploying the model as a Vertex AI endpoint with a single n1-standard-4 machine, you notice that latency exceeds 500ms at peak load and some requests time out. You have verified that the model prediction itself (excluding network overhead) takes about 50ms on average. What should you do to meet the latency and throughput requirements?

Accepted Answer

Enable autoscaling with a minimum of 2 replicas and use a larger machine type (e.g., n1-standard-8) to handle more concurrent requests.. Option C is correct because the latency bottleneck is not the model inference time (50ms) but the inability of a single n1-standard-4 machine to handle 100 concurrent requests per second without queuing. By enabling autoscaling with a minimum of 2 replicas and upgrading to n1-standard-8, you increase both the number of concurrent requests the endpoint can process and the CPU/memory resources per replica, reducing queue wait times and keeping total latency under 200ms. This directly addresses the throughput and latency requirements without changing the model or switching to batch processing.

Answer

Change the machine type to a GPU-accelerated machine like n1-standard-4 with a T4 GPU.

Answer

Prune the model to reduce size and improve prediction speed.

Answer

Switch from online prediction to batch prediction using Vertex AI Batch Prediction.

Question 9

A machine learning team has a prototype using a custom TensorFlow model trained on a small dataset stored in Cloud Storage. They want to scale the prototype to production with minimal code changes while ensuring the model can handle increased traffic and new data. The model currently loads data using tf.data.Dataset from CSV files. Which approach best meets these requirements?

Accepted Answer

Use Vertex AI Training with hyperparameter tuning and distributed training, then deploy the model to Vertex AI Prediction with autoscaling.. Vertex AI Prediction with autoscaling directly addresses the need to handle increased traffic without code changes, while Vertex AI Training with hyperparameter tuning and distributed training enables scaling to larger datasets with minimal modifications to the existing tf.data pipeline. This approach keeps the custom TensorFlow model intact and leverages managed infrastructure for both training and serving.

Answer

Deploy the model to AI Platform (Unified) Prediction with a custom container, and use AI Platform Training to retrain on larger datasets.

Answer

Migrate the model to BigQuery ML and use SQL for training and prediction to leverage BigQuery's scalability.

Answer

Package the model as a Cloud Run Function and use Cloud Scheduler to trigger retraining periodically.

Question 10

Which TWO actions are best practices when scaling a prototype ML model to production in Google Cloud?

Accepted Answer

Store and manage features in a feature store like Vertex AI Feature Store.. Vertex AI Feature Store centralizes feature management, ensuring consistency between training and serving. This eliminates training-serving skew by providing a single source of truth for features, which is critical when scaling from prototype to production.

Answer

Test the model only on a small sample of the production data to save costs.

Answer

Manually scale inference instances based on historical traffic patterns.

Answer

Use one-hot encoding for all categorical features without considering cardinality.

Question 11

A team deployed a prototype classification model to Vertex AI Prediction. After a week, they notice the metrics shown in the exhibit. What is the most likely cause of the performance degradation and latency increase?

Accepted Answer

The training data does not represent the current production data distribution, causing the model to make incorrect predictions and requiring more computation.. The exhibit shows both accuracy degradation and increased latency. Option C is correct because when the production data distribution shifts away from the training data (data drift), the model makes more incorrect predictions, which can trigger additional computation (e.g., retries, fallback logic, or increased uncertainty estimation) and cause latency spikes. Vertex AI Prediction does not inherently add computation for wrong predictions, but the model's internal confidence thresholds or post-processing steps may consume extra resources when handling out-of-distribution inputs.

Answer

The prediction endpoint's autoscaling is too slow, causing requests to queue and time out.

Answer

The prediction requests are too large, exceeding the maximum request size limit for Vertex AI.

Answer

The custom prediction container uses outdated libraries that are incompatible with Vertex AI's runtime.

Question 12

Drag and drop the steps to create and deploy a custom ML model on Vertex AI using a container in the correct order.

Question 13

Drag and drop the steps to set up model monitoring for drift detection on Vertex AI in the correct order.

Question 14

Match each ML acronym to its definition.

Question 15

Match each ML model interpretability method to its description.

Question 16

A team has a trained TensorFlow model running locally and wants to deploy it for low-latency online predictions on Google Cloud. Which service should they use?

Accepted Answer

Vertex AI Prediction. Vertex AI Prediction is the correct choice because it is a fully managed service designed specifically for deploying trained ML models for online (real-time) prediction with low latency. It supports importing TensorFlow SavedModel artifacts and automatically scales the serving infrastructure, including GPU/TPU support, to handle request traffic while providing built-in monitoring and explainability features.

Answer

AI Platform Training

Answer

Cloud Run

Answer

Cloud Functions

Question 17

An ML team is scaling a prototype to production. The data pipeline currently reads from Cloud Storage and transforms data with a custom Python script. They need to handle higher throughput and add monitoring. Which approach should they take?

Accepted Answer

Migrate the pipeline to Apache Beam on Dataflow with Cloud Monitoring. Apache Beam on Dataflow provides a unified programming model for batch and streaming data processing, enabling automatic scaling to handle higher throughput. Cloud Monitoring integrates natively with Dataflow to track pipeline metrics, latency, and error rates, addressing the monitoring requirement. This approach is purpose-built for production-grade data pipelines, unlike ad-hoc solutions.

Answer

Deploy the Python script on a large Compute Engine instance with a cron job

Answer

Rewrite the pipeline to use Pub/Sub and Cloud Functions for processing

Answer

Use Cloud Composer to orchestrate the Python script at scale

Question 18

A company has a prototype ML model that achieves 85% accuracy on historical data. In production, accuracy drops to 70% after two weeks due to data drift. They need an automated retraining pipeline with minimal manual oversight. Which solution is most cost-effective?

Accepted Answer

Set up Vertex AI Model Monitoring to detect drift, which triggers a Cloud Function that submits a Vertex AI Training job with new data. Option C is correct because it combines automated drift detection via Vertex AI Model Monitoring with a serverless retraining trigger (Cloud Function) that submits a Vertex AI Training job, minimizing manual oversight while only incurring costs when drift is detected. This avoids the expense of continuous retraining or always-on GPU instances, making it the most cost-effective solution for the described scenario.

Answer

Use Cloud Functions to trigger a Dataflow job that trains the model using custom containers

Answer

Deploy the model on a GPU-equipped Compute Engine VM and run retraining every time new data arrives

Answer

Schedule a weekly Cloud Composer DAG that runs a new training job with all available data

Question 19

A team prototypes a recommendation model using a Jupyter notebook on Vertex AI Workbench. They want to productionize the model with CI/CD. Which approach should they use to package the model for deployment?

Accepted Answer

Upload the model to Vertex AI Model Registry and use it for deployment. Vertex AI Model Registry is the central repository for managing ML models, enabling versioning, evaluation, and deployment to endpoints. This approach integrates with CI/CD pipelines via the Vertex AI SDK or Cloud Build, allowing automated model promotion and deployment without manual packaging. Option D directly leverages Vertex AI's native deployment workflow, which is the recommended path for productionizing models from Workbench.

Answer

Use Cloud Build to deploy the notebook directly as a prediction endpoint

Answer

Store the model in Cloud Source Repositories and deploy from there

Answer

Containerize the model and push to Artifact Registry, then deploy via Cloud Run

Question 20

A data scientist trains an XGBoost model on Vertex AI with a custom container. The model performs well on a held-out test set but fails to generalize in production. They suspect data leakage between training and validation. What is the best practice to prevent this?

Accepted Answer

Store and serve features using Vertex AI Feature Store with point-in-time correctness. Option A is correct because Vertex AI Feature Store with point-in-time correctness ensures that for each training example, only feature values that were known at the time of the prediction (i.e., before the label occurred) are used. This prevents future data from leaking into the training set, which is the most common cause of poor generalization when temporal ordering matters. The Feature Store automatically retrieves the latest feature value as of a specified timestamp, eliminating the need for manual joins and windowing logic.

Answer

Implement feature engineering in Vertex AI Pipelines to ensure temporal ordering

Answer

Store all features in BigQuery and join on timestamp during training and serving

Answer

Use Vertex AI AutoML instead of custom training