PMLE Serving and Scaling Models Practice Question

You are deploying a large language model on a Vertex AI endpoint. The model is loaded from a Cloud Storage bucket at container startup, which adds 3 minutes to each cold start. You want to reduce cold-start time and ensure predictable latency during scale-out. Which approach should you take?

⚠ Common exam trap

The trap here is assuming that raising the minimum replica count eliminates cold starts entirely, when it only masks them for steady-state traffic and does nothing for scale-out events.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Store the model artifacts in a custom container image and push it to Artifact Registry.

Embedding the model artifacts in a custom container image eliminates the separate download from Cloud Storage during container startup. The image layers are pulled by the node as part of standard container initialization, which is typically faster and more predictable than fetching a large model from a GCS bucket after the container starts.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✓
Store the model artifacts in a custom container image and push it to Artifact Registry.
Why this is correct
Baking the model into a custom container image pulls the artifacts when the container image is downloaded, which happens as part of standard node provisioning. This reduces the additional model-download step at startup, cutting cold-start time and making scale-out more predictable.
✗
Use a larger machine type with more vCPUs and memory for each replica.
Why it's wrong here
A larger machine type may speed up model loading slightly, but the bottleneck is the network transfer from Cloud Storage to the replica. The dominant delay remains the artifact download, and larger machines increase cost without addressing the root cause.
✗
Set the endpoint's minReplicaCount to a high value so that replicas are always warm.
Why it's wrong here
Increasing minReplicaCount keeps replicas running, avoiding cold starts for those replicas, but it does not reduce the cold-start time when the endpoint must scale out beyond the minimum. It also increases cost. The scenario asks to reduce cold-start time, not just avoid it at steady state.
✗
Enable request-response logging on the endpoint to monitor startup latency.
Why it's wrong here
Request-response logging captures prediction requests and responses for auditing and debugging. It does not affect how long the model takes to load from Cloud Storage, so it does not reduce cold-start time. Logging is an observability feature, not a performance optimization.

Go deeper

PMLE Serving and Scaling Models Practice Question

Why each option matters

Option-by-option breakdown

Related to this question