PMLE Scaling Prototypes into ML Models Practice Question

You are preparing to train a large image classification model on Vertex AI using a custom training job. You want to optimize the training job for cost and performance. The dataset is stored in Cloud Storage as TFRecords and is about 2 TB. You plan to use a machine with 4 NVIDIA V100 GPUs. Which two actions should you take to improve training efficiency? (Choose two.)

⚠ Common exam trap

The trap here is focusing on infrastructure changes like bucket location or more vCPUs, when the most impactful optimizations are at the training algorithm level, such as precision and batch size.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Use a larger batch size and scale the learning rate accordingly.

Mixed precision training and increasing batch size with learning rate scaling are both effective ways to improve training efficiency on V100 GPUs. Mixed precision leverages tensor cores for faster computation and reduced memory, while larger batches improve GPU utilization. These actions directly target performance and cost without altering the model architecture or data storage.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Use a smaller model architecture to reduce training time.
Why it's wrong here
Changing the model architecture alters the problem and may reduce accuracy, which is not the goal. The scenario is about optimizing the given model, not replacing it. This action does not address training efficiency for the existing model and could compromise results.
✗
Store the TFRecords in a regional Cloud Storage bucket instead of a multi-regional bucket.
Why it's wrong here
While regional buckets can have lower latency and cost, the difference is often negligible for training jobs that stream data. The main bottleneck is usually data loading and preprocessing, not the bucket location. This action does not directly improve training efficiency in terms of GPU utilization or speed.
✓
Use a larger batch size and scale the learning rate accordingly.
Why this is correct
Increasing batch size can improve GPU utilization and reduce training time by processing more samples per iteration. Scaling the learning rate helps maintain convergence. This is a standard technique for efficient training on multiple GPUs. It directly leverages the available GPU memory and compute, leading to better throughput and cost efficiency.
✓
Enable mixed precision training using NVIDIA Apex or TensorFlow's mixed precision API.
Why this is correct
Mixed precision training uses float16 for computations where possible, reducing memory usage and speeding up training on V100 GPUs, which have tensor cores optimized for float16. This allows larger batch sizes and faster iterations. It is a proven method to improve performance and reduce cost without sacrificing model accuracy.
✗
Increase the number of vCPUs on the machine to improve data loading throughput.
Why it's wrong here
Adding more vCPUs can help data loading, but if the input pipeline is not optimized (e.g., using tf.data with parallel reads), the extra CPUs may remain idle. It is not as direct an improvement as mixed precision or batch size tuning. Moreover, it increases cost without guaranteed performance gains.

Go deeper

PMLE Scaling Prototypes into ML Models Practice Question

Why each option matters

Option-by-option breakdown

Related to this question