PMLE Scaling Prototypes into ML Models Practice Question

An ML team is optimizing an inference model for deployment on edge devices. They need to reduce the model size and improve latency while maintaining accuracy as much as possible. Which two techniques should they use? (Choose TWO.)

⚠ Common exam trap

Candidates often think that FP16 is always better than INT8 for edge devices, but INT8 offers greater size reduction and is more widely supported on edge hardware, including Google's Edge TPU.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Post-training quantization to INT8.

Post-training quantization to INT8 reduces model size by converting 32-bit floating-point weights and activations to 8-bit integers, which also speeds up inference on edge devices with integer-optimized hardware. This technique typically maintains accuracy within 1-2% of the original model while significantly lowering memory footprint and latency.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Use a larger pre-trained model as a starting point.
Why it's wrong here
Larger model increases size and latency.
✓
Post-training quantization to INT8.
Why this is correct
Reduces size and latency with minimal accuracy loss.
✗
Use half-precision (FP16) instead of INT8.
Why it's wrong here
FP16 still uses 16 bits; INT8 is more aggressive in compression.
✓
Apply weight pruning to remove small weights.
Why this is correct
Pruning reduces model size and can improve speed on specialized hardware.
✗
Increase the number of layers in the model.
Why it's wrong here
Increases size and latency.

Go deeper

PMLE Scaling Prototypes into ML Models Practice Question

Why each option matters

Option-by-option breakdown

Related to this question