AIF-C01 Applications of Foundation Models Practice Question

Which TWO techniques can reduce the cost of running a fine-tuned foundation model on Amazon SageMaker? (Choose TWO.)

⚠ Common exam trap

AWS often tests the distinction between techniques that reduce inference cost (pruning, quantization) versus those that improve training speed or accuracy, leading candidates to mistakenly select options that increase resource usage or are irrelevant to inference cost.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Implement structured pruning to remove less important model parameters.

Option A is correct because structured pruning removes less important weights, neurons, or channels from the fine-tuned model, producing a smaller model that requires fewer compute and memory resources during SageMaker inference, which directly lowers hosting cost. Option C is correct because quantization reduces numerical precision from FP32 to FP16 or INT8, shrinking model size and memory bandwidth needs and enabling faster, cheaper inference on SageMaker endpoints, especially with GPU instances that support lower-precision arithmetic. Option B is not correct because using larger GPU instances increases the hourly cost of the endpoint rather than reducing it. Option D is not correct because keeping parameters in FP32 preserves accuracy but consumes more memory and compute, raising cost. Option E is not correct because increasing training epochs affects training time and accuracy, not the cost of running inference on the deployed model.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✓
Implement structured pruning to remove less important model parameters.
Why this is correct
Structured pruning removes entire neurons, channels or attention heads, shrinking parameter count and memory footprint. The smaller model needs fewer SageMaker instance hours and less GPU memory, directly lowering inference hosting cost while retaining most accuracy.
✗
Use larger instance types with more GPUs to speed up inference.
Why it's wrong here
Larger GPU instances increase hourly charges and do not lower the cost of running inference. It is tempting because more GPUs reduce latency per request, and would be correct when the priority is meeting strict throughput or response-time targets rather than reducing overall spend.
✓
Apply model quantization to reduce precision from FP32 to FP16 or INT8.
Why this is correct
Quantization stores weights and activations at FP16 or INT8 instead of FP32, cutting memory bandwidth and enabling faster matrix operations. This reduces the SageMaker instance size and compute time required, directly lowering the cost of hosting the fine-tuned model.
✗
Store the model parameters in FP32 to maintain accuracy during inference.
Why it's wrong here
Storing parameters in FP32 doubles memory footprint and increases inference latency and instance cost compared with lower-precision formats. It is tempting because FP32 preserves numerical accuracy, and would be correct when precision loss from quantisation would unacceptably degrade model output quality.
✗
Increase the number of training epochs to achieve higher accuracy.
Why it's wrong here
Increasing training epochs raises compute time and cost, and does not reduce inference cost at all. It is tempting because more epochs can improve model accuracy, and would be correct when the goal is maximising quality rather than minimising the cost of running the fine-tuned model.

Visual reference

Go deeper

AIF-C01 Applications of Foundation Models Practice Question

Why each option matters

Option-by-option breakdown

Related to this question