AI-900 Practice Question: Describe features of generative AI workloads on Azure
What is 'model distillation' and why might you distill a large model to a small one?
⚠ Common exam trap
Test-takers frequently confuse model distillation with model compression techniques like pruning or quantization, but distillation specifically involves training a new smaller model to mimic the larger model's output distribution, not modifying the original model's parameters.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Training a smaller model to mimic a larger model's behaviour for efficient deployment
Model distillation is a technique where a smaller 'student' model is trained to replicate the behavior of a larger 'teacher' model. This is done by using the teacher's softmax outputs (logits) as training targets, allowing the student to achieve similar accuracy with far fewer parameters, making it suitable for resource-constrained environments like edge devices or real-time inference.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Extracting the essential ideas from a model's outputs into a written summary
Why it's wrong here
This describes text summarization, a natural language processing task that condenses a model's generated content into a brief written overview. Knowledge distillation does not summarize outputs; instead, it uses the teacher model's full prediction distribution (logits) as a training signal for a compact student model. The goal is to preserve decision-making behavior, not to produce a human-readable digest.
- ✓
Training a smaller model to mimic a larger model's behaviour for efficient deployment
Why this is correct
This is the essence of knowledge distillation: a compact student model is trained to reproduce the soft predictions (probability distributions over classes) produced by a larger, more accurate teacher model. The student learns not just hard labels but also the teacher's confidence and inter-class relationships, allowing it to approach teacher-level accuracy while being much smaller. This yields a faster, lower-cost model suitable for deployment on edge devices, mobile apps, or real-time inference pipelines.
- ✗
Removing duplicate or redundant parameters from a trained model
Why it's wrong here
This is parameter pruning, a compression technique that removes redundant weights or neurons from an already-trained network to reduce its size and memory footprint. Distillation is fundamentally different because it creates a new, separate small model that learns to imitate the teacher's outputs rather than surgically deleting components from the original network. Pruning keeps the same architecture and modifies it; distillation builds a smaller and often architecturally distinct student model.
- ✗
Concentrating training data into fewer, higher-quality examples
Why it's wrong here
This describes dataset curation or data quality management, where a training set is filtered, deduplicated, or prioritized to focus on high-value examples. Distillation does not operate on raw data in this way; it operates on the teacher model's predictions, generating a rich supervisory signal derived from the model itself. The student trains on the teacher's soft targets, not on a reduced set of supposedly 'better' examples.
Go deeper
Related to this question
About these practice questions
This AI-900 question is part of Courseiva's 985-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-900 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-900 exam.