AI-900 Practice Question: Describe fundamental principles of machine learning on Azure
What is 'differential privacy' and how is it relevant to AI model training?
⚠ Common exam trap
Many candidates confuse data privacy techniques (like encryption or access control) with the formal mathematical guarantee of differential privacy, which specifically addresses information leakage from the model's outputs rather than protecting the data at rest or in transit.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
A mathematical guarantee that model training reveals negligible information about any individual's data
Differential privacy is a mathematical framework that ensures the output of a model training process does not reveal whether any specific individual's data was included in the training dataset. It achieves this by adding calibrated noise to the training process or query results, providing a formal privacy guarantee quantified by the epsilon parameter. This is directly relevant to AI model training because it allows organizations to train models on sensitive data while protecting individual privacy, which is a core requirement for compliance with regulations like GDPR and HIPAA.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
The difference in model accuracy between a private deployment and a public API
Why it's wrong here
The difference in model accuracy between a private deployment and a public API typically reflects environmental factors like hardware, batching, quantization, or serving infrastructure, not a formal characteristic of the training process. Differential privacy is a definition of privacy that constrains how much a single individual's data influences the model's learned parameters or predictions, independent of where the model is hosted. While differentially private training can reduce accuracy due to injected noise, that trade-off is a consequence of the guarantee, not the definition itself.
- ✓
A mathematical guarantee that model training reveals negligible information about any individual's data
Why this is correct
Differential privacy is a rigorous mathematical definition: it guarantees that the probability of producing any particular model output is almost identical whether any single individual's record is included in or excluded from the training dataset. This is achieved by adding calibrated random noise to the gradient updates or objective function during training, which prevents the model from memorizing or exposing individual-level information. The epsilon parameter quantifies the privacy loss, providing a formal, provable bound that attackers cannot infer specific data points with confidence, regardless of auxiliary knowledge.
- ✗
Encrypting model weights so they remain private from users accessing the model API
Why it's wrong here
Encrypting model weights protects the model as an intellectual property asset by ensuring only authorized parties can read or invoke the artifact, but it does not alter the information-theoretic properties of the training algorithm. Differential privacy addresses a different concern: the possibility that the trained model itself memorizes and later reveals sensitive attributes of individuals in the training data. Even with fully encrypted weights, a model trained without differential privacy could still leak personal data if the weights are decrypted by an authorized user.
- ✗
Using different models for different privacy tiers of customers
Why it's wrong here
Segmenting customers into different privacy tiers and serving them with separate models is a product or access-control strategy that determines which data each model is exposed to, but it does not provide any formal, quantitative guarantee about what an adversary can infer. Differential privacy is a single, mathematical property applied uniformly to the training algorithm, bounding the influence of any one record irrespective of how the user base is partitioned. Merely using different models per tier leaves each model susceptible to memorization attacks on its own training data, so it cannot substitute for a formal privacy guarantee.
Go deeper
Related to this question
About these practice questions
This AI-900 question is part of Courseiva's 985-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-900 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-900 exam.