AI-900 Practice Question: Describe features of generative AI workloads on Azure
What is 'evaluation' of generative AI models in Azure AI Foundry?
⚠ Common exam trap
Many candidates confuse the systematic, metric-driven evaluation in Azure AI Foundry (which uses automated evaluators for groundedness, relevance, and safety) with user feedback mechanisms (thumbs up/down) or training-phase diagnostics, leading candidates to pick option C or D instead of B.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Systematically measuring a generative AI application's quality (groundedness, relevance) and safety metrics
In Azure AI Foundry, evaluation refers to the systematic measurement of a generative AI application's quality and safety using predefined metrics such as groundedness (factual alignment with source data), relevance, and safety (e.g., content filtering). This process is distinct from ad-hoc user feedback or training diagnostics, as it provides structured, repeatable assessments to validate model behavior before deployment.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
The process of assessing job candidates using AI-powered assessments
Why it's wrong here
Assessing job candidates with AI-powered assessments is a business process that uses AI for HR screening, not the act of evaluating a generative AI application's output. In the Azure AI Foundry evaluation context, evaluation measures the quality and safety metrics of the application itself, such as groundedness and relevance, rather than evaluating human subjects or using AI to score people.
- ✓
Systematically measuring a generative AI application's quality (groundedness, relevance) and safety metrics
Why this is correct
This is the correct definition. Azure AI Foundry evaluation systematically runs test datasets through quality evaluators (e.g., groundedness, relevance, coherence, fluency) and safety evaluators (e.g., hateful, violent, sexual content). It produces metric scores that quantify how well the generative AI application performs, guiding iterative improvements against baseline benchmarks.
- ✗
Having users rate the AI's responses with thumbs up or thumbs down during beta testing
Why it's wrong here
Collecting thumbs up/down ratings during beta testing gathers valuable but subjective, small-scale human feedback. It is a product testing technique, not a formal evaluation because it lacks systematic, metric-based assessment against controlled test datasets. In Azure AI Foundry, evaluation runs automated evaluators over a test set to produce standardized quality and safety scores, not ad hoc user sentiment.
- ✗
Running model training and measuring loss curves to determine when to stop training
Why it's wrong here
Monitoring loss curves during training is a training-phase diagnostic used to detect convergence, overfitting, or underfitting. It does not measure the quality of a deployed generative AI application's responses. Azure AI Foundry evaluation focuses on output-level metrics like groundedness and relevance computed by evaluators on test prompts, whereas loss is about optimization dynamics, not application behavior.
Go deeper
Related to this question
Learn chapter
Azure Machine Learning Studio
Key term
Model
In IT and AI, a model is a trained mathematical representation that learns patterns from data to make predictions or decisions.
Key term
Azure OpenAI Service
Azure OpenAI Service is a cloud platform from Microsoft that lets developers use powerful artificial intelligence models, like GPT-4, to build applications that can understand and generate human-like text, code, images, and more.
About these practice questions
One of 985 original AI-900 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-900 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-900 exam.