Databricks-GenAI-Assoc Evaluation and Monitoring Practice Question
A team runs a RAG chatbot whose MLflow evaluation with the built-in groundedness judge previously scored well. After they swap the retriever for a new embedding model, groundedness scores drop sharply even though the generator model and prompt are unchanged. They confirm the judge model itself is unchanged. Which action should they take FIRST to diagnose the regression?
⚠ Common exam trap
The trap here is assuming a groundedness drop always indicates a hallucinating generator, when the metric is just as sensitive to degraded retrieved context.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Inspect the retrieved context chunks logged per evaluation row to verify whether the new retriever is returning passages that no longer support the generated claims.
Groundedness measures whether the answer is supported by the retrieved context, so a change to retrieval is the most likely culprit. Inspecting the logged retrieved chunks per evaluation row reveals whether the new embedding model returns less relevant or contradictory passages, which the judge then flags as unsupported claims. This isolates retrieval as the cause before any remediation.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Roll back the embedding model change immediately and re-run the evaluation to confirm the previous score returns.
Why it's wrong here
Rolling back may restore the score, but it destroys the evidence needed to understand why the new retriever degraded groundedness. Diagnosis should precede remediation; reverting without inspecting retrieved context leaves the root cause unknown and risks repeating the same mistake in future retriever swaps.
- ✓
Inspect the retrieved context chunks logged per evaluation row to verify whether the new retriever is returning passages that no longer support the generated claims.
Why this is correct
Groundedness judges whether each claim in the response is supported by the retrieved context. If the retriever changed, the logged retrieved chunks are the first artifact to inspect, since they directly feed the judge. Reviewing per-row traces confirms whether the regression originates in retrieval rather than generation.
- ✗
Retrain the judge model on a fresh labeled dataset so its scoring distribution aligns with the new embedding model.
Why it's wrong here
The built-in groundedness judge is not retrained on customer data, and the scenario states the judge is unchanged. Replacing or retraining the judge would mask rather than diagnose the retrieval regression, and it is not an available or appropriate first step for a managed judge metric.
- ✗
Increase the temperature of the generator model to encourage more diverse phrasing that the groundedness judge can score more reliably.
Why it's wrong here
Raising temperature increases variability and typically reduces factual grounding, which would push groundedness scores lower, not explain the regression. The generator and prompt were explicitly unchanged, so tuning generation randomness does not isolate the retriever change and would confound the diagnosis.
About these practice questions
Courseiva writes every Databricks-GenAI-Assoc question from scratch — 330 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official Databricks exam blueprint
This Databricks-GenAI-Assoc practice question is part of Courseiva's free Databricks certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the Databricks-GenAI-Assoc exam.