Databricks-GenAI-Assoc Evaluation and Monitoring Practice Question
A GenAI engineer monitors a customer-facing RAG assistant hosted on Databricks. After a routine re-indexing job, groundedness scores from the MLflow LLM Evaluation job drop sharply while answer relevance stays flat. The application prompt and the LLM serving endpoint were untouched. Which conclusion is best supported by these signals?
⚠ Common exam trap
The trap here is attributing a groundedness drop to the model or judge, when a sharp drop synchronized with re-indexing and stable relevance points to the retrieval context instead.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
The re-indexing job degraded retrieval, so the generator receives weaker supporting context
The drop begins exactly when the index was rebuilt, and relevance remains flat, which isolates the change to the evidence supplied to the generator rather than to the prompt, the LLM, or the judge. Degraded retrieval after re-indexing, from altered chunking, embedding mismatches, or an incomplete build, is the best-supported conclusion and the right place to investigate first.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
The LLM judge model was silently upgraded and is scoring more harshly
Why it's wrong here
A judge upgrade could shift scores, but the drop coincides precisely with the re-indexing job, which is a stronger causal signal. If the judge were the cause, relevance would likely move as well, since the same judge scores both metrics. The engineer should first inspect the index and retrieval outputs rather than assume judge drift.
- ✗
Users are asking harder questions than before, lowering groundedness
Why it's wrong here
A change in user behaviour would not align with the re-indexing event and would typically affect relevance too, as harder questions still need on-topic answers. More importantly, the evaluation job runs against a fixed dataset, not live traffic, so query difficulty is controlled. This explanation does not fit the evidence pattern.
- ✓
The re-indexing job degraded retrieval, so the generator receives weaker supporting context
Why this is correct
Groundedness depends on the retrieved context supporting the answer, so a sharp drop right after re-indexing implicates the retrieval pipeline. Flat relevance shows the generator still addresses the question, but now without adequate evidence, producing less grounded answers. Chunking changes, embedding mismatches, or partial index builds during re-indexing are the likely culprits to investigate first.
- ✗
The generation prompt has drifted and needs to be rewritten
Why it's wrong here
The prompt was explicitly untouched, and relevance staying flat indicates the generator still produces on-topic answers. A prompt problem would typically degrade relevance or style alongside groundedness. The timing after re-indexing plus the steady relevance score point away from the generation side and toward the evidence the generator is now receiving.
About these practice questions
Courseiva writes every Databricks-GenAI-Assoc question from scratch — 330 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official Databricks exam blueprint
This Databricks-GenAI-Assoc practice question is part of Courseiva's free Databricks certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the Databricks-GenAI-Assoc exam.