Databricks-GenAI-Assoc Evaluation and Monitoring Practice Question
A Generative AI engineer is using MLflow Tracing to monitor a RAG application deployed on Mosaic AI Model Serving. They want to capture the retrieved documents, the final prompt, and the model's response for each request to debug a quality issue. Which approach should they use to ensure all three are logged in a single trace?
⚠ Common exam trap
The trap here is assuming that inference table logging automatically includes intermediate retrieval steps and the exact prompt, when it only captures request and response payloads.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Instrument the application with the MLflow tracing SDK, creating spans for retrieval, prompt construction, and generation within the same trace context.
To capture retrieved documents, the final prompt, and the model response in a single trace, the engineer should use MLflow Tracing with manual instrumentation. Spans for each step share a trace context, providing a unified view. This is the native Databricks approach for debugging RAG applications and integrates with Mosaic AI Model Serving for production monitoring.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Instrument the application with the MLflow tracing SDK, creating spans for retrieval, prompt construction, and generation within the same trace context.
Why this is correct
MLflow Tracing allows manual instrumentation with spans that share a trace context. By wrapping retrieval, prompt construction, and generation in spans, all three are captured in one trace. This provides end-to-end visibility, which is essential for debugging RAG quality issues. It works with Mosaic AI Model Serving and integrates with the MLflow UI for inspection.
- ✗
Use the `mlflow.evaluate()` API with a custom evaluator that logs retrieved documents, prompt, and response as metrics.
Why it's wrong here
`mlflow.evaluate()` is designed for batch evaluation, not for per-request production tracing. It can log metrics and artifacts, but it does not create real-time traces for individual requests. Using it for production monitoring would be inefficient and would not provide the span-level detail needed. It is the wrong tool for this debugging scenario.
- ✗
Enable inference table logging on the serving endpoint and query the payload column for the retrieved documents and response.
Why it's wrong here
Inference tables capture request and response payloads but do not automatically include intermediate retrieval steps or the exact prompt sent to the model. They are useful for auditing but lack the span-level detail needed to see retrieved documents and the final prompt together. Relying solely on inference tables would leave gaps in the trace, making debugging harder.
- ✗
Configure the serving endpoint to log all requests to a Delta table and then join with a separate table of retrieved documents using a request ID.
Why it's wrong here
Joining separate tables by request ID is possible but fragile and does not create a unified trace. It requires custom correlation logic and may miss the exact prompt if not logged. This approach adds complexity and latency, and does not leverage MLflow's native tracing capabilities. It is not the recommended way to capture a cohesive trace.
About these practice questions
Courseiva writes every Databricks-GenAI-Assoc question from scratch — 330 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official Databricks exam blueprint
This Databricks-GenAI-Assoc practice question is part of Courseiva's free Databricks certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the Databricks-GenAI-Assoc exam.