A large enterprise is deploying a generative AI system for automated contract review. The system must provide confidence indicators for its legal analysis. How should confidence indicators be implemented to maximize transparency?
Numerical scores allow users to calibrate trust and make informed decisions.
Why this answer
Option B is correct because providing a numerical confidence score between 0 and 1 for each conclusion directly quantifies the model's certainty, enabling users to assess the reliability of each legal analysis. This approach maximizes transparency by allowing legal professionals to calibrate their trust in the AI's output, which is critical for high-stakes contract review where false positives or negatives carry significant risk.
Exam trap
Cisco often tests the misconception that binary outputs (pass/fail) are sufficient for transparency, but the trap here is that binary indicators hide the model's uncertainty, which is exactly what confidence scores are designed to reveal in responsible AI deployments.
How to eliminate wrong answers
Option A is wrong because showing the top-k most likely outcomes without probabilities omits the crucial uncertainty information; users cannot gauge how much more likely one outcome is over another, which undermines transparency in legal decision-making. Option C is wrong because a binary pass/fail indicator oversimplifies the model's output, hiding the nuanced confidence levels that are essential for evaluating ambiguous contract clauses or borderline legal interpretations. Option D is wrong because hiding confidence indicators entirely defeats the purpose of transparency, leaving users with no insight into the model's reliability and potentially leading to blind trust or unwarranted skepticism.