1Z0-1127-25 LLM Fundamentals Practice Question
A data scientist is evaluating two LLMs for a summarization task. Model X scores 45 on ROUGE-L, while Model Y scores 42. However, in human evaluation, Model Y is preferred 60% of the time. What is the most likely explanation?
⚠ Common exam trap
The 1Z0-1127 exam often tests the distinction between lexical metrics (like ROUGE) and semantic quality, trapping candidates who assume higher automated scores always indicate better performance without considering human preferences.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
ROUGE-L measures lexical overlap, which may not capture the semantic quality that humans value
ROUGE-L measures the longest common subsequence (LCS) between generated and reference summaries, focusing on lexical (word-level) overlap. It does not assess semantic meaning, fluency, or factual correctness. Human evaluators often prefer summaries that are coherent and capture key ideas, even if they use different wording, which explains why Model Y can score lower on ROUGE-L but be preferred 60% of the time.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Human evaluators are biased and cannot be trusted for objective assessment
Why it's wrong here
Human evaluation is subjective but often considered the gold standard for NLP tasks; bias is a concern but it does not explain the discrepancy here.
- ✗
Model Y overfits to the training data, causing poor generalisation
Why it's wrong here
Overfitting would likely lead to poor performance on both automatic and human metrics.
- ✓
ROUGE-L measures lexical overlap, which may not capture the semantic quality that humans value
Why this is correct
ROUGE relies on n-gram overlap; Model Y might produce more concise or coherent summaries that humans prefer but that share fewer exact n-grams with the reference.
- ✗
ROUGE-L is not a reliable metric for summarization because it only measures recall
Why it's wrong here
ROUGE-L measures F1 (precision and recall) of longest common subsequence; it is a standard automatic metric but may not align with human judgment.
Go deeper
Related to this question
About these practice questions
Courseiva writes every 1Z0-1127-25 question from scratch — 768 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This 1Z0-1127-25 practice question is part of Courseiva's free Oracle certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the 1Z0-1127-25 exam.