1Z0-1127-25 LLM Fundamentals Practice Question
A team is evaluating two LLMs for a summarization task. Model X has a BERTScore of 0.85, Model Y has a BERTScore of 0.82. However, human evaluators prefer Model Y. Which TWO reasons could explain this discrepancy?
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
BERTScore is based on BERT embeddings, which may not fully capture summary-specific qualities like conciseness or readability
BERTScore correlates with human judgment but is not perfect; it may favor certain styles. Additionally, BERTScore may be inflated if the reference summaries are similar to the model's training data.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
BERTScore is based on BERT embeddings, which may not fully capture summary-specific qualities like conciseness or readability
Why this is correct
BERTScore measures semantic similarity but may not reflect human preferences for style.
- ✗
BERTScore uses precision only, so it misses recall aspects
Why it's wrong here
BERTScore combines precision and recall via F1.
- ✗
Human evaluators were not given clear criteria for evaluation
Why it's wrong here
Unclear criteria could cause variability, but the discrepancy is systematic.
- ✗
Model Y was fine-tuned on a different dataset, causing distribution shift
Why it's wrong here
Distribution shift would affect both metrics equally, not cause a discrepancy.
- ✓
Model X overfits to the reference summaries, achieving high BERTScore but poor general quality
Why this is correct
Overfitting can inflate automatic metrics while reducing human-perceived quality.
Go deeper
Related to this question
About these practice questions
Courseiva writes every 1Z0-1127-25 question from scratch — 768 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This 1Z0-1127-25 practice question is part of Courseiva's free Oracle certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the 1Z0-1127-25 exam.