1Z0-1127-25 LLM Fundamentals Practice Question

What is the primary purpose of the self-attention mechanism in a Transformer model?

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

To compute a weighted sum of all token representations based on pairwise relevance

Self-attention allows each token to attend to every other token in the sequence, capturing contextual relationships regardless of distance.

Answer analysis

For each option: why learners choose it and why it is or isn't the right answer here.

✗
To generate token embeddings in parallel
Why it's wrong here
Parallel generation is a property of the Transformer architecture, but not specific to self-attention's purpose.
✗
To reduce the dimensionality of token embeddings
Why it's wrong here
Dimensionality reduction is not the role of self-attention.
✗
To encode positional information of tokens
Why it's wrong here
Positional encoding, not self-attention, provides positional information.
✓
To compute a weighted sum of all token representations based on pairwise relevance
Why this is correct
Self-attention computes attention scores between all pairs and aggregates information.

Go deeper