NCA-GENL Core Machine Learning and AI Knowledge Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✓

  Masked language modeling

  Why this is correct

  Masked language modeling randomly replaces a fraction of input tokens with a mask and trains the model to recover them using the full surrounding context on both sides. Because no causal mask is applied, every prediction can attend to tokens to the left and right, which is exactly the bidirectional pretraining signal the team is asking for.

• ✗

  Contrastive next-sentence prediction

  Why it's wrong here

  Contrastive next-sentence prediction trains the model to distinguish whether one sentence follows another, producing sentence-level representations rather than token-level bidirectional understanding. It was a secondary objective in early bidirectional encoders and does not by itself teach each token to attend to both neighbors across the corpus.

• ✗

  Sequence-to-sequence denoising with a causal decoder

  Why it's wrong here

  A causal decoder in a sequence-to-sequence setup still applies a left-to-right attention mask during generation, so the decoder cannot see future tokens. While the encoder may be bidirectional, the team's stated goal of bidirectional context for every token during pretraining is not achieved by this objective alone.

• ✗

  Autoregressive next-token prediction

  Why it's wrong here

  Autoregressive next-token prediction uses a causal mask that prevents each position from attending to future tokens, so the model only sees left context. This is the objective used by decoder-only models such as GPT, and it cannot provide the bidirectional context the team requires during pretraining.