AI0-001 Machine Learning and Deep Learning Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✗

  Lower the learning rate and train for more epochs

  Why it's wrong here

  A smaller learning rate may stabilize optimization but does not change the architecture's information bottleneck. The recurrent state remains a fixed-size summary that decays over long sequences, so the model continues to lose early-context fidelity for the final words no matter how long it trains.

• ✓

  Replace the recurrent layers with an attention-based transformer encoder-decoder

  Why this is correct

  The described degradation at the end of long sequences is characteristic of vanishing gradients and limited memory in plain recurrent networks. Self-attention lets every output position attend directly to every input position in constant path length, so information from early and late tokens remains accessible. Transformers are the standard architecture for long-sequence translation and directly resolve this failure.

• ✗

  Increase the batch size used during training

  Why it's wrong here

  Batch size affects gradient estimate variance and hardware utilization, not the model's ability to retain information across many timesteps. The recurrent hidden state still must compress the entire sentence, so the positional degradation pattern persists regardless of how many sentences are processed per update.

• ✗

  Increase the number of recurrent layers stacked in the encoder

  Why it's wrong here

  Adding recurrent layers deepens the unrolled computation graph and can amplify vanishing gradients through the extended time dimension. It increases capacity and training cost without providing direct long-range connections, so accuracy on the final words of long sentences is unlikely to improve and may degrade further.