NCA-GENL Core Machine Learning and AI Knowledge Practice Question
Which activation function is most commonly used in the hidden layers of deep neural networks to mitigate the vanishing gradient problem?
⚠ Common exam trap
Candidates confuse hidden-layer activation requirements with output-layer needs, incorrectly selecting sigmoid or softmax for deep hidden layers.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
ReLU
The ReLU (Rectified Linear Unit) activation function is the industry standard for hidden layers because it avoids the saturation characteristic of sigmoid or tanh functions. By outputting zero for negative inputs and linear values for positive inputs, it maintains a gradient of one during backpropagation for active neurons. This allows gradients to flow through deep networks without shrinking exponentially, which is essential for training modern, deep architectures effectively.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Sigmoid
Why it's wrong here
The sigmoid function saturates at both ends, squashing inputs into a range between 0 and 1. The derivative of sigmoid is very small, leading to the vanishing gradient problem in deep networks, where weights in earlier layers stop updating. This makes it unsuitable for deep hidden layers in modern deep learning architectures.
- ✓
ReLU
Why this is correct
ReLU provides a constant gradient of 1 for all positive inputs, preventing the gradient from diminishing during backpropagation. This simple, non-saturating nature makes it highly effective for training very deep networks. It is computationally efficient and has been a cornerstone of deep learning success across various computer vision and language tasks.
- ✗
Softmax
Why it's wrong here
Softmax is typically used in the final layer of a classification network to convert raw logits into probability distributions. It is not designed to solve the vanishing gradient problem in hidden layers. Using it in hidden layers would create computational complexity and likely lead to poor training dynamics due to normalization dependencies.
- ✗
Hyperbolic Tangent (tanh)
Why it's wrong here
The tanh function is a scaled version of sigmoid, ranging from -1 to 1. While it is zero-centered, which can be better than sigmoid, it still saturates for large positive or negative inputs. Like sigmoid, it causes vanishing gradients in very deep networks, making it less favorable than ReLU for hidden layers.
About these practice questions
This NCA-GENL question is part of Courseiva's 367-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official NVIDIA exam blueprint
This NCA-GENL practice question is part of Courseiva's free NVIDIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the NCA-GENL exam.