Machine Learning and Deep Learning practice questions

Normal equation

Why it fails: Normal equation requires computing (X^T X)^{-1}, which is computationally expensive for large datasets.

Batch gradient descent

Why it fails: Batch gradient descent uses the whole dataset for each update, which is slow for large datasets.

Principal component analysis

Why it fails: PCA reduces dimensionality but does not train a model.

Increase the learning rate

Why it fails: Increasing the learning rate may cause the loss to oscillate or diverge.

Increase the number of epochs

Why it fails: More epochs would likely increase overfitting further.

Add more convolutional layers

Why it fails: Adding more layers increases complexity and would likely worsen overfitting.

Batch normalization is disabled

Why it fails: Disabling batch normalization may slow training but would not cause constant loss.

The dataset is imbalanced

Why it fails: Imbalanced data would cause the model to predict the majority class, not constant loss at 2.3026.

The model is overfitting

Why it fails: Overfitting would show decreasing training loss and increasing validation loss, not constant values.

100,224

Why it fails: 100,224 is incorrect. This value corresponds to (784*128) - 128, which does not represent a standard parameter count.

109,258

Why it fails: 109,258 is incorrect. It does not match any realistic calculation for this architecture.

8,256

Why it fails: 8,256 is incorrect. This value is far too small; it would require only about 10 neurons, not 128.

Remove one or more hidden layers from the network

Why it fails: Removing hidden layers reduces model capacity, which would worsen the gap by underfitting; the 14-point train-validation gap signals overfitting, addressed first by regularisation or more data. Reducing capacity is tempting for overfitting, but would be correct only when the network is far larger than the task requires.

Increase the number of training epochs

Why it fails: More epochs let the network memorise training data further, widening the 99% versus 85% gap; the issue is overfitting, so regularisation or early stopping is needed. Increasing epochs is tempting because it helps when both accuracies are low, which is the underfitting scenario.

Add more features to the input data

Why it fails: Adding features increases model capacity, giving the network more ways to memorise training data and worsening the 14-point gap; overfitting calls for regularisation or more samples. Feature addition is tempting when both accuracies are low, indicating the model lacks the information needed to learn the pattern.

Accuracy

Why it fails: Accuracy counts all correct predictions, so a model predicting non-churn for every customer scores 95% while catching zero churners. It is tempting because it is intuitive and high here, but on imbalanced data it masks minority-class failure, which is why recall or precision-recall AUC is required.

Log loss

Why it fails: Log loss measures the calibration of predicted probabilities across both classes, which does not directly surface poor minority-class detection. It is tempting because it penalises confident errors, but with 95% non-churn dominance it stays low even when churn recall is poor, so precision-recall or recall is needed.

AUC-ROC

Why it fails: AUC-ROC aggregates performance across all thresholds and stays high under 95/5 imbalance, masking the poor churn-class precision and recall the F1-score exposes. It is tempting because it suits balanced datasets or ranking tasks where threshold choice is deferred, but here the minority class drives the business cost.

Increase the reward for reaching the destination to +10

Why it fails: Increasing destination reward may still not overcome the reward for circling if the agent can accumulate many steps.

Use a discount factor closer to 0

Why it fails: A low discount factor makes the agent short-sighted, which could worsen the problem.

Add a penalty for each turn the vehicle makes

Why it fails: Penalizing turns may prevent necessary navigation and does not address the core issue.

Using a larger learning rate

Why it fails: Larger learning rates can cause divergence, not prevent overfitting.

L1 regularization

Why it fails: L1 regularization is used but not as common as dropout and early stopping for overfitting.

Increasing the number of layers

Why it fails: Adding layers often increases overfitting.

Underfitting

Why it fails: Underfitting produces high training loss that plateaus early, with both training and validation error remaining poor. The exhibit shows training loss continuing to fall, indicating the model fits the training data. Underfitting would apply if the model were too simple to capture the pattern.

Data leakage

Why it fails: Data leakage inflates validation accuracy above training accuracy, since test information contaminates training. The exhibit shows the opposite relationship, so leakage is not the cause. Leakage would be the answer when validation metrics suspiciously outperform training metrics.

Vanishing gradient

Why it fails: Vanishing gradients cause early-layer weights to stop updating, producing near-static loss rather than the oscillating or diverging pattern shown. It typically affects deep networks with saturating activations. The exhibit's behaviour indicates an optimisation or data problem instead.

Undersample the majority class to balance the dataset

Why it fails: Undersampling discards majority-class images, shrinking the 47,500 negatives and forfeiting valuable training signal without correcting the loss's bias. It is tempting as a standard imbalance remedy, but class weights in the loss penalise minority misclassification directly while retaining all data.

Oversample the minority class using synthetic image generation

Why it fails: Synthetic image generation risks producing unrealistic retinal features that a clinician would reject, and GAN training on 2,500 positives is unstable; class weighting in the loss directly penalises minority misclassification without fabricating pixels. Synthetic oversampling suits tabular or text minority classes where plausible samples can be interpolated reliably.

Replace the CNN with a transformer-based architecture

Why it fails: A transformer changes the feature-extraction architecture, not the loss weighting, so the 95:5 class imbalance still drives the sigmoid output toward the majority class and recall stays near 10%. Transformers suit large, balanced datasets where long-range spatial dependencies matter; here the bottleneck is the objective function, not model capacity.

Normalize numerical features to a standard range

Why it fails: Scaling numerical features changes their magnitude but leaves missing values, outliers and categorical text untouched, so the stated defects persist. It is tempting because normalisation aids distance-based and gradient algorithms, and would be correct once imputation, outlier handling and encoding are already addressed.

Remove all features with low variance

Why it fails: Dropping low-variance features discards potentially predictive columns and does nothing about missing values, outliers or categorical encoding. It is tempting because near-constant features add little signal, and removal is defensible during feature selection after the stated cleaning steps are complete.

Increase the number of features using PCA

Why it fails: PCA reduces dimensionality, not increases features.

Apply oversampling to the majority class

Why it fails: Oversampling majority class would increase bias toward majority, likely reducing precision.

Decrease the classification threshold

Why it fails: Lowering threshold increases recall but decreases precision.

The model is overfitting

Why it fails: Overfitting shows decreasing training loss with increasing validation loss; here training loss is increasing.

The model is underfitting

Why it fails: Underfitting would show high loss but not necessarily increasing loss.

The batch size is too large

Why it fails: Large batch size can affect convergence but typically does not cause such rapid divergence.

Decrease the classification threshold

Why it fails: Decreasing threshold increases recall but may also increase false positives; however, the question asks for most likely effective, and data collection is a fundamental approach.

Increase the classification threshold

Why it fails: Increasing threshold reduces recall.

Add more features

Why it fails: Adding features may help but not as directly as addressing class imbalance.

Increase the dropout rate to 0.7

Why it fails: Increasing dropout adds more regularization, which would likely decrease training accuracy.

Increase the number of filters in the first Conv2D layer

Why it fails: More filters increase capacity but may not directly improve training accuracy; could lead to overfitting.

Add another dense layer before the output

Why it fails: Adding layers increases complexity but may not significantly boost training accuracy if current capacity is sufficient.

Increase the number of trees in the forest

Why it fails: Adding trees reduces variance in the ensemble's averaged predictions but cannot close a train-test gap caused by the model memorising training data; each tree still overfits. It is tempting because more trees usually improve accuracy, and it would be correct when validation performance is unstable rather than systematically worse than training.

Use feature scaling

Why it fails: Random forests split on feature thresholds, so monotonic rescaling leaves splits and predictions unchanged; scaling cannot reduce the overfitting gap. It is tempting because scaling is standard preprocessing, and it would be correct for distance-based or gradient-descent models such as k-NN, SVM or neural networks.

Perform PCA to reduce dimensions

Why it fails: PCA may help but hyperparameter tuning via cross-validation is more direct.

It requires the base models to be of different types

Why it fails: Bagging typically uses the same type of base model (e.g., all decision trees).

It reduces the error by combining weak learners

Why it fails: That describes boosting; bagging combines strong or unstable models to reduce variance.

Remove dropout

Why it fails: Removing dropout removes a regularizer, likely worsening overfitting.

Increase the dropout rate

Why it fails: Dropout is already 0.5; increasing it may cause underfitting and loss of signal.

Increase the learning rate

Why it fails: A higher learning rate may cause training to diverge or not converge optimally.

The learning rate is too high

Why it fails: Learning rate affects convergence, not GPU utilization directly.

The image is missing CUDA libraries

Why it fails: Missing CUDA would cause failure or fallback to CPU, not 30% utilization on GPUs.

The number of epochs is too high

Why it fails: Number of epochs does not affect per-step GPU utilization.

target_latency: 100

Why it fails: Incorrect. Target latency controls response time, not explainability.

data_retention: "90 days"

Why it fails: Incorrect. Data retention manages storage duration, not model interpretability.

drift_detection: true

Why it fails: Incorrect. Drift detection monitors model performance over time, but does not provide per-prediction explanations.