Software Troubleshooting practice questions

A/B testing framework for comparing models

Why it fails: A/B testing compares live model variants after deployment, which is monitoring and experimentation, not a pipeline design factor governing build, test and release automation. It tempts because A/B testing is central to measuring model performance in production, so it would fit a question about evaluating deployed models.

Automated unit testing of application code

Why it fails: Unit testing application code validates software logic, not the ML-specific artefacts a CI/CD pipeline must gate on, such as data validation, model evaluation and retraining triggers. It tempts because unit tests are a core CI practise for conventional software, so they would be correct for a non-ML pipeline question.

To record the model's accuracy on benchmark datasets

Why it fails: An SBOM inventories components, dependencies and licences, not performance metrics, so it cannot capture benchmark accuracy. It is tempting because accuracy evidence matters for model assurance, and benchmark results would belong in a model card or evaluation report — the correct artefact when the requirement is documenting measured performance rather than supply-chain composition.

To document the model's training hyperparameters

Why it fails: An SBOM inventories components and dependencies, not training hyperparameters such as learning rate or batch size. Those belong in model cards or experiment-tracking tools, which suit reproducibility and audit of training runs. Here the requirement is component transparency, so hyperparameter documentation addresses a different artefact entirely.

To provide a user manual for the AI model

Why it fails: An SBOM enumerates components and dependencies, not usage instructions, so it cannot serve as a user manual. Documentation of operation is tempting because AI systems need guidance, but that belongs in model cards or user documentation, not the SBOM's component inventory.

Monitoring for anomalous inputs

Why it fails: Monitoring anomalous inputs detects attacks at inference time, after the model is already integrated, so it cannot verify the artefact's provenance or integrity beforehand. It is genuinely useful for runtime drift and adversarial-input detection, but the stem asks for a pre-integration supply chain control such as signature verification.

Generating a software bill of materials (SBOM)

Why it fails: An SBOM lists component dependencies but does not validate the model artefact itself, so tampering or poisoned weights pass unnoticed. It is valuable for ongoing vulnerability tracking and licence compliance once a model has already been verified and integrated.

Performing red teaming

Why it fails: Red teaming probes a deployed system for exploitable weaknesses through adversarial testing; it does not verify a downloaded model's provenance, integrity or embedded malicious payloads. It belongs after deployment, when assessing how the running AI system resists attack.

OWASP LLM Top 10

Why it fails: The OWASP LLM Top 10 catalogues classes of application-layer vulnerability such as prompt injection, not the model's internal components or third-party dependencies. It guides threat modelling and secure design. Dependency enumeration requires a software bill of materials.

Model card

Why it fails: A model card documents intended use, training data, evaluation metrics and limitations; it does not enumerate the libraries, packages or transitive dependencies bundled into the artefact. Model cards suit transparency and fitness-for-purpose reviews. Supply chain assessment needs a software bill of materials.

Data flow diagram

Why it fails: A data flow diagram maps how data moves between systems and processes, not the model's constituent libraries, weights or dependencies. It suits privacy or data-residency reviews. Supply chain assessment needs a software bill of materials listing embedded components and versions.

Underfitting; increase the number of time steps

Why it fails: Underfitting describes poor fit on training data generally, not the specific decay of gradients across long sequences. Adding time steps lengthens the unrolled graph, worsening vanishing gradients rather than restoring long-range learning. Underfitting is addressed by increasing model capacity or training longer.

Exploding gradients; apply gradient clipping

Why it fails: Exploding gradients produce large, oscillating updates, but the stem describes failure to retain information across many steps, which is vanishing gradients. Clipping caps update magnitude and is the remedy for instability during training, not for restoring long-range signal flow; gated architectures such as LSTM address that.

Overfitting; reduce the number of layers

Why it fails: Overfitting concerns generalisation to unseen data, not the vanishing gradients that prevent learning long-range dependencies. It is tempting because reducing layers is a common remedy for overfitting, and would be correct when training accuracy far exceeds validation accuracy.

Obfuscate the model file with a proprietary packer so its internal structure is harder to reverse engineer.

Why it fails: Obfuscation raises the effort to inspect the model but does not verify its origin. An attacker can still replace the entire obfuscated file with their own packed payload, and the application has no way to distinguish it. Obfuscation is not an integrity or authenticity mechanism, so it does not ensure the loaded model came from the vendor.

Enable full-disk encryption on the device so the model file cannot be modified while at rest.

Why it fails: Full-disk encryption protects data confidentiality when the device is powered off or stolen, but it does not prevent an attacker with runtime access from replacing a file. Encryption at rest also does not establish who authored the model. It leaves the authenticity question unanswered and cannot stop a trojaned model from being loaded by the application.

Compute the model file's SHA-256 hash at install time and store it in a local text file for later comparison.

Why it fails: A stored hash can detect changes only if the stored value itself is trustworthy. An attacker who can replace the model file can also replace the local hash file, defeating the comparison. Without a signature or a trusted external reference, the hash provides no assurance of vendor authenticity and does not prevent loading a trojaned model.

Encrypt all patient data

Why it fails: Encryption protects data at rest and in transit, but adversarial inputs are crafted to manipulate model inference, so ciphertext is decrypted before the model processes it. Encryption would be correct for securing stored patient records or network interception, not for detecting or hardening against malicious input perturbations.

Use stronger authentication

Why it fails: Stronger authentication verifies user identity at login and does nothing against crafted inputs submitted to a trained model. Authentication suits protecting access to systems and APIs, whereas adversarial misdiagnosis requires input validation, adversarial training or anomaly detection on inference data.

Regular software updates

Why it fails: Software updates patch known code vulnerabilities but do not alter a model's learned decision boundaries, so manipulated inputs still produce wrong diagnoses. Updates suit remediating exploitable software flaws; adversarial robustness instead needs adversarial training, input sanitisation or detection of out-of-distribution inference inputs.

Increase the number of training epochs to ensure convergence.

Why it fails: Epoch count affects convergence and accuracy, not determinism; more epochs change learned weights rather than pinning them, so identical inputs can still yield differing outputs. It is tempting because convergence feels related to stable predictions, and it would be correct if the question asked how to improve model accuracy or training fit.

Use the same GPU hardware for both training and inference.

Why it fails: Reproducibility requires fixed random seeds, pinned library versions and identical preprocessing, not identical GPU models; the same hardware does not prevent nondeterministic kernel execution or weight initialisation differences. It is tempting because hardware consistency sounds rigorous, and it would be correct if the question asked how to reduce latency variance between environments.

Use parallel data loading to speed up inference.

Why it fails: Parallel data loading changes throughput, not the numerical result; it speeds inference without fixing seeds, versions or preprocessing, so predictions remain non-reproducible. It is tempting because data pipelines feel foundational to consistency, and it would be correct if the question asked how to increase inference throughput or reduce batch latency.

Apply input validation and sanitization techniques

Why it fails: Input validation defends against injection, not supply chain backdoors.

Use homomorphic encryption for model weights

Why it fails: Homomorphic encryption protects data in use but does not detect backdoors.

Implement differential privacy during fine-tuning

Why it fails: Differential privacy protects training data privacy, not detect backdoors.