Hardware practice questions

GPU

Why it fails: GPUs are general-purpose parallel processors from NVIDIA or AMD, not designed by Google. Google built the Tensor Processing Unit (TPU) specifically for large neural network training and inference in its cloud. A GPU would be the correct accelerator choice when running workloads on non-Google platforms or needing broad framework compatibility.

NPU

Why it fails: NPUs are fixed-function accelerators found in edge devices and mobile SoCs, not Google's cloud training platform. Google's dedicated data-centre accelerator is the TPU. An NPU would be the correct choice for on-device inference in smartphones or laptops where power efficiency matters more than large-scale training throughput.

FPGA

Why it fails: FPGAs are reconfigurable logic devices offering flexibility for custom workloads, but Google did not design them for neural network acceleration. Google's purpose-built silicon is the TPU. An FPGA would be the right choice for low-latency custom signal processing or niche workloads where reprogrammable hardware outweighs fixed-function throughput.

Roll back to the previous model version and restrict use of the AI tool to only European patients.

Why it fails: Restricting use by nationality does not address the distribution shift from the new scanner manufacturer, and rolling back reinstates a model trained on the same narrow data. It appeals because rollback is a standard safe fallback, but here the trigger is covariate shift, not a faulty deployment.

Retrain the model using the original training data but with increased regularization to avoid overfitting.

Why it fails: Retraining on the original data with added regularisation cannot correct distribution shift from new scanners and demographics; the model needs representative recent data. It is tempting because regularisation addresses overfitting, and would be correct if the model were overfitting its original training set rather than facing shifted inputs.

Adjust the model's decision threshold to reduce false positives and then monitor for two weeks.

Why it fails: Threshold tuning only trades false positives against false negatives along one curve; it cannot correct the shifted feature distribution causing both rates to rise. It is tempting because threshold adjustment is a legitimate calibration step, but that applies when the operating point, not the input distribution, is the problem.

Deploying the model on the fastest hardware available

Why it fails: Hardware speed affects training and inference latency, not security posture. It is tempting because performance is a visible engineering concern and faster GPUs are often prioritised in AI projects. Secure development instead demands threat modelling, secure data pipelines, and model validation against poisoning and evasion attacks.

Using homomorphic encryption for all data at rest

Why it fails: Homomorphic encryption protects data during computation, not at rest; applying it universally is impractical and unrelated to SDLC essentials. It is tempting because it is a genuine privacy-enhancing technique for confidential inference. Secure AI development instead requires threat modelling, data provenance controls, and adversarial testing across the lifecycle.

GPU

Why it fails: GPUs are widely used for deep learning, but TPUs are custom-designed for tensor operations and often provide higher throughput for large models.

NPU

Why it fails: NPUs are optimized for inference on edge devices, not typically used for training large models.

CPU

Why it fails: CPUs are general-purpose processors not optimized for the parallel matrix operations required in deep learning training.

Use larger datasets to improve accuracy

Why it fails: Enlarging datasets increases training compute, storage and energy consumption, directly worsening the environmental footprint the scenario seeks to reduce. It is tempting because larger corpora typically raise accuracy and generalisation, which is the correct choice when the requirement is model quality rather than green AI.

Train models only on weekends

Why it fails: Scheduling training on weekends changes only when compute runs, not how much energy is consumed or its carbon intensity; total GPU hours and grid mix remain unchanged. It is tempting because time-shifting to off-peak periods can lower cost and occasionally align with cleaner generation, which is the correct tactic when the objective is demand response or tariff optimisation.

Train models in the cloud to offload energy costs

Why it fails: Cloud migration relocates energy consumption rather than reducing it, and provider regions vary in carbon intensity, so no emissions reduction is guaranteed. It is tempting because cloud providers offer efficiency at scale and renewable-energy purchasing, which is the right answer when the goal is cost or capacity elasticity rather than measured carbon reduction.

Increase batch size

Why it fails: Increasing batch size amortises weight loading across samples, improving throughput, yet a single mobile inference request still waits for the whole batch and per-request latency does not fall. It is tempting because batching speeds server-side processing, but that scenario involves high-volume offline or server inference, not one-at-a-time mobile prediction.

Add more layers

Why it fails: Adding layers increases parameter count and sequential computation, raising inference latency further on constrained mobile hardware. It is tempting because extra depth can improve accuracy on complex tasks, but that is a training-capacity concern, not a latency-reduction one; pruning or quantisation shrinks the existing network instead.

Use a larger model

Why it fails: A larger model multiplies weights and floating-point operations, so each forward pass takes longer and consumes more memory than the current network. It is tempting because bigger architectures raise accuracy on difficult datasets, but that is a modelling-quality goal; the stem demands lower inference time on limited mobile resources.

It automatically quantizes models to INT8

Why it fails: ONNX standardises the graph representation so a model trained in one framework runs in another runtime; it does not itself quantise weights. Quantisation is a separate conversion step, typically applied afterwards with tooling such as ONNX Runtime's quantiser. It would be the right answer if the question asked how to shrink a model to INT8 for faster CPU inference.

It compresses model size by 90%

Why it fails: ONNX defines an interoperable model format; it applies no fixed compression ratio. Size reduction depends entirely on the chosen precision, pruning or operator set, so a 90% figure is arbitrary. ONNX would be the correct choice when the requirement is moving a model between training and inference frameworks without retraining, not reducing storage footprint.

It reduces training time

Why it fails: ONNX describes a serialised inference graph, so it has no bearing on the training loop, gradient computation or epoch count. Training time is governed by data, hardware and hyperparameters. ONNX would be the right answer where a model trained in PyTorch must be executed by a different runtime at deployment, which is an inference concern.

Use a high-core-count CPU with large RAM

Why it fails: CPUs execute transformer matrix multiplications serially across few vector units, so training time barely improves regardless of core count or RAM. High-core CPUs suit data preprocessing, feature engineering or small classical models, not the massively parallel tensor operations a large transformer requires.

Use a single GPU with model parallelism

Why it fails: Model parallelism on a single GPU fails to reduce training time because it partitions the model across the device’s own memory, introducing sequential computation and communication overhead that does not scale throughput. It is tempting because it enables fitting a transformer that exceeds a single GPU’s VRAM, but the correct approach for speed is data parallelism across multiple GPUs, which processes distinct micro-batches concurrently.

Use a single TPU with model parallelism

Why it fails: A single TPU cannot hold a large transformer's parameters and optimiser states, so model parallelism across one device yields no throughput gain. TPUs excel when sharded across many chips via data and model parallelism together, which is the configuration this scenario actually demands.

Train on the largest possible dataset

Why it fails: Larger datasets increase training time and energy use.

Use redundant backup servers

Why it fails: Redundant servers increase energy consumption without contributing to efficiency.

Increase batch size to maximum

Why it fails: Larger batch sizes may improve throughput but can increase peak power draw.

Apply model distillation to create a smaller student model

Why it fails: Model distillation reduces model size but still requires a neural network inference pipeline, which on severely resource-constrained IoT devices may exceed available compute cycles and battery budget; the scenario demands a model that runs locally with minimal latency, and distillation alone does not guarantee the student model meets the strict memory or energy limits of the hardware. This technique is tempting because it is a standard approach for deploying deep learning on edge devices when the target platform has moderate resources, such as a smartphone or a Raspberry Pi, where a distilled model can achieve acceptable speed and accuracy trade-offs.

Deploy the model on a cloud server and stream data

Why it fails: Cloud deployment introduces network latency and requires constant connectivity, unsuitable for real-time patient monitoring on IoT devices.

Use ONNX Runtime with a GPU backend

Why it fails: ONNX Runtime with GPU backend is designed for server-class hardware, not for low-power IoT devices.

Increase the model's classification threshold so that more cases are flagged as positive.

Why it fails: Lowering the decision threshold, not raising it, would flag more cases positive; but adjusting the threshold treats a symptom without knowing the cause. If the new scanner's pixel intensities or noise profile shift the input distribution, threshold tuning may recover some sensitivity at the cost of specificity while leaving the underlying representation mismatch unaddressed. Diagnosis must precede any threshold change.

Fine-tune the model on the most recent production images that clinicians have labeled as missed positives.

Why it fails: Fine-tuning on missed positives alone creates a biased sample that overrepresents hard cases and can degrade performance on the broader population. It also skips diagnosis, so the team may be correcting the wrong thing, and it risks overfitting to a small, non-representative label set. Establishing why the failures occur must come before any retraining decision.

Replace the triage model with a larger architecture trained from scratch on a combined dataset of all available scanners.

Why it fails: Retraining from scratch is a costly, slow response that ignores diagnosis entirely. Without first confirming that scanner-induced distribution shift is the cause, the team could spend months rebuilding and reproduce the same failure. It also discards validated behavior on the original population and requires full revalidation, making it an inappropriate first step when a targeted investigation is available.

The edge devices have less RAM and slower CPUs than the cloud inference servers used during evaluation.

Why it fails: Hardware differences affect latency and throughput, not the mathematical result of a forward pass, as long as the same numeric format is used. Slower processors do not silently change class probabilities. Since the question describes a consistent accuracy difference rather than timeouts or dropped frames, resource constraints are not the explanation.

The test images were captured with a different camera model than the ones mounted on the conveyor belts.

Why it fails: A camera mismatch would also hurt accuracy, but it would affect cloud evaluation equally if the same test images were used in both environments, so it cannot explain a gap between cloud and edge results on identical inputs. Domain shift is a real concern, yet it does not produce an environment-specific delta; it produces a uniform drop everywhere the model runs.

The cloud evaluation used a batch size of 32 while the edge device processes one image at a time.

Why it fails: Batch size changes how inputs are grouped during inference but each sample's forward computation is independent, so predictions for a given image are essentially unchanged apart from floating-point summation order. It cannot account for a several-point accuracy drop. Batch size matters for throughput and memory, not for the classification quality of individual packages.

Increase the number of training epochs to ensure convergence.

Why it fails: Epoch count affects convergence and accuracy, not determinism; more epochs change learned weights rather than pinning them, so identical inputs can still yield differing outputs. It is tempting because convergence feels related to stable predictions, and it would be correct if the question asked how to improve model accuracy or training fit.

Use the same GPU hardware for both training and inference.

Why it fails: Reproducibility requires fixed random seeds, pinned library versions and identical preprocessing, not identical GPU models; the same hardware does not prevent nondeterministic kernel execution or weight initialisation differences. It is tempting because hardware consistency sounds rigorous, and it would be correct if the question asked how to reduce latency variance between environments.

Use parallel data loading to speed up inference.

Why it fails: Parallel data loading changes throughput, not the numerical result; it speeds inference without fixing seeds, versions or preprocessing, so predictions remain non-reproducible. It is tempting because data pipelines feel foundational to consistency, and it would be correct if the question asked how to increase inference throughput or reduce batch latency.

Deploy the native PyTorch model using TorchServe

Why it fails: TorchServe serves PyTorch's native format, so the artefact stays tied to the PyTorch runtime and cannot load in other frameworks. It is tempting because TorchServe gives low-latency PyTorch inference with minimal conversion effort, but the interoperability requirement demands a framework-neutral format such as ONNX.

Quantize the model to INT8 and deploy as a TensorFlow Lite model

Why it fails: Quantising to INT8 and converting to TensorFlow Lite changes the runtime away from PyTorch and targets mobile/edge delegates, not cross-framework server interoperability. It is tempting because INT8 cuts inference latency, but the requirement is framework-agnostic portability, which ONNX provides.

Convert the model to TensorFlow SavedModel and deploy using TensorFlow Serving

Why it fails: Converting to TensorFlow SavedModel ties the artefact to TensorFlow Serving, and PyTorch-to-TensorFlow conversion is lossy for many operators. It is tempting when the estate already runs TensorFlow Serving, but interoperability across frameworks and runtimes requires the ONNX open standard instead.

Central Processing Unit (CPU)

Why it fails: CPUs execute few threads with low parallelism, so training deep networks on large image datasets would be prohibitively slow. They are tempting because they handle general-purpose sequential logic and orchestration well, and would suit lightweight inference or preprocessing, but not the massively parallel matrix operations training demands.

Neural Processing Unit (NPU)

Why it fails: NPUs are specialized for inference on edge devices, not typically for training large models.

Tensor Processing Unit (TPU)

Why it fails: TPUs are Google-designed accelerators optimised for TensorFlow matrix operations, but the question asks for the widely used general-purpose AI training component. GPUs offer the parallel processing and broad framework support expected here. TPUs are tempting because they genuinely accelerate training, yet they are cloud-specific rather than the common choice.

Full fine-tuning with gradient checkpointing

Why it fails: Full fine-tuning with gradient checkpointing still updates all 70B parameters, so optimiser states and gradients alone exceed 24 GB VRAM. Gradient checkpointing only trades compute for activation memory, not parameter memory. It would suit scenarios where the whole model fits and activations dominate, but PEFT's low-rank adapters are required here.

Instruction tuning with a smaller 7B model

Why it fails: Instruction tuning with a 7B model changes the base model rather than the fine-tuning method, so it abandons the stated 70B target entirely. It is tempting because instruction tuning genuinely adapts a model to follow prompts, and a 7B model would fit 24 GB VRAM — but that solves a different scenario, not PEFT on 70B parameters.

LoRA (Low-Rank Adaptation) alone

Why it fails: LoRA alone still trains adapters across every layer, so a 70B model's optimiser states and activations exceed 24 GB. It is tempting because LoRA is the standard PEFT method, and it would be correct on a larger GPU or with aggressive quantisation such as QLoRA.