NCP-AIO Practice Test 2 — 30 Questions

Question 1

An administrator is deploying the NVIDIA GPU Operator on a Kubernetes cluster in an air-gapped environment. The cluster nodes have no internet access, and all container images must be pulled from a private registry. Which two actions are required to ensure a successful deployment? (Choose two.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Configure the GPU Operator to use the private registry by setting the image repository in the Helm chart values.. Air-gapped deployments require that all container images are available in a private registry, and the GPU Operator must be configured to pull from that registry. Mirroring the images ensures availability, and setting the image repository in the Helm values directs the Operator to the correct location. These two actions are essential for a successful deployment without internet access.

Answer

Ensure that the private registry supports manifest lists for multi-architecture images.

Answer

Disable the driver container and install drivers manually on each node.

Answer

Set the environment variable 'AIRGAP=true' in the operator's deployment.

Question 2

When configuring a node for NVIDIA AI Enterprise in a Kubernetes environment, what is the primary function of the NVIDIA Container Toolkit?

Accepted Answer

Exposing GPUs to containers. The NVIDIA Container Toolkit allows container engines, such as Docker or containerd, to interface with physical GPUs. It enables the exposure of GPUs inside containers, ensuring that applications can access CUDA libraries and hardware acceleration. Without this toolkit, the container runtime cannot bridge the gap between the host's GPU hardware and the containerized process, effectively rendering the GPU unusable for AI workloads.

Answer

Provisioning virtual machines

Answer

Optimizing neural network layers

Answer

Managing Kubernetes network policies

Question 3

A platform team runs an NVIDIA AI cluster with the GPU Operator deployed. Users submit jobs directly with kubectl and frequently request whole GPUs even when their notebooks only need a fraction of one. The team wants Kubernetes itself to admit and queue jobs based on GPU demand without users changing their manifests. Which component should the team deploy to meet this requirement?

Accepted Answer

NVIDIA KAI Scheduler. The requirement is for Kubernetes to make admission and queueing decisions based on GPU demand while users submit normal manifests. KAI Scheduler is the NVIDIA component that provides queue-based, gang-aware scheduling for AI workloads and understands fractional GPU requests, so jobs wait rather than fail. The GPU Operator, MIG Manager, and DCGM Exporter each address installation, partitioning, or observability rather than scheduling admission.

Answer

NVIDIA MIG Manager configured for every GPU

Answer

NVIDIA GPU Operator with the device plugin enabled

Answer

NVIDIA DCGM Exporter with Prometheus alerting

Question 4

Which strategy is most effective for managing heterogeneous GPU clusters containing both older architectures (e.g., V100) and newer architectures (e.g., H100)?

Accepted Answer

Using Node Selectors and Taints/Tolerations.. Using Kubernetes Node Selectors and Taints/Tolerations allows administrators to route specific workloads to compatible hardware. This is essential because newer GPUs support features like MIG or specific precision formats that older hardware lacks. By properly tagging nodes based on architecture, AI Ops prevents 'Incompatible Device' errors and ensures that high-performance training jobs are scheduled on hardware that meets their specific computational requirements, optimizing cluster-wide performance and efficiency.

Answer

Enabling uniform scheduling across all nodes.

Answer

Configuring the NVIDIA Device Plugin to hide older GPUs.

Answer

Deploying a single unified container image for all jobs.

Question 5

A platform team is preparing a bare-metal Kubernetes cluster to run AI workloads. They want the GPU Operator to install and manage the NVIDIA driver automatically on each node. Which prerequisite must be satisfied on every worker node before the GPU Operator can succeed?

Accepted Answer

The node must have a supported Linux kernel with matching kernel headers available for the driver container to build against.. The driver container builds the NVIDIA kernel module at runtime against the host kernel, so matching kernel headers or development packages must be present on each node. The GPU Operator manages the container toolkit, DCGM, and device plugin itself, so those do not need to be preinstalled. vGPU Manager applies to virtualized deployments, not bare-metal driver automation.

Answer

The node must have the NVIDIA Container Toolkit preinstalled and configured.

Answer

The node must be running the NVIDIA Data Center GPU Manager (DCGM) as a systemd service before joining the cluster.

Answer

The node must have the NVIDIA vGPU Manager installed so the physical GPU can be partitioned before driver installation.

Question 6

A DevOps engineer needs to monitor GPU health metrics in real-time for workload management. Which tool provides the most granular visibility into GPU utilization and power consumption for individual containers?

Accepted Answer

NVIDIA DCGM Exporter.. The NVIDIA DCGM (Data Center GPU Manager) Exporter is the industry-standard tool for collecting fine-grained GPU telemetry. By integrating with Prometheus and Grafana, it allows administrators to track metrics like power usage, temperature, and utilization at the container level. This granular data is vital for proactive workload management, allowing for autoscaling based on actual hardware demand rather than generic CPU metrics.

Answer

Kubernetes Horizontal Pod Autoscaler (HPA) with CPU metrics.

Answer

Standard Linux 'top' command.

Answer

Docker stats command.

Question 7

When configuring a multi-tenant environment on an NVIDIA DGX system, how are MIG (Multi-Instance GPU) instances best provisioned?

Accepted Answer

Using the nvidia-smi command to define the desired MIG profiles.. MIG allows a single GPU to be partitioned into multiple isolated instances, each with dedicated compute and memory resources. Configuring MIG via the nvidia-smi tool at the OS level ensures that these partitions are visible to the container runtime. This is crucial for AI operations, as it enables safe, secure multi-tenancy where different users can run jobs simultaneously without resource contention or cross-tenant interference.

Answer

Through the BIOS settings, creating physical partitions on the GPU die.

Answer

By editing the /etc/nvidia/mig-config.json file directly.

Answer

Via the OS kernel boot parameters in the GRUB configuration.

Question 8

A Kubernetes cluster running the NVIDIA GPU Operator is shared by an inference team and a research team. The research team's training pods repeatedly evict the inference pods from GPUs, causing latency spikes in production. The administrator wants to guarantee that inference pods always get GPU access first. Which Kubernetes scheduling mechanism should be configured?

Accepted Answer

Assign a higher PriorityClass to the inference pods and enable preemption on the scheduler.. Priority and preemption are the native Kubernetes controls that decide which pods win when GPU resources are contended. Giving inference pods a higher PriorityClass lets the scheduler admit them first and evict lower-priority training pods when necessary, which directly satisfies the requirement that production inference always obtains GPU access.

Answer

Enable the NVIDIA MIG feature on all GPUs and dedicate a MIG instance to each inference pod.

Answer

Configure a taint on the GPU nodes and add the corresponding toleration only to the inference pods.

Answer

Create a PodDisruptionBudget for the inference deployment and set maxUnavailable to zero.

Question 9

Refer to the exhibit. The system has two GPUs. What is the most likely cause of the observed performance discrepancy?

Accepted Answer

Only one GPU is receiving the workload due to improper data distribution.. The exhibit shows one GPU heavily utilized while the second is idling despite similar memory consumption. This indicates a data parallelism imbalance, where one process is doing the bulk of the work. This is a common issue in multi-GPU setups where workload distribution is not correctly handled, leading to massive inefficiencies where expensive hardware is under-utilized, significantly increasing the time required for model training or inference tasks.

Answer

The GPUs have different amounts of physical memory.

Answer

The system is limited by the PCIe bus bandwidth.

Answer

The model is too small to be parallelized across two GPUs.

Question 10

Refer to the exhibit. The training job is showing intermittent "thermal throttling" warnings. Which configuration change is the most appropriate adjustment?

Accepted Answer

Lower the "power_limit_watts" value.. Thermal throttling occurs when the GPU reaches its maximum operating temperature and lowers its clock speed to prevent physical damage. While power limits can be adjusted to reduce heat, simply lowering the limit may negatively impact training performance. Adjusting the power limit to a value that balances thermal overhead with compute requirements is a necessary trade-off to ensure stable, consistent performance during long training runs without hardware damage.

Answer

Set "compute_mode" to "exclusive_process".

Answer

Disable ECC mode to lower the power consumption.

Answer

Set "persistence" to "disabled".

Question 11

When deploying NVIDIA containers using the NVIDIA Container Toolkit, what is the primary function of the 'nvidia-container-runtime'?

Accepted Answer

It exposes the host's NVIDIA GPUs and driver libraries to the container environment.. The nvidia-container-runtime is a crucial component that allows Docker containers to interface with host GPUs. By modifying the container runtime specification, it ensures that the necessary device nodes and NVIDIA driver libraries are injected into the container namespace at startup. This enables seamless hardware acceleration for AI applications without requiring users to manually manage drivers or complex device path configurations inside their container images.

Answer

It automatically recompiles the application code for the specific GPU architecture found on the host.

Answer

It manages the lifecycle of the GPU driver installation on the host operating system.

Answer

It monitors the temperature and power consumption of the GPUs during container execution.

Question 12

An administrator is deploying NVIDIA AI Enterprise on a bare-metal cluster. Which component must be installed first to ensure proper communication between the Kubernetes scheduler and the underlying GPU hardware?

Accepted Answer

NVIDIA GPU Operator. The NVIDIA GPU Operator is essential for automating the management of all NVIDIA software components in Kubernetes. By installing it first, the administrator ensures that the device plugin, monitoring tools, and drivers are correctly configured. This foundation is critical for scheduling GPU-accelerated pods, as the Kubernetes scheduler requires the device plugin to advertise available GPU resources to the cluster's API server, enabling seamless workload orchestration across the infrastructure.

Answer

NVIDIA Triton Inference Server

Answer

NVIDIA NeMo Framework

Answer

NVIDIA Base Command Manager

Question 13

An operations engineer notices that an inference container on an A100 is intermittently returning stale predictions after a model update. The container mounts the model directory from a host path, and the update process replaces files in place. Which change most reliably prevents the stale predictions?

Accepted Answer

Use a versioned, immutable model artifact and restart or roll the inference container so it loads the new version atomically.. When model files are overwritten in place, a running inference process keeps using the weights it already loaded, so clients see stale results until the process restarts. Treating each model as an immutable, versioned artifact and performing a container restart or rolling deployment guarantees an atomic switch to a consistent set of weights. This removes the timing race between the update and the load, which is the actual source of the stale predictions.

Answer

Increase the container's shared memory size so the model can be cached entirely in RAM.

Answer

Reduce the inference batch size so each prediction reads the model from disk again.

Answer

Enable read-only mounts for the model directory to block writes from the container.

Question 14

Which component is strictly necessary for managing NVIDIA AI Enterprise licensing across a distributed cluster of nodes?

Accepted Answer

NVIDIA License System (NLS) Instance.. The NVIDIA License System (NLS) is the centralized authority for managing product entitlements. In a distributed environment, nodes must reach this server to validate their software keys. This component is crucial because it decouples the license management from the individual worker nodes, allowing for flexible scaling and centralized compliance reporting, which are essential in enterprise AI infrastructure where hardware resources are frequently provisioned and decommissioned.

Answer

NVIDIA CUDA Toolkit.

Answer

NVIDIA Container Toolkit.

Answer

NVIDIA Deep Learning GPU Training System (DIGITS).

Question 15

An administrator manages a Kubernetes cluster where a training job repeatedly fails with an OutOfMemory error on the GPU even though the pod requests one nvidia.com/gpu. DCGM metrics show that another pod on the same node is consuming GPU memory concurrently. GPU sharing via time-slicing is enabled cluster-wide. Which action should the administrator take to prevent this cross-pod interference while preserving the ability to share GPUs among trusted inference workloads?

Accepted Answer

Disable time-slicing on the node pool hosting training jobs and use MIG or dedicated GPUs for those workloads. Time-slicing multiplexes a GPU across pods without memory partitioning, so one workload can starve another and trigger OutOfMemory errors. The reliable fix is to separate untrusted or memory-intensive training workloads onto dedicated GPUs or MIG instances, while leaving time-slicing for inference workloads where memory contention is acceptable and controlled.

Answer

Apply a ResourceQuota limiting nvidia.com/gpu to one per namespace

Answer

Set the CUDA_VISIBLE_DEVICES environment variable manually in the training pod spec

Answer

Increase the pod's nvidia.com/gpu request to two GPUs

Question 16

A platform team runs an NVIDIA GPU Operator-managed Kubernetes cluster shared by two research groups. Group A's pods request `nvidia.com/gpu: 1` and are scheduled correctly, but Group B's pods that omit any GPU resource request are also landing on GPU nodes and consuming host memory and CPU, degrading Group A's jobs. The team wants Group B's non-GPU pods to stop consuming capacity on the GPU node pool without changing Group B's manifests. Which action should the administrator take?

Accepted Answer

Apply a taint such as `nvidia.com/gpu=present:NoSchedule` to the GPU nodes and add a matching toleration to Group A's pod templates.. Dedicating a node pool to GPU consumers is done at the scheduler layer: a taint on the GPU nodes repels every pod that does not carry a matching toleration. Because Group B's manifests cannot be changed, repelling rather than attracting is the only workable direction, and adding tolerations to Group A's templates is a one-time change under the platform team's control. Quotas, driver flags, and plugin options all operate after or outside scheduling and cannot keep non-GPU pods off those nodes.

Answer

Set `NVIDIA_DRIVER_CAPABILITIES=compute,utility` on the GPU Operator DaemonSet so non-GPU pods cannot attach to the device.

Answer

Create a `ResourceQuota` in each namespace that caps `requests.nvidia.com/gpu` at zero for Group B.

Answer

Enable the `nvidia` device plugin's `--fail-on-init-error=false` flag so unresolvable GPU requests fall back to CPU scheduling.

Question 17

A research lab is deploying NVIDIA AI Enterprise on an air-gapped Kubernetes cluster. The cluster has no internet access, and all software must be installed from a local registry. The administrator plans to use the NVIDIA GPU Operator. Which two actions must be performed to ensure a successful deployment in this environment? (Choose two.)

Accepted Answer

Configure the GPU Operator to use the private registry by setting the appropriate Helm chart values.. In an air-gapped environment, the GPU Operator cannot reach public registries to pull container images. Therefore, all necessary images must be mirrored to a private registry, and the GPU Operator must be configured to use that registry via Helm values. These two actions ensure that the Operator can deploy its components without internet access. Other options are either not required or invalid for this scenario.

Answer

Install the NVIDIA GPU Operator using the --offline flag.

Answer

Disable the GPU Operator's driver management and install the driver manually on each node.

Answer

Ensure the cluster nodes have direct access to the NVIDIA licensing server for vGPU.

Question 18

A research team is submitting many short-lived experiment jobs to an NVIDIA-accelerated Kubernetes cluster. The operations team wants to reduce GPU idle time and improve overall utilization without modifying the training code. Which TWO approaches should the operations team implement? (Choose two.)

Accepted Answer

Use a gang-scheduling or batch scheduler with backfill so queued jobs start as soon as GPUs free up. Short-lived experiment jobs create idle GPU gaps. Time-slicing allows multiple such pods to share a device, and a batch scheduler with backfill keeps the queue moving so GPUs are assigned as soon as capacity frees. Both measures raise utilization without touching training code, which is exactly what the operations team needs.

Answer

Pin each experiment job to a dedicated physical GPU using node affinity

Answer

Disable the NVIDIA device plugin and mount GPUs directly via hostPath

Answer

Increase the pod's nvidia.com/gpu request to reserve more GPU memory

Question 19

An AI operations team is using NVIDIA DCGM to monitor a cluster of GPUs. They want to set up alerts for when GPUs are running at high temperatures for extended periods. Which DCGM feature should they use?

Accepted Answer

DCGM health checks with custom thresholds and alerting via DCGM exporter to Prometheus.. DCGM health checks allow defining thresholds for GPU metrics, including temperature. When combined with the DCGM exporter and Prometheus, teams can create alerting rules that trigger on sustained high temperatures. This is the recommended method for cluster-wide monitoring. Other options either misuse DCGM diagnostics, assume non-existent features, or use less scalable methods.

Answer

DCGM's built-in email alerting system configured via dcgm.conf.

Answer

DCGM diagnostics with the '-r' flag to run a comprehensive test and report temperature violations.

Answer

NVIDIA-smi's '--query-gpu=temperature.gpu' with a cron job to check and send alerts.

Question 20

An administrator is deploying NVIDIA AI Enterprise on a Kubernetes cluster and wants to verify that the GPU Operator has successfully installed all required components. Which command should the administrator use to check the status of the GPU Operator pods?

Accepted Answer

kubectl get pods -n gpu-operator. The GPU Operator installs its components as pods in the gpu-operator namespace. Checking pod status with kubectl get pods -n gpu-operator is the correct way to verify that all required components are running. Other commands provide GPU-level or node-level information but do not show the status of the operator's pods.

Answer

nvidia-smi -q

Answer

kubectl get nodes -o wide

Answer

dcgmi discovery -l