NCP-AIO Quick Quiz — 10 Questions

Question 1

Refer to the exhibit. An administrator notices that 'user_a' is consistently hitting resource limits despite having sufficient total system GPU memory. Based on the policy JSON, what is the cause?

Accepted Answer

The user is exceeding the 16GB VRAM limit.. The policy specifies a hard resource quota of '16GB' for the user. Even if the total system memory is higher, the scheduler enforces this limit per user. When the user's workload attempts to allocate more than 16GB of VRAM, the job will fail or throttle. This is a common method for preventing single users from starving the entire cluster of shared resources.

Answer

The system has reached the max_concurrent_jobs limit.

Answer

The NVIDIA driver is blocking the user's access.

Answer

The GPU memory is fragmented across the nodes.

Question 2

Refer to the exhibit. An administrator applies this security policy to a container runtime environment. What is the immediate effect on containerized AI applications within this scope?

Accepted Answer

The container is unable to detect or communicate with the NVIDIA GPU. The policy explicitly denies access to the character device files associated with the NVIDIA GPU. Without access to /dev/nvidia0, /dev/nvidiactl, and /dev/nvidia-uvm, the CUDA runtime cannot interact with the GPU hardware. Consequently, any attempt to initialize a CUDA device will fail, causing the application to crash. This policy is a common restrictive measure in high-security environments where GPU access must be strictly managed or audited.

Answer

The container gains elevated privileges to access the host GPU

Answer

The container can use the GPU but cannot perform memory mapping

Answer

The container experiences increased latency for GPU operations

Question 3

Which NVIDIA technology enables a GPU to be shared among multiple virtual machines or containers while maintaining strict hardware isolation?

Accepted Answer

NVIDIA Multi-Instance GPU (MIG). NVIDIA Multi-Instance GPU (MIG) is the core technology that enables hardware-level partitioning on supported GPUs. By creating dedicated instances that have their own memory and compute resources, MIG ensures that workloads remain isolated, providing predictable quality of service and security in multi-tenant environments. This is a foundational technology for maximizing GPU utilization in cloud and enterprise data center environments where mixed workloads are common.

Answer

NVIDIA CUDA streams

Answer

NVIDIA Collective Communications Library (NCCL)

Answer

NVIDIA GPUDirect RDMA

Question 4

An administrator is optimizing multi-GPU utilization. Which TWO of the following configurations allow multiple containers to share a single physical GPU on a supported NVIDIA architecture?

Accepted Answer

Enabling NVIDIA Time-Slicing in the GPU Operator configuration.. Multi-Instance GPU (MIG) and Time-Slicing are the primary methods for sharing physical GPU resources. MIG provides hardware-level isolation, while Time-Slicing provides software-based multiplexing. Understanding these options is essential for AI operations, as it allows administrators to maximize hardware ROI by supporting smaller workloads that do not require an entire A100 or H100 GPU, effectively increasing the density of the training or inference environment.

Answer

Increasing the CUDA_VISIBLE_DEVICES environment variable.

Answer

Setting a higher GPU limit in the Kubernetes manifest.

Answer

Deploying the NVIDIA Network Operator.

Question 5

An administrator is optimizing a cluster for AI model training using NVIDIA Base Command. Which TWO tasks are critical for ensuring consistent performance across the training nodes?

Accepted Answer

Standardizing NVIDIA driver and NCCL library versions across all nodes. Consistency in high-performance AI clusters depends on hardware synchronization and resource availability. Ensuring that all nodes run identical driver and firmware versions prevents subtle performance regressions during multi-node training. Furthermore, configuring GPUDirect RDMA is essential for reducing latency in inter-GPU communication, which is a major bottleneck in distributed training jobs. These steps are fundamental to maintaining high utilization and predictable training times across large-scale NVIDIA accelerated infrastructure.

Answer

Increasing the size of the swap partition on each node

Answer

Disabling the NVIDIA persistence mode to save power

Answer

Reducing the number of CUDA streams per training job

Question 6

An AI researcher is running a large-scale training job on an NVIDIA DGX system using Kubernetes. They observe that GPU utilization is consistently low despite high CPU load. Which workload management configuration is most likely to resolve this bottleneck by optimizing data pipeline throughput?

Accepted Answer

Optimize the data loader by increasing worker threads and implementing buffered prefetching.. Low GPU utilization often stems from data starvation where the GPU waits for the CPU to preprocess or fetch data from storage. Implementing data prefetching and increasing the number of workers in the data loader ensures the GPU remains fed with batches. This is critical in AI operations to maximize compute ROI and reduce total training time for expensive cluster resources.

Answer

Increase the GPU memory limit in the pod specification.

Answer

Enable NVIDIA Multi-Instance GPU (MIG) for this specific job.

Answer

Lower the batch size to reduce the memory footprint on the GPU.

Question 7

Which THREE components are required for a container to successfully leverage NVIDIA GPUs on a Kubernetes cluster?

Accepted Answer

NVIDIA Container Toolkit. Successful GPU integration in Kubernetes relies on the interaction between the container engine, the NVIDIA driver, and the device plugin. The driver provides the kernel-level interface, the container runtime handles the low-level injection, and the device plugin advertises the hardware to the scheduler. These three parts must be correctly configured and compatible to ensure that containers can schedule and utilize GPUs effectively without manual configuration or persistent connectivity issues.

Answer

NVIDIA Data Center GPU Manager (DCGM)

Answer

Pre-installed CUDA Toolkit inside the container

Question 8

A system administrator is troubleshooting a 'CUDA error: invalid device ordinal' when launching a job on a multi-GPU system. What is the most likely cause?

Accepted Answer

CUDA_VISIBLE_DEVICES is set to an out-of-range index.. The 'invalid device ordinal' error typically indicates that the application is attempting to access a GPU index (e.g., GPU 4) that does not exist or is not visible to the process. This is often caused by environment variables like CUDA_VISIBLE_DEVICES being configured incorrectly, mapping the application to non-existent hardware. Ensuring the logical-to-physical GPU mapping is accurate is vital for correct job execution on shared infrastructure.

Answer

The GPU driver is corrupted and requires a reinstall.

Answer

The GPU memory is full.

Answer

The InfiniBand fabric is misconfigured.

Question 9

During the deployment of an AI model training workload on a multi-node cluster, the administrator notices that inter-node communication is significantly slower than expected. Which deployment aspect should be investigated first?

Accepted Answer

Check the NCCL_DEBUG and network interface configuration for the training job.. NCCL (NVIDIA Collective Communications Library) is the primary engine for inter-node communication in distributed training. Misconfiguration of the network interface or the underlying fabric provider (e.g., InfiniBand or RoCE) will lead to significant performance bottlenecks. Investigating the NCCL configuration and the network topology ensures that the GPUs are utilizing the highest bandwidth available, which is vital for preventing training jobs from stalling during gradient synchronization across multiple nodes.

Answer

Verify that the GPU memory usage is below 50% on all nodes.

Answer

Restart the Kubernetes API server to refresh the node connection states.

Answer

Update the NVIDIA driver to the latest gaming-optimized release.

Question 10

A platform team is deploying the NVIDIA GPU Operator on a Kubernetes cluster to manage GPU nodes. They want the Operator to automatically install the NVIDIA driver, the container toolkit, and the device plugin on each GPU node. Which component of the GPU Operator is responsible for installing the NVIDIA driver on the host?

Accepted Answer

NVIDIA Driver Container. The GPU Operator automates the deployment of all NVIDIA software components on Kubernetes. The Driver Container is specifically designed to install the NVIDIA driver on the host. It runs as a DaemonSet and installs the driver into the host's filesystem, eliminating the need for manual driver installation. The other components handle container runtime integration, resource advertisement, and validation.

Answer

NVIDIA Container Toolkit

Answer

NVIDIA Device Plugin

Answer

NVIDIA GPU Operator Validator