NCP-GENL Flashcards — Free NVIDIA Certified Professional: Generative AI LLMs Study Cards

BERTScore computes token similarity matrices using contextual embeddings to capture deep semantic meaning regardless of surface-level phrasing variations.

BERTScore leverages contextual embeddings from transformer models to evaluate token-level semantic overlap rather than exact string matching, making it ideal for technical support text where phrasing varies. This automated metric correlates strongly with human judgment, significantly accelerating iteration cycles during enterprise model development workflows on NVIDIA infrastructure.

BF16 offers a larger dynamic range for gradients.

BF16 uses the same exponent range as FP32, which prevents overflow issues commonly encountered in deep learning training when using FP16. This increased dynamic range makes it more robust for gradient calculations and weight updates. By providing a wider range while maintaining the same performance advantages of half-precision, BF16 has become the industry standard for stabilizing training and inference of modern large language models.

The computational complexity of the self-attention mechanism is reduced from quadratic to linear.

Sliding window attention restricts the receptive field of each token to a local neighborhood, drastically reducing the quadratic memory complexity of standard attention to linear. This is critical for scaling LLMs to long contexts, as it prevents the O(n²) memory growth that typically causes GPU out-of-memory errors on large input sequences while maintaining local coherence.

Implementing document-aware recursive character splitting with overlapping segments and metadata tagging.

Chunking strategies and metadata tagging ensure that context retrieval is precise. By preserving technical hierarchy and associating data with specific product versions, the LLM retrieves ground-truth documentation rather than generic information. This reduces hallucinations by constraining the search space to relevant, version-controlled text blocks, directly impacting the accuracy and reliability of downstream inference tasks in enterprise NVIDIA-based AI deployments.

SM (Streaming Multiprocessor) occupancy

Monitoring GPU utilization alone is insufficient because it does not distinguish between compute saturation and memory bandwidth bottlenecks. NVML-based metrics specifically tracking SM (Streaming Multiprocessor) occupancy provide the granular insight required to identify whether the execution cores are fully utilized. This metric is essential for capacity planning and ensuring that the inference throughput meets strict service level agreements under high concurrent request loads in production environments.

The attention weight projection matrices

LoRA freezes the pre-trained model weights and injects trainable rank decomposition matrices into the transformer architecture layers. By targeting specifically the attention query, key, and value projection matrices, practitioners can achieve high performance with a fraction of the trainable parameters. This approach is critical for memory-constrained environments, allowing fine-tuning on consumer-grade NVIDIA GPUs while avoiding the massive memory requirements associated with full parameter updates.

Implement NeMo Guardrails 'dialogue rails' to detect and redirect queries related to medical diagnosis.

NeMo Guardrails provides a structured way to intercept model input and output to enforce safety boundaries. By defining specific 'rails' that detect non-compliant topics, the system can pivot the conversation or block the response entirely. This mechanism is critical in high-stakes environments where adherence to strict safety protocols is mandatory to prevent liability and ensure that generative AI tools do not cross into domains requiring human expertise.

It decomposes complex problems into verifiable logical segments.

Chain-of-Thought prompting decomposes complex problems into sequential logical steps, which is critical when using LLMs for technical reasoning tasks. By forcing the model to articulate its internal logic, the likelihood of hallucination decreases significantly. This practice is essential for NVIDIA engineers deploying agents that require high-precision output, as it creates an audit trail for the model's reasoning process and allows for better debugging of multi-turn interactions.

Enabling TensorRT-LLM custom kernel fusion for multi-head attention and activation blocks.

Kernel fusion combines multiple successive GPU operations, such as bias additions and activations, into a single CUDA kernel. This drastically reduces high-latency global memory read and write operations, directly mitigating the memory bandwidth bottleneck characteristic of autoregressive transformer decoding phases on NVIDIA hardware.

Enable Dynamic Batching in the Triton model configuration file.

Dynamic Batching is the optimal strategy for Triton Inference Server in production environments. It groups individual inference requests arriving within a short time window into a single batch, allowing the GPU to process them in parallel. This maximizes throughput by fully saturating CUDA cores, reducing the overhead of kernel launches, and ensuring that hardware utilization remains high even under variable traffic loads, effectively balancing latency and overall system capacity.