NCP-GENL Practice Test 17 — 15 Questions

Question 1

A team must serve a 70B model on a single 80 GB GPU for an internal assistant with modest concurrency. Full FP16 weights will not fit alongside the KV cache for the target context length. They want to keep accuracy loss minimal and are willing to spend additional build time. Which approach best fits these constraints?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use INT4 weight-only quantization with a group size and calibration set chosen for the model.. The binding constraint is fitting a 70B model plus KV cache on one 80 GB GPU while preserving accuracy. Weight-only INT4 quantization cuts weight memory roughly fourfold while keeping activations and cache at higher precision, and group-wise scales with a representative calibration set limit degradation. The team's willingness to accept longer build time matches the calibration and engine-build cost this method requires.

Answer

Split the model with tensor parallelism across two GPUs even though only one is available.

Answer

Apply FP8 quantization to both weights and activations and rebuild the engine.

Answer

Keep FP16 weights and reduce the KV cache by capping the context length far below the requirement.

Question 2

You are using NVIDIA NeMo Curator to filter a 600 GB web-crawl corpus before pre-training. Your team wants to remove exact duplicates and near-duplicates to reduce memorization and speed up training. Which NeMo Curator stage should you apply?

Accepted Answer

ExactDuplicates and FuzzyDuplicates stages. NeMo Curator's ExactDuplicates and FuzzyDuplicates stages are purpose-built for removing redundant documents. ExactDuplicates handles byte-level matches, while FuzzyDuplicates uses MinHash and LSH to catch near-identical texts. Together they reduce corpus size without discarding unique content, which directly lowers memorization risk and training cost on a large web-crawl dataset.

Answer

HeuristicFilter stage with a quality threshold

Answer

DocumentDownloader stage followed by DocumentResolver

Answer

ClassifierFilter stage using a domain classifier

Question 3

An ML engineer is fine-tuning a 13B LLM with LoRA on 4 NVIDIA A100 GPUs using NVIDIA NeMo. They notice that the effective batch size is very small and gradients are noisy, but increasing the per-GPU micro batch size triggers out-of-memory errors. Which technique should they apply to increase the effective batch size without increasing memory per step?

Accepted Answer

Enable gradient accumulation to sum gradients across multiple micro batches before the optimizer step.. Gradient accumulation performs multiple forward and backward passes on small micro batches and sums their gradients before one optimizer update, emulating a larger batch while keeping peak memory at the level of a single micro batch. It is the standard remedy when memory prevents enlarging the micro batch. Changing adapter rank, precision, or sequence length does not increase the number of samples per update in a memory-neutral way.

Answer

Switch from bfloat16 to float32 precision to stabilize gradient values.

Answer

Increase the LoRA rank and alpha so the adapter captures more information per step.

Answer

Reduce the sequence length by truncating all training examples to 128 tokens.

Question 4

You are preparing a large instruction-tuning dataset for an LLM using NVIDIA NeMo. You need to ensure the dataset supports efficient training and evaluation. Which TWO steps are essential in the data preparation pipeline? (Choose two.)

Accepted Answer

Convert the dataset into a format compatible with NeMo, such as JSONL with input and output fields. Formatting the dataset into a NeMo-compatible structure such as JSONL with input and output fields ensures the data loader can consume it, while splitting into training, validation, and test sets enables proper model evaluation and hyperparameter tuning. These two steps are foundational; other activities like tokenizer training or augmentation are optional and context-dependent.

Answer

Train a tokenizer from scratch on the dataset

Answer

Apply data augmentation using back-translation

Answer

Remove all punctuation from the text

Question 5

A team is preparing a supervised fine-tuning job in NVIDIA NeMo Framework for a customer-support assistant. They have a large corpus of raw support chat logs with no labels. They want the model to learn to answer customer questions in the company's tone and format. Which data preparation step is most appropriate before training?

Accepted Answer

Convert the raw chat logs into instruction-response pairs that reflect the desired tone and format, then use them for supervised fine-tuning.. Supervised fine-tuning learns behavior from explicit instruction-response pairs, so raw logs must be transformed into curated examples that demonstrate the target tone and format. This gives the model clear, aligned supervision and is the most direct way to shape how the assistant responds to customer questions.

Answer

Use the raw logs directly as a supervised fine-tuning dataset by treating each message as both instruction and response.

Answer

Apply reinforcement learning from human feedback using the raw logs as the reward model training data without any preference labels.

Answer

Run continued pretraining on the raw chat logs so the model absorbs the company's vocabulary and style without any labeling.

Question 6

A developer is preparing a dataset for instruction fine-tuning of an LLM using NVIDIA NeMo. The raw data consists of customer support transcripts with speaker labels and timestamps. Which preprocessing step is most important before training?

Accepted Answer

Convert each transcript into a structured instruction-response pair with a clear prompt and a target completion.. Instruction fine-tuning requires examples that pair a prompt with the desired response so the model learns to follow instructions. Raw support transcripts contain speaker labels and timestamps that are not part of the target behavior. Converting each transcript into a structured instruction-response pair makes the data compatible with the training objective and ensures the loss is computed on the correct tokens.

Answer

Duplicate each transcript so the model sees every example at least twice per epoch.

Answer

Remove all punctuation and capitalization so the model learns a consistent lowercase style.

Answer

Increase the learning rate to compensate for the noisy transcript data.

Question 7

A developer is using an NVIDIA NIM for a customer support chatbot. The chatbot must handle multi-turn conversations and maintain context about the user's issue. The developer notices that after several turns, the bot starts giving generic responses and forgets earlier details. Which prompt engineering approach is most effective to maintain context?

Accepted Answer

Include the entire conversation history in the prompt for each turn, ensuring the model has access to all previous messages.. Including the full conversation history in the prompt is the most direct way to maintain context across turns. The model can attend to all previous messages, ensuring it remembers details from earlier in the conversation. While summarization can reduce token usage, it risks losing critical information. Temperature and system prompts do not address context retention.

Answer

Use a summarization prompt at each turn to condense the conversation so far, and include only the summary in the next prompt.

Answer

Add a system prompt that instructs the model to always ask the user to repeat their issue if it is unsure.

Answer

Use a higher temperature setting to make the model more creative in interpreting the user's issue.

Question 8

A team is evaluating a large language model for a question-answering system using NVIDIA NeMo Evaluation. They need to assess both the relevance of the answer to the question and its factual correctness. (Choose two.)

Accepted Answer

Factual consistency score using an NLI model. The team must evaluate relevance and factual correctness. Answer relevance score directly measures how well the answer addresses the question, while factual consistency score via NLI checks whether the answer is factually supported. Exact Match, token F1, and BLEU focus on string overlap and do not separately assess relevance and factual correctness, making them less suitable for this dual requirement.

Answer

F1 score over tokens

Answer

BLEU score

Answer

Exact Match (EM)

Question 9

An LLM inference service on NVIDIA Triton Inference Server uses dynamic batching with a max_queue_delay of 500 microseconds. During a load test, p99 latency exceeds the SLA while GPU utilization remains below 40%. Which change should you make first to improve latency without reducing throughput?

Accepted Answer

Reduce max_queue_delay to a smaller value to allow quicker dispatch of requests.. The high p99 latency with low GPU utilization indicates that requests are spending too long in the dynamic batching queue. Reducing max_queue_delay allows requests to be dispatched sooner, cutting tail latency. Since GPU utilization is low, throughput is not limited by batch size, so this change improves latency without harming throughput.

Answer

Increase the max_batch_size in the model configuration to allow larger batches.

Answer

Enable model warmup to reduce first-inference latency.

Answer

Switch to a larger GPU with more memory to increase batch capacity.

Question 10

A company wants to fine-tune a 70B LLM to follow domain-specific instructions. The base model already performs well on general language tasks. They have limited labeled data and limited GPU memory. Which approach is most appropriate?

Accepted Answer

Parameter-Efficient Fine-Tuning with LoRA adapters targeting attention projection layers.. LoRA is the standard parameter-efficient method for adapting very large models under memory constraints, since it trains only small adapter matrices while freezing base weights. With limited labeled data, the reduced trainable parameter count also lowers overfitting risk and helps retain general capabilities. Full fine-tuning, tokenizer retraining, and prompt-only approaches either exceed resource limits or fail to deliver durable domain adaptation.

Answer

Retrain the tokenizer on the domain corpus and continue pretraining from scratch.

Answer

Full-parameter fine-tuning of all 70B weights on the domain dataset.

Answer

Prompt engineering with few-shot examples and no weight updates.

Question 11

You are preparing a dataset for continued pre-training of an LLM on internal engineering documents. The corpus contains many documents with boilerplate headers, footers, and legal disclaimers repeated across files. Which NeMo Curator approach best reduces this boilerplate while preserving unique technical content?

Accepted Answer

Use the n-gram or sentence-level deduplication stage to remove repeated spans. Span-level deduplication in NeMo Curator detects repeated n-grams or sentences across documents and removes them, which precisely targets boilerplate headers, footers, and legal disclaimers. Unlike document-level filtering, it preserves the unique technical content within each file, making it the best choice for a corpus where redundancy is localized rather than whole-document.

Answer

Apply a heuristic filter that drops documents below a word-count threshold

Answer

Lowercase all text and remove punctuation

Answer

Split documents into fixed-size chunks and keep only the first chunk

Question 12

An enterprise is fine-tuning a 34B model with NVIDIA NeMo Framework and observes that the validation loss begins rising after the first epoch while training loss continues to fall. The team wants to reduce this divergence and preserve downstream task quality. (Choose two.)

Accepted Answer

Apply weight decay or increase its coefficient to penalize large parameter updates and discourage overfitting.. Rising validation loss with falling training loss is classic overfitting. Early stopping on validation loss halts training before further damage, while stronger weight decay regularizes the model and narrows the generalization gap. Together they reduce divergence and protect downstream task quality without discarding useful training signal.

Answer

Remove the validation split and train on all available data so the model sees more examples per epoch.

Answer

Raise the learning rate significantly so the model converges faster and reaches a flatter minimum.

Answer

Increase the number of training epochs so the optimizer has more time to escape the overfitting region.

Question 13

You are preparing a 2 TB corpus of English and German web text for continued pretraining of a NeMo-based LLM. The German portion includes many pages with unescaped HTML entities and mixed-language sentences. Which NeMo Curator stage should you apply to remove boilerplate, fix HTML artifacts, and filter low-quality documents before tokenization?

Accepted Answer

NeMo Curator's text cleaning and heuristic filtering pipeline, including HTML unescaping, language identification, and quality classifier stages.. NeMo Curator is the correct tool because it supplies production-grade stages for HTML unescaping, language identification, and quality filtering that operate on large-scale text before tokenization. These stages directly address the German/English mixed-language and malformed HTML issues. The other options are training, runtime safety, or serving components, none of which clean a pretraining corpus.

Answer

NeMo's Megatron-LM pretraining script with an increased dropout rate on the embedding layer.

Answer

NeMo Guardrails with a custom Colang flow that blocks documents containing HTML tags.

Answer

NVIDIA Triton Inference Server with a Python backend that preprocesses each document at inference time.

Question 14

A research team is evaluating a large language model's ability to follow instructions. They have a dataset of prompts with corresponding reference outputs. They want to use an automated metric that correlates well with human judgments of instruction-following quality. Which evaluation method is most suitable?

Accepted Answer

GPT-4-based evaluation with a detailed rubric. LLM-based evaluation with a detailed rubric, such as using GPT-4 as a judge, has been demonstrated to align closely with human judgments for instruction-following. It can assess adherence to constraints, format, and correctness in a way that n-gram metrics like BLEU and ROUGE-L cannot. Perplexity measures fluency, not instruction adherence. Therefore, the LLM-as-judge approach is the most suitable.

Answer

BLEU score against reference outputs

Answer

Perplexity of the model on the reference outputs

Answer

ROUGE-L score against reference outputs

Question 15

You are preparing a dataset of customer reviews for fine-tuning an LLM to generate concise summaries. The reviews are in multiple languages, but the target summaries must be in English. You have a limited budget for translation. Which data preparation step is most critical to ensure the fine-tuned model produces high-quality English summaries?

Accepted Answer

Ensure that each training example pairs a review in its original language with a high-quality English summary.. Pairing original-language reviews with English summaries directly trains the model for cross-lingual summarization, which is the end goal. This avoids translation errors and preserves the original semantics. It also prepares the model for real-world multilingual inputs, ensuring it can generate English summaries without an intermediate translation step.

Answer

Translate all reviews into English and then train the model to summarize English text.

Answer

Use a multilingual LLM to translate all non-English reviews into English before training.

Answer

Filter the dataset to include only reviews originally written in English.