NCA-GENL Practice Test 15 — 15 Questions

Question 1

You are analyzing embedding quality for a retrieval-augmented generation system. You have 1,000 document embeddings of 4,096 dimensions and want to inspect whether semantically similar documents form visible clusters. Which dimensionality-reduction approach is most appropriate before plotting in two dimensions?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Uniform Manifold Approximation and Projection (UMAP). Embeddings are high-dimensional and their semantic structure is typically nonlinear, so a nonlinear manifold method is preferred for two-dimensional inspection. UMAP preserves local neighborhoods while retaining more global organization than t-SNE and scales efficiently, making cluster formation visible. Linear PCA and arbitrary dimension truncation either miss nonlinear structure or discard most information, while a correlation matrix analyzes features rather than documents.

Answer

Compute a correlation matrix and reorder it hierarchically

Answer

Principal Component Analysis (PCA) to two components

Answer

Reduce dimensions by selecting the first 100 raw embedding dimensions

Question 2

A team is evaluating a retrieval-augmented generation pipeline. They have a dataset of 500 queries, each with a retrieved context and a generated answer. The goal is to visualize how often the generated answer is faithful to the retrieved context versus hallucinated, and to compare this across three different retriever configurations. Which visualization best supports this comparison?

Accepted Answer

A grouped bar chart showing the proportion of faithful, partially faithful, and hallucinated answers for each retriever configuration.. The task requires comparing categorical faithfulness outcomes across three retriever configurations. A grouped bar chart places the proportions of each faithfulness category side by side for each configuration, making differences immediately visible. The other options either collapse the comparison, use continuous variables that do not represent faithfulness, or focus on a single example.

Answer

A pie chart showing the overall proportion of faithful, partially faithful, and hallucinated answers across all retrievers combined.

Answer

A heatmap of token-level attention weights between the generated answer and the retrieved context for a single query.

Answer

A scatter plot of answer length versus retrieval score, colored by retriever configuration.

Question 3

An engineer needs to track validation loss, learning rate, and GPU utilization together over training steps for a fine-tuning run, and wants the ability to compare multiple runs side by side in a web dashboard. Which approach best meets this need?

Accepted Answer

Log metrics with a framework-integrated experiment tracker that provides a web UI for run comparison.. Tracking several metrics across steps and comparing runs requires structured logging plus a visualization layer. Framework-integrated experiment trackers supply both: they capture scalars at each step and render interactive dashboards where runs can be overlaid, filtered, and compared. Manual files, console output, and one-off snapshots lack the persistence, structure, and comparison features the task demands.

Answer

Print metrics to stdout and rely on the terminal scrollback to review trends.

Answer

Write metrics to a CSV file and open it in a spreadsheet after training completes.

Answer

Capture a single screenshot of nvidia-smi output at the end of training.

Question 4

A data scientist has embedded 200,000 LLM training documents with a sentence-transformer and wants to visualize the embedding space to inspect semantic clusters. Running UMAP on the full set is too slow, so they first reduce dimensions with PCA. Which approach best preserves local cluster structure for the final visualization?

Accepted Answer

Apply UMAP directly to the first 50 principal components of the embeddings. PCA is a fast linear preprocessing step that removes noise and reduces dimensionality, and UMAP operates best on a moderate number of informative components rather than raw high-dimensional vectors. Retaining 50 components keeps semantic variance while cutting computation, so local cluster structure survives into the final nonlinear embedding.

Answer

Apply t-SNE directly to the first 2 principal components of the embeddings

Answer

Apply PCA to reduce to 2 dimensions and plot the documents directly

Answer

Apply UMAP directly to the raw 768-dimensional embeddings without PCA

Question 5

You have generated 512-dimensional embeddings for 200,000 documents using an NVIDIA NeMo embedding model and want to inspect whether semantically similar documents cluster together. Which technique should you apply first to project these embeddings into two dimensions for visual inspection?

Accepted Answer

UMAP with a small n_neighbors value to emphasize local structure.. Nonlinear manifold learning such as UMAP is designed to project high-dimensional embeddings into two or three dimensions while preserving neighborhood relationships. With a small n_neighbors setting, UMAP emphasizes local structure, making it well suited to checking whether semantically similar documents form coherent clusters. Linear methods like PCA and scalar summaries like vector norms cannot expose that local neighborhood structure.

Answer

Principal Component Analysis projecting onto the top two components.

Answer

A bar chart of the L2 norm of each embedding vector.

Answer

A Pearson correlation heatmap of all 200,000 document pairs.

Question 6

A data scientist is analyzing 2 million document embeddings from a RAG corpus on a single NVIDIA GPU. A full pairwise cosine similarity matrix would require roughly 16 TB of memory, which is infeasible. They need to identify near-duplicate documents and visualize cluster density without materializing the full matrix. Which approach is most appropriate?

Accepted Answer

Use FAISS with an IVF-PQ index to perform approximate nearest-neighbor search, then plot a 2D projection colored by local density.. At 2 million vectors, the full similarity matrix is memory-infeasible, so an approximate method is required. FAISS IVF-PQ combines inverted-file partitioning with product quantization to search a compressed index on a single GPU, returning near-duplicates efficiently. Plotting a 2D projection colored by local neighbor density then exposes cluster structure and duplicate hotspots. Exact full-matrix computation, PCA-to-2D distances, and norm sorting all fail on memory or accuracy grounds.

Answer

Sort embeddings by their L2 norm and treat documents with similar norms as duplicates.

Answer

Compute the full 2M x 2M cosine similarity matrix in float32 on the GPU to preserve exact distances.

Answer

Reduce dimensionality to 2D with PCA first, then compute exact pairwise distances in the 2D space to find duplicates.

Question 7

You are building a dashboard to monitor an LLM inference service deployed with NVIDIA Triton Inference Server. Stakeholders want to detect quality degradation and latency regressions before users complain. Which two metrics should be tracked continuously to surface these issues earliest? (Choose two.)

Accepted Answer

Distribution drift of output token log-probabilities or embedding distances against a reference set.. Early detection of LLM service problems requires pairing latency telemetry with output quality telemetry. Time to first token and inter-token latency percentiles expose responsiveness regressions as they emerge, while drift in output distributions or embedding distances catches silent quality decay. Static counters and hardware-level signals do not provide that early, actionable warning.

Answer

Cumulative count of HTTP 200 responses since service start.

Answer

Total number of model weight parameters loaded into GPU memory.

Answer

GPU clock frequency of the host at one-minute sampling intervals.

Question 8

A data scientist has a table of 500 LLM evaluation runs, each with a numerical faithfulness score from 0 to 1 and a categorical model version label. They want a compact view comparing the score distributions across model versions, including medians and spread, in a single figure. Which visualization should they choose?

Accepted Answer

A grouped box plot of faithfulness score by model version.. When comparing a continuous score across categorical groups, box plots are the standard compact choice because each box summarizes median, interquartile range, and outliers per group. They allow immediate side-by-side comparison of both center and spread across model versions, which is precisely what the data scientist needs in a single figure.

Answer

A network graph connecting runs that share the same model version.

Answer

A heatmap of faithfulness score binned by run index and model version.

Answer

A single scatter plot of faithfulness score against run index.

Question 9

A machine learning engineer is monitoring an LLM inference service deployed on NVIDIA GPUs. They want a real-time dashboard that shows GPU utilization, memory usage, and request latency, and they need to set alerts when thresholds are exceeded. Which NVIDIA tool is purpose-built for this monitoring and alerting?

Accepted Answer

NVIDIA Data Center GPU Manager (DCGM) with DCGM-Exporter and Prometheus/Grafana.. Real-time GPU monitoring with dashboards and alerts is the core purpose of NVIDIA DCGM combined with DCGM-Exporter and a metrics stack such as Prometheus and Grafana. The other tools address profiling, inference optimization, or model serving, none of which provide the required operational monitoring and alerting.

Answer

NVIDIA TensorRT.

Answer

NVIDIA Triton Inference Server.

Answer

NVIDIA Nsight Systems.

Question 10

A team is analyzing an LLM evaluation dataset with thousands of prompts and multiple scoring dimensions such as correctness, fluency, and safety. They want a single visualization that reveals how these dimensions correlate and whether any prompts score unusually on several dimensions at once. Which visualization is most suitable?

Accepted Answer

A parallel coordinates plot of the scoring dimensions across prompts.. Parallel coordinates are designed for multivariate data: each dimension gets an axis and each prompt becomes a polyline across them. This exposes correlations between correctness, fluency, and safety and makes multi-dimension outliers visually obvious. Charts that collapse dimensions into averages or single sequences cannot reveal those relationships.

Answer

A stacked bar chart of total score by prompt category.

Answer

A histogram of the overall average score.

Answer

A single line chart of correctness scores ordered by prompt index.

Question 11

A developer is profiling an LLM inference endpoint on an NVIDIA L40S and observes that time-to-first-token (TTFT) is stable but inter-token latency spikes periodically. They want to determine whether the spikes align with KV cache growth or with batch-size changes. Which visualization strategy best isolates the cause?

Accepted Answer

Plot a time series of inter-token latency with KV cache size and active batch size overlaid on the same time axis.. The goal is to determine whether periodic latency spikes align temporally with KV cache growth or batch-size changes. Only a shared time axis can establish that alignment. Plotting inter-token latency, KV cache size, and active batch size together makes coincident events visible immediately. Summary statistics, length histograms, and single-request flame graphs all discard the cross-request temporal relationship required to isolate the cause.

Answer

Generate a histogram of token lengths for all requests served during the profiling window.

Answer

Compute the mean and standard deviation of inter-token latency and report them as a bar chart.

Answer

Render a flame graph of GPU kernel execution for a single representative request.

Question 12

A team is comparing two LLM checkpoints on a summarization benchmark. They want a single visualization that shows, for each evaluation metric, both the mean score and the spread across the benchmark's document categories, while making it easy to see whether the two checkpoints overlap. Which visualization best fits this requirement?

Accepted Answer

A grouped box plot with one box per checkpoint per metric, grouped by metric.. Comparing two checkpoints across multiple metrics while preserving spread calls for a distribution-aware chart. Grouped box plots keep each metric on its own scale, show central tendency and variability, and place the two checkpoints adjacent so overlap is immediately visible. Charts that reduce to means or sums discard the variance the team needs to judge whether differences are meaningful.

Answer

A stacked bar chart of total scores per checkpoint across all metrics.

Answer

A heatmap of mean scores with checkpoints as rows and metrics as columns.

Answer

A single line chart with one line per checkpoint plotting mean score against metric name.

Question 13

A team is evaluating an LLM-based summarization service and wants a visualization that shows how the distribution of generated summary lengths compares to the reference summaries across 5,000 test articles. They want to see whether the model systematically produces shorter or longer outputs. Which visualization is best suited?

Accepted Answer

A grouped histogram or overlaid density plot of generated versus reference summary lengths.. The question is distributional: do generated summaries differ systematically in length from references across the corpus? Overlaying the two length histograms or density curves on a shared axis makes any shift in center or spread obvious at a glance. Scatter plots of quality metrics, per-epoch training lines, and single-example attention heatmaps all address different questions and cannot reveal a corpus-level length bias.

Answer

A line chart of average summary length per training epoch.

Answer

A scatter plot of BLEU score versus article length.

Answer

A heatmap of token-level attention weights for a single example.

Question 14

A data scientist is preparing a visualization to compare the performance of three different LLM fine-tuning runs on a summarization benchmark. They want to show both the central tendency and the variability of ROUGE-L scores across multiple evaluation samples. Which two visualizations are most appropriate for this goal? (Choose two.)

Accepted Answer

A box plot of ROUGE-L scores for each fine-tuning run.. To compare central tendency and variability of ROUGE-L scores across three runs, distributions must be shown per run. Box plots and violin plots both display median, quartiles, and spread, with violin plots adding density shape. The other options either collapse the score distribution, show only averages, or focus on a different relationship.

Answer

A pie chart showing the proportion of samples where each run achieved the highest ROUGE-L score.

Answer

A scatter plot of ROUGE-L versus ROUGE-1 for each sample, colored by run.

Answer

A single line chart plotting the average ROUGE-L score of each run over training epochs.

Question 15

A data scientist is preparing a visualization of an LLM evaluation suite that covers several task types with different score ranges. The audience includes both engineers and non-technical stakeholders. Which two practices best ensure the visualization is accurate and interpretable? (Choose two.)

Accepted Answer

Include sample counts or confidence intervals alongside each aggregate score.. Mixed-scale metrics must be placed on a comparable footing, and viewers must be told how that was done, so normalization with documented transformation is appropriate. Because aggregates hide variability, pairing them with sample counts or confidence intervals keeps interpretation honest. Together these practices let engineers assess reliability and stakeholders read the chart without being misled by scale or noise.

Answer

Encode the numeric score using both bar length and a diverging color gradient.

Answer

Truncate the y-axis to start just below the minimum observed value to emphasize differences.

Answer

Use a separate 3D perspective chart for each task type to maximize visual impact.