NCA-GENL Data Analysis and Visualization Practice Test 4 — 15 Questions

Question 1

A data scientist is preparing an exploratory report on a large corpus of prompt-completion pairs used to fine-tune an LLM. They want to visualize the distribution of a single numerical feature, prompt token count, to check for skew before choosing a tokenization budget. Which visualization is most appropriate?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

A histogram of prompt token counts.. When the goal is to inspect the shape of one continuous variable, a histogram is the natural choice. It reveals skew, gaps, and multiple peaks in prompt token counts, which directly informs how large a tokenization budget should be. Other charts either summarize relationships or display categories, not univariate distribution shape.

Answer

A stacked bar chart of prompt categories by split.

Answer

A t-SNE scatter plot of prompt embeddings.

Answer

A correlation heatmap of all numerical features.

Question 2

A data scientist wants to track training loss, learning rate, and GPU utilization side by side across thousands of steps in an interactive dashboard that supports comparing multiple runs. Which tool is designed specifically for this experiment-tracking and interactive visualization workflow?

Accepted Answer

Weights & Biases. Experiment-tracking platforms are purpose-built to ingest scalar metrics emitted during training and present them in interactive dashboards. Weights & Biases provides this out of the box, including live updates, run grouping, and side-by-side comparison, which are the exact capabilities requested. Generic plotting libraries and spreadsheet tools can render a single chart but cannot manage multi-run, multi-metric tracking at scale.

Answer

Excel

Answer

Pandas

Answer

Matplotlib

Question 3

A data scientist is analyzing 2 million document embeddings from a RAG corpus on a single NVIDIA GPU. A full pairwise cosine similarity matrix would require roughly 16 TB of memory, which is infeasible. They need to identify near-duplicate documents and visualize cluster density without materializing the full matrix. Which approach is most appropriate?

Accepted Answer

Use FAISS with an IVF-PQ index to perform approximate nearest-neighbor search, then plot a 2D projection colored by local density.. At 2 million vectors, the full similarity matrix is memory-infeasible, so an approximate method is required. FAISS IVF-PQ combines inverted-file partitioning with product quantization to search a compressed index on a single GPU, returning near-duplicates efficiently. Plotting a 2D projection colored by local neighbor density then exposes cluster structure and duplicate hotspots. Exact full-matrix computation, PCA-to-2D distances, and norm sorting all fail on memory or accuracy grounds.

Answer

Sort embeddings by their L2 norm and treat documents with similar norms as duplicates.

Answer

Compute the full 2M x 2M cosine similarity matrix in float32 on the GPU to preserve exact distances.

Answer

Reduce dimensionality to 2D with PCA first, then compute exact pairwise distances in the 2D space to find duplicates.

Question 4

An engineer needs to track validation loss, learning rate, and GPU utilization together over training steps for a fine-tuning run, and wants the ability to compare multiple runs side by side in a web dashboard. Which approach best meets this need?

Accepted Answer

Log metrics with a framework-integrated experiment tracker that provides a web UI for run comparison.. Tracking several metrics across steps and comparing runs requires structured logging plus a visualization layer. Framework-integrated experiment trackers supply both: they capture scalars at each step and render interactive dashboards where runs can be overlaid, filtered, and compared. Manual files, console output, and one-off snapshots lack the persistence, structure, and comparison features the task demands.

Answer

Print metrics to stdout and rely on the terminal scrollback to review trends.

Answer

Write metrics to a CSV file and open it in a spreadsheet after training completes.

Answer

Capture a single screenshot of nvidia-smi output at the end of training.

Question 5

A machine learning engineer is monitoring an LLM inference service deployed on NVIDIA GPUs. They want a real-time dashboard that shows GPU utilization, memory usage, and request latency, and they need to set alerts when thresholds are exceeded. Which NVIDIA tool is purpose-built for this monitoring and alerting?

Accepted Answer

NVIDIA Data Center GPU Manager (DCGM) with DCGM-Exporter and Prometheus/Grafana.. Real-time GPU monitoring with dashboards and alerts is the core purpose of NVIDIA DCGM combined with DCGM-Exporter and a metrics stack such as Prometheus and Grafana. The other tools address profiling, inference optimization, or model serving, none of which provide the required operational monitoring and alerting.

Answer

NVIDIA TensorRT.

Answer

NVIDIA Triton Inference Server.

Answer

NVIDIA Nsight Systems.

Question 6

You are analyzing embedding quality for a retrieval-augmented generation system. You have 1,000 document embeddings of 4,096 dimensions and want to inspect whether semantically similar documents form visible clusters. Which dimensionality-reduction approach is most appropriate before plotting in two dimensions?

Accepted Answer

Uniform Manifold Approximation and Projection (UMAP). Embeddings are high-dimensional and their semantic structure is typically nonlinear, so a nonlinear manifold method is preferred for two-dimensional inspection. UMAP preserves local neighborhoods while retaining more global organization than t-SNE and scales efficiently, making cluster formation visible. Linear PCA and arbitrary dimension truncation either miss nonlinear structure or discard most information, while a correlation matrix analyzes features rather than documents.

Answer

Compute a correlation matrix and reorder it hierarchically

Answer

Principal Component Analysis (PCA) to two components

Answer

Reduce dimensions by selecting the first 100 raw embedding dimensions

Question 7

A team is comparing two LLM fine-tuning runs on the same dataset. Run A used a cosine learning-rate schedule, and Run B used a constant learning rate. They plot validation loss versus training step for both runs on the same axes. Run A's curve is smooth, while Run B's curve shows a sharp upward spike around step 800 and then recovers. The team wants to determine whether the spike in Run B indicates a data-order artifact or a genuine optimization instability. Which additional visualization is most useful for that diagnosis?

Accepted Answer

A scatter plot of per-batch training loss for Run B colored by batch index modulo epoch length. Plotting per-batch training loss with color encoding the batch's position modulo the epoch length exposes whether the spike recurs at a fixed data position. A recurring pattern implies a data-order artifact, such as a hard shard or mislabeled examples, while a random pattern implies optimization instability. Summary charts and hardware metrics do not preserve the temporal and data-order information needed for this diagnosis.

Answer

A bar chart of final validation loss for Run A and Run B

Answer

A heatmap of attention weights for one validation example

Answer

A line chart of GPU utilization for Run A and Run B

Question 8

A team is analyzing an LLM evaluation dataset with thousands of prompts and multiple scoring dimensions such as correctness, fluency, and safety. They want a single visualization that reveals how these dimensions correlate and whether any prompts score unusually on several dimensions at once. Which visualization is most suitable?

Accepted Answer

A parallel coordinates plot of the scoring dimensions across prompts.. Parallel coordinates are designed for multivariate data: each dimension gets an axis and each prompt becomes a polyline across them. This exposes correlations between correctness, fluency, and safety and makes multi-dimension outliers visually obvious. Charts that collapse dimensions into averages or single sequences cannot reveal those relationships.

Answer

A stacked bar chart of total score by prompt category.

Answer

A histogram of the overall average score.

Answer

A single line chart of correctness scores ordered by prompt index.

Question 9

A team is evaluating an LLM-based summarization service and wants a visualization that shows how the distribution of generated summary lengths compares to the reference summaries across 5,000 test articles. They want to see whether the model systematically produces shorter or longer outputs. Which visualization is best suited?

Accepted Answer

A grouped histogram or overlaid density plot of generated versus reference summary lengths.. The question is distributional: do generated summaries differ systematically in length from references across the corpus? Overlaying the two length histograms or density curves on a shared axis makes any shift in center or spread obvious at a glance. Scatter plots of quality metrics, per-epoch training lines, and single-example attention heatmaps all address different questions and cannot reveal a corpus-level length bias.

Answer

A line chart of average summary length per training epoch.

Answer

A scatter plot of BLEU score versus article length.

Answer

A heatmap of token-level attention weights for a single example.

Question 10

You have generated 512-dimensional embeddings for 200,000 documents using an NVIDIA NeMo embedding model and want to inspect whether semantically similar documents cluster together. Which technique should you apply first to project these embeddings into two dimensions for visual inspection?

Accepted Answer

UMAP with a small n_neighbors value to emphasize local structure.. Nonlinear manifold learning such as UMAP is designed to project high-dimensional embeddings into two or three dimensions while preserving neighborhood relationships. With a small n_neighbors setting, UMAP emphasizes local structure, making it well suited to checking whether semantically similar documents form coherent clusters. Linear methods like PCA and scalar summaries like vector norms cannot expose that local neighborhood structure.

Answer

Principal Component Analysis projecting onto the top two components.

Answer

A bar chart of the L2 norm of each embedding vector.

Answer

A Pearson correlation heatmap of all 200,000 document pairs.

Question 11

A data scientist is preparing a visualization to compare the performance of three different LLM fine-tuning runs on a summarization benchmark. They want to show both the central tendency and the variability of ROUGE-L scores across multiple evaluation samples. Which two visualizations are most appropriate for this goal? (Choose two.)

Accepted Answer

A box plot of ROUGE-L scores for each fine-tuning run.. To compare central tendency and variability of ROUGE-L scores across three runs, distributions must be shown per run. Box plots and violin plots both display median, quartiles, and spread, with violin plots adding density shape. The other options either collapse the score distribution, show only averages, or focus on a different relationship.

Answer

A pie chart showing the proportion of samples where each run achieved the highest ROUGE-L score.

Answer

A scatter plot of ROUGE-L versus ROUGE-1 for each sample, colored by run.

Answer

A single line chart plotting the average ROUGE-L score of each run over training epochs.

Question 12

A data scientist is preparing a visualization of an LLM evaluation suite that covers several task types with different score ranges. The audience includes both engineers and non-technical stakeholders. Which two practices best ensure the visualization is accurate and interpretable? (Choose two.)

Accepted Answer

Include sample counts or confidence intervals alongside each aggregate score.. Mixed-scale metrics must be placed on a comparable footing, and viewers must be told how that was done, so normalization with documented transformation is appropriate. Because aggregates hide variability, pairing them with sample counts or confidence intervals keeps interpretation honest. Together these practices let engineers assess reliability and stakeholders read the chart without being misled by scale or noise.

Answer

Encode the numeric score using both bar length and a diverging color gradient.

Answer

Truncate the y-axis to start just below the minimum observed value to emphasize differences.

Answer

Use a separate 3D perspective chart for each task type to maximize visual impact.

Question 13

A team is preparing a dashboard to monitor an LLM inference service in production. They want visualizations that surface latency problems and resource saturation before users are affected. Which two visualizations are most appropriate for this goal? (Choose two.)

Accepted Answer

A gauge or time-series chart of GPU memory utilization and KV cache occupancy. Effective production monitoring pairs a latency view with a resource view. Percentile time series expose tail latency changes that averages conceal, while GPU memory and KV cache occupancy show the underlying pressure that causes those changes. Together they let the team act on leading indicators. Traffic-share pies, token word clouds, and input-output scatter plots describe workload characteristics rather than service health, so they do not support preemptive detection of degradation.

Answer

A word cloud of the most frequent prompt tokens received today

Answer

A scatter plot of prompt length versus generated response length for a random sample

Answer

A pie chart of total requests grouped by model version for the current day

Question 14

A data analyst is preparing a dashboard for an LLM inference service. The service logs per-request latency in milliseconds. The team wants a single chart that lets an on-call engineer quickly see both the typical latency and how often requests exceed the service-level objective of 500 ms. Which visualization best supports that goal?

Accepted Answer

A histogram of latency with a vertical marker at 500 ms. A histogram shows the latency distribution in ordered bins, so the typical range and the tail are both visible. Adding a vertical marker at 500 ms turns the chart into a threshold monitor, making SLO exceedance immediately apparent. Mean-only line charts, pie charts, and scatter plots each omit either the distribution shape or the threshold context needed for fast operational decisions.

Answer

A pie chart of requests grouped by latency bucket

Answer

A time-series line chart of mean latency only

Answer

A scatter plot of request size versus latency

Question 15

A developer is profiling an LLM inference endpoint on an NVIDIA L40S and observes that time-to-first-token (TTFT) is stable but inter-token latency spikes periodically. They want to determine whether the spikes align with KV cache growth or with batch-size changes. Which visualization strategy best isolates the cause?

Accepted Answer

Plot a time series of inter-token latency with KV cache size and active batch size overlaid on the same time axis.. The goal is to determine whether periodic latency spikes align temporally with KV cache growth or batch-size changes. Only a shared time axis can establish that alignment. Plotting inter-token latency, KV cache size, and active batch size together makes coincident events visible immediately. Summary statistics, length histograms, and single-request flame graphs all discard the cross-request temporal relationship required to isolate the cause.

Answer

Generate a histogram of token lengths for all requests served during the profiling window.

Answer

Compute the mean and standard deviation of inter-token latency and report them as a bar chart.

Answer

Render a flame graph of GPU kernel execution for a single representative request.