NCA-GENL Data Analysis and Visualization Practice Test 3 — 15 Questions

Question 1

A data scientist is analyzing a large corpus of LLM training documents and wants to visualize which topics appear together across documents. After computing TF-IDF vectors, they apply non-negative matrix factorization (NMF) to reduce dimensionality. Which visualization best shows the relationships between the discovered topics and the documents?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

A heatmap of the document-topic matrix with documents on one axis and topics on the other. The document-topic matrix produced by NMF is a two-dimensional array of weights, and a heatmap is the natural visualization for such data. It lets the analyst see which topics are active in each document and which topics tend to co-occur, directly answering the question about topic relationships across the corpus.

Answer

A pie chart of the top 10 most frequent words in the corpus

Answer

A line chart of the reconstruction error across NMF iterations

Answer

A scatter plot of documents positioned by their first two TF-IDF principal components

Question 2

You are comparing the inference throughput of an LLM served with two different batching strategies across a range of request arrival rates. You want a single visualization that shows both the median throughput and the variability at each arrival rate. Which visualization is most appropriate?

Accepted Answer

A box plot of throughput for each batching strategy, with arrival rate on the x-axis. A box plot is designed to show median and spread simultaneously, and placing one box per arrival rate per strategy makes the comparison direct. It handles skewed throughput distributions better than mean-only line charts and avoids the clutter of plotting every individual measurement.

Answer

A line chart of mean throughput only, with one line per batching strategy

Answer

A stacked bar chart of total throughput summed across arrival rates

Answer

A scatter plot of individual throughput measurements colored by batching strategy

Question 3

You are analyzing token frequency distribution across a 50 GB pretraining corpus before fine-tuning an NVIDIA NIM-deployed Llama model. The raw frequency histogram is heavily right-skewed, making it impossible to compare low-frequency tokens. Which transformation should you apply to the x-axis to make the distribution easier to compare across the full vocabulary?

Accepted Answer

Plot token rank on a logarithmic x-axis against frequency on a logarithmic y-axis.. Token frequencies in natural-language corpora follow a Zipfian power law, so both rank and frequency span several orders of magnitude. Plotting rank and frequency on logarithmic axes linearizes that relationship and makes the entire vocabulary comparable in a single chart. Linear or weakly transformed axes cannot compress the dynamic range enough to reveal the tail behavior that matters for vocabulary and sampling decisions.

Answer

Normalize each token count by the total corpus token count and keep a linear x-axis.

Answer

Bin tokens into deciles and plot only the top ten most frequent tokens.

Answer

Apply a square-root transform to token counts and keep the vocabulary index on the x-axis.

Question 4

You are analyzing token probability distributions from an LLM inference service to detect hallucination risk. You need a single visualization that shows, for one generated response, how the model's confidence evolved token-by-token and where it suddenly dropped. Which visualization is most appropriate?

Accepted Answer

A line chart of the top-1 token probability across generation steps, with a marked threshold line.. Tracking the top-1 token probability across generation steps preserves sequence order and directly encodes model confidence at each step. Adding a threshold line turns the chart into an alerting view where abrupt dips flag possible hallucination regions. Aggregated or embedding-based views lose the temporal resolution needed for this per-response diagnosis.

Answer

A confusion matrix of predicted versus reference tokens for the entire batch.

Answer

A t-SNE scatter plot of the final hidden state embeddings for the batch.

Answer

A histogram of maximum token probabilities across all requests in the last hour.

Question 5

A data scientist is analyzing the output of a Llama 3 8B model on a summarization task. The token-level log-probabilities are extracted, and the goal is to visualize how confident the model is in each generated token across the summary. Which visualization is most appropriate for showing the per-token probability distribution and identifying tokens where the model is uncertain?

Accepted Answer

A line chart plotting the log-probability of each generated token against its position in the output sequence.. The scenario asks for per-token confidence across a generated summary. A line chart of log-probability by token position preserves the sequential order and directly displays the model's confidence at each step. The other options either aggregate away position information or use dimensionality reduction that does not represent output probabilities.

Answer

A bar chart of the top-5 predicted tokens for the entire summary, aggregated across all positions.

Answer

A t-SNE scatter plot of the hidden states of all generated tokens, colored by token ID.

Answer

A confusion matrix comparing predicted tokens to ground-truth tokens for the entire summary.

Question 6

You are monitoring a production LLM inference service on an NVIDIA GPU. The service's request latency distribution is heavily right-skewed, and a small fraction of requests take far longer than the rest. You need a visualization that shows the full distribution shape, including the median and the extreme tail, to decide whether the GPU is under-provisioned. Which visualization should you use?

Accepted Answer

A box plot of request latency with whiskers extending to the 1.5 IQR and outliers plotted individually.. Because the latency distribution is right-skewed, summary statistics like the mean hide the tail. A box plot exposes the median, quartiles, and individual outliers, so the extreme slow requests remain visible. This lets you decide whether the long tail is severe enough to justify additional GPU capacity.

Answer

A single line chart of mean request latency sampled every minute.

Answer

A pie chart showing the proportion of requests in each latency bucket.

Answer

A scatter plot of GPU utilization against time for the last 24 hours.

Question 7

You are analyzing the output of an LLM inference endpoint that returns a JSON payload containing a top-k token probability distribution for a single generated step. Which visualization most directly communicates the model's confidence ranking across the returned tokens?

Accepted Answer

A horizontal bar chart with tokens on the y-axis sorted by probability descending.. Ranking data is best shown with sorted bar lengths because position and length are preattentively processed. A descending horizontal bar chart lets a reviewer instantly see the top token and the margin over runners-up, which is exactly the confidence information the JSON payload contains. Other chart types either distort magnitude judgments or plot against irrelevant dimensions.

Answer

A scatter plot of probability versus vocabulary index position.

Answer

A line chart plotting probability against token string length.

Answer

A pie chart showing each token's probability as a slice of the total mass.

Question 8

A team is fine-tuning an LLM and wants to detect whether individual training examples are causing unusually large gradient updates. They plan to visualize per-example gradient norms alongside other diagnostics. Which two visualizations are most appropriate for identifying these influential examples? (Choose two.)

Accepted Answer

A sorted bar chart of per-example gradient norms with examples ranked from largest to smallest. Detecting influential examples requires per-example gradient information. A sorted bar chart ranks examples so the largest norms stand out, while a scatter plot of gradient norm versus loss adds context about why those norms are large. Together they let the team isolate and inspect the examples driving unusually large updates.

Answer

A line chart of the moving average of total training loss across epochs

Answer

A pie chart showing the proportion of examples in each loss decile

Answer

A heatmap of the model's attention weights for a single randomly chosen example

Question 9

You are analyzing a dataset of 50,000 LLM training samples and want to visualize how sample lengths are distributed to decide on a maximum sequence length cutoff. The lengths range from 10 to 8,000 tokens with a long right tail. Which visualization should you use to best reveal the shape, central tendency, and outliers of this single continuous variable?

Accepted Answer

A histogram with 50 bins and a log-scaled x-axis. A histogram is the standard univariate visualization for a continuous variable because bin heights encode frequency and expose shape, center, and outliers. With a range spanning three orders of magnitude, a log-scaled x-axis prevents the low-token region from collapsing into a single bar. This combination lets the data scientist choose a cutoff where sample density drops, balancing truncation loss against compute cost.

Answer

A scatter plot of token length versus sample index

Answer

A line chart of cumulative sample count sorted by token length

Answer

A pie chart of samples grouped into five equal token-length ranges

Question 10

A team is fine-tuning an NVIDIA NIM-hosted Llama 3 8B model and wants a single visualization that tracks per-step training loss, learning rate, and GPU memory utilization together, so they can correlate a mid-run loss spike with resource pressure. They need a framework that integrates natively with the NVIDIA NeMo training stack and requires minimal custom plotting code. Which visualization approach best meets these requirements?

Accepted Answer

Use the NVIDIA NeMo training logs with TensorBoard, logging loss, learning rate, and GPU memory as scalars on the same step axis.. The scenario requires a single, integrated view that ties training loss, learning rate, and GPU memory together on the same step axis. NeMo's built-in TensorBoard logging emits these as scalars during training, letting the team visually align a loss spike with resource pressure without writing custom plotting code. Post-hoc CSV plotting, embedding projections, and confusion matrices either lack the required metrics or cannot correlate them in real time.

Answer

Render a t-SNE projection of the model's embedding layer each epoch to observe representation drift.

Answer

Generate a confusion matrix from the validation set after each checkpoint to detect training instability.

Answer

Export per-step metrics to CSV and build a custom Matplotlib multi-panel figure after training completes.

Question 11

A data scientist has a table of 500 LLM evaluation runs, each with a numerical faithfulness score from 0 to 1 and a categorical model version label. They want a compact view comparing the score distributions across model versions, including medians and spread, in a single figure. Which visualization should they choose?

Accepted Answer

A grouped box plot of faithfulness score by model version.. When comparing a continuous score across categorical groups, box plots are the standard compact choice because each box summarizes median, interquartile range, and outliers per group. They allow immediate side-by-side comparison of both center and spread across model versions, which is precisely what the data scientist needs in a single figure.

Answer

A network graph connecting runs that share the same model version.

Answer

A heatmap of faithfulness score binned by run index and model version.

Answer

A single scatter plot of faithfulness score against run index.

Question 12

A team is comparing two LLM checkpoints on a summarization benchmark. They want a single visualization that shows, for each evaluation metric, both the mean score and the spread across the benchmark's document categories, while making it easy to see whether the two checkpoints overlap. Which visualization best fits this requirement?

Accepted Answer

A grouped box plot with one box per checkpoint per metric, grouped by metric.. Comparing two checkpoints across multiple metrics while preserving spread calls for a distribution-aware chart. Grouped box plots keep each metric on its own scale, show central tendency and variability, and place the two checkpoints adjacent so overlap is immediately visible. Charts that reduce to means or sums discard the variance the team needs to judge whether differences are meaningful.

Answer

A stacked bar chart of total scores per checkpoint across all metrics.

Answer

A heatmap of mean scores with checkpoints as rows and metrics as columns.

Answer

A single line chart with one line per checkpoint plotting mean score against metric name.

Question 13

A team is evaluating a retrieval-augmented generation pipeline. They have a dataset of 500 queries, each with a retrieved context and a generated answer. The goal is to visualize how often the generated answer is faithful to the retrieved context versus hallucinated, and to compare this across three different retriever configurations. Which visualization best supports this comparison?

Accepted Answer

A grouped bar chart showing the proportion of faithful, partially faithful, and hallucinated answers for each retriever configuration.. The task requires comparing categorical faithfulness outcomes across three retriever configurations. A grouped bar chart places the proportions of each faithfulness category side by side for each configuration, making differences immediately visible. The other options either collapse the comparison, use continuous variables that do not represent faithfulness, or focus on a single example.

Answer

A pie chart showing the overall proportion of faithful, partially faithful, and hallucinated answers across all retrievers combined.

Answer

A heatmap of token-level attention weights between the generated answer and the retrieved context for a single query.

Answer

A scatter plot of answer length versus retrieval score, colored by retriever configuration.

Question 14

A data scientist is analyzing token-level loss values produced by an LLM evaluation run on a summarization dataset. Losses are stored as a list of floats, and most values cluster around 2.1, but a few exceed 9.0. The team wants a visualization that shows the shape of the loss distribution, including those extreme values, without hiding them through bin aggregation. Which visualization is most appropriate?

Accepted Answer

A strip plot (jittered scatter) of token-level loss. A strip plot plots every token-level loss as an individual mark, so extreme values remain visible instead of being merged into a bin or smoothed away. It reveals the distribution shape and the location of outliers simultaneously. Histograms, box plots, and violin plots each aggregate or summarize the data, which would obscure the few very high losses the team needs to inspect.

Answer

A histogram with 50 equal-width bins

Answer

A violin plot of token-level loss

Answer

A box plot of token-level loss

Question 15

A data scientist has embedded 200,000 LLM training documents with a sentence-transformer and wants to visualize the embedding space to inspect semantic clusters. Running UMAP on the full set is too slow, so they first reduce dimensions with PCA. Which approach best preserves local cluster structure for the final visualization?

Accepted Answer

Apply UMAP directly to the first 50 principal components of the embeddings. PCA is a fast linear preprocessing step that removes noise and reduces dimensionality, and UMAP operates best on a moderate number of informative components rather than raw high-dimensional vectors. Retaining 50 components keeps semantic variance while cutting computation, so local cluster structure survives into the final nonlinear embedding.

Answer

Apply t-SNE directly to the first 2 principal components of the embeddings

Answer

Apply PCA to reduce to 2 dimensions and plot the documents directly

Answer

Apply UMAP directly to the raw 768-dimensional embeddings without PCA