Databricks-GenAI-Assoc Data Preparation Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✗

  Drop every review shorter than 100 characters to reduce noise in the corpus.

  Why it's wrong here

  Short reviews may still be relevant and valid; discarding them removes potential retrieval evidence without solving HTML noise or context-window overflow. Filtering by length is a judgment call about corpus composition, not a fix for the stated preprocessing problems. It also risks deleting useful signal from the RAG index.

• ✗

  Store the raw HTML in a separate column so it remains available for future parsing needs.

  Why it's wrong here

  Preserving raw HTML does not clean the text that will be embedded; the embedded field would still contain tags unless a separate cleaning step is applied. Keeping a raw copy is a reasonable audit practice, but it does not by itself address markup noise or paragraph length. The question asks for actions that directly resolve the quality issues.

• ✓

  Split long paragraphs into overlapping chunks sized to the embedding model's token limit.

  Why this is correct

  The reviews contain paragraphs that exceed the model context window, so chunking with overlap ensures no content is silently truncated and context is preserved across boundaries. Sizing chunks to the token limit directly addresses the stated overflow problem. Overlap helps retrieval quality by preventing answers that straddle a boundary from being split apart.

• ✓

  Strip HTML tags and normalize whitespace using a Spark transformation before chunking.

  Why this is correct

  Removing HTML tags and collapsing whitespace directly handles the markup noise and inconsistent formatting described in the reviews. Doing this in a Spark transformation keeps the operation distributed and ensures every downstream chunk contains clean prose. It is a prerequisite for meaningful chunk boundaries, since tags and stray whitespace can corrupt token counts and embeddings.

• ✗

  Convert all review text to uppercase to standardize casing across the corpus.

  Why it's wrong here

  Uppercasing is a destructive normalization that removes case information useful for named entities and sentence boundaries, and it does not address HTML tags or length limits. Standardizing casing usually means lowercasing only when the embedding model expects it, but blanket uppercase harms tokenization quality. It leaves the actual quality issues untouched.