The correct next step is to repartition the clinical notes data into 2000 partitions before TF-IDF vectorization. This directly addresses the out-of-memory errors in Spark for machine learning by increasing parallelism, which reduces the data volume each executor must process during the memory-intensive TF-IDF stage. With a 50 GB dataset on a 10-node cluster (64 GB RAM each), the default partition count is too low, causing individual partitions to exceed executor memory limits despite tuning shuffle partitions and serialization. On the CompTIA AI+ AI0-010 exam, this scenario tests your understanding of Spark memory management in ML pipelines, specifically how partition sizing impacts executor memory pressure—a common trap is focusing only on shuffle tuning while ignoring data distribution before transformations. Remember the mnemonic: “Partition before transformation to prevent memory starvation.”
AI0-001 AI Models and Data Engineering Practice Question
This AI0-001 practice question tests your understanding of ai models and data engineering. The scenario asks you to isolate a root cause — eliminate options that address a different problem before choosing. After answering, compare your reasoning against the explanation and wrong-answer breakdown below. Once you have made your selection, read the full explanation to reinforce the concept and understand why each distractor is designed to mislead on exam day.
A healthcare startup is deploying a machine learning model to predict patient readmission within 30 days using electronic health records (EHR). The data pipeline uses Apache Spark for preprocessing and training on an Amazon EMR cluster. The training dataset is 50 GB and composed of structured numeric and categorical features, along with unstructured clinical notes. The data scientist observes that training takes over 12 hours and frequently fails due to out-of-memory (OOM) errors, especially when processing the clinical notes via TF-IDF vectorization. The cluster has 10 nodes with 64 GB RAM each. The data engineer has already tried increasing spark.sql.shuffle.partitions to 400 and using Kryo serialization, but OOM persists. Which action should the data engineer take next to resolve the OOM errors?
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Repartition the clinical notes data into 2000 partitions before TF-IDF
Option B is correct because repartitioning the clinical notes data into 2000 partitions before TF-IDF vectorization increases parallelism and reduces the memory pressure per partition. The default partition count (often based on spark.default.parallelism) is too low for 50 GB of data, causing individual partitions to exceed executor memory limits. By increasing partitions, each executor processes smaller chunks, preventing OOM errors during the memory-intensive TF-IDF stage.
Key principle: Answer the scenario, not the keyword: identify the specific constraint before choosing the most familiar-sounding option.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
✗
Broadcast the TF-IDF model to all executors to avoid shuffling
Why it's wrong here
Broadcasting is for small data; clinical notes are large, causing driver OOM.
✓
Repartition the clinical notes data into 2000 partitions before TF-IDF
Why this is correct
More partitions reduce the data per executor, mitigating OOM during vectorization.
Related concept
Read the scenario before looking for a memorised answer.
✗
Add 10 more nodes to the cluster to increase total memory
Why it's wrong here
Adding nodes increases resources but may not fix skewed partitions and is costly.
✗
Use a single executor with 64 GB and increase driver memory to 128 GB
Why it's wrong here
Single executor limits parallelism and still risks OOM on large data.
Common exam traps
Common exam trap: answer the scenario, not the keyword
CompTIA often tests the misconception that increasing cluster resources (nodes or memory) alone solves OOM errors, when the real fix is to optimize data partitioning and parallelism within Spark's execution model.
Detailed technical explanation
How to think about this question
Under the hood, TF-IDF vectorization in Spark (via HashingTF or CountVectorizer) first tokenizes text and then computes term frequencies per document, which can create large intermediate objects. If partitions are too large, the executor's heap is exhausted even with Kryo serialization. Repartitioning to 2000 partitions (roughly 25 MB per partition for 50 GB) aligns with the recommended partition size of 100-200 MB, balancing memory usage and task overhead. In real-world scenarios, data skew in clinical notes (e.g., some records with extremely long text) can still cause OOM; combining repartitioning with a custom partitioner or bucketing can further mitigate skew.
KKey Concepts to Remember
Read the scenario before looking for a memorised answer.
Find the constraint that changes the correct option.
Eliminate answers that are true in general but not in this case.
TExam Day Tips
→Watch for words such as best, first, most likely and least administrative effort.
→Review why wrong options are wrong, not only why the correct option is correct.
Key takeaway
Answer the scenario, not the keyword: identify the specific constraint before choosing the most familiar-sounding option.
Real-world example
How this comes up in practice
A small business has 20 workstations on the 192.168.1.0/24 network and one public IP from its ISP. The router uses PAT (NAT overload) so all 20 devices share one public address using different source ports. NAT questions test whether you understand the four address terms and which direction each translation applies.
What to study next
Got this wrong? Here's your next step.
Identify which exam domain this question belongs to, review the core concept, then practise similar questions from the same domain.
AI Models and Data Engineering — This question tests AI Models and Data Engineering — Read the scenario before looking for a memorised answer..
What is the correct answer to this question?
The correct answer is: Repartition the clinical notes data into 2000 partitions before TF-IDF — Option B is correct because repartitioning the clinical notes data into 2000 partitions before TF-IDF vectorization increases parallelism and reduces the memory pressure per partition. The default partition count (often based on spark.default.parallelism) is too low for 50 GB of data, causing individual partitions to exceed executor memory limits. By increasing partitions, each executor processes smaller chunks, preventing OOM errors during the memory-intensive TF-IDF stage.
What should I do if I get this AI0-001 question wrong?
Identify which exam domain this question belongs to, review the core concept, then practise similar questions from the same domain.
What is the key concept behind this question?
Read the scenario before looking for a memorised answer.
About these practice questions
Courseiva creates original exam-style practice questions with explanations and wrong-answer analysis. It does not publish real exam questions, exam dumps, or protected exam content. Learn why practice questions differ from exam dumps →
Share a tip, memory trick, or ask about the reasoning behind this question. Do not post real exam questions, leaked content, braindumps, or copyrighted exam material. Comments are moderated and may be removed without notice.
This AI0-001 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI0-001 exam.
Question Discussion
Share a tip, memory trick, or ask about the reasoning behind this question. Do not post real exam questions, leaked content, braindumps, or copyrighted exam material. Comments are moderated and may be removed without notice.
Sign in to join the discussion.