AI0-001 AI Models and Data Engineering Practice Question
A healthcare startup is deploying a machine learning model to predict patient readmission within 30 days using electronic health records (EHR). The data pipeline uses Apache Spark for preprocessing and training on an Amazon EMR cluster. The training dataset is 50 GB and composed of structured numeric and categorical features, along with unstructured clinical notes. The data scientist observes that training takes over 12 hours and frequently fails due to out-of-memory (OOM) errors, especially when processing the clinical notes via TF-IDF vectorization. The cluster has 10 nodes with 64 GB RAM each. The data engineer has already tried increasing spark.sql.shuffle.partitions to 400 and using Kryo serialization, but OOM persists. Which action should the data engineer take next to resolve the OOM errors?
⚠ Common exam trap
CompTIA often tests the misconception that increasing cluster resources (nodes or memory) alone solves OOM errors, when the real fix is to optimize data partitioning and parallelism within Spark's execution model.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Repartition the clinical notes data into 2000 partitions before TF-IDF
Repartitioning the clinical notes data into 2000 partitions before TF-IDF vectorization increases parallelism and reduces the memory pressure per partition. The default partition count (often based on spark.default.parallelism) is too low for 50 GB of data, causing individual partitions to exceed executor memory limits. By increasing partitions, each executor processes smaller chunks, preventing OOM errors during the memory-intensive TF-IDF stage.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Broadcast the TF-IDF model to all executors to avoid shuffling
Why it's wrong here
Broadcasting is for small data; clinical notes are large, causing driver OOM.
- ✓
Repartition the clinical notes data into 2000 partitions before TF-IDF
Why this is correct
More partitions reduce the data per executor, mitigating OOM during vectorization.
- ✗
Add 10 more nodes to the cluster to increase total memory
Why it's wrong here
Adding nodes increases resources but may not fix skewed partitions and is costly.
- ✗
Use a single executor with 64 GB and increase driver memory to 128 GB
Why it's wrong here
Single executor limits parallelism and still risks OOM on large data.
Visual reference
About these practice questions
Courseiva writes every AI0-001 question from scratch — 754 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI0-001 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI0-001 exam.