DEA-C01 Data Ingestion and Transformation Practice Test 13 — 15 Questions

Question 1

A data engineering team is designing a data ingestion pipeline that will receive millions of small JSON files per hour from external partners via API. The files should be stored in Amazon S3 and then transformed into Parquet for querying. Which approach is MOST cost-effective and scalable?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use Amazon Kinesis Data Firehose to buffer and deliver data to S3, then use AWS Glue to convert to Parquet.. Amazon Kinesis Data Firehose is the most cost-effective and scalable approach because it can buffer millions of small JSON files per hour, automatically batch them, and deliver them to S3 without requiring any server management. After delivery, AWS Glue can efficiently convert the JSON data to Parquet format for optimized querying, leveraging its serverless, pay-per-use model that scales with data volume.

Answer

Use AWS Lambda to process each file as it arrives and write to S3.

Answer

Use AWS Direct Connect to establish a dedicated network for file uploads.

Answer

Use Amazon EMR to process the files as they arrive in S3.

Question 2

A data engineer is troubleshooting a slow AWS Glue ETL job that reads from Amazon S3 and writes to Amazon Redshift. The job processes 10 GB of CSV data. The engineer notices that the job runs with a single DPU and takes longer than expected. Which change would MOST likely improve performance?

Accepted Answer

Increase the number of DPUs and configure the job to use the S3 list implementation for parallel reads.. The job runs with a single DPU, which limits parallelism. Increasing the number of DPUs allows the job to process data in parallel across multiple workers. Additionally, configuring the S3 list implementation enables the job to list objects in parallel, reducing the overhead of discovering input files. This combination directly addresses the bottleneck of a single-threaded read from S3.

Answer

Replace Redshift with Amazon Redshift Spectrum.

Answer

Change the input format to Parquet and enable predicate pushdown.

Answer

Use a JDBC connection to read data directly from S3.

Question 3

A company uses Amazon Kinesis Data Streams to ingest clickstream data from a website. The data must be transformed (e.g., enrich with user location) before being stored in Amazon S3. Which architecture is MOST efficient for this transformation?

Accepted Answer

Use Amazon Kinesis Data Analytics to transform the stream and output to Amazon Kinesis Data Firehose, which writes to S3.. Amazon Kinesis Data Analytics (KDA) can perform real-time transformations (e.g., enriching clickstream data with user location via SQL or Flink) on the stream, then output the transformed data to Kinesis Data Firehose, which can batch and compress records before writing to S3. This architecture minimizes operational overhead and is purpose-built for streaming transformations, avoiding the latency and complexity of Lambda cold starts or the provisioning overhead of Glue/EMR.

Answer

Use AWS Glue to run a streaming ETL job.

Answer

Use Amazon EMR to consume the stream using Spark Streaming.

Answer

Use AWS Lambda to process each record from the stream and write to S3.

Question 4

A company uses AWS Glue to transform data from Amazon S3 into Parquet format. The job fails with an out-of-memory error for large files. Which TWO actions can resolve this issue? (Choose TWO.)

Accepted Answer

Increase the number of DPUs allocated to the job.. Increasing the number of DPUs (Data Processing Units) allocated to the Glue job provides more memory and compute capacity, which directly addresses out-of-memory errors when processing large files. AWS Glue uses Apache Spark under the hood, and each DPU provides 4 vCPU and 16 GB of memory, so adding DPUs scales the available resources for in-memory transformations.

Answer

Change the input format from CSV to JSON.

Answer

Use the Glue streaming ETL feature.

Answer

Enable CloudWatch logs for detailed error analysis.

Question 5

A company is building a data lake on Amazon S3 and needs to ingest data from multiple sources. The ingestion must be automated and handle schema changes. Which THREE services can be used together to achieve this? (Choose THREE.)

Accepted Answer

AWS Glue Crawler. AWS Glue Crawler (B) automatically discovers and catalogs schemas from data sources. Amazon Kinesis Data Firehose (C) ingests streaming data into S3. AWS Lambda (E) can transform data on the fly. Together, they automate ingestion and handle schema changes. Amazon Redshift (A) is a data warehouse, not a data lake ingestion service. Amazon EMR (D) is a big data processing framework, not primarily for automated schema change handling.

Answer

Amazon Redshift

Answer

Amazon EMR

Question 6

An IAM policy is attached to an AWS Glue job. The job needs to read from and write to S3 buckets, and also trigger other Glue jobs. The job is failing with an AccessDenied error when trying to write to a bucket named 'example-bucket'. What is the MOST likely cause?

Accepted Answer

The bucket name in the resource ARN does not match the actual bucket name.. The AccessDenied error when writing to 'example-bucket' most likely occurs because the resource ARN in the IAM policy specifies a different bucket name, causing the S3 service to deny the s3:PutObject action. IAM policies require exact ARN matches for resource-based permissions, and a mismatch between the ARN bucket name and the actual bucket name will result in an implicit deny, even if the action is allowed.

Answer

The policy does not include s3:PutObject action.

Answer

The policy uses a resource ARN with a wildcard, which is not allowed.

Answer

The policy does not allow Glue actions.

Question 7

A company needs to ingest data from an on-premises Oracle database into Amazon Redshift for analytics. The data volume is 500 GB and the network bandwidth is limited. Which AWS service should be used for the initial one-time data migration?

Accepted Answer

AWS Snowball. AWS Snowball is the correct choice for the initial one-time migration of 500 GB of data from an on-premises Oracle database to Amazon Redshift when network bandwidth is limited. Snowball provides a physical storage device that bypasses network constraints entirely, allowing you to transfer large volumes of data securely and quickly by shipping the device to AWS for ingestion into Amazon S3, which can then be loaded into Redshift. This approach avoids the prolonged transfer times and potential bottlenecks associated with limited bandwidth, making it ideal for a one-time, large-scale data migration.

Answer

AWS Direct Connect

Answer

Amazon S3 Transfer Acceleration

Answer

AWS Database Migration Service (DMS)

Question 8

A company uses Amazon Kinesis Data Firehose to deliver streaming data to Amazon S3. The data is in JSON format and needs to be converted to Parquet. However, the conversion is failing. What is the most likely cause?

Accepted Answer

The schema is not defined in the AWS Glue Data Catalog. Kinesis Data Firehose requires a schema to convert to Parquet. This schema can be provided by a Glue Data Catalog table. If the schema is not defined, the conversion fails. Data size is not an issue for conversion. Kinesis Data Streams is not involved here. Lambda transformation can convert to Parquet but is not required.

Answer

The data transformation Lambda function is not converting to Parquet

Answer

The data size exceeds the 1 MB limit per record

Answer

The delivery stream is configured to use Kinesis Data Streams as source

Question 9

A company is designing a data ingestion pipeline for real-time analytics. The source is a relational database, and the target is Amazon Redshift. The pipeline must handle schema changes in the source database automatically. Which combination of services should be used?

Accepted Answer

AWS DMS and AWS Glue. AWS DMS (Database Migration Service) can capture ongoing changes from a relational database source using Change Data Capture (CDC) and replicate them to Amazon Redshift. AWS Glue can be used to handle schema evolution by crawling the source and updating the target schema, ensuring the pipeline adapts to schema changes automatically. Together, they provide a real-time ingestion pipeline with schema flexibility.

Answer

Amazon S3 and Amazon Athena

Answer

AWS Glue and Amazon Redshift COPY

Answer

Amazon Kinesis Data Streams and AWS Lambda

Question 10

Which THREE factors should be considered when selecting a data ingestion service for a high-volume, real-time streaming pipeline that requires exactly-once processing semantics? (Choose 3.)

Accepted Answer

Ability to replay records from a checkpoint. In high-volume streaming pipelines, the ability to replay records from a checkpoint is essential for achieving exactly-once processing semantics. Services like Amazon Kinesis Data Streams and Apache Kafka allow consumers to reset to a specific sequence number or offset, enabling reprocessing of unacknowledged records after a failure without data loss or duplication.

Answer

Support for schema evolution

Answer

Ability to transform data in-flight

Question 11

Which TWO AWS services can be used to transform data in an Amazon S3 data lake before loading into Amazon Redshift? (Choose 2.)

Accepted Answer

AWS Glue. AWS Glue (Option D) is correct because it provides a fully managed ETL service that can crawl, catalog, and transform data in Amazon S3 data lakes. With AWS Glue, you can create ETL jobs using a serverless Spark environment to transform raw data into optimized formats (e.g., Parquet) and load it into Amazon Redshift via JDBC connections, making it ideal for preparing data before ingestion.

Answer

AWS Lambda

Answer

Amazon Athena

Answer

Amazon EMR

Question 12

The exhibit shows the output of describing an Amazon Kinesis Data Stream. A producer is sending records but the consumer is not receiving all records. What is the most likely cause?

Accepted Answer

The stream has only one shard, causing write throttling. The most likely cause is that the stream has only one shard, causing write throttling (A). A Kinesis Data Stream with a single shard has a write capacity of 1 MB/s and 1,000 records/s. If the producer exceeds this, writes are throttled, leading to lost or delayed records that the consumer does not receive. The exhibit likely shows a stream with one shard.

Answer

The stream is in ACTIVE status, which prevents reading

Answer

The retention period is too short

Answer

The hash key range is too wide

Question 13

A data pipeline ingests CSV files from an S3 bucket into a Redshift table using the COPY command. Recently, files with inconsistent column delimiters (some use pipes, others use commas) have been arriving. The pipeline must handle both delimiters without manual intervention. What is the MOST efficient solution?

Accepted Answer

Create an AWS Lambda function triggered by S3 events that reads the first line of each file, detects the delimiter, and runs the COPY command with the appropriate DELIMITER option.. It provides an automated, event-driven solution that dynamically detects the delimiter of each incoming CSV file and executes the COPY command with the appropriate DELIMITER option. This approach eliminates manual intervention and leverages AWS Lambda's ability to process S3 events in near real-time, making it the most efficient for handling inconsistent delimiters.

Answer

Configure the COPY command with a fixed delimiter (e.g., comma) and manually convert files with pipes before ingestion.

Answer

Use AWS Glue to crawl the S3 bucket and automatically detect the schema and delimiter before writing to Redshift.

Answer

Use Amazon Athena to query the files with the OpenCSVSerDe, which automatically detects delimiters, and then write the results to Redshift.

Question 14

A data pipeline uses Amazon Kinesis Data Firehose to ingest log data from web servers and deliver it to Amazon S3. The data is then transformed by an AWS Glue job before being loaded into Amazon Redshift. The pipeline must handle a sudden spike in log volume without data loss. Which configuration change is MOST appropriate?

Accepted Answer

Configure Kinesis Data Firehose to back up all data to S3 in case of delivery failures.. Kinesis Data Firehose can be configured to back up all data to Amazon S3 in case of delivery failures, ensuring no data loss during spikes. This feature writes incoming data to a separate S3 bucket as a safety net when the primary destination (e.g., Redshift via Glue) is unavailable or overwhelmed, directly addressing the requirement to handle sudden volume spikes without data loss.

Answer

Increase the AWS Glue job timeout and allocate more DPUs.

Answer

Increase the number of nodes in the Redshift cluster to handle higher load.

Answer

Increase the S3 bucket size limit and enable versioning.

Question 15

A company needs to ingest data from a self-managed Apache Kafka cluster running on EC2 into Amazon S3. The data must be delivered in near real-time. Which AWS service is BEST suited for this task?

Accepted Answer

Use an AWS Glue streaming ETL job that reads from the Kafka cluster and writes to S3.. AWS Glue streaming ETL jobs can connect directly to an Apache Kafka cluster (including self-managed) as a source and write data to Amazon S3 in near real-time, making it a fully managed and suitable solution. Option A (Amazon MSK) is a managed Kafka service but does not directly ingest into S3; Option B (S3 Transfer Acceleration) only accelerates uploads, not ingestion; Option C (Kinesis Data Streams) adds unnecessary complexity as an intermediary.

Answer

Use Amazon MSK to replicate the Kafka cluster and then use a connector to S3.

Answer

Use Amazon S3 Transfer Acceleration to speed up the transfer from Kafka brokers to S3.

Answer

Use Amazon Kinesis Data Streams as an intermediary to buffer data before writing to S3.