DEA-C01 Data Ingestion and Transformation Practice Test 36 — 15 Questions

Question 1

A data engineering team needs to ingest streaming data from an application into Amazon S3 for analytics. The data volume is moderate and the team wants the lowest operational overhead. Which AWS service should they use?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is a fully managed service for loading streaming data into S3 with no code required and minimal operational overhead. Option A is incorrect because Amazon SQS is a message queue service, not designed for streaming data ingestion into S3. Option B is incorrect because AWS Glue is primarily a batch ETL service, not suitable for real-time streaming. Option C is incorrect because Amazon Kinesis Data Streams requires custom consumers and more management, increasing operational overhead.

Answer

Amazon SQS

Answer

AWS Glue

Answer

Amazon Kinesis Data Streams

Question 2

A company uses Amazon Kinesis Data Analytics to process real-time data. The application needs to aggregate data over a 10-minute window. The team notices that late-arriving events are being dropped. Which configuration should they adjust?

Accepted Answer

Set the allowed_lateness parameter in the application's windowed aggregation.. Kinesis Data Analytics (now part of Amazon Managed Service for Apache Flink) provides an `allowed_lateness` parameter in windowed aggregations to handle late-arriving events. Option A is incorrect because Kinesis Firehose is used for data delivery and buffering, not for handling late events within Kinesis Data Analytics; the application itself must manage lateness. Option B is incorrect because increasing shard count increases parallelism and throughput but does not affect how late events are treated; they will still be dropped unless a lateness tolerance is set. Option D is incorrect because `RecordColumn` count defines the schema of the input stream and has no impact on late-arriving event handling.

Answer

Configure a Kinesis Firehose delivery stream to buffer the late events.

Answer

Increase the shard count of the source Kinesis stream.

Answer

Increase the RecordColumn count in the input stream mapping.

Question 3

A data pipeline uses AWS Glue to read CSV files from an S3 bucket, transform them, and write Parquet back to S3. The pipeline runs daily and processes about 500 GB per run. The team wants to reduce costs without increasing runtime. Which approach is most effective?

Accepted Answer

Optimize the Glue script to select only required columns and filter rows early.. Optimizing the Glue script to use column pruning and predicate pushdown reduces the amount of data scanned, lowering costs without increasing runtime. Option A (pre-converting CSV to Parquet) adds an extra processing step and cost, making it less effective. Option B (enabling job bookmarks) tracks already processed data but does not reduce the cost per run. Option C (increasing DPUs) may actually increase costs and can cause unpredictable runtime.

Answer

Pre-convert the CSV files to Parquet in S3 using a separate process.

Answer

Enable job bookmarks to skip already processed data.

Answer

Increase the number of DPUs for the Glue job to improve parallelism.

Question 4

A company needs to ingest data from an on-premises Oracle database into Amazon S3 for analytics. The data volume is about 1 TB initially, with daily incremental updates of about 10 GB. Which TWO services can be combined to achieve this with minimal custom code?

Accepted Answer

Amazon S3. The correct combination is AWS DMS (Database Migration Service) for migration and Amazon S3 as the target. DMS can perform full load and ongoing replication with minimal custom code, making it suitable for the initial 1 TB load and daily 10 GB increments. AWS Glue (option A) could be used but often requires more custom code for change data capture (CDC) compared to DMS. Amazon Kinesis Data Streams (option B) is designed for real-time streaming data, not database migration. Amazon Athena (option C) is a query service, not a data movement service. Therefore, the correct answers are D (Amazon S3) and E (AWS DMS).

Answer

AWS Glue

Answer

Amazon Kinesis Data Streams

Answer

Amazon Athena

Question 5

A data engineer is designing a streaming pipeline using Amazon Kinesis Data Streams. The data must be transformed in real-time and then stored in Amazon S3 for long-term retention. Which THREE services can be used together to achieve this?

Accepted Answer

Amazon Kinesis Data Analytics. Options A, B, and E are correct: Kinesis Data Streams ingests data, Kinesis Data Analytics performs real-time transformations, and Kinesis Data Firehose delivers the transformed data to S3. Option C (AWS Glue) is a batch ETL service, not suitable for real-time streaming. Option D (Amazon Athena) is a query service, not for data ingestion or transformation in a streaming pipeline.

Answer

AWS Glue

Answer

Amazon Athena

Question 6

A data pipeline uses AWS Glue to process large CSV files. The team notices that some jobs fail with out-of-memory errors. Which TWO configuration changes can help mitigate this issue?

Accepted Answer

Increase the number of DPUs for the Glue job.. Options B and C are correct: increasing the number of DPUs provides more memory, and enabling autoscaling allows the job to automatically scale resources as needed. Option A (reducing DPUs) would worsen the problem by limiting resources. Option D (converting to Parquet) can improve performance but is not a direct configuration change for the Glue job itself. Option E (job bookmarks) is for incremental processing and does not affect memory.

Answer

Reduce the number of DPUs to limit concurrency.

Answer

Convert input files from CSV to Parquet.

Answer

Enable job bookmarks.

Question 7

Refer to the exhibit. A Glue ETL job failed. What is the most likely cause?

Accepted Answer

Some source files have a different schema than others.. The error 'Cannot merge incompatible schemas' indicates that some source files have a different schema than others, preventing the Glue ETL job from merging them. Option D is wrong because insufficient DPUs cause memory or performance errors, not schema conflicts. Option B is wrong because partition pruning is about filtering data, not schema mismatches. Option C is wrong because job bookmarks track processed files to avoid reprocessing, not schema issues.

Answer

The job bookmarks are misconfigured.

Answer

The source data has inconsistent partitioning.

Answer

The job ran out of memory due to insufficient DPUs.

Question 8

Refer to the exhibit. A Lambda function with this IAM policy is used to process records from a Kinesis stream and write to S3. The function is failing with access denied errors when writing to S3. What is the issue?

Accepted Answer

The S3 bucket name in the policy does not match the actual bucket used by the function.. The IAM policy grants s3:PutObject permission on the resource 'my-bucket/*', but the Lambda function is attempting to write to an S3 bucket with a different name. Since the bucket name in the policy does not match the actual bucket used by the function, access is denied. Option A is incorrect because the function does not need Kinesis Data Analytics for transformation. Option B is incorrect because the policy includes GetRecords and other Kinesis actions, so the Kinesis stream ARN is not the issue (if it were invalid, the error would occur when reading, not writing). Option C is incorrect because the policy explicitly allows Kinesis actions like GetRecords and DescribeStream, so the function does have permission to read from the stream.

Answer

The function needs to use Kinesis Data Analytics for transformation.

Answer

The Kinesis stream ARN is incorrect.

Answer

The Lambda function does not have permission to read from the Kinesis stream.

Question 9

Refer to the exhibit. A data engineer is troubleshooting a Glue job that reads objects from this S3 bucket. The job runs successfully but produces no output. The Glue catalog table points to the same S3 path. What is the most likely cause?

Accepted Answer

The S3 key does not follow Hive-style partitioning (e.g., year=2024/month=01).. AWS Glue Data Catalog relies on Hive-style partitioning (e.g., year=2024/month=01) to automatically discover and register partitions. The S3 key in the exhibit does not follow this pattern, so Glue cannot identify the partitions, resulting in no data being read even though the job runs. Option B is incorrect because object metadata size does not affect Glue's ability to read data. Option C is incorrect because the STANDARD storage class is fully supported by Glue. Option D is incorrect because ContentType is not a factor in Glue catalog table definitions.

Answer

The object metadata is too large.

Answer

The StorageClass is not supported by Glue.

Answer

The ContentType is not supported by Glue.

Question 10

A company wants to move data from an Amazon RDS for MySQL database to Amazon Redshift for analytics. The data needs to be refreshed daily. Which AWS service is best suited for this?

Accepted Answer

AWS Glue. (AWS Glue) is correct because it is a fully managed ETL service that can connect to RDS for MySQL as a source and Amazon Redshift as a target, and it supports scheduling for daily refreshes. Option A (AWS DMS) is designed for database migration and continuous replication, not for scheduled batch loads. Option C (Amazon EMR) is for big data processing using Hadoop/Spark, which is overkill for this simple transfer. Option D (Amazon Athena) is an interactive query service for data in Amazon S3, not for moving data between databases.

Answer

AWS Database Migration Service (DMS)

Answer

Amazon EMR

Answer

Amazon Athena

Question 11

A data pipeline ingests JSON data from an S3 bucket using AWS Glue. The JSON files contain nested structures, and the team wants to flatten them for analysis in Amazon Athena. Which Glue transformation is most appropriate?

Accepted Answer

Relationalize. Relationalize is specifically designed to flatten nested JSON into relational tables. Option A (Map) applies a function to each record. Option B (Filter) removes records. Option C (Join) combines datasets.

Answer

Filter

Answer

Join

Answer

Map

Question 12

A company uses Amazon Kinesis Data Firehose to ingest data into an S3 bucket. The data is in JSON format and the team wants to convert it to Parquet before storage. Which TWO configurations are required?

Accepted Answer

Create a Glue table with the schema of the data.. Amazon Kinesis Data Firehose can automatically convert JSON data to Parquet format before delivery to S3. To enable this, you must: (1) Enable data format conversion in the Firehose delivery stream settings and set the output format to Parquet (Option E). (2) Provide a schema for the data, which is done by referencing an AWS Glue table that defines the schema (Option B). Option A is incorrect because Kinesis Data Analytics is not required; Firehose handles conversion natively. Option C (Lambda) is a valid way to transform data but is not required and not listed as a configuration for this purpose. Option D (Athena) is for querying data already in S3, not for conversion during ingestion.

Answer

Use Kinesis Data Analytics to transform data to Parquet.

Answer

Configure a Lambda function to convert data on the fly.

Answer

Set up an Athena table to read the data.

Question 13

A data engineering team is designing a batch processing workflow using AWS Glue. The job reads from an S3 bucket, transforms data, and writes to another S3 bucket. The job runs daily and processes new data incrementally. Which THREE features should they use to optimize performance and cost?

Accepted Answer

Enable Glue job autoscaling.. Options B, D, and E are correct. Glue job autoscaling (B) dynamically adjusts resources, optimizing cost and performance. Predicate pushdown and column pruning (D) reduce data scanned, improving efficiency. Job bookmarks (E) enable incremental processing, avoiding reprocessing of old data. Option A (converting to Parquet) is a good practice but not a Glue feature. Option C (manually increasing DPUs) is not optimal because it lacks flexibility and may waste resources.

Answer

Convert all input data to Parquet format before processing.

Answer

Manually increase the number of DPUs for each run.

Question 14

A company wants to ingest data from thousands of IoT devices into AWS for real-time analytics. The data is in JSON format and each device sends about 1 KB every second. Which service should be used as the primary ingestion point?

Accepted Answer

Amazon Kinesis Data Streams. Amazon Kinesis Data Streams (D) is the correct choice because it is designed for real-time streaming of large amounts of data from many producers, such as thousands of IoT devices. Each device sends 1 KB per second, resulting in ~1 MB/s total throughput, which Kinesis Data Streams can handle with sharding. It supports multiple consumers for real-time analytics. AWS IoT Core (A) is for device management and MQTT messaging, not a general-purpose ingestion point for analytics. Kinesis Data Firehose (B) is for loading streaming data into storage, but it does not support multiple real-time consumers and has a minimum 60-second buffer latency. Amazon SQS (C) is a message queue for decoupled applications, not built for high-throughput streaming analytics.

Answer

AWS IoT Core

Answer

Amazon Kinesis Data Firehose

Answer

Amazon SQS

Question 15

A data engineer runs an AWS Glue job that reads from a JDBC connection to a PostgreSQL database. The job fails with a 'Connection timed out' error. The Glue job runs in a VPC with the appropriate security group. What is the most likely cause?

Accepted Answer

The network ACL associated with the Glue job's subnet is blocking outbound traffic.. The 'Connection timed out' error indicates a network connectivity issue. Since the Glue job runs in a VPC with a security group that likely allows outbound traffic, the most likely cause is that the network ACL (which is stateless) associated with the Glue job's subnet is blocking outbound traffic to the database. Option A is correct. Option B is incorrect because the error is not an authentication or permission issue. Option C is incorrect because the security group's inbound rule does not affect outbound traffic from the Glue job. Option D is incorrect because the error is network-related, not a credentials issue.

Answer

The Glue job does not have permission to access the database.

Answer

The security group does not allow inbound traffic from the Glue job.

Answer

The database credentials are incorrect.