DEA-C01 Data Ingestion and Transformation Practice Test 17 — 15 Questions

Question 1

A company uses AWS Glue ETL jobs to transform data in Amazon S3. The data arrives in JSON format but needs to be converted to Parquet for efficient querying. Which AWS Glue feature should be used to infer the schema and generate transformation code?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

AWS Glue crawlers. AWS Glue crawlers are the correct feature because they automatically connect to data stores (like S3), infer the schema of JSON data by sampling it, and populate the AWS Glue Data Catalog with table definitions. This catalog schema can then be used by AWS Glue ETL jobs to generate transformation code (e.g., converting JSON to Parquet) without manual schema definition.

Answer

Amazon S3 Select

Answer

Amazon Athena

Answer

Amazon Kinesis Data Analytics

Question 2

A data engineer is ingesting streaming data from an IoT fleet into Amazon Kinesis Data Streams. The data must be transformed in real-time and loaded into an Amazon Redshift cluster. Which solution minimizes operational overhead?

Accepted Answer

Use Kinesis Data Firehose with a Lambda transformation function. Kinesis Data Firehose is the fully managed service for loading streaming data into Redshift with near-real-time latency. By attaching a Lambda transformation function, you can perform lightweight data transformations (e.g., JSON flattening, field masking) without managing any compute infrastructure. This combination eliminates the need to provision or tune any servers, clusters, or consumer applications, minimizing operational overhead.

Answer

Use AWS Glue ETL jobs running continuously

Answer

Use Kinesis Client Library (KCL) to consume and transform data, then write to Redshift using COPY

Answer

Use AWS Direct Connect to stream data directly into Redshift

Question 3

A company ingests clickstream data into Amazon S3 via Kinesis Data Firehose. The data arrives in 20 MB files every 2 minutes. The data engineering team needs to transform nested JSON into a flat structure before loading into Amazon Redshift. Which approach is most cost-effective and scalable?

Accepted Answer

Create an AWS Glue ETL job that runs on a schedule, using dynamic frames to flatten the data and write to S3 in Parquet. AWS Glue ETL jobs are designed for serverless, scalable data transformation, and using dynamic frames to flatten nested JSON and write to Parquet is both cost-effective (pay per DPU-hour) and scalable (automatically handles data volume). The 20 MB files arriving every 2 minutes are well-suited for Glue's batch processing, and Parquet output optimizes Redshift loading via COPY commands.

Answer

Run an Amazon EMR cluster with Spark to flatten the data and write back to S3

Answer

Use AWS Lambda to transform each file as it arrives in S3

Answer

Use Amazon Redshift Spectrum to query the nested JSON directly and create a view

Question 4

A data engineer needs to capture change data capture (CDC) events from an Amazon RDS for PostgreSQL database and stream them to Amazon S3 in near real-time. Which AWS service should be used?

Accepted Answer

AWS Database Migration Service (AWS DMS). AWS DMS supports continuous replication from PostgreSQL source databases using logical replication slots to capture CDC events in near real-time. It can directly stream these changes to Amazon S3 as a target, making it the correct choice for this use case.

Answer

Amazon S3 Transfer Acceleration

Answer

Amazon Athena

Answer

Amazon Kinesis Data Streams

Question 5

A company uses AWS Glue to process data in Amazon S3. The Glue job fails with an error indicating that the partition keys in the catalog do not match the actual S3 partition structure. What is the most likely cause?

Accepted Answer

The Glue Data Catalog partition metadata is outdated after the S3 structure changed. The Glue Data Catalog stores partition metadata separately from the actual S3 partition layout. When the S3 partition structure changes (e.g., new partitions are added or existing ones are renamed) without updating the catalog, the Glue job reads stale partition metadata, leading to a mismatch error. The job fails because it expects partitions based on the catalog, not the live S3 structure.

Answer

The IAM role does not have permissions to read the S3 data

Answer

The data files are encrypted with SSE-KMS

Answer

The table name in the catalog is different from the one used in the job

Question 6

A data engineer is designing a streaming pipeline using Amazon Kinesis Data Streams with a shard count of 10. The incoming data rate is 1 MB/second. The consuming application uses the Kinesis Client Library (KCL) with a single worker. What is the most likely performance bottleneck?

Accepted Answer

The single KCL worker cannot process all shards in parallel. The Kinesis Client Library (KCL) uses a 1:1 mapping between shards and record processors by default. With 10 shards and only a single KCL worker, that worker must run all 10 record processors sequentially on a single host, creating a bottleneck. The worker cannot process records from multiple shards in parallel, so the throughput is limited by the single worker's processing capacity, not the stream's write capacity.

Answer

The Lambda function invoked by the stream has a cold start issue

Answer

The data stream has insufficient write capacity

Answer

The shard count is too low to handle the data rate

Question 7

A company needs to ingest data from multiple on-premises databases into Amazon S3 for analytics. The databases include Oracle, MySQL, and PostgreSQL. The data must be continuously replicated with minimal latency. Which AWS service should be used?

Accepted Answer

AWS Database Migration Service (AWS DMS). AWS DMS supports continuous replication (change data capture, CDC) from Oracle, MySQL, and PostgreSQL to S3 as a target, enabling near-real-time data ingestion with minimal latency. It handles schema conversion and can replicate ongoing changes without interrupting source databases, making it the correct choice for this use case.

Answer

Amazon Kinesis Data Streams

Answer

AWS Snowball

Answer

AWS Glue

Question 8

A data engineer is building a data pipeline that ingests data from Amazon S3 into Amazon Redshift. The data is in CSV format and includes a timestamp column. The pipeline should load only new data incrementally. Which approach is most efficient?

Accepted Answer

Use the COPY command with a manifest file that lists only the new S3 objects. Using the COPY command with a manifest file allows you to explicitly list only the new S3 objects to be loaded, enabling incremental loading without scanning or loading the entire bucket. This approach is efficient as it avoids the overhead of deduplication or full-bucket scans, and it leverages Redshift's native high-speed parallel ingestion from S3.

Answer

Use the COPY command to load the entire bucket and rely on Redshift to deduplicate

Answer

Use Amazon Redshift Spectrum to query the S3 data directly without loading

Answer

Use INSERT statements within a loop to load each new file

Question 9

A company uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The data is compressed with GZIP and partitioned by year, month, day, and hour. The delivery stream is configured to buffer up to 5 MB or 60 seconds. Some records are missing from S3. What is the most likely cause?

Accepted Answer

The Lambda transformation function timed out and records were skipped. When a Lambda transformation function times out, Kinesis Data Firehose will skip the affected records by default. The delivery stream configuration (5 MB or 60 seconds) only controls buffering, not Lambda invocation failures. If the Lambda function exceeds its timeout limit, Firehose treats the invocation as failed and, depending on the error handling configuration, may drop the records entirely, leading to missing data in S3.

Answer

The S3 bucket does not have sufficient permissions

Answer

The data compression format is incompatible with S3

Answer

The partition key configuration is incorrect

Question 10

Which TWO AWS services can be used to ingest streaming data into Amazon S3? (Choose two.)

Accepted Answer

Amazon Managed Streaming for Apache Kafka (Amazon MSK). Amazon Kinesis Data Firehose is the easiest way to reliably load streaming data into Amazon S3. It can capture, transform, and deliver streaming data to S3 destinations in near real-time with no code required. Amazon MSK (Managed Streaming for Apache Kafka) can also ingest streaming data into S3 by using Kafka Connect with an S3 sink connector, which writes data from Kafka topics directly to S3.

Answer

Amazon S3 Transfer Acceleration

Answer

Amazon Elastic Block Store (Amazon EBS)

Answer

AWS Snowball

Question 11

Which TWO practices improve the performance of AWS Glue ETL jobs? (Choose two.)

Accepted Answer

Use pushdown predicates to filter data at the source. Pushdown predicates (Option A) improve AWS Glue ETL performance by filtering data at the source before it is read into the job. This reduces the volume of data transferred and processed, which is especially effective when using formats like Parquet or ORC that support predicate pushdown natively. By applying filters early, Glue avoids scanning unnecessary partitions or rows, leading to faster execution and lower costs.

Answer

Increase the number of DPUs to the maximum allowed

Answer

Use the smallest possible file size for input data

Answer

Enable AWS Glue job metrics and debug logging

Question 12

Which THREE factors should be considered when choosing between Amazon Kinesis Data Streams and Amazon Kinesis Data Firehose for real-time data ingestion? (Choose three.)

Accepted Answer

The need for a fully managed delivery destination. Amazon Kinesis Data Firehose is a fully managed service that automatically delivers streaming data to destinations like Amazon S3, Amazon Redshift, Amazon OpenSearch Service, and Splunk, making it ideal when you need a fully managed delivery destination without managing the ingestion pipeline. In contrast, Amazon Kinesis Data Streams requires you to build and manage consumers to process and deliver data, so if you need a fully managed destination, Firehose is the correct choice.

Answer

The ability to compress data before storage

Answer

The maximum throughput supported per shard

Question 13

Refer to the exhibit. A data engineer has attached this IAM policy to an AWS Glue job role. The Glue job fails when trying to write transformed data to an S3 bucket located in a different AWS account. What is the most likely reason?

Accepted Answer

The policy does not grant s3:ListBucket, and the bucket policy may not allow cross-account access. The IAM policy shown does not include the s3:ListBucket permission, which is required for the Glue job to list objects in the S3 bucket before writing. Additionally, cross-account access requires both the source account's IAM policy (this one) to grant write permissions and the target account's S3 bucket policy to explicitly allow the source account's role, which may not be configured. Without s3:ListBucket, the Glue job cannot verify the bucket's existence or structure, causing the write operation to fail.

Answer

The policy does not allow lambda:InvokeAsync

Answer

The Glue job role does not have permissions to write to S3

Answer

The policy does not include kinesis:DescribeStream

Question 14

Refer to the exhibit. A data engineer runs the AWS CLI command and observes the output. The stream has two shards. A producer sends a record with a partition key that hashes to 150000000000000000000000000000000000000. To which shard will the record be written?

Accepted Answer

shardId-000000000001. The record will be written to shardId-000000000001 because the hash key range for shardId-000000000000 is [0, 170141183460469231731687303715884105727) and for shardId-000000000001 is [170141183460469231731687303715884105727, 340282366920938463463374607431768211455). The partition key hash value of 150000000000000000000000000000000000000 falls within the second shard's range, so the record is routed to shardId-000000000001.

Answer

The record will be rejected because it does not match any shard

Answer

shardId-000000000000

Answer

The record will be written to both shards

Question 15

Refer to the exhibit. A data engineer creates an AWS Glue job using this CloudFormation template. The job processes new data files in S3 and uses job bookmarks to track processed files. After initial success, the job runs again but processes all files again instead of only new ones. What is the most likely cause?

Accepted Answer

The S3 input path does not have a partitioning scheme or timestamp to identify new files. AWS Glue job bookmarks rely on the structure of the input data to identify new files. Without a partitioning scheme or a timestamp-based naming convention in the S3 path, Glue cannot determine which files are new; it falls back to reprocessing all files. The job bookmark feature tracks processed files by examining the S3 path and file metadata, so a flat or non-partitioned structure prevents it from distinguishing new files from old ones.

Answer

The job bookmark option is set to 'job-bookmark-disable'

Answer

The enable-metrics parameter is set to true

Answer

The MaxRetries parameter is set to 0