DEA-C01 Practice Test 40 — 15 Questions

Question 1

A data engineer is designing a data pipeline that ingests data from an on-premises system into Amazon S3 using AWS Transfer Family. The data must be encrypted at rest using a customer-managed key in AWS KMS. The S3 bucket policy must allow only encrypted connections. Which policy condition should be used?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

aws:SecureTransport. The correct condition is 'aws:SecureTransport' because it ensures that requests to the S3 bucket are made over HTTPS (TLS), thereby enforcing encrypted connections. This condition key is a boolean that indicates whether the request used SSL/TLS. Setting it to 'true' in a bucket policy denies any non-encrypted (HTTP) requests.

Answer

kms:EncryptionContext

Answer

s3:x-amz-server-side-encryption-aws-kms-key-id

Answer

s3:x-amz-server-side-encryption

Question 2

A data engineer is using AWS Glue to transform data from Amazon S3 and load it into Amazon Redshift. The job runs daily and processes only new data added since the last run. The engineer notices that the job is reprocessing old data, increasing runtime and cost. Which AWS Glue feature should be enabled to ensure only new data is processed?

Accepted Answer

Job bookmarks. Job bookmarks are the AWS Glue feature designed to track processed data and enable incremental processing. When enabled, the job reads only new data based on S3 object metadata. This reduces runtime and cost by avoiding reprocessing. The other options do not provide this capability: DataBrew is for data preparation, crawler incremental crawl updates the catalog, and S3 event notifications are for triggering actions.

Answer

AWS Glue DataBrew

Answer

Amazon S3 event notifications

Answer

AWS Glue Crawler incremental crawl

Question 3

A data engineer maintains an AWS Glue ETL job that reads JSON files from Amazon S3, applies transformations using a DynamicFrame, and writes Parquet to another S3 bucket. The job currently reads the entire source prefix on every run, but the source is now partitioned by year/month/day and only new partitions need processing. The engineer wants to process only new partitions without changing the job's transformation logic. Which change should the engineer make?

Accepted Answer

Enable job bookmarks on the Glue job so it tracks previously processed S3 objects and skips them on subsequent runs. Job bookmarks are the native Glue mechanism for incremental processing. When enabled, Glue tracks the state of previously processed S3 objects and partitions, so each run reads only new data. This meets the requirement to process only new partitions without rewriting the transformation logic or manually specifying dates.

Answer

Create a Glue crawler that catalogs only the new partitions and point the job at the crawler's output table

Answer

Configure the job to use a smaller number of DPUs so each run scans less data

Answer

Add a pushdown predicate to the DynamicFrame that filters on the year, month, and day partition columns using hardcoded values

Question 4

A data engineer is using AWS Glue DataBrew to clean a dataset stored in Amazon S3. The dataset contains a column with inconsistent date formats (for example, '2023-01-15', '01/15/2023', and '15-Jan-2023'). The engineer needs to standardize all values to ISO 8601 format and then write the cleaned data back to S3. Which approach should the engineer use?

Accepted Answer

Use the 'Standardize date' transform in DataBrew, select the column, and choose the target format as ISO 8601.. AWS Glue DataBrew includes a 'Standardize date' transform that parses multiple date formats and converts them to a specified target format such as ISO 8601. This transform is designed for exactly this scenario, where a column contains mixed date representations. Using it avoids custom code and ensures consistent, repeatable cleaning across the dataset.

Answer

Use the 'Format' transform in DataBrew to change the column data type to date.

Answer

Use the 'Replace value' transform to manually replace each non-ISO date string with its ISO equivalent.

Answer

Write a custom Python function in a DataBrew recipe step using the 'Custom transform' option.

Question 5

A data engineer is designing a streaming ingestion pipeline using Amazon Kinesis Data Streams. The stream receives records from thousands of IoT devices, and the engineer must ensure that records from the same device are processed in order. The engineer also needs to scale the stream to handle peak loads without manual intervention. Which two actions should the engineer take? (Choose two.)

Accepted Answer

Use the device ID as the partition key when putting records into the stream.. To ensure per-device ordering, records from the same device must go to the same shard, which is achieved by using the device ID as the partition key. To handle peak loads without manual intervention, the stream should use on-demand capacity mode, which automatically scales shards. Together, these actions meet both ordering and scaling requirements for the IoT streaming pipeline.

Answer

Enable enhanced fan-out for consumers to increase read throughput.

Answer

Use a random partition key to evenly distribute records across shards.

Answer

Increase the retention period of the stream to 365 days.

Question 6

A data engineer is using AWS Glue DataBrew to profile a dataset stored in Amazon S3. The profile shows that a column named country contains values such as 'US', 'usa', 'United States', and 'U.S.A.' The engineer needs to standardize these values to a single canonical form before loading the data into Amazon Redshift. Which DataBrew transformation should the engineer apply?

Accepted Answer

Use the 'Replace value or pattern' transform to replace each variant with 'United States'.. Standardizing a column with several representations of the same entity requires explicit value mapping, which the 'Replace value or pattern' transform provides. Case conversion, punctuation removal, and duplicate flagging each address only part of the problem or change nothing in the stored values. The replace transform is the only option that yields one canonical value for all variants.

Answer

Use the 'Remove special characters' transform to strip punctuation.

Answer

Use the 'Change case' transform to convert all values to uppercase.

Answer

Use the 'Flag duplicate values' transform to identify repeated entries.

Question 7

A data engineer is using AWS Glue to transform data from Amazon S3 and load it into Amazon Redshift. The job uses the write_dynamic_frame.from_jdbc_conf method. The engineer notices that the job is slow and sometimes fails due to connection timeouts. Which action should the engineer take to improve performance and reliability?

Accepted Answer

Use the Amazon Redshift COPY command via a staging area in Amazon S3 instead of JDBC.. For large data loads into Amazon Redshift, the COPY command is significantly faster and more reliable than JDBC. AWS Glue can write transformed data to Amazon S3 and then invoke the COPY command to load it into Redshift. This approach leverages Redshift's parallel processing, reduces connection overhead, and avoids timeouts. Other options do not address the fundamental performance issue.

Answer

Increase the number of DPUs for the Glue job to provide more parallelism.

Answer

Enable job bookmarks to avoid reprocessing previously loaded data.

Answer

Increase the JDBC connection timeout and retry settings in the Glue job script.

Question 8

A data engineer runs an AWS Glue ETL job that reads semi-structured JSON from Amazon S3, flattens nested arrays, and writes Parquet to a partitioned S3 location. Job runs are becoming expensive because Glue reprocesses all historical partitions on every run. The engineer wants subsequent runs to process only newly arrived data. Which approach should the engineer take with the LEAST operational overhead?

Accepted Answer

Create an AWS Glue job bookmark and enable it on the S3 data source, then let the job read only new objects on each run.. Glue job bookmarks maintain run state and cause the reader to process only new or changed S3 objects since the last successful run. This eliminates repeated scanning of historical partitions with no custom code, unlike runtime filters or schedule changes, which still read all objects. Moving data to RDS is a much heavier architectural change than needed.

Answer

Schedule the job with a shorter interval so each run has less data to process.

Answer

Move the source data to an Amazon RDS table and use a Glue JDBC connection with a watermark column.

Answer

Add a WHERE clause to the DynamicFrame that filters records by the current date at runtime.

Question 9

A data engineer is using AWS Glue Studio to create a visual ETL job that reads from an Amazon S3 bucket containing JSON files, applies a filter transformation, and writes the output to Amazon Redshift. The job must run daily. The engineer notices that the job is taking a long time to complete and wants to improve performance. Which action should the engineer take to optimize the job?

Accepted Answer

Convert the source JSON files to Parquet format before running the ETL job.. Converting JSON to Parquet improves ETL performance because Parquet is columnar, compressed, and requires less I/O and CPU to parse. AWS Glue jobs benefit significantly from columnar formats when reading from S3. While increasing DPUs or enabling bookmarks can help in some cases, the most direct and effective optimization for slow JSON processing is to use a more efficient file format.

Answer

Use a larger number of smaller files instead of fewer large files.

Answer

Increase the number of AWS Glue DPUs allocated to the job.

Answer

Enable job bookmarks to track processed files and avoid reprocessing.

Question 10

A data engineer is configuring an AWS Glue crawler to catalog data stored in Amazon S3. The data is organized as Parquet files under prefixes named by year, month, and day, such as s3://analytics/events/year=2024/month=05/day=17/. Queries in Amazon Athena must use partition pruning to limit scanned data. Which crawler configuration should the engineer choose?

Accepted Answer

Create a crawler with the S3 path pointing to s3://analytics/events/ and enable 'Update all new and existing partitions with metadata from the table' so partition metadata stays current.. Hive-style prefixes such as year=/month=/day= are recognized as partition keys when an AWS Glue crawler targets the parent S3 prefix. To keep the catalog current as new date prefixes appear, the crawler should be configured to update all new and existing partitions with metadata from the table. Athena then reads the partition keys from the Data Catalog and prunes to only the relevant prefixes, reducing scanned bytes and query cost.

Answer

Create a crawler for each individual day prefix, such as s3://analytics/events/year=2024/month=05/day=17/, so each day becomes its own table.

Answer

Create a crawler with the S3 path pointing to s3://analytics/events/ and set the 'Schema change policy' to 'Delete tables and columns' so stale partitions are removed.

Answer

Create a crawler with the S3 path pointing to s3://analytics/events/ and enable the 'Add new columns only' option so new partitions are detected automatically.

Question 11

A data engineer needs to transform data in Amazon S3 using SQL statements without managing any infrastructure. The transformations are simple projections and filters, and the engineer wants the results written back to S3 in Parquet. Which AWS service should be used?

Accepted Answer

Amazon Athena with a CREATE TABLE AS SELECT statement. Amazon Athena is fully serverless and executes SQL directly against S3 data. A CREATE TABLE AS SELECT statement reads the source, applies projections and filters, and writes the result to S3 in a chosen format like Parquet. This satisfies the SQL and no-infrastructure requirements without cluster or job management.

Answer

AWS Glue ETL with a PySpark script

Answer

Amazon EMR with Hive on a transient cluster

Answer

Amazon Redshift Spectrum with an external schema

Question 12

A data engineering team needs to ingest streaming data from an application into Amazon S3 for analytics. The data volume is moderate and the team wants the lowest operational overhead. Which AWS service should they use?

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is a fully managed service for loading streaming data into S3 with no code required and minimal operational overhead. Option A is incorrect because Amazon SQS is a message queue service, not designed for streaming data ingestion into S3. Option B is incorrect because AWS Glue is primarily a batch ETL service, not suitable for real-time streaming. Option C is incorrect because Amazon Kinesis Data Streams requires custom consumers and more management, increasing operational overhead.

Answer

Amazon SQS

Answer

AWS Glue

Answer

Amazon Kinesis Data Streams

Question 13

A company uses AWS Lake Formation to manage data lake permissions. The data engineer notices that a user with SELECT permission on a table can also query the underlying data in Amazon S3 directly. How can the engineer enforce that access to the S3 data is only through Lake Formation?

Accepted Answer

Grant the user permissions only through Lake Formation and remove any IAM policies that allow direct S3 access to the data location. Lake Formation permissions are enforced only when the caller accesses data through a Lake Formation-integrated engine (Athena, Redshift Spectrum, EMR, Glue). If the user also has IAM permissions on the underlying S3 path, they can bypass Lake Formation entirely by reading S3 directly. The fix is to remove those direct S3 IAM permissions so the only path to the data is through Lake Formation-governed services.

Answer

Use S3 Access Points with a policy that restricts access to only Lake Formation

Answer

Enable S3 Block Public Access on the bucket

Answer

Change the S3 bucket policy to deny all access except from Lake Formation

Question 14

A data engineer needs to ingest data from an Amazon Kinesis Data Stream into an Amazon S3 bucket for long-term storage. The data is in JSON format and must be delivered within 60 seconds of arrival. The engineer wants a fully managed solution that requires minimal code. Which service should the engineer use?

Accepted Answer

Amazon Kinesis Data Firehose with an Amazon S3 destination.. Amazon Kinesis Data Firehose is the fully managed service designed to deliver streaming data from Kinesis Data Streams to destinations like Amazon S3. It requires no code, can buffer for up to 60 seconds, and handles scaling automatically. The other options involve custom code or are not suited for near-real-time streaming ingestion.

Answer

AWS Glue ETL job scheduled to run every minute to pull from Kinesis.

Answer

AWS Lambda to read from the Kinesis stream and write to Amazon S3.

Answer

Amazon Kinesis Client Library (KCL) application running on Amazon EC2.

Question 15

A company wants to schedule a nightly batch job to copy data from an on-premises PostgreSQL database to Amazon S3. The solution must minimize operational overhead. Which AWS service should be used?

Accepted Answer

AWS Glue. AWS Glue is the correct choice because it provides a fully managed ETL service that can connect to on-premises PostgreSQL via JDBC, extract data, and write it to Amazon S3 with minimal configuration. Glue's built-in scheduler can run the job nightly, eliminating the need to manage servers or orchestration infrastructure, which directly meets the requirement to minimize operational overhead.

Answer

AWS Data Pipeline

Answer

Amazon EMR

Answer

AWS Database Migration Service (AWS DMS) with ongoing replication