DEA-C01 Data Ingestion and Transformation Practice Test 29 — 15 Questions

Question 1

A company is ingesting streaming data from social media feeds using Amazon Kinesis Data Streams. The data volume peaks at 10,000 records per second, and each record is up to 1 KB. The company needs to archive the raw data in Amazon S3 in near real-time and also make it available for real-time analytics using Amazon Kinesis Data Analytics. What is the MOST efficient architecture to meet these requirements?

Accepted Answer

Use Kinesis Data Streams as the ingestion point. Use Kinesis Data Firehose to read from the stream and write to S3. Use Kinesis Data Analytics to read directly from the same stream.. Option D is correct because Kinesis Data Firehose can read from a Kinesis Data Stream and write to S3, while Kinesis Data Analytics can read from the same stream for real-time analytics. Option A is incorrect because Lambda adds cost and complexity. Option B is incorrect because two separate streams are wasteful. Option C is incorrect because Firehose cannot convert to Parquet without a schema; also Lambda is avoidable.

Answer

Use Kinesis Data Streams as the ingestion point. Use Kinesis Data Firehose to read from the stream, convert to Parquet, and write to S3. Use a Lambda function to send data to Kinesis Data Analytics.

Answer

Use Kinesis Data Streams as the ingestion point. Use a Lambda function to read from the stream, write to S3, and send data to Kinesis Data Analytics.

Answer

Use two Kinesis Data Streams: one for S3 delivery and one for Kinesis Data Analytics.

Question 2

A data engineer needs to design a data ingestion pipeline that ingests CSV files from an Amazon S3 bucket, transforms the data by adding a timestamp column, and loads it into an Amazon Redshift table. The pipeline should run automatically whenever a new file is uploaded to the S3 bucket. Which AWS service should be used to trigger the transformation?

Accepted Answer

AWS Lambda. Option B is correct because S3 events can trigger a Lambda function directly. Option A is incorrect because SQS is not triggered by S3 events without a separate setup. Option C is incorrect because EventBridge cannot directly trigger Lambda from S3 events; S3 events go to Lambda via S3 notification. Option D is incorrect because Step Functions would need an event source; Lambda is simpler.

Answer

AWS Step Functions

Answer

Amazon EventBridge

Answer

Amazon Simple Queue Service (SQS)

Question 3

A company uses Amazon Kinesis Data Firehose to deliver data to an Amazon S3 bucket. The data is in JSON format and contains a 'timestamp' field with a Unix epoch value. The company wants to partition the S3 objects by year, month, day, and hour based on the timestamp. What is the MOST efficient method to achieve this?

Accepted Answer

Use the dynamic partitioning feature of Kinesis Data Firehose with inline parsing to extract the timestamp and create the S3 prefix.. Option A is correct because Firehose supports dynamic partitioning using inline parsing or Lambda to extract the timestamp. Option B is incorrect because Glue ETL would add latency. Option C is incorrect because custom partitioning is not a Firehose feature; dynamic partitioning is. Option D is incorrect because Athena would need a Lambda function to partition, not efficient.

Answer

Configure a custom S3 prefix in Firehose using the 'YYYY/MM/dd/HH' format based on the current time.

Answer

Use an AWS Glue ETL job to read from Firehose, partition, and write to S3.

Answer

Use Amazon Athena to run a CTAS query that partitions the data by timestamp.

Question 4

A company needs to ingest data from a relational database into Amazon S3 for analytics. The database is an Amazon RDS MySQL instance. Which AWS service should be used for a one-time historical data load?

Accepted Answer

AWS Database Migration Service (DMS). Option C is correct because AWS Database Migration Service (DMS) can perform a one-time full load from RDS MySQL to S3. Option A is incorrect because Glue ETL can do it but DMS is specialized for databases. Option B is incorrect because Kinesis Data Firehose is for streaming, not database. Option D is incorrect because Athena queries data, not ingest.

Answer

AWS Glue ETL

Answer

Amazon Athena

Answer

Amazon Kinesis Data Firehose

Question 5

A company is using Amazon Kinesis Data Streams with a Lambda consumer to process clickstream data. The data rate is high and the Lambda function is falling behind, resulting in increased processing latency. What is the MOST effective way to improve throughput?

Accepted Answer

Increase the number of shards in the Kinesis stream.. Option C is correct because increasing the number of shards increases the stream's capacity and allows more concurrent Lambda invocations. Option A is incorrect because Lambda concurrency might be a limit but the root cause is shard count. Option B is incorrect because increasing Lambda memory may help but not as much as more shards. Option D is incorrect because Firehose would add a separate step.

Answer

Increase the memory allocated to the Lambda function.

Answer

Increase the Lambda function timeout.

Answer

Use Kinesis Data Firehose instead of Lambda.

Question 6

A data engineer is designing a data ingestion pipeline for JSON files landing in an Amazon S3 bucket. The pipeline must transform the data (e.g., flatten nested structures) and load it into Amazon Redshift. The transformation logic is complex and may evolve frequently. Which approach provides the MOST flexibility and ease of maintenance?

Accepted Answer

Use AWS Glue ETL jobs to transform the data and load into Redshift.. Option A is correct because Glue ETL (Apache Spark) provides a flexible, code-based environment for complex transformations. Option B is incorrect because Athena is for querying, not ETL. Option C is incorrect because Lambda is limited by execution time and memory for large files. Option D is incorrect because Redshift COPY does not transform.

Answer

Use AWS Lambda functions to transform each file and load into Redshift.

Answer

Use the Amazon Redshift COPY command to load raw JSON directly.

Answer

Use Amazon Athena to query the raw data and insert into Redshift.

Question 7

A company needs to ingest streaming data from multiple sources and store it in Amazon S3. The data volume is up to 5 GB per hour. What is the MOST cost-effective ingestion service?

Accepted Answer

Amazon Kinesis Data Firehose. Option C is correct because Kinesis Data Firehose is a fully managed service that scales automatically and charges based on data volume. Option A is incorrect because Kinesis Data Streams charges per shard, which can be more expensive. Option B is incorrect because Lambda is not a streaming ingestion service. Option D is incorrect because Glue is for ETL, not real-time ingestion.

Answer

AWS Glue

Answer

Amazon Kinesis Data Streams

Answer

AWS Lambda

Question 8

A company is using AWS Glue ETL to transform and load data from Amazon S3 to Amazon Redshift. The data engineer notices that the job is taking longer than expected. Which TWO actions can improve the job performance?

Accepted Answer

Partition the source data in S3.. Options B and D are correct because increasing the number of DPUs and using partitioning in S3 improve parallelism and reduce data scanned. Option A is incorrect because S3 Transfer Acceleration is for uploads, not ETL. Option C is incorrect because Redshift Spectrum is for querying, not Glue ETL. Option E is incorrect because a larger instance type for Redshift does not affect Glue job performance.

Answer

Use Amazon Redshift Spectrum to query data directly.

Answer

Enable S3 Transfer Acceleration.

Answer

Use a larger Redshift node type.

Question 9

A company needs to ingest data from a MySQL database into Amazon S3 using AWS DMS. The data changes frequently and the requirement is to capture changes in near real-time. Which THREE configurations are necessary?

Accepted Answer

Create an S3 target endpoint in DMS.. Options A, B, and E are correct because the database must have binary logging enabled, DMS requires a replication instance, and the target endpoint should be S3. Option C is incorrect because S3 events are not needed for DMS. Option D is incorrect because a VPC endpoint is not required if using public S3.

Answer

Create a VPC endpoint for S3.

Answer

Configure an S3 event notification to trigger DMS.

Question 10

A company is building a data lake on Amazon S3 and needs to ingest data from various on-premises sources. Which TWO AWS services can be used to transfer data securely over the internet?

Accepted Answer

AWS DataSync. Options A and C are correct because AWS DataSync can transfer data over the internet, and the AWS CLI can be used for uploads. Option B is incorrect because Direct Connect is a physical connection. Option D is incorrect because Snowcone is a physical device. Option E is incorrect because Kinesis Data Firehose is for streaming, not file transfer.

Answer

AWS Snowcone

Answer

Amazon Kinesis Data Firehose

Answer

AWS Direct Connect

Question 11

A company is ingesting streaming data from IoT devices into Amazon Kinesis Data Streams. The data must be transformed in real-time using custom Python code before being stored in Amazon S3. Which AWS service should be used to perform this transformation?

Accepted Answer

Kinesis Data Analytics for Apache Flink. Option C is correct because Kinesis Data Analytics for Apache Flink allows running custom Apache Flink applications for real-time data processing. Option A (Kinesis Data Firehose) does not support custom Python code; it uses built-in transformations. Option B (AWS Lambda) can be used but requires custom code and is typically used for simple transformations. Option D (Amazon EMR) is for large-scale batch processing, not real-time.

Answer

Amazon EMR with Spark Streaming

Answer

AWS Lambda function triggered by Kinesis Data Streams

Answer

Kinesis Data Firehose with custom data transformation

Question 12

A data engineer needs to ingest data from an on-premises Oracle database into Amazon S3 using AWS DMS. The change data capture (CDC) must be enabled to capture ongoing changes. Which additional AWS service is required to store the transaction logs for CDC?

Accepted Answer

Amazon S3. Option A is correct because AWS DMS CDC requires Oracle transaction logs to be stored in S3 or a custom S3 bucket for replication. Option B (CloudWatch Logs) is for monitoring, not storage. Option C (Amazon RDS) is a database, not storage. Option D (Amazon EBS) is block storage attached to EC2, not suitable for DMS CDC.

Answer

Amazon RDS

Answer

Amazon EBS

Answer

Amazon CloudWatch Logs

Question 13

A company wants to ingest data from multiple SaaS applications into Amazon S3 using a fully managed service that supports schema discovery and transformation. Which AWS service should they use?

Accepted Answer

Amazon AppFlow. Option B is correct because Amazon AppFlow is a fully managed integration service that supports SaaS sources, schema discovery, and data transformation. Option A (AWS Glue) is for ETL but not for SaaS ingestion directly. Option C (Amazon Kinesis) is for streaming data. Option D (AWS Data Pipeline) is not fully managed for SaaS.

Answer

Amazon Kinesis Data Firehose

Answer

AWS Glue

Answer

AWS Data Pipeline

Question 14

A data engineer is troubleshooting an AWS Glue ETL job that fails with an OutOfMemory error when processing large JSON files from Amazon S3. The files contain deeply nested structures. Which approach should the engineer take to resolve this issue?

Accepted Answer

Increase the DPU (Data Processing Unit) per worker. Option D is correct because using the `recurse` option with `getResolvedOptions` is not relevant; the issue is memory. Option A increases parallelism but not memory per worker. Option B reduces memory. Option C (increase number of workers) adds more workers but each still has the same memory; option D (increase DPU per worker) allocates more memory per worker, solving the OOM issue.

Answer

Use the `recurse` option with `getResolvedOptions` to limit recursion

Answer

Increase the number of workers in the Glue job configuration

Answer

Decrease the number of partitions while reading the data

Question 15

A company is using AWS Lake Formation to manage permissions on data in Amazon S3. They need to ingest data from an external source into a new database 'sales_db' and a table 'transactions' using AWS Glue. The IAM role used by Glue must have the minimal permissions to create the database and table in the Data Catalog and write data to the S3 location. Which combination of permissions should be granted?

Accepted Answer

Lake Formation permissions: `CREATE_DATABASE` on the catalog, `CREATE_TABLE` on `sales_db`, and S3 location permission. Option C is correct because Lake Formation permissions are required to create databases and tables in the Data Catalog. IAM permissions alone are insufficient; the Glue role must have Lake Formation `CreateDatabase` and `CreateTable` permissions. Option A lacks Lake Formation. Option B is too broad. Option D lacks S3 write.

Answer

IAM policy with `glue:CreateDatabase`, `glue:CreateTable`, and `s3:PutObject`

Answer

IAM policy with `lakeformation:GrantPermissions` on the database and table

Answer

IAM policy with `s3:GetObject` and `s3:PutObject` on the target location