DEA-C01 Data Ingestion and Transformation Practice Test 37 — 15 Questions

Question 1

A company ingests streaming data from multiple sources into a single Kinesis Data Streams stream. Each source produces records with a different schema. The data must be routed to different S3 prefixes based on the source. Which approach minimizes transformation overhead?

Accepted Answer

Use a single Kinesis Data Firehose with a Lambda transformation that reads schema metadata from DynamoDB to determine the S3 prefix.. Option A is correct because using a DynamoDB table to store schema metadata and a Lambda function to route records to the appropriate Firehose delivery stream is efficient and scalable. Option B is wrong because it requires multiple Kinesis streams, increasing cost and complexity. Option C is wrong because ingesting into S3 and then using Glue to route adds latency. Option D is wrong because using Kinesis Data Analytics for routing is overkill and not designed for this purpose.

Answer

Ingest all data into S3 and use AWS Glue ETL jobs to partition and route data to different prefixes.

Answer

Use separate Kinesis Data Streams for each source and configure separate Firehose delivery streams.

Answer

Use Kinesis Data Analytics to run SQL queries that route data to different Firehose streams.

Question 2

A company needs to ingest data from an external FTP server into AWS S3. The FTP server is not accessible from the internet. Which AWS service should be used to securely transfer the data?

Accepted Answer

AWS Transfer Family with SFTP endpoint in a VPC. Option B is correct because AWS Transfer Family supports SFTP and can be used with a VPC endpoint to transfer data from an FTP server in a private network to S3. Option A is wrong because Snowball Edge is for large offline data transfers, not regular FTP transfers. Option C is wrong because DataSync is for moving data between on-premises storage and AWS, but it requires network access. Option D is wrong because Kinesis Data Firehose is for streaming data, not FTP transfers.

Answer

Kinesis Data Firehose

Answer

AWS DataSync

Answer

AWS Snowball Edge

Question 3

A data engineer is using AWS Glue ETL to transform data from an S3 data lake. The job fails with a memory error. Which approach should be used to resolve this issue without major code changes?

Accepted Answer

Increase the number of DPUs allocated to the Glue job. Option C is correct because increasing the number of DPUs (Data Processing Units) allocated to the Glue job provides more memory and parallelism. Option A is wrong because using a smaller file format may not address memory issues. Option B is wrong because rewriting in PySpark is a major code change. Option D is wrong because using a different service is unnecessary.

Answer

Rewrite the ETL script in PySpark instead of Scala

Answer

Change the input file format from CSV to Parquet

Answer

Use Amazon EMR instead of AWS Glue

Question 4

A company uses Kinesis Data Firehose with a Lambda function for data transformation. The transformation is failing intermittently due to Lambda timeouts. The maximum record size is 1 MB. What is the most cost-effective way to reduce failures without losing data?

Accepted Answer

Increase the Lambda function timeout and memory allocation. Option A is correct because increasing the Lambda timeout and memory reduces timeouts while remaining cost-effective. Option B is wrong because sending to a dead-letter queue (DLQ) loses data. Option C is wrong because decreasing batch size may increase costs. Option D is wrong because using Kinesis Data Analytics adds unnecessary complexity and cost.

Answer

Use Kinesis Data Analytics to pre-process data before Firehose

Answer

Decrease the Firehose batch size to reduce the number of records per invocation

Answer

Configure the Firehose delivery stream to send failed records to an S3 dead-letter bucket

Question 5

A company wants to ingest data from an on-premises SQL Server database into Amazon Redshift. They need to transform the data during ingestion, such as masking PII columns. Which approach meets these requirements with minimal operational overhead?

Accepted Answer

Use AWS Database Migration Service (DMS) with transformation rules. Option C is correct because AWS DMS can perform transformation tasks, including data masking, during migration and can load directly into Redshift. Option A is wrong because Glue ETL adds an extra step. Option B is wrong because Kinesis Data Firehose is for streaming data. Option D is wrong because using EC2 adds operational overhead.

Answer

Use AWS Glue ETL jobs to extract data from SQL Server, transform it, and load into Redshift

Answer

Use a custom application on EC2 to extract, transform, and load

Answer

Use Kinesis Data Firehose to stream data from SQL Server to Redshift

Question 6

A company stores IoT sensor data in S3 as JSON files. They need to convert the data to Parquet format for efficient querying with Amazon Athena. Which AWS service can perform this transformation with minimal effort?

Accepted Answer

AWS Glue ETL job. Option B is correct because AWS Glue ETL can easily convert JSON to Parquet. Option A is wrong because Athena is a query engine, not a transformation service. Option C is wrong because Lambda is for small, event-driven transformations. Option D is wrong because Kinesis Data Firehose is for streaming data.

Answer

Kinesis Data Firehose

Answer

Amazon Athena

Answer

AWS Lambda

Question 7

A company ingests streaming data from social media APIs into Kinesis Data Streams. Each record is approximately 5 KB. The data must be enriched with geolocation information from a DynamoDB table before being stored in S3. The enrichment process takes about 200 ms per record. Which architecture minimizes latency and cost?

Accepted Answer

Use Kinesis Data Firehose with a Lambda function that queries DynamoDB. Option D is correct because using Kinesis Data Firehose with a Lambda function for enrichment is efficient for moderate-size records. Option A is wrong because Kinesis Data Analytics is for real-time analytics, not enrichment. Option B is wrong because EC2 adds significant operational overhead. Option C is wrong because Glue ETL is for batch processing, not streaming.

Answer

Use an EC2 instance running a custom application to consume from Kinesis, enrich, and write to S3

Answer

Use AWS Glue ETL jobs running continuously on the stream

Answer

Use Kinesis Data Analytics to perform enrichment with SQL

Question 8

A company is designing a data ingestion pipeline for clickstream data from a website. The data must be ingested in near real-time. Which TWO services can be used together to build this pipeline?

Accepted Answer

Amazon Kinesis Data Streams. Option A and Option D are correct because Kinesis Data Streams can ingest the clickstream data, and Kinesis Data Firehose can deliver it to S3. Option B is wrong because S3 is not a real-time ingestion service. Option C is wrong because DynamoDB is a database, not a streaming ingestion service. Option E is wrong because SQS is for message queuing, not streaming ingestion.

Answer

Amazon Simple Queue Service (SQS)

Answer

Amazon S3

Answer

Amazon DynamoDB

Question 9

A company is migrating a legacy on-premises ETL pipeline to AWS. The pipeline processes daily batch files from an FTP server. The data must be transformed using complex business logic before being loaded into Amazon Redshift. Which THREE AWS services should be used for this migration?

Accepted Answer

Amazon Redshift. Option A, Option C, and Option D are correct because AWS Transfer Family can ingest files from FTP, AWS Glue can perform complex transformations, and Amazon Redshift is the target. Option B is wrong because Kinesis Data Streams is for streaming data. Option E is wrong because Athena is for querying, not ETL.

Answer

Amazon Athena

Answer

Amazon Kinesis Data Streams

Question 10

A company uses Kinesis Data Firehose to deliver streaming data to S3. They need to transform the data by adding a timestamp and removing sensitive fields. Which TWO approaches can achieve this?

Accepted Answer

Use AWS Glue ETL to process data after delivery to S3. Option A and Option D are correct because Lambda transformation can modify records, and Glue ETL can process data from Firehose's S3 destination. Option B is wrong because Kinesis Data Analytics is for analytics, not transformation. Option C is wrong because S3 Select is for retrieving subsets of data, not transformation. Option E is wrong because Redshift Spectrum is for querying data in S3.

Answer

Use Kinesis Data Analytics to transform the stream

Answer

Use S3 Select to transform data at rest

Answer

Use Amazon Redshift Spectrum to transform data

Question 11

A company runs a data pipeline using AWS Glue ETL jobs to process daily files from an S3 bucket. The files are in CSV format and range from 1 GB to 10 GB. The Glue job runs successfully for small files but fails with an 'Out of Memory' error for files larger than 5 GB. The job uses a single G.1X DPU (16 GB memory). The company needs to process these large files without changing the existing ETL script. Which solution should the company implement?

Accepted Answer

Increase the number of DPUs and use the G.2X worker type to provide more memory per worker.. Option A is correct because increasing the DPU count and using G.2X worker type provides more memory per worker, resolving the memory issue without script changes. Option B is wrong because converting to Parquet may reduce data size but does not guarantee memory issues are resolved and may require script changes. Option C is wrong because using a different file format may not address memory issues. Option D is wrong because using Spark on EMR requires rewriting the script.

Answer

Convert the input files from CSV to Parquet format to reduce memory usage.

Answer

Use the Optimus format in AWS Glue to compress data.

Answer

Use Amazon EMR with Spark instead of AWS Glue.

Question 12

A company has a Kinesis Data Firehose delivery stream that receives JSON data from IoT devices. The data is delivered to an S3 bucket. The company notices that the data in S3 is delayed by up to 30 minutes. The Firehose stream is configured with a buffer size of 1 MB and a buffer interval of 60 seconds. The incoming data rate is approximately 100 KB per second. The company needs to reduce the delivery latency to under 5 minutes. Which action should the company take?

Accepted Answer

Decrease the buffer size to 256 KB.. Option B is correct because reducing the buffer interval to 60 seconds (already set) is not enough; the actual issue is that buffer size is too large relative to data rate. Decreasing buffer interval to 60 seconds is already set, but they need to also reduce buffer size or increase data rate. Actually, the correct action is to decrease the buffer size to 1 MB (already) and decrease buffer interval to 60 seconds (already). Wait, the latency is due to buffer interval of 60 seconds? No, the problem states latency up to 30 minutes. That suggests that the buffer interval is not the only factor; maybe the data rate is low. Actually, with 100 KB/s, it takes about 10 seconds to fill 1 MB buffer. So buffer interval of 60 seconds should cause latency up to 60 seconds. The 30-minute delay suggests another issue. Perhaps the Firehose is waiting for more data or there is a backlog. The correct answer is to decrease the buffer interval to 60 seconds (already) and also decrease the buffer size? Actually, option B says decrease buffer size to 256 KB? That would cause more frequent deliveries and reduce latency. Option A is wrong because increasing buffer interval would increase latency. Option C is wrong because changing compression format does not affect latency. Option D is wrong because using Lambda adds processing time.

Answer

Enable Lambda transformation to process data faster.

Answer

Increase the buffer interval to 300 seconds.

Answer

Change the compression format from GZIP to Snappy.

Question 13

A company needs to ingest data from an on-premises MySQL database into Amazon S3 for analytics. The database is 2 TB in size. The company has a low-bandwidth internet connection (10 Mbps). They need to perform an initial full load and then incremental updates every hour. Which approach should they use?

Accepted Answer

Use AWS Database Migration Service (DMS) to perform the full load and ongoing replication.. Option B is correct because AWS Database Migration Service (DMS) supports full load and ongoing replication, and can be used with limited bandwidth. Option A is wrong because Snowball Edge is for offline transfer, not ongoing replication. Option C is wrong because Glue ETL is not optimized for continuous replication. Option D is wrong because Kinesis Data Firehose is for streaming data, not database replication.

Answer

Use Kinesis Data Firehose to stream data from MySQL to S3.

Answer

Use AWS Glue ETL jobs to extract data and load into S3.

Answer

Use AWS Snowball Edge to transfer the initial full load, then use AWS DataSync for incremental updates.

Question 14

A company is streaming IoT sensor data to Amazon Kinesis Data Streams. The data is JSON with a schema that changes occasionally. They want to load the data into Amazon S3 in Parquet format partitioned by date and sensor_id. Which approach is MOST cost-effective and operationally efficient?

Accepted Answer

Use Amazon Kinesis Data Firehose with a schema from AWS Glue Data Catalog to convert to Parquet and enable dynamic partitioning by date and sensor_id.. Option B is correct because Kinesis Data Firehose can directly convert JSON to Parquet using a Glue table schema and also supports custom partitioning (e.g., by date and sensor_id) without needing a separate transformation step. Option A (Lambda) adds cost and complexity. Option C (EMR) is overkill for this streaming case. Option D (Kinesis Client Library + EC2) is custom and not managed. Option B is the most cost-effective and operationally efficient.

Answer

Use Amazon EMR to read from Kinesis Data Streams and write to S3 in Parquet format.

Answer

Use a Lambda function to transform records to Parquet and write to S3.

Answer

Use a custom Kinesis Client Library application on EC2 to buffer and write Parquet files to S3.

Question 15

A data engineer needs to ingest daily CSV files from an external FTP server into Amazon S3. The files are 5 GB each. Which service is MOST suitable to automate this ingestion?

Accepted Answer

AWS DataSync. Option A is correct because AWS DataSync supports scheduled transfers from FTP servers to S3. Option B (S3 Transfer Acceleration) only speeds up uploads to S3, not fetch from FTP. Option C (AWS Glue) is for ETL, not file transfer. Option D (AWS AppSync) is for real-time APIs, not batch file transfer.

Answer

AWS AppSync

Answer

Amazon S3 Transfer Acceleration

Answer

AWS Glue