DEA-C01 Mock Exam 95 — 25 Questions

Question 1

A financial services company has an Amazon DynamoDB table named 'Transactions' with provisioned read capacity of 10,000 RCU and write capacity of 5,000 WCU. The table stores transaction records for the past 90 days. The application performs point reads by transaction ID (partition key) and range queries by customer ID and timestamp (GSI). Recently, the company started a new marketing campaign, causing a sudden spike in write traffic. The write capacity is now at 4,500 WCU, and the application is experiencing occasional throttling on writes. The data engineer needs to ensure that writes are not throttled during future campaigns, while keeping costs low. The table currently has auto scaling enabled with a maximum capacity of 10,000 WCU. Which solution should the engineer implement?

Accepted Answer

Switch the table to DynamoDB on-demand capacity mode.. Option C is correct. DynamoDB on-demand mode automatically scales to accommodate traffic spikes, eliminating throttling without manual intervention. Option A is wrong because increasing max auto scaling setting still has a limit and may not react quickly enough. Option B is wrong because using SQS adds latency and complexity; not ideal for real-time. Option D is wrong because DAX is for reads, not writes.

Answer

Use DynamoDB Accelerator (DAX) to cache write requests.

Answer

Implement an Amazon SQS queue to buffer write requests and process them in batches.

Answer

Increase the maximum write capacity in the auto scaling configuration to 20,000 WCU.

Question 2

A startup is building a ride-sharing application that uses Amazon DynamoDB to store trip data. The table has a partition key of 'trip_id' and a sort key of 'status'. The application writes a new item when a trip starts and updates the status when the trip ends. The development team is experiencing high write latency during peak hours. The table is provisioned with 5,000 write capacity units (WCU) and 5,000 read capacity units (RCU). CloudWatch metrics show that WriteThrottleEvents are occurring frequently, but the consumed write capacity is never above 4,000 WCU. The team suspects that the issue is due to hot partitions. How should the data engineer resolve this issue?

Accepted Answer

Modify the application to add a random suffix to the partition key when writing items.. Option C is correct because using a random prefix for the partition key distributes writes across partitions, avoiding hot spots. Option A is incorrect because increasing WCU does not solve the hot partition issue; throttling happens at the partition level. Option B is incorrect because DAX is a cache for reads, not writes. Option D is incorrect because decreasing RCU is unrelated to write throttling.

Answer

Enable DynamoDB Accelerator (DAX) to cache write operations.

Answer

Decrease the provisioned RCU to 2,000 to reduce costs.

Answer

Increase the provisioned WCU to 10,000 to handle the spikes.

Question 3

A healthcare company uses Amazon RDS for PostgreSQL to store patient records. The database has a size of 1 TB and is running on a db.r5.large instance. The company requires that the database be highly available and have automated backups with point-in-time recovery (PITR) for the last 35 days. The operations team has configured Multi-AZ deployment and automated backups with a 35-day retention period. During a recent disaster simulation, the team attempted to restore the database to a point in time from 30 days ago. The restore operation failed because the backup was not available. On investigation, the team found that the automated backups were being deleted before the retention period ended. The team also noticed that the database has a large number of transaction logs generating a high volume of write activity. What is the most likely cause of the backups being deleted prematurely?

Accepted Answer

The RDS instance was deleted, which automatically deletes all automated backups.. Option D is correct because automated backups are retained based on the backup retention period. However, if the database instance is deleted, all automated backups are also deleted. Option A is wrong because manual snapshots are separate from automated backups. Option B is wrong because the backup window does not affect retention. Option C is wrong because Multi-AZ automatically performs backups from the standby, but the retention is still enforced.

Answer

The automated backup window was set to a time that conflicted with the database maintenance window.

Answer

The Multi-AZ deployment was not enabled during the backup process, causing backups to fail.

Answer

The database had manual snapshots that were deleted manually by the operations team.

Question 4

A gaming company uses Amazon DynamoDB to store player profiles and game state. The table has a partition key of 'player_id' and no sort key. The table is provisioned with 5,000 RCUs and 5,000 WCUs. The application performs frequent reads and writes to update player scores. Recently, the company introduced a new feature that allows players to form guilds. The guild data is stored in a separate DynamoDB table with a partition key of 'guild_id'. The application often needs to retrieve all members of a guild. The data engineer is encountering high latency when querying the guild table because the guilds can have up to 100 members. The engineer wants to reduce latency without changing the application architecture. What should the data engineer do?

Accepted Answer

Create a global secondary index (GSI) on the guild table with partition key guild_id and sort key member_id.. Option B is correct because adding a global secondary index (GSI) on the guild table with guild_id as the partition key and member_id as the sort key allows efficient queries for all members of a guild. Option A is wrong because increasing capacity may not solve the access pattern issue. Option C is wrong because DynamoDB Streams are for change data capture, not for query optimization. Option D is wrong because DAX caches read results; if the query pattern is inefficient, DAX won't help much.

Answer

Increase the provisioned read and write capacity for the guild table to 10,000 RCUs and 10,000 WCUs.

Answer

Enable DynamoDB Streams on the guild table and process the stream to populate a separate read table.

Answer

Use DynamoDB Accelerator (DAX) to cache the results of the guild queries.

Question 5

A company uses Amazon EMR to run Spark jobs on a cluster of 20 nodes. The cluster stores intermediate data on Amazon S3 using EMRFS. The company's data engineering team notices that the Spark jobs are running slower than expected. Upon investigating, they find that the cluster is experiencing high network I/O and that the S3 storage costs have increased significantly. The team suspects that the Spark jobs are writing too much intermediate data to S3. The jobs are performing many shuffle operations. The team wants to optimize the job performance and reduce costs without modifying the Spark application code. What should the data engineer do?

Accepted Answer

Configure the EMR cluster to use instance store volumes for intermediate data instead of EMRFS.. Option C is correct because enabling automatic encryption on the bucket does not affect performance; the issue is about shuffle data. Option B is wrong because increasing EBS volumes for shuffle storage on instance nodes is not standard; EMR uses instance store or EMRFS. Option A is correct because using instance store for shuffle data reduces S3 I/O and cost. Option D is wrong because increasing node count may increase cost and network I/O.

Answer

Enable S3 server-side encryption on the S3 bucket to reduce storage costs.

Answer

Increase the size of the EBS root volumes on the cluster nodes to store more intermediate data locally.

Answer

Add more nodes to the cluster to distribute the shuffle load.

Question 6

A company runs a transactional database on Amazon RDS for PostgreSQL with Multi-AZ deployment. The database size is 2 TB and experiences moderate write load. The company recently enabled RDS Performance Insights and noticed a high number of 'TupleLock' wait events during peak hours. The development team reports that a batch update job runs every hour, updating millions of rows in a large table. The job takes longer than expected. The DBA suspects that excessive row-level locking is causing contention. The team wants to minimize lock contention without changing the application code. Which solution should be implemented?

Accepted Answer

Tune the autovacuum settings (e.g., autovacuum_vacuum_scale_factor and autovacuum_vacuum_threshold) to run more frequently and aggressively.. Option D (autovacuum tuning) is correct because frequent updates generate dead tuples, leading to increased lock contention. Tuning autovacuum ensures timely cleanup, reducing lock escalation and wait events. Option A (increasing RDS instance size) may alleviate CPU/memory pressure but does not address lock contention. Option B (enabling RDS Proxy) helps with connection pooling, not lock contention. Option C (using pg_partman for partitioning) reduces row-level contention by splitting the table but requires code changes to queries, which is prohibited by the stem.

Answer

Increase the RDS instance size to a larger instance class with more vCPUs and memory.

Answer

Enable RDS Proxy to manage database connections and reduce connection overhead.

Answer

Implement table partitioning using the pg_partman extension to split the large table into smaller partitions.

Question 7

A large e-commerce company uses Amazon DynamoDB to store shopping cart data. The table has a partition key of 'user_id' and a sort key of 'item_id'. The application performs frequent updates to the 'quantity' attribute for items in a user's cart. Recently, the operations team noticed that write requests are being throttled during peak shopping hours. The table is provisioned with 10,000 write capacity units (WCUs) and uses DynamoDB Accelerator (DAX) for read caching. The data engineer suspects that the throttling is due to hot partitions. The application uses a single AWS SDK client configured with retries. After reviewing the Amazon CloudWatch metrics, the engineer sees that the WriteThrottleEvents metric spikes for a few partition keys. The table has a high number of partitions. What should the data engineer do to resolve the throttling issue with minimal application changes?

Accepted Answer

Configure DynamoDB Auto Scaling with a maximum WCU setting of 20,000 and a target utilization of 70%.. Option D is correct because using DynamoDB Auto Scaling with a higher maximum WCUs allows the table to scale up during peak demand, addressing hot partitions without code changes. Option A is wrong because increasing WCUs manually does not adapt to variable traffic. Option B is wrong because DAX is for reads, not writes. Option C is wrong because Global Tables replicate data but do not increase write capacity.

Answer

Increase the provisioned write capacity to 20,000 WCUs permanently.

Answer

Enable DynamoDB Global Tables to distribute writes across regions.

Answer

Add more nodes to the DAX cluster to offload write traffic.

Question 8

A media company stores video files in Amazon S3 buckets organized by content type. The company has a requirement to automatically archive files that are older than 90 days to Amazon S3 Glacier Deep Archive to reduce costs. However, the company wants to retain the ability to restore files within 12 hours if needed. The data engineer creates an S3 Lifecycle policy to transition objects to Glacier Deep Archive after 90 days. After deploying the policy, the engineer notices that the storage costs have not decreased significantly. On reviewing the bucket metrics, the engineer sees that many objects are being deleted directly by users before the lifecycle policy takes effect. The company needs to enforce the lifecycle policy and prevent premature deletions. What should the data engineer do to enforce the lifecycle policy?

Accepted Answer

Enable S3 Versioning on the bucket and add a lifecycle rule to transition noncurrent versions to Glacier Deep Archive after 90 days.. Option A is correct because an S3 Lifecycle policy with a noncurrent version transition will move older versions to Deep Archive, and enabling S3 Versioning ensures that deletions create delete markers rather than permanently deleting objects. Option B is wrong because MFA Delete is for extra security, not for lifecycle enforcement. Option C is wrong because Object Lock prevents object deletion entirely, which may not be desired. Option D is wrong because S3 Glacier Select is for querying archived data, not for lifecycle management.

Answer

Apply an S3 Object Lock retention policy with a default retention mode of GOVERNANCE and a retention period of 90 days.

Answer

Enable S3 Multi-Factor Authentication (MFA) Delete on the bucket to require MFA for all delete operations.

Answer

Use S3 Glacier Select to query the objects and restore them if needed.

Question 9

A startup is building a real-time analytics application using Amazon Kinesis Data Streams and Amazon Kinesis Data Analytics. The application processes clickstream data from a website. The data is also stored in Amazon S3 for historical analysis. The company uses an S3 bucket with a lifecycle policy that transitions objects to Amazon S3 Glacier Deep Archive after 30 days. The data engineering team has configured a Kinesis Data Firehose delivery stream to write data to the S3 bucket. The team notices that the data in S3 is not being transitioned to Glacier Deep Archive after 30 days. The lifecycle policy is correctly configured and has been verified. What is the most likely cause of this issue?

Accepted Answer

The S3 lifecycle rule is configured with a filter that does not match the prefix used by Kinesis Data Firehose.. Option C is correct because Kinesis Data Firehose writes data with a prefix that includes the date, but it often uses the delivery stream's timestamp rather than the object creation date. The lifecycle rule is based on the object creation date. If the prefix does not match the rule filter, the rule may not apply. Option A is wrong because S3 Batch Operations are not related to lifecycle transitions. Option B is wrong because Glacier Deep Archive does have a minimum 90-day storage charge, but the lifecycle rule can still transition after 30 days (though you'll pay a penalty). Option D is wrong because S3 Intelligent-Tiering is an alternative storage class, not a requirement.

Answer

The Glacier Deep Archive storage class requires a minimum 90-day storage period, so the lifecycle policy cannot transition objects after 30 days.

Answer

The S3 bucket is not enabled for S3 Intelligent-Tiering, which is required for lifecycle transitions to Glacier Deep Archive.

Answer

The S3 bucket does not have S3 Batch Operations enabled to invoke the lifecycle policy.

Question 10

A financial services company uses Amazon Redshift for its data warehouse. The cluster has two nodes and is used for complex analytical queries. The company recently migrated from a single-node cluster to a two-node cluster to improve performance. After the migration, the data engineer notices that query performance has not improved as expected. Some queries are even slower than before. The engineer checks the workload management (WLM) queue configuration and sees that there is only one queue with a concurrency level of 5. The queries are mostly large scans and aggregations. The cluster's CPU utilization is low, but disk I/O is high. What should the data engineer do to improve query performance?

Accepted Answer

Add more nodes or upgrade to a larger node type to increase memory and reduce disk spills.. Option C is correct because high disk I/O and low CPU utilization indicate that the cluster is spilling to disk due to insufficient memory. Increasing the number of nodes or upgrading to a larger node type (e.g., dc2.large to dc2.8xlarge) increases memory. Option A is wrong because increasing concurrency would increase contention. Option B is wrong because distribution style is unlikely the main issue. Option D is wrong because manual compression is not needed; Redshift automatically applies compression.

Answer

Apply compression to the tables to reduce the amount of data scanned.

Answer

Increase the concurrency level in the WLM queue to allow more queries to run simultaneously.

Answer

Change the distribution style of large tables to DISTSTYLE ALL to avoid data redistribution.

Question 11

A data engineering team uses AWS Glue ETL jobs to process data from Amazon S3 and load it into an Amazon Redshift cluster. The cluster has a single node of type dc2.large. The team notices that the ETL jobs are failing intermittently with errors related to disk space. The Redshift cluster shows that the disk is nearly full. The team needs to resolve the disk space issue and ensure the ETL jobs can complete successfully without increasing costs significantly. Which solution should the team implement?

Accepted Answer

Convert the cluster to use RA3 node types (e.g., ra3.xlarge) with managed storage.. Option C (converting to a ra3.xlarge node) is correct because RA3 nodes use managed storage, allowing scaling of compute and storage independently. This resolves disk space issues without over-provisioning. Option A (adding more dc2.large nodes) increases cost and still has fixed storage. Option B (using diststyle ALL) may improve query performance but does not add disk space. Option D (loading into a staging table and using VACUUM) helps with reclaiming space but does not address the underlying insufficient storage capacity.

Answer

Load data into a staging table first, then perform a VACUUM and ANALYZE on the target tables.

Answer

Add more nodes to the Redshift cluster by resizing to a multi-node dc2.large configuration.

Answer

Set the table's distribution style to ALL for fact tables to avoid data redistribution during joins.

Question 12

A financial services company ingests real-time stock trade data from multiple exchanges into Amazon Kinesis Data Streams. Each trade record is a JSON object with fields: trade_id, symbol, price, quantity, timestamp. The stream has 5 shards. The data is consumed by an AWS Lambda function that aggregates trades per symbol every minute and writes the results to an Amazon DynamoDB table for a real-time dashboard. Recently, the dashboard has been showing outdated data, and the Lambda function is experiencing high error rates. The CloudWatch logs show 'ProvisionedThroughputExceededException' errors from DynamoDB. The DynamoDB table has 10 read capacity units (RCU) and 10 write capacity units (WCU). The average trade volume is 5,000 trades per second across all symbols, and there are 100 symbols. The Lambda function is configured with a batch size of 100 and a 1-minute window. The data volume is expected to double in the next month. As a data engineer, what is the most appropriate course of action?

Accepted Answer

Increase the DynamoDB write capacity units to 100 and enable auto scaling. Option C is correct because the DynamoDB table is throttling due to insufficient write capacity. With 5,000 trades/s and updating per symbol per minute, the write rate is about 100 writes per minute (one per symbol), but the aggregation may cause bursts. However, the 'ProvisionedThroughputExceededException' indicates WCU is too low. Increasing WCU to 100 resolves the immediate issue; auto scaling handles future growth. Option A (increase shards) addresses Lambda concurrency but not DynamoDB throttling. Option B (use S3) changes the architecture and loses real-time capabilities. Option D (use Firehose) is for delivery to S3, not real-time dashboard.

Answer

Switch the storage from DynamoDB to Amazon S3 and use Amazon Athena for the dashboard

Answer

Increase the number of Kinesis shards to 10 to increase Lambda concurrency

Answer

Use Amazon Kinesis Data Firehose to deliver data to S3 and use Amazon QuickSight for the dashboard

Question 13

A financial services company ingests real-time stock trade data using Amazon Kinesis Data Streams with 10 shards. Each shard receives about 500 records per second, each record approximately 1 KB. The data is consumed by a single AWS Lambda function that transforms the data and writes to Amazon S3. The Lambda function is configured with 1024 MB memory and a timeout of 5 minutes. The company notices that the Lambda function is frequently throttled, and data ingestion lags behind. The Lambda function's CloudWatch metrics show that the iterator age is increasing, and the function's concurrency is maxed out at 1000. The data engineer needs to resolve the throttling issue without changing the Lambda function code. What should the data engineer do?

Accepted Answer

Increase the number of shards in the Kinesis data stream to increase parallelism.. Option B is correct. Increasing the number of shards increases the number of Lambda consumers, allowing more parallel processing and reducing the iterator age. Option A is wrong because increasing Lambda memory may not improve throughput if the function is CPU-bound; also, the issue is concurrency, not memory. Option C is wrong because decreasing batch size may increase overhead; the function is already maxing concurrency. Option D is wrong because reducing memory would likely make the function slower.

Answer

Reduce the Lambda function memory to 512 MB to increase concurrency limit.

Answer

Decrease the batch size to 10 records to reduce processing time per invocation.

Answer

Increase the Lambda function memory to 2048 MB to improve processing speed.

Question 14

A media company ingests video files from content partners into an Amazon S3 bucket. Each video file is 10-50 GB. Upon upload, an AWS Lambda function is triggered to extract metadata (e.g., resolution, duration) and store it in DynamoDB. The company now wants to also generate a thumbnail image for each video. The thumbnail generation is CPU-intensive and can take up to 5 minutes per video. The Lambda function has a maximum execution time of 15 minutes. The company has noticed that some thumbnail generation tasks are timing out. What should the data engineer do to reliably generate thumbnails for all videos?

Accepted Answer

Use AWS Batch to run a containerized thumbnail generation job triggered by S3 events. Option C is correct because AWS Batch is designed for long-running, compute-intensive jobs like video processing. Option A (increase Lambda timeout) is not a good practice for such heavy tasks. Option B (EC2 instance) requires manual management. Option D (Glue) is for ETL, not video processing.

Answer

Provision an EC2 instance to run a script that polls S3 for new videos and generates thumbnails

Answer

Use AWS Glue with a Python shell job to generate thumbnails

Answer

Increase the Lambda timeout to 15 minutes and allocate more memory

Question 15

An e-commerce company uses AWS Glue to process clickstream data from its website. The data is stored in Amazon S3 in partitioned Parquet format by date and hour. A recent increase in traffic has caused the Glue job to fail with 'Java heap space' errors. The job runs with 10 DPUs and uses Spark's default configurations. The data engineer needs to resolve the memory issue without modifying the ETL script. What should the data engineer do?

Accepted Answer

Increase the Spark configuration 'spark.sql.shuffle.partitions' to 500.. Option B is correct. Increasing Spark's shuffle partitions reduces the amount of data handled per partition, preventing heap space errors. Option A is wrong because increasing DPUs may not directly address the heap space issue if the problem is within a single executor. Option C is wrong because reducing partitions may increase memory pressure. Option D is wrong because changing worker type to G.1X may not help if the issue is shuffle-related.

Answer

Decrease the Spark configuration 'spark.sql.shuffle.partitions' to 50.

Answer

Change the worker type to G.1X.

Answer

Increase the number of DPUs to 20.

Question 16

A media company ingests large video files (up to 100 GB each) from content creators via Amazon S3 multipart uploads. After upload, the company needs to transcode the videos into multiple formats using AWS Elemental MediaConvert. The current pipeline uses S3 event notifications to trigger an AWS Lambda function that starts a MediaConvert job. However, for very large files, the Lambda function times out (15-minute limit) before the upload completes because the event is sent when the multipart upload is initiated, not when it completes. How should the engineer fix this issue?

Accepted Answer

Change the S3 event notification to listen for s3:ObjectCreated:Put events instead of s3:ObjectCreated:*, which fires only after the object is fully written.. Option D is correct: Use S3 event notifications for s3:ObjectCreated:Put, which triggers only after the object is completely written. Then Lambda can start MediaConvert. Option A (increase timeout) does not solve the root cause. Option B (Step Functions) adds complexity. Option C (SQS) does not address the event timing.

Answer

Increase the Lambda timeout to 30 minutes.

Answer

Use Amazon SQS to queue the event and have a Lambda function process it later.

Answer

Use AWS Step Functions to poll the S3 bucket for object existence and then start MediaConvert.

Question 17

A retail company uses AWS Glue ETL jobs to process sales data from an S3 data lake. The source data is partitioned by year/month/day in CSV format. The Glue job reads the latest day's data, performs transformations (e.g., cleaning, aggregating), and writes the results to a separate S3 bucket. The job runs on a schedule every day at 2 AM. Recently, the job has been failing intermittently with the error 'AnalysisException: Path does not exist: s3://source-bucket/year=2024/month=02/day=30/'. The engineer verifies that the folder 'day=30' does not exist because February has only 28 days in 2024. The job is reading data from a hardcoded path. The company expects the job to handle variable days per month automatically. What should the engineer do to fix the issue?

Accepted Answer

Use a Glue crawler to populate the Data Catalog and use dynamic frame from_catalog with partition predicates. Option B is correct because using a Glue crawler to update the partition metadata and then using dynamic frame with from_catalog allows Glue to automatically discover all existing partitions. This eliminates the need for hardcoded paths. Option A (try-catch) is a workaround but not a proper solution. Option C (increase retries) does not fix the root cause. Option D (use Spark SQL with manual partition pruning) still requires knowing which partitions exist.

Answer

Modify the script to use Spark SQL with manual partition pruning based on current date

Answer

Add a try-catch block in the script to skip missing partitions

Answer

Increase the job's retry count and set a timeout

Question 18

A data engineer is responsible for ingesting log files from a fleet of on-premises servers into Amazon S3 for central analysis. Each server generates log files that are rotated every hour, resulting in files of about 500 MB each. The total daily data volume is approximately 1 TB. The network connection between the on-premises data center and AWS is a 100 Mbps VPN. The engineer needs to ensure that all log files are transferred to S3 within 24 hours of generation without data loss. The engineer is considering using AWS DataSync. However, the initial setup shows that the transfer speed is insufficient to meet the 24-hour SLA. What should the engineer do to meet the requirement?

Accepted Answer

Configure AWS DataSync to run on a schedule with incremental transfers and enable data compression.. Option C is correct because AWS DataSync supports scheduling tasks incrementally; after the initial full sync, only new and changed files are transferred, which will fit within the bandwidth. Option A is wrong because ordering a Snowball Edge would take days to arrive and is not suitable for ongoing daily transfers. Option B is wrong because increasing bandwidth may not be feasible or cost-effective. Option D is wrong because compressing files may reduce size but adds complexity and does not solve the network bottleneck; incremental sync is more effective.

Answer

Use AWS CLI with multipart uploads and parallel threads to maximize throughput.

Answer

Contact the network team to upgrade the VPN bandwidth to at least 1 Gbps.

Answer

Order an AWS Snowball Edge device to transfer the initial data and then use DataSync for incremental changes.

Question 19

A media company uses Amazon Kinesis Data Firehose to ingest log data from web servers into Amazon S3. The data is then processed by AWS Glue jobs. The company wants to ensure that data is delivered to S3 within 5 minutes of ingestion. Currently, the Firehose delivery stream is configured with a buffer interval of 300 seconds and a buffer size of 5 MB. The log data arrives at a rate of 2 MB per second. The data engineer notices that some log files are delayed by up to 10 minutes. The company cannot change the buffer size due to downstream requirements. What should the data engineer do to meet the 5-minute delivery requirement?

Accepted Answer

Decrease the buffer interval to 120 seconds.. Option A is correct. Decreasing the buffer interval to 120 seconds will cause Firehose to deliver data more frequently, meeting the 5-minute SLA. Option B is wrong because increasing buffer interval would increase delay. Option C is wrong because increasing buffer size would also increase delay. Option D is wrong because compressing data does not reduce buffer interval.

Answer

Increase the buffer interval to 600 seconds to reduce the number of delivery attempts.

Answer

Increase the buffer size to 10 MB to ensure data is delivered in larger chunks.

Answer

Enable GZIP compression on the Firehose stream to reduce data size.

Question 20

A retail company uses Amazon Kinesis Data Firehose to ingest clickstream data from its website into an Amazon S3 bucket. The data includes fields: user_id, event_type, timestamp, page_url. Recently, the data engineering team noticed that some records have malformed JSON (missing commas, extra brackets) causing delivery failures to S3. The Firehose delivery stream is configured to retry failed records for 300 seconds, after which the records are sent to an S3 bucket for failed records. The team wants to transform the data to correct malformed JSON before delivery to the main S3 bucket. They need a solution that does not require managing servers and can handle high throughput. What should the team do?

Accepted Answer

Configure an AWS Lambda function as a data transformation in Kinesis Data Firehose to correct malformed JSON.. Option B is correct: Lambda transformation in Firehose can process each record and fix JSON errors. Option A (Kinesis Data Analytics) is for real-time analytics, not record-level transformation. Option C (Glue streaming) adds complexity and latency. Option D (EMR) requires cluster management.

Answer

Set up an Amazon EMR cluster with Apache Spark to process the data in micro-batches and fix JSON errors.

Answer

Use an AWS Glue streaming ETL job to read from Firehose and write corrected data to S3.

Answer

Use Amazon Kinesis Data Analytics with a SQL application to parse and fix JSON.