AWS Certified Data Engineer Associate DEA-C01 (DEA-C01) — Questions 1501–1575

Increasing the number of shards increases the write capacity, directly addressing the throughput exceeded error. Option B is wrong because the error is not from the producer side. Option C is wrong because the error is about write throughput, not read.

Option D is wrong because the error is not about data format.

Option A is correct because the lifecycle rule in the CloudFormation template is missing the `NoncurrentVersionExpiration` or `NoncurrentVersionTransition` properties. When S3 bucket versioning is enabled, lifecycle rules that only specify `ExpirationInDays` or `Transition` apply exclusively to the current version of objects. To manage older (noncurrent) versions, you must explicitly include `NoncurrentVersionExpirationInDays` or `NoncurrentVersionTransitionInDays` in the rule.

Without these, noncurrent versions accumulate indefinitely, causing the bucket size to grow larger than expected.

Option A is correct. Views in Athena do not automatically enforce row-level security; the filter must be applied via the view definition. Option B is wrong because the function exists.

Option C is wrong because Athena executes the view's SQL. Option D is wrong because the tag is present.

Option C is correct because using a cluster with managed scaling and a larger initial core node count can reduce the time waiting for resource allocation. Option A (spot instances) may cause interruptions. Option B (reserved instances) is for long-term use, not transient clusters.

Option D (increasing instance size) may reduce processing time but not idle time.

For SSE-KMS, the IAM role must have kms:Decrypt permission on the KMS key. Option A is wrong because s3:GetObject is already allowed. Option B is wrong because SSE-KMS does not require s3:GetObjectAcl.

Option D is wrong because kms:GenerateDataKey is used for writes. Option C is correct.

Options A, C, and D are correct. Enable binary logging for CDC, use a bookmark to track processed records, and create a job bookmark in Glue. Option B is wrong because querying entire table daily is inefficient.

Option E is wrong because Kinesis Data Streams is for real-time, not batch CDC.

Option C is correct because the explicit deny in the bucket policy overrides any allow in the Lambda role. Option A is wrong because the VPC endpoint policy is not mentioned. Option B is wrong because KMS permissions are not relevant.

Option D is wrong because the bucket is in the same account.

Option B is correct because AWS Transfer Family supports SFTP and can automatically transfer files to S3, with no need to manage servers. Option A is wrong because AWS DataSync requires an agent on-premises. Option C is wrong because S3 Transfer Acceleration is for speeding up transfers to S3, not for ingesting from SFTP.

Option D is wrong because Kinesis Data Firehose is for streaming data, not file-based SFTP transfers.

DynamoDB Global Tables provide multi-region replication with low latency. Option A is correct. Option B: DAX is a cache, not replication.

Option C: TTL is for automatic item expiration. Option D: Streams capture changes but do not automatically replicate to other regions without custom code.

Option C is correct because increasing the number of shards increases the stream's capacity and allows more parallel processing. Option A is wrong because reducing the number of records per shard per second would decrease throughput. Option B is wrong because using a single worker per shard is the default; adding more workers per shard is not recommended.

Option D is wrong because decreasing the number of shards reduces capacity, worsening lag.

Option A is correct because using an S3 VPC Endpoint with a gateway endpoint ensures traffic stays within AWS and can enforce encryption via policies. Option C is correct because mounting S3 as a file system using AWS Storage Gateway with HTTPS ensures encryption in transit. Option B is incorrect because default encryption is for data at rest, not in transit.

Option D is incorrect because S3 Transfer Acceleration does not enforce encryption; it uses HTTPS but is optional. Option E is incorrect because SSE-S3 is for data at rest.

Increasing the number of DPUs (Data Processing Units) allocated to the Glue job provides more parallelism and memory, which can significantly speed up processing. Using G.1X worker type with more memory can also help. Using S3 Select is for filtering within S3, not for DynamoDB.

Changing to JSON format may reduce performance due to larger file size. Reducing batch size could slow down the job.

Options A, C, and E are correct. The IAM role must have s3:PutObject, and also the S3 bucket policy must allow the role's access (option A). Option C is needed for writing temporary files.

Option E is needed for the Glue service principal to write to the bucket. Option B is wrong because the error is about write access, not read. Option D is wrong because the error is about the staging bucket, not the Glue job script.

Option C is correct because storedBytes: 0 means no log events were stored, likely because the job failed before writing any logs or logging was not enabled. Option A is wrong because logs are not expired; they were never written. Option B is wrong because the stream exists.

Option D is wrong because the command retrieved the latest stream.

Option D is correct because Kinesis Data Firehose can use an AWS Lambda function for data transformation, and converting to Parquet can be done via a Lambda function. Option A is wrong because Firehose can deliver to S3 directly. Option B is wrong because Firehose can transform without a separate Glue job.

Option C is wrong because Firehose does not natively support Parquet conversion without a transformation.

For cross-account access with SSE-KMS, the KMS key policy must grant the partner account access to use the key. The bucket policy alone is insufficient. The partner account does not need VPC endpoints, and the bucket policy for decryption is not needed.

The partner account does not need access to the S3 bucket's encryption configuration.

Option A is correct because the KeyManager is "AWS", indicating it is an AWS managed key. Option B is wrong because the key is enabled. Option C is wrong because the key was created in us-east-1 (implied by ARN).

Option D is wrong because the key state is Enabled, not PendingDeletion.

Option A is correct because increasing DPUs and using the S3 list implementation can parallelize reading. Option B is wrong because Parquet is more efficient, but the bottleneck may be parallelism. Option C is wrong because JDBC connections are for databases, not S3.

Option D is wrong because Redshift Spectrum queries data in place, but the job is an ETL, not querying.

Option A is correct because Kinesis Data Firehose can be used as a HTTP endpoint (via API Gateway or directly with Firehose API), automatically buffers data, converts to Parquet, and writes to S3 with partitioning by date. This handles spikes without custom code. Option B (Lambda + SQS) adds complexity and still faces cold starts.

Option C (EMR) is overkill. Option D (Kinesis Data Streams + Lambda) still requires Lambda and has cold start issues.

Option D is correct because a bucket policy that denies requests without the aws:SecureTransport condition enforces HTTPS for all access. Option A is wrong because S3 default encryption only encrypts data at rest, not in transit. Option B is wrong because CloudFront does not enforce HTTPS by default; it can be configured but is not the direct solution.

Option C is wrong because SSE-C is server-side encryption at rest, not in transit.

Option D is correct. Redshift Spectrum allows querying data directly in S3 without loading it into the cluster, offloading storage and compute. Option A is wrong because increasing concurrency scaling adds compute but not storage relief.

Option B is wrong because distribution style affects performance but not storage costs. Option C is wrong because Dense Compute nodes are not compatible with RA3; RA3 uses managed storage.

Option B is correct because increasing the number of workers adds more parallelism but does not increase memory per worker; the error is heap space per worker. Option A is wrong because G.1X has less memory. Option C is correct: increasing MaxCapacity (DPUs) per worker increases memory per worker.

Option D is wrong because reducing workers exacerbates the issue.

Option B is correct because partition projection requires explicit S3 permissions to list the partition location, which Lake Formation may not automatically grant. Option A is wrong because SELECT permission is granted. Option C is wrong because encryption settings don't cause zero rows.

Option D is wrong because the table is registered.

With Multi-AZ, RDS automatically fails over to the standby in another AZ. The CNAME record is updated to point to the standby, so the endpoint remains the same. Option A is correct.

Option B is wrong because failover is automatic and typically completes within minutes, not requiring manual DNS update. Option C is wrong because RDS does not create a new endpoint. Option D is wrong because the standby is promoted to primary, not a read replica.

Option B is correct because Amazon AppFlow is a fully managed integration service that supports SaaS sources, schema discovery, and data transformation. Option A (AWS Glue) is for ETL but not for SaaS ingestion directly. Option C (Amazon Kinesis) is for streaming data.

Option D (AWS Data Pipeline) is not fully managed for SaaS.

The error indicates that the data type detected by Athena's schema inference does not match the actual data. Since the Lambda function converts timestamps to Unix epoch (a number), Athena may infer the column as a string due to some records not being converted properly. Option A is wrong because partitions do not cause this error.

Option B is wrong because S3 permissions would cause a different error. Option D is wrong because data format is likely compatible.

Option C is correct because Glue's DynamicFrame has a `recurse` option in `withErrorThreshold` or can use `from_options` with `multiLine` and `allowQuotedRecordDelimiters`. However, the best approach is to use the `from_catalog` or `from_options` with `withErrorThreshold` to skip bad records. Option A is wrong because modifying the data source is not always possible.

Option B is wrong because using a different file format does not fix malformed JSON. Option D is wrong because AWS Glue Schema Registry validates schemas, not malformed records.

Option D is correct because the expiration action deletes objects after 365 days. Option A is wrong because the policy transitions to Glacier, not Deep Archive. Option B is wrong because the transition to Standard IA occurs at 30 days, not 365.

Option C is wrong because the transition to Glacier occurs at 90 days, not 365.

Options A and B are correct. Sort keys help the query optimizer prune blocks, and distribution keys reduce data movement. Option C is wrong because VARCHAR is variable-length and may not improve performance.

Option D is wrong because GROUP BY is not a design practice. Option E is wrong because adding more slices (via cluster resize) is not a table design change.

Options A and D are correct. Set 'BufferIntervalInSeconds' to 60 seconds to flush every minute, and 'BufferSizeInMBs' to 1 MB to ensure small files are delivered quickly. Option B is wrong because enabling S3 compression does not affect delivery frequency.

Option C is wrong because converting to Parquet is a transformation, not a delivery trigger. Option E is wrong because enabling error logging does not reduce latency.

Option D is correct because using Redshift's COPY command with S3 is optimized for bulk loads and avoids transformation delays. Option A is wrong because increasing Glue workers may help but doesn't address Redshift load speed. Option B is wrong because columnar storage is already used.

Option C is wrong because S3 Select reduces data scanned but does not accelerate the Glue job.

The correct answer is to use column pruning and predicate pushdown. Reading only necessary columns and filtering early reduces data scanned and processing time. Option B (increasing worker count) increases cost and may not be needed.

Option C (switching to CSV) would increase data size and slow performance. Option D (using a single large file) reduces parallelism and may harm performance.

Options B and D are correct. AWS CloudTrail logs API calls to S3. Amazon S3 server access logs provide detailed records of requests.

Option A is wrong because Amazon CloudWatch Logs can receive logs but doesn't generate S3 access logs directly. Option C is wrong because AWS Config tracks configuration changes, not access. Option E is wrong because VPC Flow Logs capture network traffic, not S3 access.

Option B is correct because S3 Intelligent-Tiering automatically moves objects between frequent and infrequent access tiers, and lifecycle policies can transition to S3 Glacier Deep Archive for long-term retention. Option A is wrong because S3 Standard is expensive for 7 years. Option C is wrong because S3 Glacier Instant Retrieval is not cost-effective for the first 90 days.

Option D is wrong because S3 One Zone-IA is not recommended for durability.

Option B is correct because the Glue job needs permissions to access the Data Catalog. The IAM role associated with the job must have 'glue:GetTable' permission. Option A is wrong because the error is about the table not existing, not about the S3 path.

Option C is wrong because the script is correct syntactically. Option D is wrong because the job bookmark setting does not affect table discovery.

Option C is correct because a Global Secondary Index with 'GameId' as partition key and 'Score' as sort key allows efficient querying with ScanIndexForward=false to get top scores. Option A is wrong because the base table's sort key is 'Score', but querying by 'GameId' and sorting descending would work but is less efficient if the table has other attributes. However, GSI is specifically for this access pattern.

Option B is wrong because DynamoDB Streams is for change data capture, not querying. Option D is wrong because DAX is a caching layer, not a querying feature.

Option D is correct because Redshift uses a read-only transaction when connected via a read-only workload or if the connection string specifies auto_commit=False and the job tries to write without committing. Option A is wrong because insufficient permissions would cause a different error. Option B is wrong because Redshift is not in read-only mode.

Option C is wrong because Glue connections do not have a read-only setting.

Option A is correct because the error message indicates the GlueServiceRole does not have s3:PutObject permission on the bucket. Option B is wrong because the error is about permissions, not the file name. Option C is wrong because Parquet format is not the issue.

Option D is wrong because the bucket exists (error shows it in the ARN).

Option B is correct because using a 'query' parameter with a WHERE clause that filters on the bookmark key (e.g., a timestamp column) allows Glue to read only the incremental data, reducing the amount of data transferred. Option A is wrong because increasing the number of DPUs adds parallelism but the bottleneck may be the database's ability to serve data. Option C is wrong because increasing the fetch size may cause memory issues.

Option D is wrong because job bookmark already tracks processed data; disabling it would cause reprocessing.

Option B is correct because QuickSight RLS requires a permissions dataset that maps users to the values in the restricted column. If the RLS rules are not properly set, all rows are visible. Option A is wrong because the column exists.

Option C is wrong because SPICE supports RLS. Option D is wrong because RLS is dataset-level, not dashboard-level.

Option C is correct because the policy allows s3:PutObject but does not allow kms:GenerateDataKey or kms:Encrypt, which are needed to write to an SSE-KMS encrypted bucket. Option A is wrong because ListBucket is allowed. Option B is wrong because GetObject is allowed.

Option D is wrong because the key ARN is correct.

By default, Lake Formation uses IAMAllowedPrincipals group which grants full access to IAM users and roles. If this group is present, Lake Formation permissions are bypassed. Option A is wrong because IAM policy granting Athena access is not sufficient without Lake Formation permissions.

Option C is wrong because S3 permissions alone do not allow Athena queries. Option D is wrong because Lake Formation is a different service.

Option B is correct because Lambda has a default timeout of 3 seconds, but can be increased to 15 minutes. If the function times out at 15 seconds, the timeout is set to 15 seconds. Increasing it resolves the issue.

Option A is wrong because 10 MB is within Lambda limits. Option C is wrong because memory may need increase but timeout is the immediate cause. Option D is wrong because VPC configuration can cause delays but not specifically timeout.

Option A is correct because after failover, the standby becomes the new primary, and a new standby is created in a different AZ, which may have higher latency. Option B is incorrect because the DNS record is updated. Option C is incorrect because read replicas are not affected.

Option D is incorrect because failover is automatic.

Option C is correct because Redshift audit logging captures queries, and column-level access control can be enforced using views with restricted columns. Option A is wrong because IAM policies do not control column-level access in Redshift. Option B is wrong because row-level security is not yet supported.

Option D is wrong because CloudWatch Logs do not control access.

Amazon Kinesis Data Streams is designed for real-time streaming data ingestion from many sources. Option A is wrong because S3 is for object storage, not real-time streaming. Option B is wrong because Lambda processes data but is not a primary ingestion service.

Option D is wrong because RDS is a relational database, not for streaming ingestion.

Option A is correct because AWS Database Migration Service (DMS) supports ongoing replication from MySQL to S3 with change data capture (CDC), enabling incremental updates. Option B is wrong because AWS Glue can do batch jobs but does not natively support CDC as seamlessly as DMS. Option C is wrong because Amazon Kinesis is for streaming data, not database snapshots.

Option D is wrong because Amazon EMR is overkill for simple database ingestion.

Option B is correct because Amazon RDS provides managed relational databases with support for complex joins and transactions. Option A is wrong because DynamoDB is NoSQL and does not support complex joins. Option C is wrong because Redshift is for analytics and is overkill for 500 GB.

Option D is wrong because S3 is not a relational database.

Option A and D are correct. AWS Lake Formation provides data lineage and versioning. AWS Glue Data Catalog stores metadata and can be integrated with Lake Formation.

Option B is wrong because S3 does not provide lineage. Option C is wrong because CloudTrail logs API calls but not lineage. Option E is wrong because Data Pipeline is for data movement.

Option A is correct because CloudWatch can collect custom metrics from EMR via the CloudWatch agent or EMR metrics integration. Option B is incorrect because CloudTrail records API calls, not resource utilization. Option C is incorrect because AWS Config tracks configuration changes.

Option D is incorrect because Athena is a query service, not a monitoring service.

Option A (Glue ETL) is serverless and can handle large files. Option C (Step Functions with Athena) can orchestrate Parquet conversion via CTAS. Option B (EMR) is not serverless.

Option D (Redshift Spectrum) is for querying. Option E (EC2) requires management.

Option A is correct because Amazon Timestream is a fast, scalable, serverless time-series database. Option B is wrong because Amazon RDS is not optimized for time-series workloads. Option C is wrong because Amazon DynamoDB is a key-value store, not purpose-built for time-series.

Option D is wrong because Amazon ElastiCache is a caching layer, not a primary datastore for time-series.

Option B is correct because the Lambda function should not wait for the transcoding job to complete; it should submit the job and exit. The current design synchronously waits, causing timeouts. Option A is wrong because increasing the timeout to 20 minutes may still fail if jobs take longer.

Option C is wrong because the issue is not about concurrency. Option D is wrong because DMS is for database migration, not file processing.

Option B is correct because Redshift does not automatically maintain materialized views; they need to be refreshed to reflect changes in the base tables. Option A is wrong because Redshift does not have a cache that needs clearing in this context. Option C is wrong because VACUUM reclaims space and sorts tables, but does not refresh materialized views.

Option D is wrong because ANALYZE updates statistics, not the data content.

Option B is correct because AWS DMS supports live migration with minimal downtime. Option A is wrong because S3 is for object storage, not database migration. Option C is wrong because Snowball is for large data transfers, not continuous replication.

Option D is wrong because AWS Glue is for ETL, not live database migration.

Option C is correct because Kinesis Data Analytics for Apache Flink allows running custom Apache Flink applications for real-time data processing. Option A (Kinesis Data Firehose) does not support custom Python code; it uses built-in transformations. Option B (AWS Lambda) can be used but requires custom code and is typically used for simple transformations.

Option D (Amazon EMR) is for large-scale batch processing, not real-time.

Option A (AppFlow) connects to SaaS sources. Option C (Glue DataBrew) can clean and deduplicate data. Option E (EventBridge) can trigger Glue jobs on schedule or events.

Option B (Kinesis) is for streaming, not SaaS. Option D (DMS) is for databases.