CCNA Data Ingestion and Transformation Questions

Option A is correct because Kinesis Data Streams provides sequence numbers, and Kinesis Data Analytics can use Flink's exactly-once semantics with idempotent sinks. Option B (Firehose) provides at-least-once. Option C (SQS) provides at-least-once.

Option D (MSK) can achieve exactly-once but requires more configuration and is not as straightforward.

A (Kinesis Data Streams) provides the high-throughput ingestion layer, and C (Kinesis Data Analytics for Apache Flink) provides low-latency stream processing. B (SQS) is a queue, not optimized for high-throughput streams. D (Firehose) has higher latency (buffering).

E (Glue) is batch-oriented.

Option A: AWS Glue is serverless and can transform data. Option C: S3 event notifications can trigger a Lambda function to start the Glue job. Option B (Amazon EMR) is server-based and requires cluster management.

Option D (AWS Data Pipeline) is not serverless. Option E (Amazon Redshift Spectrum) queries data in place but does not transform file formats.

Option B is correct because AppFlow is a fully managed service for securely transferring data from SaaS applications to AWS. Option A (Glue) can connect to JDBC sources but not directly to many SaaS APIs. Option C (DataSync) is for file-based transfers.

Option D (Transfer Family) is for SFTP.

Option D is the best approach: S3 event notifications trigger Lambda to start a Glue job passing the file key; the job processes only that file, using the file key to avoid reprocessing. Option A still processes all files and may cause duplicates if a file arrives after the job starts. Option B (Glue Workflows) does not solve the file-level tracking.

Option C (EMR) is overkill and still scans the entire input folder.

Options A and C are correct because AWS DataSync can transfer data over the internet, and the AWS CLI can be used for uploads. Option B is incorrect because Direct Connect is a physical connection. Option D is incorrect because Snowcone is a physical device.

Option E is incorrect because Kinesis Data Firehose is for streaming, not file transfer.

Options A, B, and D are correct. Kinesis Firehose for streaming, Glue for batch ETL, and DMS for database migration. Option C is wrong because Athena is a query service, not ingestion.

Option E is wrong because Redshift Spectrum is for querying S3 from Redshift.

Amazon Kinesis Data Streams ingests streaming data reliably, and Amazon Kinesis Data Firehose delivers it to S3 with retry logic. Option A (Amazon SQS) is for message queues, not streaming ingestion. Option B (Amazon RDS) is a database.

Option D (AWS Lambda) can process records but is not the primary ingestion service for high-throughput streaming. Option E (Amazon DynamoDB Streams) is for CDC from DynamoDB, not general streaming.

Option B is correct because missing 'kinesis:DescribeStream' is required for Glue to read from the stream. Option A is wrong because the S3 policy is fine. Option C is wrong because resource ARN is correctly formatted.

Option D is wrong because the policy does not allow 'kinesis:GetRecords' or 'kinesis:DescribeStream'.

Option C is correct because AWS DataSync is designed for large-scale data transfers from on-premises to AWS. Option A is wrong because Snowball is for offline transfer, not real-time. Option B is wrong because Direct Connect is a dedicated network connection, not a data transfer service.

Option D is wrong because S3 Transfer Acceleration speeds up uploads over the internet but is not specifically designed for on-premises bulk transfer.

Duplicate PK errors often occur when the target table has an auto-increment column that conflicts with DMS inserts. Option A is correct. Option B is wrong because LOB mode doesn't cause duplicates.

Option C is wrong because full LOB mode is for large objects. Option D is wrong because parallel threads can cause duplicates if not properly configured.

Option A is correct because increasing the number of Glue workers (DPUs) directly increases parallelism and reduces runtime, and you can choose a worker type with more memory if needed. Option B is wrong because RDS read replica does not help Glue processing speed. Option C is wrong because S3 Transfer Acceleration is for uploads, not Glue processing.

Option D is wrong because reducing DPUs would increase runtime.

Option A is correct: Increase shard count to handle peak throughput, and enable automatic scaling for Kinesis Data Streams. Option B (increase buffer size) would increase latency. Option C (use Lambda) may not handle the throughput.

Option D (use SQS) would not reduce latency.

Options A and D are correct. Option A: Lambda needs an event source mapping to poll from Kinesis. Option D: Lambda needs an IAM role with permissions to read from Kinesis and write to S3.

Option B is wrong because Lambda does not write directly to S3 via Kinesis; it writes via the Lambda function. Option C is wrong because Lambda does not use SQS to trigger from Kinesis. Option E is wrong because Lambda does not write to S3 via Firehose; Firehose can directly write to S3.

Option A is correct because Kinesis Data Streams ingests data in real-time, and Kinesis Data Firehose can buffer, transform (e.g., convert to Parquet), and deliver to S3 with low latency. Option B is wrong because Kinesis Data Analytics is for running SQL on streams, not for simple transformation. Option C is wrong because Glue is batch-oriented, not real-time streaming.

Option D is wrong because Lambda can transform but may not scale cost-effectively for thousands of devices.

Option A is correct because AWS DMS with ongoing replication (change data capture) can capture changes from Oracle with minimal impact and replicate to S3, then COPY into Redshift. Option B is wrong because AWS Glue is batch-oriented and does not support real-time CDC natively. Option C is wrong because Lambda can process events but is not designed for continuous CDC from a database.

Option D is wrong because Kinesis Data Streams requires custom producers and does not directly integrate with Oracle for CDC.

Option B is correct because Glue jobs use a version of the table schema that may be stale if the schema was updated after the job started; the job may be referencing an old version that no longer exists. Option A is wrong because table location is irrelevant if the table exists. Option C is wrong because concurrent jobs do not cause 'Resource Not Found' errors.

Option D is wrong because the job name does not affect table access.

Option C is correct because AWS Glue is a serverless ETL service that can run scheduled jobs to convert data to Parquet. Option A is wrong because Lambda has a 15-minute timeout. Option B is wrong because EMR requires managing clusters.

Option D is wrong because Redshift is a warehouse, not for file conversion.

Option C is correct because a network ACL can block outbound traffic from the VPC to the database. Option A is wrong because the error is network, not authentication. Option B is wrong because the security group allows inbound, but need outbound.

Option D is wrong because it's a network issue, not schema.

Enabling DynamoDB auto scaling increases write capacity automatically when needed. Using Kinesis Data Firehose would change the architecture but does not address throttling directly. Increasing Lambda memory does not help with DynamoDB throttling.

Using SQS would add a queue but does not increase DynamoDB capacity.

Options A, C, and D are correct. Increasing the number of DPUs provides more parallelism. Partitioning the input data improves read performance.

Using a JDBC connection with appropriate batch size improves write performance. Option B is wrong because reducing the number of files may reduce parallelism. Option E is wrong because switching to a single node cluster reduces parallelism.

Option B is correct because using DynamoDB Streams with AWS Lambda allows incremental change data capture (CDC), reducing the need to read the entire table. Option A is wrong because increasing read capacity might help but still reads the whole table. Option C is wrong because reducing parallelism increases job duration but still reads everything.

Option D is wrong because exporting to S3 via the console is a one-time operation, not incremental.

Option A and Option C are correct because Kinesis Data Streams ingests the clickstream, and Kinesis Data Analytics for Apache Flink can enrich streams with DynamoDB lookups. Option B is wrong because Lambda can enrich but is limited by concurrency. Option D is wrong because Firehose can transform via Lambda but not DynamoDB enrichment directly.

Option E is wrong because Glue is batch, not streaming.

Option A is correct for ingestion, Option C is correct for delivery to S3. Option B is for batch ETL, not real-time. Option D is for ad-hoc querying.

Option E is for batch data transfer.

Options A, C, and E are correct. Option A enables continuous replication using CDC. Option C uses a transformation to convert to Parquet.

Option E uses a target endpoint for S3 with parquet format. Option B is wrong because SCT is for schema conversion, not for DMS replication. Option D is wrong because a full load only is not continuous replication.

Option B is correct because S3 events can trigger a Lambda function directly. Option A is incorrect because SQS is not triggered by S3 events without a separate setup. Option C is incorrect because EventBridge cannot directly trigger Lambda from S3 events; S3 events go to Lambda via S3 notification.

Option D is incorrect because Step Functions would need an event source; Lambda is simpler.

Option D is correct. AWS Snowball Edge is a physical device that can transfer large volumes quickly, bypassing network limitations. Option A is wrong because 1 Gbps would take ~111 hours for 50 TB, and with overhead it may exceed 10 days.

Option B is wrong because DataSync also uses network and would be too slow. Option C is wrong because S3 Transfer Acceleration is for existing internet transfers, not bulk offline.

The policy allows s3:PutObject on my-data-lake/*, but if the output bucket is different (e.g., my-output-bucket), the policy does not cover it. The error is due to missing permissions on the output bucket. The Glue service role may not have permissions to write to the output bucket.

The policy does not restrict resource to only one bucket, but the ARN specifies my-data-lake. The job might be trying to write to a different bucket. There is no issue with Glue actions.

A is correct because enabling partition indexing in the Glue Data Catalog allows partition pruning and automatic updates. C is correct because configuring a Glue crawler with a schedule will automatically discover new partitions. B is wrong because setting up an S3 event notification to trigger Lambda for manual updates is not as efficient as crawler scheduling.

D is wrong because using AWS Glue ETL jobs to update the catalog is not automatic. E is wrong because Amazon Athena does not update the catalog.

Option D is correct because using 'coalesce' or 'repartition' with a number based on the target file size (e.g., 128 MB) reduces files without a single partition bottleneck. Option A is wrong because lowering shuffle partitions reduces parallelism and may cause OOM. Option B is wrong because increasing shuffle partitions increases files.

Option C is wrong because S3 multipart upload is automatic and does not affect file count.

Options A, B, and D are correct. Increasing DPUs provides more parallelism. Using pushdown predicates reduces data scanned.

Using JDBC fetch size reduces round trips. Option C is wrong because using a single connection per partition is the default and may cause contention; increasing connections can help but the default is fine. Option E is wrong because Parquet is already efficient; converting to CSV would make it worse.

Option D is correct because dynamic partitioning requires a separate configuration (enabling dynamic partitioning and specifying the partitioning keys). The prefix with timestamp expressions is used for S3 prefix customization but not for dynamic partitioning. Option A (time zone) could cause wrong hour but not single partition.

Option B (buffer size) affects file size. Option C (IAM) would cause errors, not mispartitioning. The key is that dynamic partitioning must be explicitly enabled.

Amazon Kinesis Data Firehose can directly ingest streaming data and deliver to S3. Amazon Kinesis Data Streams can also ingest data, but requires a consumer to write to S3. AWS Glue is for batch ETL.

Amazon SQS is for message queuing. Amazon DynamoDB Streams captures changes in DynamoDB, not from mobile apps.

Option C is correct because Glue jobs need an IAM role that has Lake Formation permissions to access the table; the error indicates the job's IAM role lacks necessary permissions. Option A is wrong because S3 permissions are managed by Lake Formation, but access is granted via Lake Formation, not direct S3 bucket policies. Option B is wrong because the table already exists; there is no need to register.

Option D is wrong because Lake Formation does not require a VPC endpoint.

Option A (use coalesce) reduces the number of output files by merging partitions. Option C (enable partition pruning) helps queries skip irrelevant partitions, improving performance. Option B is wrong because increasing the number of workers may increase the number of output files.

Option D is wrong because disabling compression increases file size. Option E is wrong because using dynamic frame with larger partition size does not directly minimize file count.

Option A (Use CloudWatch Logs subscription filter with Kinesis Data Firehose to deliver to S3) is correct because Firehose can deliver to S3 with low latency (typically <1 minute). Option B (CloudWatch Logs export to S3) is a batch process that can take hours. Option C (Use a Lambda function to read from CloudWatch Logs and write to S3) is slower and less reliable.

Option D (Use CloudWatch Logs Insights) is for querying, not delivery.

Option B is correct. Increasing the number of shards increases the number of Lambda consumers, allowing more parallel processing and reducing the iterator age. Option A is wrong because increasing Lambda memory may not improve throughput if the function is CPU-bound; also, the issue is concurrency, not memory.

Option C is wrong because decreasing batch size may increase overhead; the function is already maxing concurrency. Option D is wrong because reducing memory would likely make the function slower.

Option C is correct. If the function times out, increasing the timeout gives it more time to complete. Option A is wrong because the event is correct.

Option B is wrong because the function can access DynamoDB with the right permissions. Option D is wrong because the event is from S3.

Option C is correct because the Lambda function needs permission to list the bucket (s3:ListBucket) to verify the bucket exists and possibly to check for existing objects. The 's3:PutObject' action is allowed, but without 's3:ListBucket', the function may fail when trying to write if the key does not exist or for bucket-level operations. Option A is wrong because 's3:GetObject' is already present.

Option B is wrong because 's3:DeleteObject' is not needed. Option D is wrong because 's3:PutObjectAcl' is not required.

Option A is correct because S3 Event Notifications can invoke Lambda on PutObject events. Option B is wrong because Kinesis is for streaming, not event-based S3 triggers. Option C is wrong because SQS would require polling.

Option D is wrong because EventBridge is more complex for simple S3 object creation triggers.

Option A is correct because the error indicates the output path does not exist; the Glue job may not have permission to create the directory if the bucket or prefix does not exist. Option B (IAM role) would cause a permissions error, not 'No such file or directory'. Option C (schema mismatch) would cause a different error.

Option D (memory) would cause out-of-memory or timeout errors.

Option C is correct because AWS Glue supports Python shell jobs with extra Python modules via S3 paths. Option A is wrong because --additional-python-modules is not valid. Option B is wrong because Lambda has a deployment package limit.

Option D is wrong because Glue does not support pip install directly.

Amazon AppFlow natively integrates with Salesforce and can write to S3. AWS Glue can also connect via JDBC. Option A is correct.

Option B is wrong because Kinesis Data Streams is for real-time streaming, not batch from SaaS. Option C is correct. Option D is wrong because Transfer Family is for FTP.

Option E is wrong because DataSync is for file/object storage, not SaaS.

Option C is correct because Lake Formation permissions are required to create databases and tables in the Data Catalog. IAM permissions alone are insufficient; the Glue role must have Lake Formation `CreateDatabase` and `CreateTable` permissions. Option A lacks Lake Formation.

Option B is too broad. Option D lacks S3 write.

Option B is correct. The event notification sends to SQS, not directly to Lambda. To invoke Lambda, the queue must be configured as an event source for Lambda.

Option A is wrong because the prefix is correct. Option C is wrong because permissions are needed but not the primary cause; the Lambda function is not triggered by SQS events unless Lambda polls the queue. Option D is wrong because the queue exists and is configured.

Option C is correct. DMS may re-apply changes from the transaction log if the target tables have no unique constraints, leading to duplicate rows. Option A is wrong because DMS uses CDC, not a full load.

Option B is wrong because PostgreSQL is case-sensitive, but that would cause missing rows, not extras. Option D is wrong because binary replication is not relevant.

Option D is correct because Lambda functions may be invoked multiple times for the same record if they fail or time out, leading to duplicates if the operation is not idempotent. Option A is wrong because DynamoDB streams guarantee at-least-once delivery, not exactly-once. Option B is wrong because Kinesis shard rebalancing does not cause duplicates.

Option C is wrong because Lambda concurrency limits would cause throttling, not duplicates.

Option D is correct because DynamoDB Streams captures changes (inserts, updates, deletes) and can be read by AWS Lambda to write to S3, enabling incremental processing. Option A (Scan with LastEvaluatedKey) still scans entire table. Option B (GSI) does not capture changes automatically.

Option C (DMS) is overkill and not as seamless with DynamoDB Streams.

Option B is correct because AWS Glue's 'Relationalize' transform flattens nested JSON into a relational format. Option A is wrong because 'FindMatches' is for deduplication. Option C is wrong because 'DropNullFields' removes null fields.

Option D is wrong because 'Map' is for simple field mappings.

Option B is correct because Glue job bookmarks automatically track the last processed files in S3 and only process new data in subsequent runs. Option A is wrong because S3 Event Notifications can trigger the job but do not track processed files; the job would need to handle deduplication. Option C is wrong because a DynamoDB table would require custom code to track state.

Option D is wrong because checking last modified timestamp manually is error-prone and not built-in.

Option A and D are correct. Kinesis Data Streams provides low-latency ingestion, and Kinesis Data Analytics performs real-time aggregations. Option B is wrong because Kinesis Data Firehose has higher latency (minutes).

Option C is wrong because S3 is storage, not stream processing. Option E is wrong because Glue ETL is batch-oriented.

Increasing DPUs or enabling auto-scaling can improve performance. Option A is correct. Option B is wrong because reducing DPUs would slow the job.

Option C is wrong because using a different engine may not be compatible. Option D is wrong because Spark UI is for monitoring, not performance improvement.

Option A, Option C, and Option D are correct. Kinesis Data Streams requires custom consumers; Firehose is fully managed. Streams provide sub-second latency; Firehose has buffer intervals.

Streams allow custom processing; Firehose has limited transformation options. Option B is wrong because both scale. Option E is wrong because both support compression for destinations.

Implementing a dead-letter queue and retry logic with exponential backoff is best practice. Option A is correct. Option B is wrong because increasing Lambda concurrency may worsen throttling.

Option C is wrong because KCL adds complexity. Option D is wrong because DynamoDB Streams is for change data capture, not for throttling errors.

Option A is correct because increasing the number of workers (DPUs) provides more memory and processing capacity. Option B (increasing S3 partitions) may help with parallelism but not directly with memory. Option C (using Spark SQL) is not a direct fix.

Option D (switching to Athena) changes the architecture.

Option D is correct because AWS DMS with change data capture (CDC) can continuously replicate changes with minimal setup. Option A (full export) is wasteful. Option B (Glue with JDBC) does not natively support CDC.

Option C (custom script) increases maintenance.

Correct options: A, C, and D. Parquet, ORC, and Avro all support compression and are commonly used with Spark. Option B is wrong because JSON can be compressed but is not a columnar format and is less efficient; however, it does support compression, but the question asks for formats commonly used with Spark for analytics; Parquet, ORC, and Avro are typical.

Option E is wrong because CSV can be compressed but is not efficient for Spark processing. The answer is A, C, D.

The policy allows GetObject and PutObject on the bucket, but the Glue job also needs 's3:ListBucket' permission to list objects in the bucket. Without ListBucket, the job cannot enumerate files. Option A is wrong because the resource is correct ('/*' includes all objects).

Option C is wrong because the action 'glue:UpdateTable' is allowed. Option D is wrong because there is no condition key issue.

Option B is correct because using a Glue crawler to update the partition metadata and then using dynamic frame with from_catalog allows Glue to automatically discover all existing partitions. This eliminates the need for hardcoded paths. Option A (try-catch) is a workaround but not a proper solution.

Option C (increase retries) does not fix the root cause. Option D (use Spark SQL with manual partition pruning) still requires knowing which partitions exist.

Option A is correct because AWS Step Functions can orchestrate multiple Glue jobs with error handling and retries. Option B is wrong because Amazon MWAA (Airflow) can also orchestrate but is more complex. Option C is wrong because Amazon CloudWatch Events schedules jobs but does not handle dependencies.

Option D is wrong because AWS Lambda is not designed for orchestration of long-running jobs.