CCNA Data Ingestion and Transformation Questions

Option C is correct because AWS DMS can continuously replicate changes from a source database to S3. Option A is wrong because Kinesis Data Streams is for custom streaming applications. Option B is wrong because S3 Transfer Acceleration speeds up uploads but does not capture CDC.

Option D is wrong because Athena is a query service.

Option C is correct because using a Job Bookmark in Glue tracks processed data and skips already processed files, processing only new ones. Option A is wrong because it would reprocess all files. Option B is wrong because partitioning alone does not prevent reprocessing.

Option D is wrong because an SQS event can trigger a job, but without bookmarks, it may still reprocess.

Option A is correct because AWS Lambda can run code in response to Kinesis Data Streams events and perform transformations before writing to S3. Option B is incorrect because Kinesis Data Analytics is for running SQL/Java on streams, not simple transforms. Option C is incorrect because Kinesis Data Firehose is for loading data to destinations with optional simple transformations via Lambda.

Option D is incorrect because Amazon S3 is storage, not a transformation service.

Option B is correct because Amazon Kinesis Data Firehose is a fully managed service for loading streaming data into S3, and it can convert data to columnar formats like Parquet and ORC. Option A (Kinesis Data Streams) requires custom consumers. Option C (Kinesis Data Analytics) processes data but does not load to S3 directly.

Option D (SQS) is a message queue, not designed for streaming ingestion.

Option C is correct because AWS DMS can perform ongoing replication with minimal impact, and S3 as a target is supported. Option A (AWS Glue) is a batch ETL tool but may put load on the source. Option B (AWS Data Pipeline) is older and less efficient.

Option D (Amazon Kinesis) is for streaming, not batch database loads.

Kinesis Data Firehose requires a schema to convert to Parquet. This schema can be provided by a Glue Data Catalog table. If the schema is not defined, the conversion fails.

Data size is not an issue for conversion. Kinesis Data Streams is not involved here. Lambda transformation can convert to Parquet but is not required.

Option B is correct because Kinesis Data Firehose can subscribe to a Kinesis Data Stream, transform data using Lambda or its built-in transformations, and automatically deliver to S3 with no code required for delivery. Option A is wrong because Kinesis Data Analytics is for real-time analytics, not direct delivery to S3. Option C is wrong because AWS Glue is a batch ETL service, not real-time.

Option D is wrong because Amazon EMR is a managed Hadoop cluster that requires significant overhead.

Option A is correct because Glue's built-in transform 'Relationalize' converts nested JSON into flat tables. Option B is wrong because 'FindMatches' is for deduplication. Option C is wrong because 'Spigot' is for sampling.

Option D is wrong because 'DropFields' is for removing fields, not handling nesting.

AWS Glue can run ETL jobs to transform S3 data and load into Redshift. Amazon Redshift Spectrum can query S3 data directly and load results into Redshift tables. Amazon Athena is for querying, not loading.

Amazon EMR requires cluster management. AWS Lambda is for small data volumes.

Option B (Use AWS DMS with ongoing replication from Oracle to S3) is correct because DMS supports change data capture (CDC) with low latency. Option A (Full export using AWS Glue) does not capture incremental changes in near real-time. Option C (Use Kinesis Data Firehose with Oracle as source) is not directly possible without a connector.

Option D (Use AWS DataSync) is for file transfers, not database CDC.

Option B is correct because reducing the buffer interval to 60 seconds (minimum 60s for Firehose) forces Firehose to deliver data more frequently, reducing latency. Option A (increasing buffer size) would increase latency. Option C (increasing shards) does not apply to Firehose directly; shards are for Kinesis Data Streams.

Option D (buffering hints) is not a direct configuration in Firehose.

Option A and Option D are correct because API Gateway can receive webhooks and route to Firehose or S3. Option B is wrong because Glue cannot directly receive webhooks. Option C is wrong because Kinesis Data Streams requires custom producers.

Option E is wrong because Lambda can receive but not directly store to S3 without code.

Options A, B, and D are correct. Bookmarks track processed data for incremental jobs. DynamicFrames are flexible for schema evolution.

Partition pruning helps read only relevant partitions. Option C (Spark SQL) is not specific to incremental. Option E (JobWatch) is not an AWS Glue feature.

Options A, C, and D are correct. Implementing retry logic with exponential backoff handles transient throttling. Increasing the service quota for Glue API calls if the limit is reached.

Using Glue job bookmarking reduces reprocessing, thus lowering API calls. Option B (increasing DPUs) addresses resource issues but not API throttling. Option E (changing to Spark SQL) does not affect API calls.

Option A is correct because if the crawler is configured with a custom classifier that expects a fixed schema, it will ignore new columns. Option B is wrong because partition discovery does not prevent schema updates. Option C is wrong because S3 event notifications are not related.

Option D is wrong because IAM permissions would cause a different error.

Option B is correct because AWS DMS can perform a full load from on-premises PostgreSQL to S3 efficiently, and it's a managed service. Option A (CLI) would be manual and slow. Option C (Snowball) is for larger datasets or slow networks; here 50 GB at 1 Gbps takes ~7 minutes, so Snowball is overkill.

Option D (Lambda + pg_dump) adds custom code and complexity.

Correct options: A and C. The number of shards determines the maximum number of concurrent batches (one per shard). The batch size (records per invocation) also affects concurrency because smaller batches lead to more invocations.

Option B is wrong because DynamoDB table type does not affect Lambda concurrency. Option D is wrong because Lambda memory is not directly related to concurrency. Option E is wrong because stream name does not affect concurrency.

Option D is correct: Increase the buffer size to 100 MB and the buffer interval to 300 seconds to allow more data to accumulate before writing. Option A (Lambda) would not aggregate efficiently. Option B (Glue) adds latency and cost.

Option C (EMR) is overkill.

A is correct because using a partition key of stock symbol ensures records with the same symbol go to the same shard, preserving order. D is correct because enabling the Kinesis Client Library (KCL) with sequence number ordering ensures records are processed in order within a shard. B is wrong because increasing shards does not guarantee ordering.

C is wrong because using a random partition key would distribute records across shards, breaking order. E is wrong because DynamoDB Streams is for CDC, not for streaming order.

Using Amazon S3 file grouping or converting to columnar format like Parquet reduces the number of files and improves read performance. Increasing workers helps, but file consolidation is more impactful. Using G.1X worker type may help, but grouping files is key.

Using Spark SQL directly does not address the small files problem.

Option C is correct because AWS Snowball is a physical device that can transfer large amounts of data faster than over the network given the bandwidth limitation. Option A (DataSync) can use the network but may not complete within a week due to bandwidth constraints. Option B (Glue) is for ETL, not raw transfer.

Option D (Transfer Family) is for SFTP.

Options A and C are correct. To enable ongoing replication, DMS needs to have full load completed (A) and the source database must have continuous change capture enabled (e.g., binary logs for MySQL) (C). Option B is wrong because the target table must exist before replication.

Option D is wrong because DMS does not need a VPC endpoint; it connects via the internet or VPN. Option E is wrong because transformation rules are optional.

Option D is correct because the error 'Cannot merge incompatible schemas' indicates differing schemas in the source data. Option A is wrong because insufficient DPUs cause memory errors, not schema conflicts. Option B is wrong because partition pruning is about filtering, not schema.

Option C is wrong because bookmarks track processed files, not schema issues.

AWS DMS can continuously replicate data from the source database to S3 in CDC mode. AWS Glue can then detect schema changes and transform the data before loading into Redshift. Kinesis Data Streams is for streaming data, not database CDC.

Athena is for querying, not for handling schema evolution. Lambda alone would require custom code for schema detection.

Glue's 'Update schema' option in the job's output target allows the job to automatically incorporate new columns. Option A is wrong because a crawler after the job would add lag. Option B is wrong because schema-on-read is not automatic.

Option D is wrong because catalog partitions are for partitioning, not schema evolution.

Option C is correct because Kafka Connect with S3 Sink Connector supports exactly-once semantics and custom transformations via Single Message Transforms (SMTs) or custom connectors. Option A (Kinesis Data Analytics for Apache Flink) is for Flink, not Kafka. Option B (AWS Lambda) does not provide exactly-once semantics from MSK.

Option D (Amazon EMR) is for batch processing.

Option D is correct: Use S3 event notifications for s3:ObjectCreated:Put, which triggers only after the object is completely written. Then Lambda can start MediaConvert. Option A (increase timeout) does not solve the root cause.

Option B (Step Functions) adds complexity. Option C (SQS) does not address the event timing.

Option D is correct because AWS Glue ETL jobs can read from S3, connect to RDS via JDBC, perform joins and transformations, and write to Redshift. Option A (Athena) can query S3 but cannot join with RDS natively. Option B (EMR) is possible but more complex to set up.

Option C (Data Pipeline) is older and less integrated.

Option B is correct because Kinesis Data Firehose can convert the input data format to Parquet using a schema from AWS Glue. Option A is incorrect because Lambda can do this but incurring additional compute cost. Option C is incorrect because Athena queries raw data and would not help with ingestion.

Option D is incorrect because Glue ETL would add cost and latency.

The correct answer is to increase the parallelism of the Flink application to match the number of shards (8). When parallelism is lower than the number of shards, some shards are underutilized, causing backpressure and latency. Option A (increasing checkpointing interval) would reduce overhead but not address the parallelism mismatch.

Option C (using a larger instance type) could help but is less effective than matching parallelism. Option D (decreasing batch size) is not applicable to Flink. Option B directly fixes the bottleneck.

Option B is correct because increasing the number of shards increases the ingestion capacity. Option A is wrong because Kinesis Data Firehose would add latency. Option C is wrong because the partition key is already random (UUID), which distributes data well.

Option D is wrong because increasing the retention period does not affect throughput.

Option A is correct. Using DynamoDB Streams with a Lambda function that writes to S3 is a common pattern for real-time replication with minimal impact. Option B is wrong because Export to S3 is a one-time or scheduled export, not continuous.

Option C is wrong because using Scan would consume read capacity and impact performance. Option D is wrong because adding a secondary index does not help with exporting data.

Option D is correct because increasing the number of DPUs (Data Processing Units) in AWS Glue ETL jobs can significantly reduce runtime by parallelizing the transformation. Option A is wrong because Auto Scaling is enabled by default and may not help if DPUs are already maxed. Option B is wrong because converting to Parquet can help with reading from S3, but the source is RDS, not S3.

Option C is wrong because writing to S3 using dynamic frames does not directly improve performance of reading from RDS.

Option A is correct because using a DynamoDB table to store schema metadata and a Lambda function to route records to the appropriate Firehose delivery stream is efficient and scalable. Option B is wrong because it requires multiple Kinesis streams, increasing cost and complexity. Option C is wrong because ingesting into S3 and then using Glue to route adds latency.

Option D is wrong because using Kinesis Data Analytics for routing is overkill and not designed for this purpose.

Option A is correct because with 2 MB/s, the buffer size of 5 MB will be reached in 2.5 seconds, but the buffer interval is 60 seconds. Firehose writes when either condition is met first. Since the buffer size is reached much earlier, writes will occur approximately every 2.5 seconds.

Option B, C, D are incorrect because they are based on the interval or miscalculations.

Option B is correct: S3 event notifications can be configured to invoke Lambda when a new object is created, and Lambda can start the Glue job. Option A (CloudWatch Events) cannot directly monitor S3 object creation. Option C (SQS) is not needed.

Option D (Step Functions) adds unnecessary complexity.

Option B is correct because AWS DMS supports heterogeneous database migrations and continuous replication to S3. Option A is wrong because Glue is batch-oriented. Option C is wrong because Kinesis Data Streams is for streaming data, not database replication.

Option D is wrong because Snowball is for large offline transfers.

Option B is correct because AWS Transfer Family supports SFTP and can be used with a VPC endpoint to transfer data from an FTP server in a private network to S3. Option A is wrong because Snowball Edge is for large offline data transfers, not regular FTP transfers. Option C is wrong because DataSync is for moving data between on-premises storage and AWS, but it requires network access.

Option D is wrong because Kinesis Data Firehose is for streaming data, not FTP transfers.

Option D is correct. Kinesis Data Streams ingests streaming data, Kinesis Data Firehose delivers data to S3 with built-in Parquet conversion and partitioning, and optionally uses Lambda for lightweight transformations. Option A is wrong because EC2-based ingestion is high overhead.

Option B is wrong because S3 Transfer Acceleration is for large file transfers, not streaming. Option C is wrong because Kafka and Flink require more operational overhead.

Option B is correct because the error 'Rate exceeded' indicates the Glue API rate limit has been reached. The most likely cause is multiple concurrent job runs or too many API calls. Implementing retry logic with exponential backoff in the job or script that triggers the job will handle transient throttling.

Option A (increasing DPUs) does not affect API rate. Option C (deleting old job runs) might free up quota but is not the best practice. Option D (using reserved capacity) is not applicable for Glue API calls.

Option C is correct because AWS Batch is designed for long-running, compute-intensive jobs like video processing. Option A (increase Lambda timeout) is not a good practice for such heavy tasks. Option B (EC2 instance) requires manual management.

Option D (Glue) is for ETL, not video processing.

Option C is correct because increasing the number of shards increases the stream's capacity and allows more concurrent Lambda invocations. Option A is incorrect because Lambda concurrency might be a limit but the root cause is shard count. Option B is incorrect because increasing Lambda memory may help but not as much as more shards.

Option D is incorrect because Firehose would add a separate step.

Amazon Kinesis Data Firehose can buffer and batch data to S3 with partitioning. Option A is correct. Option B is wrong because Lambda cold starts can cause latency.

Option C is wrong because Glue is for batch ETL, not real-time. Option D is wrong because KCL requires custom application.

Options B and D are correct. Enabling schema update in the Glue crawler (B) allows the crawler to update the existing table schema when new columns are found. Running the crawler on a schedule (D) ensures that changes are captured periodically.

Option A (manual schema update) is not automated. Option C (using a single schema) is not practical for evolving data. Option E (disabling schema update) would not update the catalog.

Option A and Option D are correct because Lambda transformation can modify records, and Glue ETL can process data from Firehose's S3 destination. Option B is wrong because Kinesis Data Analytics is for analytics, not transformation. Option C is wrong because S3 Select is for retrieving subsets of data, not transformation.

Option E is wrong because Redshift Spectrum is for querying data in S3.

Option A is correct because Kinesis Data Streams can handle high throughput spikes, Lambda can process and write to S3, and Athena can query directly. Option B is wrong because SQS is pull-based and does not support push to S3 directly; Lambda would need to poll, adding latency. Option C is wrong because DMS is for database migration, not real-time clickstream.

Option D is wrong because Glue is batch-oriented and not suitable for low-latency streaming.

Option B is correct because Amazon Kinesis Data Firehose is a fully managed service that can ingest streaming data and deliver it to S3 with near-real-time latency (typically under 60 seconds). Option A is wrong because AWS Database Migration Service is for database migration, not streaming ingestion. Option C is wrong because Amazon Kinesis Data Streams requires custom consumers and does not directly deliver to S3.

Option D is wrong because Amazon S3 Transfer Acceleration speeds up uploads but does not provide streaming ingestion capabilities.

The correct answer is AWS Snowball Edge, a physical device for large data transfers over slow networks. 50 TB at 100 Mbps would take over 46 days, exceeding the 2-week deadline. Option A (AWS DataSync) uses the network and would be too slow. Option B (AWS Glue) is for ETL, not transfer.

Option D (Amazon Kinesis) is for streaming.

Options A and D are correct. Increase the checkpoint interval to reduce frequency, and increase parallelism with more KPUs. Option B is wrong because reducing parallelism would worsen lag.

Option C is wrong because decreasing checkpoint interval increases failures. Option E is wrong because Kinesis Data Firehose is not a direct solution to checkpoint failures.

Options A and C are correct. Kinesis Data Streams ingests streaming data, and Kinesis Data Firehose can deliver data from the stream to S3. Option B is wrong because SQS is a message queue, not for real-time processing.

Option D is wrong because Glue is batch. Option E is wrong because Direct Connect doesn't apply.

Option C is correct because increasing the number of DPUs provides more memory. Option A is wrong because reducing data volume is not a solution. Option B is wrong because using a smaller file size may help but not directly address memory.

Option D is wrong because Glue does not support memory configuration directly.

A is correct because a DMS replication instance is needed to run the migration tasks. B is correct because a source endpoint for Oracle is required. E is correct because a target endpoint for RDS PostgreSQL is required.

C is wrong because an S3 bucket is not required unless using S3 as a target. D is wrong because a Snowball device is for large offline transfers, not for DMS.

Option C is correct because increasing the number of DPUs provides more memory for processing. Option A is wrong because reducing parallelism may increase memory per worker but typically reduces overall throughput. Option B is wrong because S3 Select filters data server-side but does not reduce memory usage in Glue.

Option D is wrong because using a smaller instance type would exacerbate the memory issue.

Option C is correct because Kinesis Data Firehose can directly accept HTTP POST requests via the Kinesis Agent or direct API calls, and it automatically delivers data to S3 with buffering, requiring no servers to manage. Option A is wrong because running an EC2 instance adds operational overhead and cost even if it's small. Option B is wrong because SQS also requires a consumer (e.g., Lambda) to move data to S3, adding complexity.

Option D is wrong because API Gateway and Lambda add unnecessary layers and cost for simple ingestion.

Options A, C, and D are correct. Option A: Using a read replica avoids load on the primary database. Option C: Using JDBC connection with a WHERE clause to extract only new/changed data reduces the amount of data read.

Option D: Scheduling Glue jobs during off-peak hours minimizes impact. Option B is wrong because increasing DPUs does not reduce database load; it may increase parallelism and actually increase load. Option E is wrong because reducing job bookmarks would cause full scans, increasing load.

Options B and D are correct. AWS Transfer Family provides a fully managed SFTP service that can transfer files directly to S3 with encryption in transit. AWS Glue can then be used to process the CSV files in S3.

Option A is wrong because Lambda has a 15-minute timeout and may not handle large files. Option C is wrong because Kinesis is for streaming data, not batch file transfers. Option E is wrong because Athena is a query service, not a transfer service.

Option D (Use Kinesis Data Firehose as the delivery stream with Lambda for transformation and configure error handling with retries and a backup S3 bucket) is correct because Firehose provides built-in retry and error handling. Option A (Increase Lambda timeout) does not solve transient errors. Option B (Use a DLQ) helps capture failures but does not automatically retry.

Option C (Use Kinesis Client Library) requires more custom code.

Option A is correct because Firehose can natively convert JSON to Parquet using a schema from AWS Glue Data Catalog, without custom code. Option B (Lambda) requires writing and maintaining code. Option C (Kinesis Data Analytics) is overkill.

Option D (EC2) adds management overhead.

Option A is correct because the buffer interval determines how often data is delivered to the destination. Changing it from 300 seconds to 60 seconds will deliver data every minute. Option B (buffer size) affects delivery based on data volume, not time.

Option C (compression) does not affect frequency. Option D (partitioning) does not affect delivery frequency.

Option B is correct because the policy allows s3:PutObject on 'my-bucket/*' but the function might be writing to a different bucket or the resource ARN is missing permissions for the bucket itself (e.g., s3:ListBucket). Option A is wrong because the policy includes GetRecords. Option C is wrong because the policy has kinesis actions.

Option D is wrong because Kinesis Data Analytics is not mentioned.