CCNA Data Operations and Support Questions

In Kinesis Data Analytics with Flink, the default parallelism is 1. With 10 shards in the source Kinesis stream, a single parallel task must read from all shards, creating a bottleneck and causing the 'MillisBehindLatest' metric to be consistently high (above 60000). Option A is wrong because while a throttling sink can cause backpressure, the question specifically states the application is falling behind and the metric is high, which is more directly explained by insufficient parallelism.

Option C is wrong because 10 shards is typically sufficient; the issue is how they are consumed. Option D is wrong because the retention period does not affect the lag metric; it only controls how long data is stored.

Enabling Apache Flink metrics in Amazon CloudWatch provides visibility into heap usage, checkpoint sizes, and backpressure, which can help diagnose the root cause of memory failures. After increasing parallelism and TaskManager memory without success, the next best step is to monitor these metrics to identify the specific bottleneck. Option A changes processing semantics from exactly-once to at-least-once, which can reduce overhead but does not help diagnose the memory issue and may alter data delivery guarantees.

Option B reduces the number of shards in the source stream, which decreases throughput and might not address the underlying memory consumption. Option D increases the buffer timeout for the S3 sink, which could accumulate more data in memory before writing, potentially worsening the memory problem.

DynamoDB on-demand capacity automatically scales based on traffic but has a maximum throughput limit determined by the table's previous peak usage. When a new peak exceeds this limit, throttling occurs until the table adapts. Contacting AWS Support to pre-warm the table raises the initial throughput limit, allowing higher bursts immediately.

Option B is incorrect because increasing partition count does not directly increase the overall throughput limit; it affects distribution of throughput. Option A is incorrect because DAX is a caching layer that improves read performance, not write throughput. Option D is incorrect because switching to provisioned capacity would require setting WCU to 50,000, but this changes the billing model and may still require a limit increase; pre-warming is the direct solution for on-demand throttling.

Using the AWS Glue 'dropDuplicates' transformation on the 'id' column, ordering by 'timestamp' descending, will remove duplicate 'id' values, keeping the latest record. This is the direct approach within the Glue ETL script. Option B is incorrect because using 'overwrite' mode would replace the entire Redshift table, which would discard existing data and is not a targeted solution for handling duplicates.

Option C is incorrect because loading into a staging table and using MERGE requires additional setup and is not directly part of the Glue ETL script, making it less efficient for the stated requirement. Option D is incorrect because disabling primary key constraints only defers the duplicate key error, and it would compromise data integrity rather than properly deduplicating records.

Options A, B, and E are correct. Option A: Glue's 'recast' transformation can handle type changes by converting data types as needed. Option B: Setting the Glue crawler to update the table's schema in the Data Catalog ensures that new columns or changes in the source DynamoDB table are reflected.

Option E: Using Spark's 'mergeSchema' option when writing to S3 allows Parquet files to have differing schemas, enabling schema evolution. Option C is incorrect because converting Parquet to CSV does not help with schema evolution and may introduce data loss or inefficiency. Option D is incorrect because partitioning by date and deleting old partitions is a data retention strategy, not a schema evolution technique.

The correct first step is to verify the S3 event notification configuration (Option C). Since there are no execution logs in CloudWatch, the Lambda function is not being triggered at all. This indicates a problem with the trigger mechanism, not with the function's code, permissions, or network configuration.

Option A (checking code) is premature because code errors would appear in logs after invocation. Option B (IAM role) affects what the function can do during execution, not whether it is invoked. Option D (VPC) affects network access but does not prevent triggering.

Therefore, the S3 bucket's event notification must be checked to ensure it is correctly set up to invoke the Lambda function.

Enabling pg_stat_statements in the PostgreSQL parameter group and querying the pg_stat_statements view provides detailed query execution statistics (e.g., total execution time, calls, rows) and helps identify high‑load queries. The execution plan can then be obtained by running EXPLAIN on the identified query. Option A is incorrect because CloudWatch Logs captures logs but does not provide real‑time query performance details.

Option C is incorrect because Enhanced Monitoring shows OS‑level metrics (CPU, memory), not query‑level plans or statistics. Option D is incorrect because RDS Performance Insights identifies top queries by wait events and load but does not directly expose the execution plan; the execution plan is best obtained via EXPLAIN on the specific query identified from pg_stat_statements.

Options B and C are correct. Multi-AZ deployment with a standby instance enables automatic failover, typically achieving an RTO of under 1-2 minutes. Combined with automated backups and point-in-time recovery, which allow database restoration to any point within seconds (RPO of less than 1 minute), the requirements are met.

Option A is incorrect because manual snapshots are not frequent enough to guarantee RPO under 1 minute and restoring from a snapshot takes longer than 5 minutes. Option D is incorrect because a read replica in a different Availability Zone is not designed for automatic failover; it is for read scaling and requires manual promotion. Option E is incorrect because Amazon Aurora Global Database is a different service; this question is about Amazon RDS for PostgreSQL, and cross-Region replication for RDS does not provide automatic failover and typically has higher RTO.

High memory utilization (90%) with low CPU (<20%) indicates that the data does not fit in memory, causing frequent spills to disk. Using memory-optimized instances (r5.xlarge) provides more memory per vCPU compared to m5.xlarge, allowing more data to be kept in memory and reducing spills. Option B is incorrect because increasing the number of core nodes adds more CPU and memory overall, but each node still has the same memory-to-CPU ratio (8 GB per 4 vCPUs for m5.xlarge), so memory pressure per node remains.

Option C is incorrect because the issue is insufficient memory, not the number of shuffle partitions; adjusting shuffle partitions does not increase available memory. Option D is incorrect because decreasing the number of core nodes reduces total cluster memory, worsening the memory bottleneck.

Options A, D, and E are correct. To recover an Amazon Redshift cluster in a different region (us-west-2) from us-east-1, you need to have snapshots available in the target region. Manual snapshots (option A) can be copied to us-west-2 and are retained even if the cluster is deleted.

Automated snapshots with a retention period (option D) ensure regular backups, and by configuring automated snapshot copy to us-west-2 (option E), these snapshots are automatically replicated to the target region for cross-region recovery. Option B (multi-AZ) provides high availability within a single region but does not help with cross-region disaster recovery. Option C (concurrent scaling) improves query performance under load, not disaster recovery.

The error indicates that transaction_id is being nullified or dropped during the Glue job's mapping transformation. Even though source files have non-null values, the mapping could be incorrectly mapping or omitting the column, causing nulls to be written to Redshift. The failure during peak hours is coincidental; the root cause is the mapping logic.

Option A is incorrect because filtering nulls would not fix the mapping error and could discard valid data. Option B is incorrect because increasing Redshift WLM concurrency scaling does not address the null constraint violation. Option D is incorrect because more Glue workers do not fix the transformation issue; the problem is data quality, not capacity.

The COPY command is the most efficient way to load data from Amazon S3 into Redshift, enabling high-speed parallel ingestion. Option C is correct because using columnar formats like Parquet minimizes data scanned and reduces storage costs, speeding up data transfer. Option D is correct because running VACUUM and ANALYZE after loading reorganizes data and updates statistics, optimizing query performance.

Option A is incorrect because increasing WLM queues does not improve COPY performance; COPY operations bypass WLM. Option E is incorrect because disabling distribution keys can cause data skew and degraded performance; proper distribution keys are essential for efficient cluster operation.

S3 server-side encryption (SSE-S3) encrypts data at rest in S3. HTTPS (TLS) encrypts data in transit between the data source and Kinesis Data Firehose, and between Firehose and S3. AWS CloudTrail logs S3 API calls (e.g., GetObject, PutObject) for auditing data access.

Option A is incorrect because Kinesis Data Analytics does not provide encryption in transit; it processes data but does not handle encryption. Option B is incorrect because CloudWatch Logs is for monitoring and storing logs, not for auditing data access; CloudTrail is the appropriate service for auditing. Option D is incorrect because client-side encryption requires manual key management and does not use AWS-managed encryption; AWS Config tracks configuration changes, not data access.

Options C and E are correct. The Amazon Redshift Data API has a default timeout of 10 minutes for a single API call. To handle queries that take longer, you can run the query asynchronously by using the StatementName parameter (option E).

This allows the query to continue running even if the initial API call times out, and you can poll for completion using the DescribeStatement operation. Once the query completes, you can retrieve the results using the GetStatementResult operation (option C). Option A is incorrect because QueryExecutionTimeout is not a valid parameter for the Data API.

Option B is incorrect because increasing the timeout parameter in the Redshift cluster configuration does not affect the Data API timeout. Option D is incorrect because there is no max_execution_time parameter in Redshift; the relevant parameter is statement_timeout, which controls how long a query can run before being canceled, but it does not address the Data API timeout.

Options B, D, and E are correct. B: S3 Select can retrieve only a subset of data, reducing the amount scanned and improving read performance. D: Larger average object sizes allow better throughput by leveraging parallel requests and reducing overhead.

E: Storing data in compressed formats reduces the amount of data transferred, improving read performance. Option A is incorrect because S3 Transfer Acceleration is designed to improve upload speed over long distances, not read performance. Option C is incorrect because S3 Object Lock prevents object deletion or overwrite but does not affect read performance.

The access denied error occurs because the IAM role used by Athena lacks the necessary S3 permissions, even though Lake Formation has granted SELECT and DESCRIBE permissions. Lake Formation handles permissions at the metadata level, but the actual data access still requires S3 bucket policy or IAM policy to allow s3:GetObject and s3:ListBucket for the specific S3 location. If the bucket policy does not grant the Athena IAM role access to the S3 location, the request will be denied.

Option B correctly identifies this issue. Option A is incorrect because Lake Formation tag-based access control is not required for this scenario; the data scientist already has direct table permissions. Option C is incorrect because lakeformation:GetEffectivePermissions is not needed; the role already has lakeformation:GetDataAccess.

Option D is incorrect because Athena uses the IAM role assigned to the workgroup, not the data scientist's individual IAM user.

Using Amazon S3 and AWS Lambda to buffer and batch records before writing to Kinesis smooths out traffic spikes, reducing throttling without manual intervention. Option A is incorrect because the Kinesis Client Library (KCL) is used by consumers, not producers; it cannot throttle write throughput. Option B is incorrect because manually increasing shards during peak hours requires manual intervention and is not a long-term automated solution.

Option C is incorrect because Amazon Kinesis Data Firehose is designed to deliver streaming data to destinations like S3, Redshift, or Elasticsearch, not to buffer records before writing to a Kinesis stream.

The IAM policy shown in the exhibit grants s3:ListBucket on the bucket and s3:GetObject on objects, which should allow listing. However, if the user still cannot list objects, the most likely reason is that a bucket policy explicitly denies access to that user. Bucket policies are evaluated separately from IAM policies, and an explicit deny in a bucket policy overrides any allow.

Options B, C, and D are incorrect because the IAM policy in the exhibit includes the correct resource ARN (arn:aws:s3:::example-bucket for the bucket and arn:aws:s3:::example-bucket/* for objects), includes s3:ListBucket action, and does not require s3:GetObject for listing.

In AWS Lake Formation, even when a user has SELECT permission on a table, the IAM role that Athena uses must have S3 GetObject permission on the underlying data files. Lake Formation manages permissions at the metadata level, but the actual data access is enforced by S3 bucket policies and IAM. Without GetObject permission, Athena fails with 'Access Denied'.

Option A is incorrect because Athena typically has access to the Glue Data Catalog if the table is visible; the error is about data access. Option C is incorrect because SELECT permission usually includes DESCRIBE, and the error message is about access denied, not missing describe. Option D is incorrect because the table is registered with Lake Formation since the analyst has SELECT permission.

Options B and E are correct. AWS Snowball Edge is a physical device suitable for transferring large amounts of data (like 50 TB) when network bandwidth is limited (500 Mbps). AWS DataSync can also transfer data over the network with built-in optimization, making it appropriate for this scenario.

Options A and C are incorrect: AWS Direct Connect provides a dedicated connection but does not increase bandwidth beyond 500 Mbps, and S3 Transfer Acceleration speeds up transfers but still relies on the same limited bandwidth. Option D is incorrect: Amazon Kinesis Data Firehose is designed for real-time streaming, not bulk data transfer.

The 'ThrottlingException' error occurs when the rate of API requests exceeds the allowed limit. In this scenario, the Glue job writes to Amazon Redshift. Redshift has a provisioned write throughput capacity; if the Glue job attempts to write data faster than Redshift can handle, Redshift throttles the requests, resulting in a ThrottlingException.

This is the most likely cause. Option A is incorrect because S3 throttling would manifest as a different error (e.g., 'SlowDown' or 'RequestTimeout'). Option C is incorrect because Glue job concurrency limits would cause a 'ConcurrentRunsExceededException' or similar, not ThrottlingException.

Option D is incorrect because insufficient memory would typically lead to an 'OutOfMemoryError' or job failure, not a ThrottlingException.

The error 'InvalidArgumentException: The number of partitions exceeds the limit' occurs when Kinesis Data Firehose dynamic partitioning receives more distinct partition key values than the allowed limit (default 500 distinct values per stream). Option A correctly identifies this cause. Option B is wrong because S3 bucket policy restrictions would cause AccessDenied errors, not partition limit errors.

Option C is wrong because buffer size and interval affect file size and delivery frequency, not partition count. Option D is wrong because throughput limits cause ProvisionedThroughputExceededException, not partition limit errors.

Firehose uses SSE-S3 by default unless configured otherwise. If the S3 bucket has default encryption enabled with SSE-KMS, Firehose will use that encryption, but the Glue job's IAM role may lack kms:Decrypt permission for the KMS key. The error 'Access Denied' when reading from S3 often indicates encryption permission issues.

Option A is wrong because the Glue job can read from S3 with the current permissions if no encryption is involved. Option B is wrong because the error is about access, not schema. Option C is wrong because the Glue job can use the same role as Firehose, but the role may not have KMS permissions.

Options B and D are correct. Increasing Spark shuffle partitions (B) reduces the amount of data shuffled per partition, lowering memory pressure and preventing heap overflow. Increasing the number of DPUs (D) allocates more memory and compute resources to the Glue job, directly addressing heap space limitations.

Option A is incorrect because changing the output format from Parquet to CSV does not reduce memory usage and may increase it due to lack of compression. Option C is incorrect because reducing the number of source partitions can increase partition size, worsening memory issues. Option E is incorrect because disabling job bookmarks does not affect memory usage; it may cause processing of already processed data but doesn't resolve heap space.

The error 'server closed the connection unexpectedly' typically indicates that the Redshift cluster rejected the connection, often because SSL is required but not enabled in the Glue JDBC connection. Redshift clusters can be configured with the `require_ssl` parameter set to true, forcing all connections to use SSL. Option B is correct because without SSL, the server terminates the connection.

Option A is incorrect because the 'redshift' compatibility mode is not a standard requirement; the JDBC driver works without it. Option C is incorrect because security group issues would cause a timeout or 'connection refused', not a server-side close. Option D is incorrect because Glue supports Redshift as a data source.

Increasing the number of shards in the Kinesis stream increases the number of concurrent Lambda invocations, distributing the write load across more Lambda functions and reducing the number of writes per second to DynamoDB from each invocation. This helps mitigate ProvisionedThroughputExceededException. Option A is incorrect: increasing Lambda memory does not directly affect DynamoDB write throttling.

Option B is incorrect: increasing DynamoDB read capacity does not help with write throttling. Option C is incorrect: decreasing the batch size to 1 reduces the number of records per invocation, but since each shard still invokes Lambda, it may increase the number of invocations and potentially increase the write frequency, worsening the throttling.

Enabling concurrency scaling (Option A) is the most effective action because it automatically adds transient cluster capacity during peak loads, allowing more queries to run concurrently without increasing wait times. This is specifically designed to reduce WLM query wait time. Option B (manual WLM) requires tuning and does not add capacity.

Option C (increasing max queries per queue) could increase concurrency but may lead to resource contention and longer wait times if the cluster is already saturated. Option D (short query acceleration) prioritizes short queries, which does not address overall wait times for all queries. Therefore, A is correct.

The ORA-1555 'snapshot too old' error occurs when a long-running query (such as Change Data Capture (CDC) in AWS DMS) needs to read consistent data from undo segments, but the undo information has been overwritten or retained for too short a period. Increasing the UNDO tablespace size and setting UNDO_RETENTION to a higher value ensures that undo data is preserved longer, allowing CDC to read consistent snapshots without encountering this error. Option A (disabling supplemental logging) would prevent CDC from capturing changes, thus is incorrect.

Option B (increasing redo logs) does not address the undo retention issue. Option C (enabling batch optimized apply) may improve apply performance but does not resolve the source-side undo problem.

Multi-AZ deployment provides automatic failover to a standby in another Availability Zone, meeting the RTO of 2 hours. Option B is correct because automated backups enable point-in-time recovery within the retention period, supporting the RPO of 5 minutes (default backup retention is 1 day, which is sufficient). Option E is correct because a cross-region read replica can be promoted to a standalone database in another region for faster disaster recovery if the primary region fails.

Option C is wrong because manual snapshots exported to S3 are for long-term archival and not fast enough for a 2-hour RTO. Option D is wrong because disabling automated backups would prevent point-in-time recovery and violate the RPO of 5 minutes.

Options B, D, and E are correct. B: Increasing buffer size and interval allows Firehose to batch more records before delivery, reducing the rate of PUT requests and alleviating temporary capacity issues. D: Requesting a service quota increase for Kinesis Data Firehose raises the default limits on data delivery throughput, directly addressing insufficient capacity errors caused by throttling.

E: Increasing the number of shards in the source Kinesis data stream provides higher write throughput to Firehose, reducing backpressure and 'Insufficient capacity' errors. Option A is incorrect because S3 bucket versioning handles object versioning, not write capacity or Firehose throughput. Option C is incorrect because CloudWatch alarms only monitor and alert, they do not resolve capacity issues.

Partition pruning requires the WHERE clause to filter on the partition column to reduce data scanned. Option E is correct because enabling predicate pushdown in Athena allows the query engine to push filtering conditions down to the data source, further reducing the amount of data scanned. Option A is incorrect because while Avro is a row-oriented format, it is not more efficient for analytics than Parquet; Parquet is columnar and better for selective queries.

Option C is incorrect because CSV is not compressed and would increase data scanned. Option D is incorrect because simply increasing the number of partitions without proper filtering does not improve performance; excessive partitions can even degrade performance due to metadata overhead.

Options B and D are correct. B: The EMR service role (and instance profile) must have an IAM policy granting S3 permissions; verifying these ensures the role has the necessary access. D: S3 bucket policies can include explicit deny statements that override IAM permissions, even if the IAM role allows access; checking the bucket policy can reveal such denies.

Option A is wrong because S3 versioning does not affect access permissions. Option C is wrong because disabling S3 Block Public Access is unrelated to IAM-based access from EMR. Option E is wrong because an S3 VPC endpoint is not required if the cluster can access S3 via the internet or a NAT gateway.

The correct answers are D and E. Optimizing file sizes to be around 64 MB to 256 MB (D) reduces the overhead of Athena reading many small files or processing overly large files, balancing parallelism and efficiency. Partitioning data by frequently filtered columns (E) allows Athena to prune partitions and scan less data, improving query performance.

Option A (single large file) hinders parallelism and query performance. Option B (GZIP compression) is already supported by Parquet and does not inherently optimize query performance beyond the file size considerations. Option C (many small files) increases metadata overhead and degrades performance.

The SELECT * query may include columns with data types (e.g., arrays, hstore, or geometric types) that psycopg2 does not fully support, causing those columns to be read as NULL or silently skipped during extraction. This would result in fewer rows being inserted into Redshift if the unsupported columns are part of a unique constraint or if NULL handling causes row drops. Option B is incorrect because SSL/TLS packet drops would cause connection errors, not consistent missing rows.

Option C is incorrect because autocommit=True ensures each query is a standalone transaction, preventing incomplete transactions. Option D is incorrect because the problem states the Redshift table has no constraints that would reject rows, and distribution keys do not silently discard rows.

Options A, D, and E are correct. Job bookmarks may skip new data if not configured properly (A). Dynamic frame transformations can drop rows if errors occur during processing (D).

Incorrect mapping of source columns to target columns can cause data loss (E). Option B is incorrect because disabling server-side encryption on S3 does not cause data loss; it affects data at rest encryption but not data integrity during ETL. Option C is incorrect because setting the job timeout too low may cause the job to fail prematurely, but it does not directly cause data loss; data can be reprocessed after adjusting the timeout.

Options B, C, and E are correct. To configure Kinesis Data Firehose with Lambda transformation, you need to create a Lambda function that transforms the data (B), attach an IAM role to the Firehose delivery stream that allows Firehose to invoke the Lambda function (C), and configure the Firehose delivery stream to use the Lambda function as a data transformation source (E). Option A is wrong because Lambda@Edge is used with CloudFront, not Firehose.

Option D is wrong because S3 event notifications are not used for Firehose transformation; they are used for other purposes like triggering Lambda on new S3 objects.