Troubleshooting practice questions

The user's session has timed out.

Why it fails: A timed-out session produces a connection or authentication failure, not a relation-level permission error; the query would not reach permission checking. It is tempting because session expiry is a common Redshift annoyance, but it applies when reconnecting is needed, not when SELECT has been granted yet denied.

The user does not have CONNECT permission on the database.

Why it fails: Without CONNECT on the database the user cannot log in or run any query at all, so the failure would occur before reaching the table. It is tempting because CONNECT is a prerequisite permission, but the stem shows the query executing and failing specifically on the relation.

The table is in a different schema than expected.

Why it fails: If the table sits in another schema, the unqualified name resolves elsewhere or fails to resolve, and SELECT granted on the intended table does not apply. It is tempting because search_path issues are common, but the error names the relation, indicating the table was found and permission was checked.

The Deny statement blocks all decrypt requests

Why it fails: A blanket Deny on decrypt requests would be visible in the attached policy, and the stem gives no such statement. It is tempting because an explicit Deny always overrides Allow in IAM evaluation, making it the correct diagnosis when a policy genuinely contains a Deny for kms:Decrypt on that key.

The Lambda function does not have permission to call kms:GenerateDataKey

Why it fails: GenerateDataKey is called during encryption to produce a data key, not during decryption of an existing object, so its absence cannot cause this error. It is tempting because GenerateDataKey and Decrypt are both KMS cryptographic actions, and it is correct when the failure occurs while encrypting rather than decrypting.

The IAM policy does not include kms:Decrypt permission

Why it fails: The stem states the execution role already carries a policy, and decryption failures from a missing kms:Decrypt statement would be visible in that policy's contents. It is tempting because kms:Decrypt is genuinely required to decrypt data keys, so it is the right answer when the attached policy omits that action.

Grant the crawler's IAM user access to the bucket

Why it fails: Granting the crawler's IAM user access is unnecessary: Glue crawlers assume an IAM role, not a user, so attaching S3 permissions to a user leaves the crawler still unauthorised. It is tempting because identity permissions are the usual fix, but the role's policy is what must be amended.

Attach a VPC endpoint to the S3 bucket

Why it fails: VPC endpoints provide private network routing to S3; they grant no IAM permissions, so the crawler's role still lacks s3:GetObject and the failure persists. Endpoints are tempting because they solve private-subnet connectivity to S3 without NAT, which would be the right fix if the error were a timeout rather than an authorisation denial.

Enable S3 default encryption on the bucket

Why it fails: S3 default encryption governs data at rest and does not grant the crawler's role permission to list or read objects. It is tempting because bucket-level settings often cause access failures, but the logged error is an IAM authorisation problem, not an encryption configuration one.

The Glue job is missing the s3:ListBucket permission on the bucket

Why it fails: ListBucket governs listing objects in the bucket, not the PutObject write itself, so its absence cannot cause the write denial. It is tempting because ListBucket is frequently omitted from policies, and it would be correct if the job failed while enumerating bucket contents.

The S3 bucket is encrypted with AWS KMS and the policy does not include kms:Decrypt permissions

Why it fails: KMS Decrypt applies to reading KMS-encrypted objects; the failure occurs when writing to S3, which requires kms:GenerateDataKey instead. It is tempting because KMS key policies commonly cause access denied, and it would be correct if the job were reading encrypted data.

The Glue job does not have permission to call glue:StartJobRun

Why it fails: StartJobRun governs triggering the job, not writing objects to S3, so it cannot explain the write failure. It is tempting because Glue permissions are commonly misconfigured, and it would be correct if the job failed to start rather than failing during the S3 write.

Modify the Redshift cluster parameter group to enable public accessibility.

Why it fails: Modifying the cluster parameter group does not control network access; public accessibility is a separate setting.

Verify that the cluster's security group is attached to the Redshift cluster.

Why it fails: Verifying security group attachment is redundant since the engineer already verified the group allows traffic; the issue is more likely at the subnet level.

Check the IAM role associated with the Redshift cluster.

Why it fails: IAM roles control permissions, not network-level access.

The table's read capacity is set too low

Why it fails: Write throttling is governed by write capacity units, so lowering read capacity would not cause it; reads and writes consume separate capacity pools. Read capacity is the tempting answer because under-provisioned RCU causes throttling on read-heavy tables, and that would be the correct diagnosis if the errors were on GetItem or Query calls.

The table's auto scaling is disabled

Why it fails: The stem states auto scaling is already enabled, so this contradicts the given scenario rather than explaining the throttling. Auto scaling is the tempting answer because disabled scaling genuinely causes capacity to stay fixed during traffic spikes, which would be the correct diagnosis if the question had not confirmed it was active.

The table is using global tables without conflict resolution

Why it fails: Global tables replicate writes across regions but do not throttle local write requests; conflict resolution affects data consistency, not capacity. It is tempting because multi-region replication does consume write capacity units, so a global table with heavy replication traffic would be the correct diagnosis if the stem mentioned cross-region replication load.

Check the network connectivity between Glue and the EMR cluster.

Why it fails: Connectivity failures produce timeouts or connection refused errors, not ClassNotFoundException, which is a classpath problem. It is tempting because Glue reaching an EMR metastore does depend on network paths, and checking connectivity is correct when the job cannot establish a TCP session at all.

Modify the Python script to import the Hive libraries manually.

Why it fails: Python shell jobs run without the Spark/Hive runtime, so importing Hive libraries manually cannot supply the absent JVM classes. It is tempting because import errors superficially resemble missing modules, and adding imports is right for pure-Python dependency gaps, not for Hadoop classpath requirements.

Update the IAM role to allow 'hive:Describe*' actions.

Why it fails: IAM permissions govern authorisation, not classpath resolution; the missing Hive classes cannot be granted via hive:Describe* actions. It is tempting because Glue-to-metastore failures often stem from access misconfiguration, and such policies are correct when the job is denied rather than throwing ClassNotFoundException.

Enable encryption at rest.

Why it fails: Encryption at rest protects stored objects via KMS keys; it does not reduce bytes scanned, so it cannot speed up this Athena query. It is tempting because it is a common S3 security setting, and it would be the right choice when the requirement is compliance or data protection rather than query latency.

Increase the Athena query timeout.

Why it fails: The timeout governs how long Athena waits before aborting, not how much data is scanned; a slow query still reads the same bytes and simply runs longer. It is tempting when queries fail with timeout errors, but that is a symptom of excessive scanning, addressed by partitioning and columnar pruning.

Use SELECT * to retrieve all columns.

Why it fails: SELECT * reads every column, including those the query never uses, increasing bytes scanned from Amazon S3 and thus cost and runtime. It is tempting for convenience during exploration, but specifying only required columns lets Athena's columnar Parquet and ORC formats skip unneeded data.

The VPC endpoint policy for S3 is too restrictive.

Why it fails: VPC endpoints do not cause access denied errors for encryption.

The IAM role does not have s3:PutObject permission.

Why it fails: Glue requires permissions on the S3 bucket and KMS key.

The S3 bucket uses S3 Block Public Access which denies all writes.

Why it fails: S3 Block Public Access prevents public access, not authorized writes.

The Glue job is using Amazon Kinesis Data Streams as a source, which has a shard throughput limit.

Why it fails: The stem specifies Amazon RDS for MySQL as the source, so Kinesis shard throughput cannot produce this error. Kinesis Data Streams is the right choice when ingesting real-time streaming data, where each shard's 1 MB/s ingest limit genuinely triggers 'Rate exceeded' during scaling.

The Amazon S3 bucket has a bucket policy that limits the number of objects written per second.

Why it fails: S3 bucket policies govern access permissions, not request rates; S3 scales request throughput per prefix automatically, so no policy caps objects written per second. Bucket policies are the correct control when restricting which principals or VPC endpoints may access specific prefixes.

The IAM role attached to the Glue job does not have sufficient permissions to read from RDS.

Why it fails: Missing IAM permissions return AccessDenied errors, not 'Rate exceeded'. IAM roles are the right mechanism when the job needs scoped, temporary credentials to call AWS services — but throttling stems from exceeding API request quotas, not from authorisation gaps.

firehose:DescribeDeliveryStream on the Firehose delivery stream

Why it fails: DescribeDeliveryStream is a read-only metadata action; the failure occurs when invoking PutRecord or PutRecordBatch, so describing the stream cannot resolve it. DescribeDeliveryStream would be the missing permission if the function needed to inspect stream configuration, such as destination or status, before writing.

logs:CreateLogGroup and logs:CreateLogStream on the CloudWatch log group

Why it fails: The failure is an access denied error on Kinesis or Firehose operations, so CloudWatch Logs permissions are unrelated to that denial. Adding logging permissions is tempting because Lambda needs them to emit logs, but missing logs permissions cause silent logging failures, not access denied errors on stream reads or writes.

s3:PutObjectAcl on the S3 bucket

Why it fails: Firehose writes objects to S3 using its own service role, not the Lambda execution role, so s3:PutObjectAcl is irrelevant to the Lambda's access-denied error. PutObjectAcl would be required if the function itself wrote objects directly to S3 while setting object-level ACLs.

kms:GenerateDataKey

Why it fails: kms:GenerateDataKey is needed for uploads to produce a data key, not for reading; decryption requires kms:Decrypt. It is tempting because both permissions appear in SSE-KMS policies, and GenerateDataKey would be the missing grant if the job were writing encrypted objects instead of reading them.

s3:ListBucket

Why it fails: s3:ListBucket governs listing objects in a bucket, not decrypting object data, so it cannot resolve a KMS decryption failure. It is tempting because ListBucket errors often surface during ETL job setup, and it would be the correct missing permission if the job failed to enumerate bucket contents.

s3:GetObject

Why it fails: s3:GetObject grants read access to the object but does not authorise use of the KMS key; decryption still fails without kms:Decrypt. It is tempting because GetObject is the obvious S3 read permission, and it would be correct if the error were an access-denied on the object itself.

The ETL job is processing more than 10 TB of data.

Why it fails: The attached policy grants only s3:GetObject and s3:ListBucket on the input bucket, so writing to the output bucket fails because s3:PutObject is absent from the role. Data volume is irrelevant; AccessDenied stems from missing write permissions, not from processing more than 10 TB.

The output bucket has a bucket policy that denies access to the IAM role.

Why it fails: The attached identity policy grants only s3:GetObject and s3:ListBucket on input-bucket, so no statement permits s3:PutObject on the output bucket; an explicit deny is not required for AccessDenied. It is tempting because bucket policies can deny access, and they are the right control when restricting cross-account or public S3 access.

The IAM role used by the job does not exist.

Why it fails: A non-existent role would prevent the Glue job from starting at all, producing a role-assumption or entity-not-found error rather than AccessDenied on a PutObject call. It is tempting because missing roles do cause job failures, and creating a dedicated IAM role is the correct setup when configuring a new Glue job.

Change the SORT KEY on all tables to match the join column.

Why it fails: Sort keys order rows within a slice for range filtering and merge joins; they do not control which slice stores each row, so broadcast behaviour is unchanged. It is tempting because matching the join column aids zone-map pruning, which would be correct for slow range-scan queries rather than redistribution.

Change the DISTSTYLE to EVEN on all tables.

Why it fails: EVEN distributes rows round-robin, so join keys never co-locate and Redshift must still broadcast or shuffle to complete each join. It is tempting because EVEN removes skew when no natural distribution key exists, but that scenario concerns uneven data, not broadcast-heavy joins.

Change the DISTSTYLE to ALL on all large tables.

Why it fails: DISTSTYLE ALL replicates every table to every node, multiplying storage and load cost while making broadcasts unnecessary only because full copies already exist. It is tempting for small dimension tables joined repeatedly, but applying it to large tables is precisely what the stem warns against.

Change the worker type to G.1X

Why it fails: G.1X workers offer the same 16 GB memory as the default G.1X already in use, so heap pressure from many small files persists; the fix is G.2X or grouping files. G.1X suits memory-light, CPU-bound jobs, not this out-of-memory scenario.

Increase the number of workers in the Glue job

Why it fails: Does not reduce per-worker memory pressure.

Use a G.2X worker type with more memory

Why it fails: G.2X adds memory per worker, but the heap error stems from too many small files being listed and read concurrently; compaction or grouping into larger objects reduces task overhead. G.2X is correct when per-worker memory is genuinely the constraint, not file-count-driven heap pressure.

Add s3:PutObject permission to the policy

Why it fails: s3:PutObject governs writes to S3, yet the failure occurs on kinesis:PutRecords, so this permission is irrelevant to the error. It is tempting because the function reads from S3, but reading requires s3:GetObject, and the denied call targets Kinesis.

Change the resource ARN for Kinesis to a wildcard

Why it fails: Wildcarding the Kinesis resource ARN broadens access to every stream, which is a security regression rather than the fix. It is tempting because it would resolve the mismatch, but the correct change is correcting the ARN to the specific stream the function writes to.

Add kinesis:PutRecords permission to the policy

Why it fails: The policy already grants kinesis:PutRecords, so adding it again changes nothing and the AccessDeniedException persists. It is tempting because the error names that API, but the actual cause is the Kinesis resource ARN not matching the stream the function writes to.

Add redshift:DataAPI

Why it fails: redshift:DataAPI is not a valid IAM action; the Redshift Data API is invoked through redshift-data actions such as ExecuteStatement, which authorise SQL execution. It is tempting because the Data API does run SQL, but it would be the correct choice when calling ExecuteStatement against a cluster.

Add redshift:ModifyCluster

Why it fails: redshift:ModifyCluster changes cluster configuration such as node type or maintenance settings; it does not authorise writing rows into a table. It is tempting because it is a Redshift action, but it would be the correct choice when resizing or reconfiguring the cluster itself, not for ETL writes.

Add redshift:DescribeStatement

Why it fails: redshift:DescribeStatement returns metadata about a previously submitted SQL statement; it grants no write access to the Redshift table, so the permission error persists. It is tempting because it sounds like a data-movement permission, but it is the correct choice when polling the status of an asynchronous statement.

s3:PutObject

Why it fails: s3:PutObject allows writing objects but not enumerating a bucket, so the ListBucket AccessDenied persists. It is tempting because the job's purpose is writing results to S3, and PutObject is the natural write permission. The error is on listing, which requires s3:ListBucket.

s3:DeleteObject

Why it fails: s3:DeleteObject permits object removal, not bucket listing, so the AccessDenied on ListBucket persists. It is tempting because Glue jobs often manage S3 output, but deletion is unrelated to enumeration. s3:ListBucket is the action the failing API call requires.

s3:GetObject

Why it fails: s3:GetObject authorises reading object data, not listing a bucket's contents, so the ListBucket AccessDenied remains. Engineers reach for it because Glue commonly reads source objects, making it a familiar grant. The failing call is a bucket-level list, which s3:ListBucket covers.

Check the Glue job's DPU allocation; increase if too low.

Why it fails: DPU allocation governs compute and memory for transformation, not TCP reachability to a private-subnet RDS endpoint; a 'Communications link failure' indicates the JDBC socket cannot be established. Raising DPUs is the right lever when jobs fail with out-of-memory or executor timeouts, not network-level connection errors.

Review the Glue job script for data type mismatches.

Why it fails: Data type mismatches surface as conversion or schema errors during read/write, not as a 'Communications link failure', which is a JDBC socket-level fault occurring before any row is parsed. Script review is the correct first step when jobs fail with casting exceptions or malformed records.

Increase the RDS instance's max_connections parameter.

Why it fails: max_connections governs concurrent session limits, and exceeding it returns 'Too many connections', not a 'Communications link failure'. Raising it is the right step when the RDS instance rejects new sessions under load, not when the Glue job cannot reach the private-subnet endpoint at all.

The job does not have permission to write to the bucket at all.

Why it fails: Writes to other prefixes succeed, proving the role does have bucket write permission, so a total lack of access cannot explain the failure. It is tempting when a job fails broadly and permissions seem suspect, and would be correct if every write attempt, not just the secrets/ prefix, returned AccessDenied.

The resource ARN in the Allow statement does not include the bucket itself.

Why it fails: An Allow scoped to objects under a prefix still permits writes to those objects; omitting the bucket ARN does not block prefix-level PutObject calls. It is tempting because bucket-level and object-level ARNs look similar, and would be correct if the operation required bucket-level permissions such as ListBucket.

The Deny statement is not effective because it is placed after the Allow.

Why it fails: IAM evaluates all statements together, and an explicit Deny always overrides any Allow regardless of order, so placement does not nullify it. It is tempting because statement sequence appears to matter in policy documents, and would be correct if AWS processed statements sequentially with later ones overriding earlier ones.