MLS-C01 Data Engineering Practice Test 5 — 15 Questions

Question 1

A company uses AWS Glue ETL jobs to transform CSV data from an S3 bucket into Parquet. The jobs often fail with memory errors when processing large datasets. They want to minimize cost and improve reliability. What should they do?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use G.1X or G.2X worker types and increase the number of DPUs per worker.. The G.1X and G.2X worker types provide more memory per worker (16 GB and 32 GB, respectively) compared to the standard G.0X worker (4 GB). By using these worker types and increasing the number of DPUs per worker, you allocate more memory to each task, reducing out-of-memory errors when processing large datasets. This approach also optimizes cost by using fewer, more powerful workers instead of many underpowered ones, improving reliability without unnecessary scaling.

Answer

Use Amazon Athena with CTAS queries to convert the data to Parquet.

Answer

Switch to S3 Batch Operations with AWS Lambda to process the files individually.

Answer

Increase the number of workers in the Glue job configuration.

Question 2

A company uses an Amazon SageMaker notebook to train a model using data from an S3 bucket. The IAM role attached to the notebook has the following policy. What is the MOST specific change needed to allow the notebook to read from the bucket 'ml-data-123'?

Accepted Answer

Remove the Deny statement from the IAM policy.. The existing IAM policy includes an explicit Deny statement that blocks all s3:GetObject access to the bucket 'ml-data-123'. In IAM, an explicit Deny overrides any Allow, so even if other policies grant read access, the Deny prevents it. Removing the Deny statement is the most specific change because it eliminates the blocking condition without requiring additional permissions or resources.

Answer

Add an Allow statement for 's3:GetObject' on 'ml-data-123' to the IAM policy.

Answer

Create an S3 access point and update the IAM policy to use the access point ARN.

Answer

Add a bucket policy on 'ml-data-123' that grants access to the notebook's IAM role.

Question 3

A data engineer needs to transform raw clickstream data (JSON files) stored in S3 into a partitioned Parquet dataset for querying with Athena. The transformation includes cleaning, deduplication, and enrichment. The pipeline should run daily. Which solution is MOST cost-effective and requires the least operational overhead?

Accepted Answer

Use an AWS Glue ETL job with a schedule trigger to perform the transformation and write to S3.. AWS Glue ETL jobs are serverless, require no cluster management, and can be easily scheduled for daily runs. Glue also integrates with the Data Catalog for partitioning. Option A (Amazon EMR) is not the most cost-effective or least operational overhead because it requires managing a cluster, even if it can be terminated after completion. Option C (AWS Lambda) is not suitable for large-scale clickstream data due to execution time limits and lack of built-in support for complex transformations like deduplication. Option D (Amazon Athena CTAS) is not appropriate because Athena is primarily a query engine, not a transformation tool; CTAS queries are good for converting data formats but lack the flexibility for cleaning and enrichment.

Answer

Launch an Amazon EMR cluster with Spark, transform the data, and terminate the cluster after completion.

Answer

Use AWS Lambda functions triggered by S3 events to transform each file incrementally.

Answer

Use Amazon Athena to run CTAS queries to convert and partition the data daily.

Question 4

A company uses Kinesis Data Streams to ingest real-time sensor data. The data is consumed by a Lambda function that writes to DynamoDB. During peak hours, the Lambda function throws ProvisionedThroughputExceededException. The team wants to decouple the write operation and improve resilience. What should they do?

Accepted Answer

Use Kinesis Firehose as a consumer of the stream, with a Lambda transformation to write to DynamoDB, and enable error handling.. Kinesis Firehose can consume data from a Kinesis Data Stream and invoke a Lambda function for transformation before delivering to destinations like DynamoDB. By using Firehose with error handling, the team decouples the write operation from the Lambda consumer, allowing Firehose to buffer data and retry failed writes, which improves resilience against ProvisionedThroughputExceededException without losing data.

Answer

Increase the Lambda function's reserved concurrency and provision more DynamoDB write capacity.

Answer

Place the Lambda function's output into an Amazon SQS queue, and have a second Lambda function write to DynamoDB.

Answer

Use Kinesis Data Analytics to process the stream and write results directly to DynamoDB.

Question 5

A data engineer needs to design a data pipeline that ingests CSV files from an SFTP server daily, transforms them, and loads them into Amazon Redshift. The files are typically 2-3 GB. Which combination of AWS services is MOST appropriate?

Accepted Answer

Use AWS Transfer Family to automate SFTP file retrieval to S3, then use Redshift COPY to load data.. The most appropriate because AWS Transfer Family provides a fully managed, serverless solution for automating SFTP file retrieval directly into Amazon S3. Once the CSV files are in S3, the Redshift COPY command can efficiently load the 2-3 GB files using parallel processing, which is far more performant and cost-effective than alternatives like Lambda or Glue for large file sizes.

Answer

Use AWS Glue ETL with a JDBC connection to the SFTP server to read files directly.

Answer

Use AWS Lambda to download the files from SFTP, transform them in memory, and write to Redshift using the Data API.

Answer

Use Amazon Kinesis Data Firehose with an HTTP endpoint source to receive files from SFTP.

Question 6

A team stores raw data in S3 and uses a Glue Data Catalog for metadata. They want to allow data scientists to query the data with Amazon Athena using their existing IAM roles. What is the MINIMUM set of permissions required?

Accepted Answer

Grant the IAM role permissions for Athena actions, Glue Data Catalog actions, and S3 read access.. Athena requires read access to S3 for querying data, permissions to the Glue Data Catalog for schema/metadata resolution, and Athena-specific actions to run queries. Write access to S3 is not required for querying, making B the minimum set.

Answer

Grant the IAM role permissions for Athena, Glue, and S3 (read and write).

Answer

Grant the IAM role permissions for Athena and Amazon Redshift Spectrum.

Answer

Grant the IAM role permissions for Athena and Amazon Kinesis.

Question 7

A company is building a near-real-time dashboard using data from multiple sources. They need to aggregate millions of events per second with sub-second latency. The architecture must be fully managed and minimize operational overhead. Which service should they use for the aggregation layer?

Accepted Answer

Amazon Kinesis Data Analytics for Apache Flink.. Amazon Kinesis Data Analytics for Apache Flink is the correct choice because it provides a fully managed, serverless runtime for Apache Flink, which is designed for stateful stream processing at scale. It can aggregate millions of events per second with sub-second latency using exactly-once semantics and built-in checkpointing, meeting the near-real-time dashboard requirements without any infrastructure management.

Answer

AWS Lambda functions triggered by Kinesis Data Streams.

Answer

Amazon EMR with Spark Streaming.

Answer

Amazon Redshift with materialized views refreshed frequently.

Question 8

A data engineer needs to set up a data lake on S3 that supports both batch and streaming ingestion. The data must be queryable by Athena, Redshift Spectrum, and EMR. Which TWO configurations are essential? (Choose two.)

Accepted Answer

Store data in columnar formats like Parquet or ORC.. Columnar formats like Parquet and ORC are optimized for analytical queries, reducing I/O by reading only the necessary columns. This is essential for Athena, Redshift Spectrum, and EMR, which all benefit from the efficient compression and predicate pushdown capabilities of these formats, enabling faster query performance and lower costs.

Answer

Enable S3 Select on the target buckets.

Answer

Enable S3 versioning on all buckets.

Answer

Set up Kinesis Data Firehose for streaming ingestion.

Question 9

A team wants to move data from an on-premises Oracle database to Amazon S3 for analytics. The pipeline must run daily and handle incremental updates. Which THREE services should they use together? (Choose three.)

Accepted Answer

Amazon S3. Amazon S3 is the correct destination for storing the data because it provides a scalable, durable, and cost-effective object storage solution ideal for analytics workloads. The pipeline requires daily incremental updates, and S3 integrates seamlessly with AWS DMS for continuous replication and AWS Glue for ETL processing, making it the central storage layer for the analytics pipeline.

Answer

Amazon SageMaker

Answer

Amazon Athena

Question 10

A company uses Amazon Kinesis Data Streams to ingest clickstream data. They need to archive raw data to S3 every hour and also enable real-time processing with sub-second latency. Which TWO actions should they take? (Choose two.)

Accepted Answer

Use Kinesis Data Analytics to write output to S3.. AWS Lambda can be configured as a consumer of a Kinesis Data Stream using event source mapping, enabling real-time processing with sub-second latency. Option A is correct because Kinesis Data Analytics (now Amazon Managed Service for Apache Flink) can read from a Kinesis stream and write output to S3, making it suitable for archiving raw data every hour—e.g., using a Flink sink with a tumbling window of 1 hour. Option D is incorrect because Kinesis Data Firehose has a maximum buffer interval of 900 seconds (15 minutes), not 3600 seconds; thus it cannot archive data exactly every hour as specified. Option C is incorrect because S3 events trigger Lambda on object creation in S3, not on real-time stream data. Option E is incorrect because the Kinesis Agent is used to send data from EC2 to Kinesis, not to S3 directly.

Answer

Use S3 events to trigger a Lambda function that reads from the stream.

Answer

Create a Kinesis Data Firehose delivery stream with S3 as destination and set a buffer interval of 3600 seconds.

Answer

Install the Kinesis Agent on an EC2 instance to write data to S3.

Question 11

An IAM policy attached to a SageMaker notebook role is shown. The data engineer tries to run an Athena query on a table in the 'my_database' Glue database. The query fails with an access denied error. What is the MOST likely cause?

Accepted Answer

The policy does not allow athena:StartQueryExecution on the Athena workgroup.. The IAM policy does not include the `athena:StartQueryExecution` action on the Athena workgroup, which is required to submit a query. Even if the role has permissions for Glue and S3, Athena will deny the request if the workgroup-level permission to start queries is missing, resulting in an access denied error.

Answer

The policy does not allow s3:PutObject on the query results location.

Answer

The policy does not allow glue:GetTable on the specific database.

Answer

The policy does not allow s3:ListBucket on the bucket.

Question 12

A data engineer runs the AWS CLI command above to inspect a file in S3. They need to determine if the file was modified after a Glue ETL job processed it. What additional information could they obtain from this command?

Accepted Answer

The object's last modified timestamp.. The AWS CLI command `aws s3api head-object --bucket my-bucket --key my-key` returns metadata about an S3 object without downloading it. The `LastModified` field in the response provides the exact timestamp of the last modification, which can be compared to the Glue ETL job's execution time to determine if the file was modified after processing. This is the only field that directly answers the question about modification timing.

Answer

The object's content type.

Answer

The object's storage class.

Answer

The object's ETag.

Question 13

A data engineer created a CloudFormation template for a Glue ETL job as shown. The job processes 500 GB of data and takes 90 minutes to complete. However, the job fails after 60 minutes. What is the MOST likely cause?

Accepted Answer

The Timeout property is set to 60 minutes, but the job requires more time.. The Glue ETL job has a `Timeout` property set to 60 minutes, but the job requires 90 minutes to complete. When the timeout is reached, AWS Glue forcibly terminates the job, causing it to fail. This is the most direct cause of the failure after exactly 60 minutes of execution.

Answer

The IAM role does not have sufficient permissions.

Answer

The ScriptLocation S3 bucket is in a different region.

Answer

The MaxRetries property is set to 0, so the job does not retry on failure.

Question 14

A data science team needs to process streaming data from thousands of IoT devices and perform real-time anomaly detection. The data must be persisted in Amazon S3 for batch processing later. Which combination of AWS services should be used to meet these requirements?

Accepted Answer

Amazon Kinesis Data Streams for ingestion, Amazon Kinesis Data Analytics for anomaly detection, and Amazon Kinesis Data Firehose to deliver data to Amazon S3.. Amazon Kinesis Data Streams provides durable, real-time ingestion for high-throughput IoT data. Kinesis Data Analytics can perform SQL-based anomaly detection on the stream, and Kinesis Data Firehose reliably delivers the processed or raw data to Amazon S3 for batch processing. This combination meets all requirements for streaming ingestion, real-time analytics, and persistent storage.

Answer

Amazon Kinesis Data Streams for ingestion, AWS Glue for anomaly detection, and Amazon S3 for storage.

Answer

AWS Lambda for both ingestion and anomaly detection, and Amazon S3 for storage.

Answer

Amazon Simple Queue Service (SQS) for ingestion, AWS Lambda for anomaly detection, and Amazon S3 for storage.

Question 15

A company uses Amazon Redshift for its data warehouse. The data engineering team notices that queries are slow and wants to improve performance without changing the schema. Which action is most likely to improve query performance?

Accepted Answer

Increase the number of nodes in the cluster.. Increasing the number of nodes in an Amazon Redshift cluster distributes data and query processing across more compute resources, which directly improves parallel execution and reduces query execution time. This is the most effective way to boost performance without altering the schema, as it scales the cluster's CPU, memory, and I/O capacity.

Answer

Decrease the number of nodes to reduce network overhead.

Answer

Disable compression on all tables to reduce CPU overhead.

Answer

Change the distribution style from AUTO to EVEN.