MLS-C01 Data Engineering Practice Test 8 — 15 Questions

Question 1

A data engineering team is using Apache Spark on Amazon EMR to process streaming data from Amazon Kinesis Data Streams. The Spark application uses structured streaming to read from Kinesis, perform transformations, and write to Amazon S3 in Parquet format. The team notices that the application is falling behind and the processing latency is increasing. The Kinesis stream has 5 shards, and the EMR cluster has 5 core nodes of type r5.xlarge. The Spark application is configured with 5 executors, each with 2 cores and 8 GB memory. The team wants to reduce processing latency. Which change would be most effective?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Increase the number of shards in the Kinesis stream to 10 and increase the number of core nodes to 10.. The number of shards (5) matches the number of executors (5), but each shard can be processed by a single executor. To increase parallelism, the team should increase the number of shards in the Kinesis stream and correspondingly increase the number of executors or cores. Alternatively, they can increase the number of cores per executor to allow parallel processing of multiple shards per executor.

Answer

Increase the executor memory to 16 GB.

Answer

Use a larger instance type for the core nodes, such as r5.4xlarge.

Answer

Change the output format from Parquet to CSV to reduce write time.

Question 2

A data engineering team is designing a data lake on Amazon S3. They need to enforce encryption at rest for all data stored in the bucket. The security policy requires that the encryption keys be managed by the organization using AWS Key Management Service (KMS), and that the bucket must deny uploads of unencrypted objects. Which bucket policy should be applied?

Accepted Answer

A bucket policy that denies PutObject unless the request includes the 'x-amz-server-side-encryption-aws-kms-key-id' header matching the desired KMS key ID. The security policy requires that encryption keys be managed by the organization using AWS KMS, and that unencrypted uploads be denied. A bucket policy that denies PutObject unless the request includes the 'x-amz-server-side-encryption-aws-kms-key-id' header matching the desired KMS key ID enforces both conditions: it ensures server-side encryption with a customer-managed KMS key (SSE-KMS) and blocks any upload that does not specify the exact key ID, thereby preventing unencrypted objects or objects encrypted with other keys.

Answer

A bucket policy that denies PutObject unless the request includes the 'x-amz-server-side-encryption' header with value 'AES256'

Answer

A bucket policy that denies PutObject if the 'x-amz-server-side-encryption' header is not present

Answer

Enable default encryption on the bucket with AWS-KMS

Question 3

A company uses Amazon RDS for its transactional database and needs to export a daily snapshot of a table to Amazon S3 in Parquet format for analytics. Which AWS service can perform this export without writing custom code?

Accepted Answer

AWS Glue. AWS Glue is correct because it provides a fully managed ETL service that can natively read from Amazon RDS and write to Amazon S3 in Parquet format using a scheduled job, without requiring any custom code. Glue's built-in transform capabilities and crawlers can convert the data to columnar Parquet format efficiently for analytics workloads.

Answer

Amazon Redshift

Answer

AWS Database Migration Service (DMS)

Answer

Amazon Athena

Question 4

A company is streaming data from thousands of devices using Amazon Kinesis Data Streams. The data is consumed by a AWS Lambda function that processes each record. The Lambda function is experiencing high error rates and throttling due to the volume of data. Which action would MOST effectively improve the processing throughput and reduce errors?

Accepted Answer

Increase the number of shards in the Kinesis stream. Increasing the number of shards in the Kinesis stream directly increases the stream's capacity for data ingestion and processing parallelism. Each shard supports up to 1 MB/s or 1,000 records/s for writes, and Lambda processes records from each shard concurrently. By adding more shards, you distribute the load across more Lambda invocations, reducing throttling and error rates caused by exceeding the per-shard throughput limits.

Answer

Send the data to Amazon SQS first and then process with Lambda

Answer

Use Amazon Kinesis Data Firehose instead of Kinesis Data Streams

Answer

Increase the Lambda function's batch size and reduce the batch window

Question 5

A data scientist needs to run complex ETL transformations on a large dataset stored in Amazon S3. The transformations are written in PySpark and require occasional access to Hive metastore. The solution should minimize operational overhead and allow the data scientist to focus on code development. Which AWS service should be used?

Accepted Answer

Amazon EMR. Amazon EMR is the correct choice because it natively supports PySpark and Hive metastore integration, allowing the data scientist to run complex ETL transformations on large datasets stored in S3 with minimal operational overhead. EMR provides managed clusters that automatically scale and handle infrastructure, enabling the data scientist to focus on code development rather than cluster management.

Answer

Amazon Redshift

Answer

AWS Glue

Answer

Amazon SageMaker

Question 6

A company wants to use Amazon SageMaker to train a model using data stored in Amazon S3. The data is sensitive and must be encrypted at rest and in transit. Which THREE steps should be taken to ensure data security?

Accepted Answer

Configure the SageMaker training job to use an IAM role with least privilege and enable network isolation. Option A is correct because configuring the SageMaker training job with an IAM role scoped to least privilege limits what AWS resources the job can access, and enabling network isolation prevents the training container from making any outbound network calls, which enforces data-in-transit security and reduces exfiltration risk. Option B is correct because enabling default encryption on the S3 bucket using AWS KMS ensures all objects are encrypted at rest with a customer-managed or AWS-managed KMS key, satisfying the at-rest encryption requirement for sensitive data. Option C is correct because using an S3 VPC endpoint (specifically a Gateway endpoint for S3) routes traffic between the VPC and S3 over the private AWS network rather than the public internet, keeping data in transit within AWS and avoiding exposure. Option D is incorrect because the scenario specifies data is stored in Amazon S3, and moving it to Redshift does not address the stated encryption requirements. Option E is incorrect because allowing internet access to the SageMaker notebook instance increases exposure and contradicts the goal of keeping sensitive data secure and isolated.

Answer

Store the data in Amazon Redshift instead of S3

Answer

Allow internet access for the SageMaker notebook instance

Question 7

A data engineer needs to collect and analyze log data from multiple EC2 instances in real-time. The solution should be serverless and scalable. Which TWO AWS services should be used?

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose (A) is correct because it is a fully managed, serverless streaming ingestion service that can capture log data from EC2 instances in real time and reliably deliver it to destinations such as Amazon OpenSearch Service, S3, or Splunk without managing any infrastructure. Amazon OpenSearch Service (D) is correct because it provides scalable, near-real-time search and analytics capabilities for log data, making it the natural analysis and visualization layer for the streamed logs. Together, Firehose handles the serverless, scalable ingestion while OpenSearch Service handles the real-time analysis, which matches the scenario's requirements. Amazon EMR (B) is not serverless and is designed for batch big-data processing with Hadoop/Spark, so it does not fit the real-time, serverless requirement. Amazon Athena (C) is a serverless query service but it queries data at rest in S3, not real-time streaming log data. Amazon S3 (E) is scalable storage but by itself provides no real-time collection or analysis capability.

Answer

Amazon EMR

Answer

Amazon Athena

Answer

Amazon S3

Question 8

A data engineering team needs to ingest streaming data from thousands of IoT devices into Amazon S3 for near-real-time analytics. The solution must handle data that arrives in bursts and must be able to reprocess failed records automatically. Which combination of AWS services should the team use?

Accepted Answer

Amazon Kinesis Data Streams with AWS Lambda. Amazon Kinesis Data Streams is designed for real-time ingestion of large volumes of streaming data, such as from thousands of IoT devices, and can handle bursty traffic by scaling shards. AWS Lambda can be used as a consumer to process records in near-real-time, and Kinesis Data Streams supports automatic retries and checkpointing, enabling reprocessing of failed records without data loss.

Answer

AWS Glue with Amazon S3

Answer

Amazon SQS with AWS Lambda

Answer

Amazon DynamoDB Streams with AWS Lambda

Question 9

A data engineering team needs to build a data lake on Amazon S3 that will be queried by Amazon Athena and Amazon Redshift Spectrum. The data will be ingested from multiple sources in various formats (CSV, JSON, Parquet). Which partitioning strategy will provide the best query performance for date-range queries?

Accepted Answer

Partition by year, month, and day in a hierarchical structure.. Partitioning by year, month, and day in a hierarchical structure minimizes the amount of data scanned by Amazon Athena and Redshift Spectrum for date-range queries. Athena and Redshift Spectrum both charge per byte scanned, so reducing the scan size directly improves performance and reduces cost. A hierarchical partition structure (e.g., s3://bucket/year=2023/month=11/day=01/) allows the query engine to prune partitions at each level, efficiently skipping irrelevant directories for queries like WHERE date BETWEEN '2023-11-01' AND '2023-11-30'.

Answer

Partition by date with one partition per day in a flat structure.

Answer

Do not partition; let Athena scan the entire dataset.

Answer

Partition by source system first, then by date.

Question 10

A company has an AWS Glue ETL job that reads data from an Amazon RDS for MySQL table and writes to Amazon S3 in Parquet format. The job runs daily and processes 500 GB of data. Recently, the job has been failing with memory errors during the write phase. The data schema is wide (200 columns). Which change should a data engineer make to the Glue job to resolve the memory issue?

Accepted Answer

Configure the write operation with 'groupSize' to limit records per file.. The memory error occurs because the wide schema (200 columns) and large data volume (500 GB) cause the Spark executors to run out of memory when writing Parquet files, as each executor attempts to buffer entire partitions. Configuring 'groupSize' limits the number of records written per file, reducing the per-executor memory footprint and preventing out-of-memory errors during the write phase.

Answer

Increase the number of DPUs for the Glue job.

Answer

Change the output format from Parquet to CSV.

Answer

Use the JDBC connection with fetchSize parameter.

Question 11

A data engineering team is building a real-time fraud detection system. Transactions are ingested via Amazon Kinesis Data Streams, and a machine learning model (deployed on Amazon SageMaker) scores each transaction. The team needs to store the raw transactions and the model's predictions in Amazon S3 for later analysis. Which architecture should the team use?

Accepted Answer

Use Amazon Kinesis Data Analytics for Apache Flink to enrich records with SageMaker predictions, then output to Firehose for S3.. It uses Amazon Kinesis Data Analytics for Apache Flink to perform real-time enrichment by invoking the SageMaker endpoint for each transaction, then streams the enriched records to Kinesis Data Firehose for reliable, batched delivery to S3. This architecture handles the asynchronous nature of model inference without blocking the ingestion stream, and Firehose provides automatic retry and compression for S3 storage.

Answer

Use AWS Lambda to read from Kinesis, invoke SageMaker, and write directly to S3.

Answer

Use Amazon Kinesis Data Firehose with a transformation Lambda to call SageMaker.

Answer

Use AWS Lambda to invoke the SageMaker endpoint for each record, then write to S3 via Firehose.

Question 12

A company uses Amazon Athena to query a data lake in Amazon S3. The data is partitioned by year, month, day, and hour. The team notices that queries are slow and expensive. The team wants to improve performance and reduce costs. Which THREE actions should the team take?

Accepted Answer

Ensure queries filter on partition columns (year, month, day, hour).. Option A is correct because Athena prunes partitions only when the WHERE clause filters on the partition key columns (year, month, day, hour), so filtering on them scans far less data from S3, cutting both query latency and bytes-scanned cost. Option C is correct because converting CSV to a columnar format like Parquet lets Athena read only the needed columns and benefits from compression and predicate pushdown, dramatically reducing data scanned and improving performance. Option E is correct because moving the underlying S3 objects to cost-optimized storage classes such as S3 Intelligent-Tiering lowers storage costs for the data lake without changing query behavior. Option B is not appropriate because adding a minute-level partition would create excessive small partitions and metadata overhead, worsening rather than improving performance. Option D is wrong because CSV with GZIP is still row-based and non-columnar, so Athena must scan and decompress entire rows, which is slower and more expensive than Parquet.

Answer

Increase the number of partitions by adding a partition for minute.

Answer

Use CSV format with GZIP compression.

Question 13

A data engineer uses the IAM policy above for an AWS Lambda function that processes data in S3 and triggers an AWS Glue job. The Lambda function is unable to start the Glue job. What is the most likely cause?

Accepted Answer

The policy does not include iam:PassRole permission.. The Lambda function needs to pass an IAM role to AWS Glue when starting a job, which requires the `iam:PassRole` permission. Without this permission, the `StartJobRun` API call fails even if the Lambda has permissions to invoke Glue. The policy shown lacks this critical permission, making option C the correct answer.

Answer

The Glue job name in the resource ARN is misspelled.

Answer

The policy does not allow s3:PutObject on the bucket.

Answer

The policy does not include s3:GetObject on the bucket.

Question 14

A data engineer runs the AWS CLI command above to inspect an object in S3. The engineer wants to query this metadata (kafka-offset) using Amazon Athena to track processing progress. How can the engineer make this metadata available for Athena queries without modifying the existing data pipeline?

Accepted Answer

Use AWS Glue to create a table that includes the metadata as a column by running an ETL job.. AWS Glue ETL jobs can read the S3 object's user-defined metadata (e.g., 'kafka-offset') and write it as a column in a new or transformed dataset, which Athena can then query. This approach does not modify the existing data pipeline, as the original objects remain unchanged; the metadata is extracted and stored in a queryable format (e.g., Parquet or CSV) in a separate location. Glue's ability to access S3 object metadata via the `getObjectMetadata` API during ETL processing makes this a clean, pipeline-agnostic solution.

Answer

Use S3 object tags instead of metadata and query the tags using Athena.

Answer

Use an AWS Lambda function to copy the metadata into the object's content as a new line.

Answer

Use Amazon Athena to query the object metadata directly by referencing the metadata field.

Question 15

A data engineer configures an S3 event notification to trigger an AWS Lambda function when a new object is created in 'my-input-bucket'. The Lambda function processes the CSV file and writes results to 'my-output-bucket'. The engineer notices that the Lambda function is not triggered for some objects. Which step should the engineer take to diagnose the issue?

Accepted Answer

Check the Lambda function's resource-based policy to ensure S3 has permission to invoke the function.. The most likely cause of the Lambda function not being triggered for some objects is that S3 lacks the necessary permission to invoke the function. S3 event notifications require a resource-based policy (also known as a Lambda function policy) that explicitly grants the S3 service principal permission to invoke the function. Without this policy, S3 will not be able to trigger the Lambda function, even if the event notification configuration is correct.

Answer

Check the Lambda function's execution role for permissions to write to the output bucket.

Answer

Review the CloudWatch Logs for the Lambda function to see if there are errors.

Answer

Verify that the S3 event notification is configured with the correct prefix and suffix filters.