MLS-C01 Data Engineering Practice Test 15 — 15 Questions

Question 1

A machine learning team is preparing a large dataset for training. The dataset consists of 10,000 CSV files, each about 100 MB, stored in Amazon S3. The team wants to transform the data using AWS Glue ETL jobs. The transformation involves filtering rows, adding new columns, and joining with a small reference table (100 KB). The team is concerned about job performance and cost. They currently have a Glue job with 10 DPU (Data Processing Units) and it takes about 2 hours to complete. The team wants to reduce the runtime and cost. Which approach should they take?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Convert the CSV files to Parquet format and partition the data by a column.. Converting the CSV files to Parquet format and partitioning the data by a column significantly reduces the amount of data scanned and processed by AWS Glue. Parquet is a columnar storage format that allows Glue to read only the necessary columns, and partitioning enables predicate pushdown to skip irrelevant partitions. This directly reduces I/O and compute requirements, leading to faster job runtime and lower cost without increasing DPU count.

Answer

Use Amazon Athena to transform the data.

Answer

Increase the number of DPUs to 100.

Answer

Use Amazon EMR with Spot Instances instead of AWS Glue.

Question 2

A data engineer is tasked with building a data pipeline that moves data from an on-premises database to Amazon S3 for analytics. The database is a MySQL instance that is 2 TB in size. The company has a 1 Gbps dedicated network connection to AWS (AWS Direct Connect). The data must be transferred once daily. The engineer needs to choose the most efficient and reliable service for this task. Which service should they use?

Accepted Answer

AWS Database Migration Service (DMS). AWS Database Migration Service (DMS) is the correct choice because it is purpose-built for migrating databases to AWS, supporting continuous replication and one-time migrations from MySQL to Amazon S3. It can handle the 2 TB dataset efficiently over a 1 Gbps Direct Connect link by using change data capture (CDC) for ongoing replication and parallel tasks for throughput, ensuring reliability with built-in monitoring and restart capabilities.

Answer

AWS DataSync

Answer

AWS Glue

Answer

Amazon S3 Transfer Acceleration

Question 3

A data engineering team is using Apache Spark on Amazon EMR to process streaming data from Amazon Kinesis Data Streams. The Spark application uses structured streaming to read from Kinesis, perform transformations, and write to Amazon S3 in Parquet format. The team notices that the application is falling behind and the processing latency is increasing. The Kinesis stream has 5 shards, and the EMR cluster has 5 core nodes of type r5.xlarge. The Spark application is configured with 5 executors, each with 2 cores and 8 GB memory. The team wants to reduce processing latency. Which change would be most effective?

Accepted Answer

Increase the number of shards in the Kinesis stream to 10 and increase the number of core nodes to 10.. The number of shards (5) matches the number of executors (5), but each shard can be processed by a single executor. To increase parallelism, the team should increase the number of shards in the Kinesis stream and correspondingly increase the number of executors or cores. Alternatively, they can increase the number of cores per executor to allow parallel processing of multiple shards per executor.

Answer

Increase the executor memory to 16 GB.

Answer

Use a larger instance type for the core nodes, such as r5.4xlarge.

Answer

Change the output format from Parquet to CSV to reduce write time.

Question 4

A data engineer needs to continuously ingest streaming data from thousands of IoT devices and store the raw data in Amazon S3 for archival processing. The data volume varies significantly throughout the day, and the solution must be serverless, scalable, and cost-effective. Which AWS service should be used to capture and buffer the streaming data before writing to S3?

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the correct choice because it is a fully managed, serverless service designed to reliably capture, buffer, and automatically load streaming data into Amazon S3 without requiring any custom code or infrastructure management. It handles variable data volumes by scaling automatically and provides built-in buffering (up to 128 MB or 900 seconds) before writing to S3, making it cost-effective for archival storage.

Answer

Amazon Kinesis Data Streams

Answer

AWS Glue

Answer

Amazon Simple Queue Service (SQS)

Question 5

A company is running a machine learning training job on Amazon SageMaker that reads training data from an S3 bucket. The job fails intermittently with an S3 throttling error. The data is partitioned across thousands of small files (average 100 KB). Which strategy is MOST effective to resolve the throttling issue?

Accepted Answer

Combine the small files into larger files (e.g., 100 MB) using a preprocessing step. S3 throttling errors (HTTP 503) occur when many small files cause a high request rate per prefix. By combining thousands of 100 KB files into fewer 100 MB files, you drastically reduce the number of GET requests, staying within S3's 5,500 GET requests per second per prefix limit. This preprocessing step directly addresses the root cause of the throttling without changing the training infrastructure.

Answer

Use Amazon Athena to query the data and output results to a new S3 location

Answer

Enable S3 Transfer Acceleration on the bucket

Answer

Increase the number of SageMaker training instances to distribute the load

Question 6

A data scientist wants to explore a large dataset stored in Amazon S3 using SQL queries without moving the data. The dataset is in CSV format and is updated daily with new partitions. Which AWS service should be used to directly query the data in S3?

Accepted Answer

Amazon Athena. Amazon Athena is a serverless interactive query service that allows you to analyze data directly in Amazon S3 using standard SQL, without needing to load or transform the data. It supports CSV format and can automatically discover new partitions when used with Hive-style partitioning and the MSCK REPAIR TABLE command or by enabling partition projection. This makes it the ideal choice for directly querying a large, daily-updated CSV dataset stored in S3.

Answer

Amazon Redshift Spectrum

Answer

Amazon EMR

Answer

AWS Glue

Question 7

A company is building a data pipeline that ingests data from multiple sources into a centralized data lake on Amazon S3. The data must be transformed before it is available for analysis. The pipeline should be event-driven, automatically triggering transformation jobs when new data arrives. Which combination of AWS services should be used?

Accepted Answer

Amazon S3 event notifications to invoke AWS Lambda, which triggers an AWS Glue job. Amazon S3 event notifications can be configured to invoke an AWS Lambda function when new objects are created in an S3 bucket. The Lambda function can then trigger an AWS Glue job to perform the necessary data transformations. This creates an event-driven, serverless pipeline that automatically processes data as it arrives, meeting the requirements for a centralized data lake on S3.

Answer

Amazon Kinesis Data Analytics for transformation

Answer

Amazon EMR with automatic scaling

Answer

AWS Step Functions to orchestrate the pipeline

Question 8

A data engineering team is designing a data lake on Amazon S3. They need to enforce encryption at rest for all data stored in the bucket. The security policy requires that the encryption keys be managed by the organization using AWS Key Management Service (KMS), and that the bucket must deny uploads of unencrypted objects. Which bucket policy should be applied?

Accepted Answer

A bucket policy that denies PutObject unless the request includes the 'x-amz-server-side-encryption-aws-kms-key-id' header matching the desired KMS key ID. The security policy requires that encryption keys be managed by the organization using AWS KMS, and that unencrypted uploads be denied. A bucket policy that denies PutObject unless the request includes the 'x-amz-server-side-encryption-aws-kms-key-id' header matching the desired KMS key ID enforces both conditions: it ensures server-side encryption with a customer-managed KMS key (SSE-KMS) and blocks any upload that does not specify the exact key ID, thereby preventing unencrypted objects or objects encrypted with other keys.

Answer

A bucket policy that denies PutObject unless the request includes the 'x-amz-server-side-encryption' header with value 'AES256'

Answer

A bucket policy that denies PutObject if the 'x-amz-server-side-encryption' header is not present

Answer

Enable default encryption on the bucket with AWS-KMS

Question 9

A company uses Amazon RDS for its transactional database and needs to export a daily snapshot of a table to Amazon S3 in Parquet format for analytics. Which AWS service can perform this export without writing custom code?

Accepted Answer

AWS Glue. AWS Glue is correct because it provides a fully managed ETL service that can natively read from Amazon RDS and write to Amazon S3 in Parquet format using a scheduled job, without requiring any custom code. Glue's built-in transform capabilities and crawlers can convert the data to columnar Parquet format efficiently for analytics workloads.

Answer

Amazon Redshift

Answer

AWS Database Migration Service (DMS)

Answer

Amazon Athena

Question 10

A company is streaming data from thousands of devices using Amazon Kinesis Data Streams. The data is consumed by a AWS Lambda function that processes each record. The Lambda function is experiencing high error rates and throttling due to the volume of data. Which action would MOST effectively improve the processing throughput and reduce errors?

Accepted Answer

Increase the number of shards in the Kinesis stream. Increasing the number of shards in the Kinesis stream directly increases the stream's capacity for data ingestion and processing parallelism. Each shard supports up to 1 MB/s or 1,000 records/s for writes, and Lambda processes records from each shard concurrently. By adding more shards, you distribute the load across more Lambda invocations, reducing throttling and error rates caused by exceeding the per-shard throughput limits.

Answer

Send the data to Amazon SQS first and then process with Lambda

Answer

Use Amazon Kinesis Data Firehose instead of Kinesis Data Streams

Answer

Increase the Lambda function's batch size and reduce the batch window

Question 11

A data scientist needs to run complex ETL transformations on a large dataset stored in Amazon S3. The transformations are written in PySpark and require occasional access to Hive metastore. The solution should minimize operational overhead and allow the data scientist to focus on code development. Which AWS service should be used?

Accepted Answer

Amazon EMR. Amazon EMR is the correct choice because it natively supports PySpark and Hive metastore integration, allowing the data scientist to run complex ETL transformations on large datasets stored in S3 with minimal operational overhead. EMR provides managed clusters that automatically scale and handle infrastructure, enabling the data scientist to focus on code development rather than cluster management.

Answer

Amazon Redshift

Answer

AWS Glue

Answer

Amazon SageMaker

Question 12

A company wants to perform real-time analytics on streaming data from clickstreams. The data needs to be ingested, processed, and made available for querying within seconds. Which AWS service should be used for the processing step?

Accepted Answer

Amazon Kinesis Data Analytics. Amazon Kinesis Data Analytics is the correct choice because it enables real-time processing and analysis of streaming data using SQL or Apache Flink. It can ingest data from Kinesis Data Streams or Kinesis Data Firehose, process it with sub-second latency, and output results to destinations like Kinesis Data Streams or Firehose for further querying, meeting the requirement for analytics within seconds.

Answer

AWS Glue

Answer

Amazon Redshift

Answer

Amazon Athena

Question 13

A company is using AWS Glue to catalog metadata from various data sources. The crawler is configured to run daily. However, the catalog is not reflecting new partitions added to an S3 bucket during the day. What is the MOST likely cause?

Accepted Answer

The crawler is not scheduled frequently enough to capture changes. The crawler is configured to run daily, but new partitions are being added to the S3 bucket throughout the day. Since the crawler only runs once per day, it will not detect and catalog those new partitions until its next scheduled run. To capture changes more frequently, the crawler schedule should be increased or an event-driven trigger (e.g., using Amazon S3 Events and AWS Lambda) should be implemented.

Answer

The S3 bucket has insufficient permissions for the Glue crawler

Answer

The table schema has changed and the crawler does not update it

Answer

The data format is not supported by AWS Glue

Question 14

A data engineer is designing a data pipeline that ingests data from a relational database into a data lake on Amazon S3. The data must be incrementally loaded daily. Which TWO AWS services can be used together to achieve this?

Accepted Answer

AWS Glue. AWS Glue is correct because it provides a managed ETL service that can extract data from a relational database using JDBC connections, transform it, and write it incrementally to Amazon S3. Glue's built-in job bookmarking feature tracks processed data, enabling incremental loads by automatically skipping already-processed records during subsequent runs.

Answer

Amazon Kinesis Data Streams

Answer

Amazon Athena

Answer

Amazon Redshift

Question 15

A company wants to use Amazon SageMaker to train a model using data stored in Amazon S3. The data is sensitive and must be encrypted at rest and in transit. Which THREE steps should be taken to ensure data security?

Accepted Answer

Configure the SageMaker training job to use an IAM role with least privilege and enable network isolation. Configuring the SageMaker training job with an IAM role that follows least privilege ensures that only necessary permissions are granted, reducing the risk of unauthorized access. Enabling network isolation prevents the training job from accessing the internet, which mitigates data exfiltration risks and ensures that data remains within the controlled AWS environment.

Answer

Store the data in Amazon Redshift instead of S3

Answer

Allow internet access for the SageMaker notebook instance