PMLE Automating and Orchestrating ML Pipelines — 15 Questions

Question 1

An ML pipeline must run a set of preprocessing tasks for each data shard in parallel. Which KFP SDK features should they use to implement this? (Choose two.)

Accepted Answer

dsl.ParallelFor. (dsl.ParallelFor) is correct because it allows you to iterate over a list of items (e.g., data shard identifiers) and execute a set of tasks for each item in parallel within a KFP pipeline. Option C (dsl.Collected) is correct because it collects the outputs from each parallel iteration of a dsl.ParallelFor loop into a single list, enabling downstream tasks to consume all shard results as a single artifact or parameter.

Answer

dsl.PipelineParam

Answer

dsl.Condition

Answer

dsl.ExitHandler

Question 2

A data scientist creates a custom Python function component for a Vertex AI pipeline using the Kubeflow Pipelines SDK v2. The component takes a string parameter 'input_text' and outputs a Metrics artifact. The scientist wants to include a lightweight Python function without building a container. Which code snippet correctly defines this component?

Accepted Answer

@dsl.component(base_image='python:3.9')
def my_component(input_text: str) -> Metrics:
    from kfp.dsl import Metrics
    metrics = Metrics()
    metrics.log_metric('length', len(input_text))
    return metrics. It uses the `@dsl.component` decorator with a `base_image` parameter, which is required for lightweight Python function components in Kubeflow Pipelines SDK v2. The decorator enables the component to run without a custom container by specifying a base image (here, `python:3.9`), and the function correctly returns a `Metrics` artifact after logging a metric. Without the decorator or with an incorrect decorator, the component would not be recognized as a pipeline component.

Answer

@dsl.component
def my_component(input_text: str) -> Metric:
    metrics = Metric()
    metrics.log_metric('length', len(input_text))

Answer

@dsl.pipeline
def my_pipeline(input_text: str):
    metrics = Metrics()
    metrics.log_metric('length', len(input_text))

Answer

def my_component(input_text: str) -> Metrics:
    from kfp.dsl import Metrics
    metrics = Metrics()
    metrics.log_metric('length', len(input_text))
    return metrics

Question 3

A machine learning engineer is building a Vertex AI pipeline that uses a pre-built Google Cloud Pipeline Components (GCPC) to train a custom model. Which component should the engineer use to submit a custom training job to Vertex AI?

Accepted Answer

CustomJob. The CustomJob component is the correct choice because it is the pre-built GCPC component specifically designed to submit a custom training job to Vertex AI. It allows the engineer to specify a custom container image or a Python training script, along with machine configuration and hyperparameters, directly within a Vertex AI pipeline. Other components serve different purposes, such as hyperparameter tuning, batch predictions, or model deployment.

Answer

HyperparameterTuningJob

Answer

BatchPredictionJob

Answer

ModelDeploy

Question 4

A team has a Vertex AI pipeline that includes a container component for data preprocessing. The team notices that the component is re-executed every time the pipeline runs, even when the inputs and code haven't changed. They want to leverage pipeline caching to avoid redundant executions. What should they do to enable caching for this component?

Accepted Answer

Ensure that the component does not have 'dsl.cache_options(enable_cache=False)' set.. Vertex AI pipeline caching is enabled by default for all components unless explicitly disabled using `dsl.cache_options(enable_cache=False)`. The component re-executing every time indicates that caching was likely disabled on that specific component. Removing or ensuring this setting is not present will allow the pipeline to reuse cached outputs when inputs and code have not changed.

Answer

Set the 'caching' flag to 'True' in the pipeline definition using 'pipeline.caching = True'.

Answer

Set the environment variable 'ENABLE_CACHE' to 'true' on the pipeline run request.

Answer

Re-compile the pipeline with the '--enable-cache' flag.

Question 5

A machine learning team uses Vertex AI Pipelines to orchestrate their training pipeline. They want to trigger the pipeline automatically in response to new data arriving in a Cloud Storage bucket, and also support a scheduled run every day at 6 AM. Which combination of services should they use to achieve both event-driven and schedule-based triggers?

Accepted Answer

Cloud Scheduler for the schedule, and Cloud Functions triggered by Cloud Storage events to call the Vertex AI API for event-driven.. Cloud Scheduler can trigger the pipeline at 6 AM daily via a cron job, while Cloud Functions, triggered by Cloud Storage events (e.g., object finalize), can call the Vertex AI API to start the pipeline when new data arrives. This combination provides both schedule-based and event-driven triggers without requiring custom infrastructure.

Answer

Cloud Scheduler for the schedule, and Cloud Pub/Sub with Push subscription to Vertex AI for event-driven.

Answer

Cloud Functions for both schedule and event-driven, using cron trigger.

Answer

Vertex AI Pipelines built-in scheduler for schedule, and Cloud Pub/Sub for event-driven.

Question 6

A company is using Vertex AI Pipelines to automate model retraining. They have a component that creates a BigQuery table with training data. To ensure idempotency, the component should check if the table already exists and recreate it if necessary. What is the best practice for passing data between pipeline components?

Accepted Answer

Store data as artifacts in Cloud Storage and pass the GCS URI between components.. Vertex AI Pipelines is designed to pass data between components via Cloud Storage artifacts. By storing the BigQuery table metadata or training data as a file in Cloud Storage and passing the GCS URI as an artifact, the pipeline ensures idempotency and decouples components. This approach aligns with Kubeflow Pipelines' artifact-based I/O model, where each component's outputs are materialized as URIs rather than in-memory objects.

Answer

Pass data in-memory as Python objects between components.

Answer

Use BigQuery table names as component outputs and inputs.

Answer

Use Cloud SQL to store intermediate results and pass connection strings.

Question 7

A data engineer wants to orchestrate a complex workflow that includes running a Vertex AI pipeline, then a BigQuery job, and finally a Dataflow pipeline. The workflow must handle dependencies, retries, and monitoring. Which Google Cloud service is most suitable for this orchestration?

Accepted Answer

Cloud Composer. Cloud Composer (based on Apache Airflow) is the most suitable service for orchestrating a complex workflow with dependencies, retries, and monitoring across Vertex AI, BigQuery, and Dataflow. It provides a managed Airflow environment that natively supports DAG-based orchestration, built-in retry logic, and integration with Google Cloud services via operators like VertexAIPipelineOperator, BigQueryOperator, and DataflowTemplatedJobStartOperator.

Answer

Cloud Tasks

Answer

Cloud Scheduler

Answer

Workflows

Question 8

A machine learning engineer is building a Vertex AI pipeline that uses a pre-built AutoML Tables component to train a classification model. The pipeline also includes a conditional step that deploys the model to an endpoint only if the evaluation metrics exceed a threshold. Which KFP feature should be used to implement the conditional deployment?

Accepted Answer

dsl.Condition. The `dsl.Condition` feature from KFP (Kubeflow Pipelines) is specifically designed to conditionally execute pipeline steps based on the output of a previous component. In this scenario, the AutoML Tables component produces evaluation metrics; `dsl.Condition` allows the pipeline to check whether those metrics exceed a threshold and, if true, run the deployment step. This is the correct, native KFP construct for implementing branching logic within a pipeline.

Answer

dsl.ParallelFor

Answer

dsl.ExitHandler

Answer

dsl.Collected

Question 9

A team uses Cloud Build to automatically trigger a Vertex AI pipeline when changes are pushed to the model code repository. They have a cloudbuild.yaml file that builds a container image and submits the pipeline. However, they want to run the pipeline only if the commit includes changes to the 'training/' directory. Which Cloud Build configuration option should be used to filter the trigger?

Accepted Answer

Set the 'includedFiles' field to 'training/**' in the trigger configuration.. Cloud Build triggers support an `includedFiles` field that specifies a glob pattern. When set to `training/**`, the trigger will only fire if the commit includes changes to files under the `training/` directory. This is the native, declarative way to filter triggers based on changed file paths without additional infrastructure.

Answer

Add a 'ignoreFiles' field with 'training/**' to the trigger.

Answer

Use a 'substitutions' field with a regex pattern to filter commits.

Answer

Configure a Cloud Function to check the commit diff and call Cloud Build API conditionally.

Question 10

A data scientist wants to create a Vertex AI pipeline component that uses a custom container image stored in Artifact Registry. The component should accept a dataset artifact as input and output a model artifact. Which component type should they use?

Accepted Answer

A container component defined with the ContainerSpec class.. A container component defined with the `ContainerSpec` class is the only Vertex AI component type that allows you to specify a custom container image from Artifact Registry. This component type directly wraps a Docker container, enabling you to define inputs (e.g., a dataset artifact) and outputs (e.g., a model artifact) via the `ContainerSpec` interface, which maps to the container's command-line arguments and environment variables. Lightweight Python components and Python function components cannot use a custom container image without a base image, and pre-built components from GCPC are fixed and do not support custom containers.

Answer

A Lightweight Python component without base_image.

Answer

A Python function component with a custom base_image.

Answer

A pre-built component from GCPC.

Question 11

A machine learning engineer needs to pass a large dataset between two components in a Vertex AI pipeline. What is the recommended way to pass this data?

Accepted Answer

Store the dataset as a Dataset artifact and pass the artifact between components.. In Vertex AI Pipelines, the recommended way to pass large datasets between components is to use a `Dataset` artifact. Artifacts are metadata references that point to the underlying data stored in Cloud Storage, enabling efficient, scalable, and type-safe data passing without serialization overhead or size limits. This approach leverages the Kubeflow Pipelines SDK's artifact tracking, which automatically handles lineage and versioning.

Answer

Write the dataset to a temporary BigQuery table and pass the table name.

Answer

Serialize the dataset to a string and pass it as a pipeline parameter.

Answer

Use a Cloud Storage bucket and pass the bucket name as a parameter.

Question 12

A team is using Vertex AI Pipelines to deploy a model. They have a component that evaluates the model and produces a ClassificationMetrics artifact. The pipeline should deploy the model only if the precision is greater than 0.9. They use dsl.If to check the metric. However, the condition always evaluates to False. What is the most likely cause?

Accepted Answer

The ClassificationMetrics artifact is not accessible in the condition context.. In Vertex AI Pipelines, `ClassificationMetrics` artifacts are not directly accessible as primitive values within the `dsl.If` condition context. The `dsl.If` condition can only evaluate pipeline parameters or primitive outputs (like strings, integers, floats) that are explicitly passed as pipeline-level parameters or task outputs. A `ClassificationMetrics` artifact is a complex object that must be parsed or have its specific metric values extracted (e.g., via a custom component or `dsl.Metrics`) before they can be used in a conditional check.

Answer

The precision value is stored as a float but the condition expects a string.

Answer

The evaluation component did not output the metric correctly.

Answer

The dsl.If block is placed incorrectly in the pipeline definition.

Question 13

A company wants to implement continuous training for their ML model. The pipeline should be triggered when new training data arrives in Cloud Storage, and after training, the model should be automatically deployed to a staging endpoint if evaluation metrics pass a threshold. They also need to detect skew between training data and serving data. Which two services should they use for skew detection?

Accepted Answer

Vertex AI Model Monitoring and Vertex AI Pipelines. Vertex AI Model Monitoring provides built-in skew detection by comparing training data statistics with serving data statistics, alerting when distribution shifts exceed thresholds. Vertex AI Pipelines orchestrates the continuous training workflow, including triggering on new data arrival, model evaluation, and conditional deployment to a staging endpoint, making C the correct pair for both skew detection and pipeline automation.

Answer

Cloud Monitoring and Cloud Logging

Answer

Cloud DLP and Cloud KMS

Answer

BigQuery and Dataflow

Question 14

A machine learning team uses Vertex AI Pipelines to run a multi-step training pipeline. They want to implement a continuous delivery (CD) process where a model is automatically promoted from staging to production only if it passes an evaluation gate. Which TWO actions should they include in their CI/CD pipeline? (Choose two.)

Accepted Answer

Store model versions in Vertex AI Model Registry with version aliases (e.g., 'staging', 'production').. Vertex AI Model Registry supports version aliases like 'staging' and 'production', which allow the CI/CD pipeline to automatically promote a model from staging to production only after it passes the evaluation gate. This enables a controlled, automated CD process without manual intervention.

Answer

Use Cloud Functions to periodically check for new model versions and deploy.

Answer

Manually approve each model version before deployment.

Answer

Deploy every model version directly to production without evaluation.

Question 15

A company runs a Vertex AI pipeline that uses a container component to preprocess data. The component downloads a large file from a public URL and saves the output to Cloud Storage. The pipeline fails intermittently with a 'timeout' error. Which THREE steps should the team take to improve reliability? (Choose three.)

Accepted Answer

Make the component idempotent by checking for existing output before processing.. Making the component idempotent by checking for existing output before processing prevents redundant work and avoids timeout failures when the file has already been downloaded. In Vertex AI pipelines, idempotent components can safely skip processing if the output already exists in Cloud Storage, reducing the risk of hitting timeout limits on subsequent pipeline runs.

Answer

Reduce the size of the file being downloaded.

Answer

Increase the machine type for the component.