PMLE Automating and Orchestrating ML Pipelines — 30 Questions

Question 1

A data scientist creates a custom Python function component for a Vertex AI pipeline using the Kubeflow Pipelines SDK v2. The component takes a string parameter 'input_text' and outputs a Metrics artifact. The scientist wants to include a lightweight Python function without building a container. Which code snippet correctly defines this component?

Accepted Answer

@dsl.component(base_image='python:3.9')
def my_component(input_text: str) -> Metrics:
    from kfp.dsl import Metrics
    metrics = Metrics()
    metrics.log_metric('length', len(input_text))
    return metrics. Option D is correct because it uses the `@dsl.component` decorator with a `base_image` parameter, which is required for lightweight Python function components in Kubeflow Pipelines SDK v2. The decorator enables the component to run without a custom container by specifying a base image (here, `python:3.9`), and the function correctly returns a `Metrics` artifact after logging a metric. Without the decorator or with an incorrect decorator, the component would not be recognized as a pipeline component.

Answer

@dsl.component
def my_component(input_text: str) -> Metric:
    metrics = Metric()
    metrics.log_metric('length', len(input_text))

Answer

@dsl.pipeline
def my_pipeline(input_text: str):
    metrics = Metrics()
    metrics.log_metric('length', len(input_text))

Answer

def my_component(input_text: str) -> Metrics:
    from kfp.dsl import Metrics
    metrics = Metrics()
    metrics.log_metric('length', len(input_text))
    return metrics

Question 2

A machine learning engineer is building a Vertex AI pipeline that uses a pre-built Google Cloud Pipeline Components (GCPC) to train a custom model. Which component should the engineer use to submit a custom training job to Vertex AI?

Accepted Answer

CustomJob. The CustomJob component is the correct choice because it is the pre-built GCPC component specifically designed to submit a custom training job to Vertex AI. It allows the engineer to specify a custom container image or a Python training script, along with machine configuration and hyperparameters, directly within a Vertex AI pipeline. Other components serve different purposes, such as hyperparameter tuning, batch predictions, or model deployment.

Answer

HyperparameterTuningJob

Answer

BatchPredictionJob

Answer

ModelDeploy

Question 3

A team has a Vertex AI pipeline that includes a container component for data preprocessing. The team notices that the component is re-executed every time the pipeline runs, even when the inputs and code haven't changed. They want to leverage pipeline caching to avoid redundant executions. What should they do to enable caching for this component?

Accepted Answer

Ensure that the component does not have 'dsl.cache_options(enable_cache=False)' set.. Option D is correct because Vertex AI pipeline caching is enabled by default for all components unless explicitly disabled using `dsl.cache_options(enable_cache=False)`. The component re-executing every time indicates that caching was likely disabled on that specific component. Removing or ensuring this setting is not present will allow the pipeline to reuse cached outputs when inputs and code have not changed.

Answer

Set the 'caching' flag to 'True' in the pipeline definition using 'pipeline.caching = True'.

Answer

Set the environment variable 'ENABLE_CACHE' to 'true' on the pipeline run request.

Answer

Re-compile the pipeline with the '--enable-cache' flag.

Question 4

A machine learning team uses Vertex AI Pipelines to orchestrate their training pipeline. They want to trigger the pipeline automatically in response to new data arriving in a Cloud Storage bucket, and also support a scheduled run every day at 6 AM. Which combination of services should they use to achieve both event-driven and schedule-based triggers?

Accepted Answer

Cloud Scheduler for the schedule, and Cloud Functions triggered by Cloud Storage events to call the Vertex AI API for event-driven.. Option C is correct because Cloud Scheduler can trigger the pipeline at 6 AM daily via a cron job, while Cloud Functions, triggered by Cloud Storage events (e.g., object finalize), can call the Vertex AI API to start the pipeline when new data arrives. This combination provides both schedule-based and event-driven triggers without requiring custom infrastructure.

Answer

Cloud Scheduler for the schedule, and Cloud Pub/Sub with Push subscription to Vertex AI for event-driven.

Answer

Cloud Functions for both schedule and event-driven, using cron trigger.

Answer

Vertex AI Pipelines built-in scheduler for schedule, and Cloud Pub/Sub for event-driven.

Question 5

A company is using Vertex AI Pipelines to automate model retraining. They have a component that creates a BigQuery table with training data. To ensure idempotency, the component should check if the table already exists and recreate it if necessary. What is the best practice for passing data between pipeline components?

Accepted Answer

Store data as artifacts in Cloud Storage and pass the GCS URI between components.. Option D is correct because Vertex AI Pipelines is designed to pass data between components via Cloud Storage artifacts. By storing the BigQuery table metadata or training data as a file in Cloud Storage and passing the GCS URI as an artifact, the pipeline ensures idempotency and decouples components. This approach aligns with Kubeflow Pipelines' artifact-based I/O model, where each component's outputs are materialized as URIs rather than in-memory objects.

Answer

Pass data in-memory as Python objects between components.

Answer

Use BigQuery table names as component outputs and inputs.

Answer

Use Cloud SQL to store intermediate results and pass connection strings.

Question 6

A data engineer wants to orchestrate a complex workflow that includes running a Vertex AI pipeline, then a BigQuery job, and finally a Dataflow pipeline. The workflow must handle dependencies, retries, and monitoring. Which Google Cloud service is most suitable for this orchestration?

Accepted Answer

Cloud Composer. Cloud Composer (based on Apache Airflow) is the most suitable service for orchestrating a complex workflow with dependencies, retries, and monitoring across Vertex AI, BigQuery, and Dataflow. It provides a managed Airflow environment that natively supports DAG-based orchestration, built-in retry logic, and integration with Google Cloud services via operators like VertexAIPipelineOperator, BigQueryOperator, and DataflowTemplatedJobStartOperator.

Answer

Cloud Tasks

Answer

Cloud Scheduler

Answer

Workflows

Question 7

A machine learning engineer is building a Vertex AI pipeline that uses a pre-built AutoML Tables component to train a classification model. The pipeline also includes a conditional step that deploys the model to an endpoint only if the evaluation metrics exceed a threshold. Which KFP feature should be used to implement the conditional deployment?

Accepted Answer

dsl.Condition. The `dsl.Condition` feature from KFP (Kubeflow Pipelines) is specifically designed to conditionally execute pipeline steps based on the output of a previous component. In this scenario, the AutoML Tables component produces evaluation metrics; `dsl.Condition` allows the pipeline to check whether those metrics exceed a threshold and, if true, run the deployment step. This is the correct, native KFP construct for implementing branching logic within a pipeline.

Answer

dsl.ParallelFor

Answer

dsl.ExitHandler

Answer

dsl.Collected

Question 8

A team uses Cloud Build to automatically trigger a Vertex AI pipeline when changes are pushed to the model code repository. They have a cloudbuild.yaml file that builds a container image and submits the pipeline. However, they want to run the pipeline only if the commit includes changes to the 'training/' directory. Which Cloud Build configuration option should be used to filter the trigger?

Accepted Answer

Set the 'includedFiles' field to 'training/**' in the trigger configuration.. Option D is correct because Cloud Build triggers support an `includedFiles` field that specifies a glob pattern. When set to `training/**`, the trigger will only fire if the commit includes changes to files under the `training/` directory. This is the native, declarative way to filter triggers based on changed file paths without additional infrastructure.

Answer

Add a 'ignoreFiles' field with 'training/**' to the trigger.

Answer

Use a 'substitutions' field with a regex pattern to filter commits.

Answer

Configure a Cloud Function to check the commit diff and call Cloud Build API conditionally.

Question 9

A data scientist wants to create a Vertex AI pipeline component that uses a custom container image stored in Artifact Registry. The component should accept a dataset artifact as input and output a model artifact. Which component type should they use?

Accepted Answer

A container component defined with the ContainerSpec class.. Option C is correct because a container component defined with the `ContainerSpec` class is the only Vertex AI component type that allows you to specify a custom container image from Artifact Registry. This component type directly wraps a Docker container, enabling you to define inputs (e.g., a dataset artifact) and outputs (e.g., a model artifact) via the `ContainerSpec` interface, which maps to the container's command-line arguments and environment variables. Lightweight Python components and Python function components cannot use a custom container image without a base image, and pre-built components from GCPC are fixed and do not support custom containers.

Answer

A Lightweight Python component without base_image.

Answer

A Python function component with a custom base_image.

Answer

A pre-built component from GCPC.

Question 10

A machine learning engineer needs to pass a large dataset between two components in a Vertex AI pipeline. What is the recommended way to pass this data?

Accepted Answer

Store the dataset as a Dataset artifact and pass the artifact between components.. In Vertex AI Pipelines, the recommended way to pass large datasets between components is to use a `Dataset` artifact. Artifacts are metadata references that point to the underlying data stored in Cloud Storage, enabling efficient, scalable, and type-safe data passing without serialization overhead or size limits. This approach leverages the Kubeflow Pipelines SDK's artifact tracking, which automatically handles lineage and versioning.

Answer

Write the dataset to a temporary BigQuery table and pass the table name.

Answer

Serialize the dataset to a string and pass it as a pipeline parameter.

Answer

Use a Cloud Storage bucket and pass the bucket name as a parameter.

Question 11

A team is using Vertex AI Pipelines to deploy a model. They have a component that evaluates the model and produces a ClassificationMetrics artifact. The pipeline should deploy the model only if the precision is greater than 0.9. They use dsl.If to check the metric. However, the condition always evaluates to False. What is the most likely cause?

Accepted Answer

The ClassificationMetrics artifact is not accessible in the condition context.. Option C is correct because in Vertex AI Pipelines, `ClassificationMetrics` artifacts are not directly accessible as primitive values within the `dsl.If` condition context. The `dsl.If` condition can only evaluate pipeline parameters or primitive outputs (like strings, integers, floats) that are explicitly passed as pipeline-level parameters or task outputs. A `ClassificationMetrics` artifact is a complex object that must be parsed or have its specific metric values extracted (e.g., via a custom component or `dsl.Metrics`) before they can be used in a conditional check.

Answer

The precision value is stored as a float but the condition expects a string.

Answer

The evaluation component did not output the metric correctly.

Answer

The dsl.If block is placed incorrectly in the pipeline definition.

Question 12

A company wants to implement continuous training for their ML model. The pipeline should be triggered when new training data arrives in Cloud Storage, and after training, the model should be automatically deployed to a staging endpoint if evaluation metrics pass a threshold. They also need to detect skew between training data and serving data. Which two services should they use for skew detection?

Accepted Answer

Vertex AI Model Monitoring and Vertex AI Pipelines. Vertex AI Model Monitoring provides built-in skew detection by comparing training data statistics with serving data statistics, alerting when distribution shifts exceed thresholds. Vertex AI Pipelines orchestrates the continuous training workflow, including triggering on new data arrival, model evaluation, and conditional deployment to a staging endpoint, making C the correct pair for both skew detection and pipeline automation.

Answer

Cloud Monitoring and Cloud Logging

Answer

Cloud DLP and Cloud KMS

Answer

BigQuery and Dataflow

Question 13

A machine learning team uses Vertex AI Pipelines to run a multi-step training pipeline. They want to implement a continuous delivery (CD) process where a model is automatically promoted from staging to production only if it passes an evaluation gate. Which TWO actions should they include in their CI/CD pipeline? (Choose two.)

Accepted Answer

Store model versions in Vertex AI Model Registry with version aliases (e.g., 'staging', 'production').. Option B is correct because Vertex AI Model Registry supports version aliases like 'staging' and 'production', which allow the CI/CD pipeline to automatically promote a model from staging to production only after it passes the evaluation gate. This enables a controlled, automated CD process without manual intervention.

Answer

Use Cloud Functions to periodically check for new model versions and deploy.

Answer

Manually approve each model version before deployment.

Answer

Deploy every model version directly to production without evaluation.

Question 14

A company runs a Vertex AI pipeline that uses a container component to preprocess data. The component downloads a large file from a public URL and saves the output to Cloud Storage. The pipeline fails intermittently with a 'timeout' error. Which THREE steps should the team take to improve reliability? (Choose three.)

Accepted Answer

Make the component idempotent by checking for existing output before processing.. Option A is correct because making the component idempotent by checking for existing output before processing prevents redundant work and avoids timeout failures when the file has already been downloaded. In Vertex AI pipelines, idempotent components can safely skip processing if the output already exists in Cloud Storage, reducing the risk of hitting timeout limits on subsequent pipeline runs.

Answer

Reduce the size of the file being downloaded.

Answer

Increase the machine type for the component.

Question 15

A data scientist is creating a Vertex AI pipeline using the Kubeflow Pipelines SDK v2. Which TWO statements about pipeline parameters are correct? (Choose two.)

Accepted Answer

Pipeline parameters are defined as inputs to the pipeline function decorated with @dsl.pipeline.. Option A is correct because in the Kubeflow Pipelines SDK v2, pipeline parameters are explicitly defined as input arguments to the pipeline function that is decorated with @dsl.pipeline. These parameters serve as the primary mechanism for passing configuration values (e.g., model name, learning rate, number of epochs) into the pipeline at creation time and can be consumed by downstream components.

Answer

Pipeline parameters must be serialized to JSON before use.

Answer

Pipeline parameters can only be of type str.

Answer

Pipeline parameters can be used to pass large datasets between components.

Question 16

A data science team wants to build a Vertex AI pipeline that trains a model, evaluates it, and conditionally deploys it if the accuracy exceeds 0.9. They want to use the Kubeflow Pipelines SDK v2. Which construct allows them to conditionally execute the deployment step based on the evaluation metric?

Accepted Answer

dsl.If. In Kubeflow Pipelines SDK v2, `dsl.If` is the correct construct for conditionally executing pipeline steps based on runtime metrics or parameters. It allows you to define a condition that, when evaluated to true, triggers the deployment step only if the model accuracy exceeds 0.9. This is the standard way to implement branching logic in v2 pipelines.

Answer

dsl.Conditional

Answer

dsl.ExitHandler

Answer

dsl.Collected

Question 17

A machine learning engineer needs to schedule a Vertex AI pipeline to run daily at midnight. Which approach should they use?

Accepted Answer

Create a Cloud Function that calls the Vertex AI API, triggered by a Pub/Sub message from Cloud Scheduler.. Option D is correct because Vertex AI Pipelines does not natively support cron scheduling. The recommended pattern is to use Cloud Scheduler to publish a message to a Pub/Sub topic at the desired time, which then triggers a Cloud Function that calls the Vertex AI API to create and run the pipeline. This decoupled architecture ensures reliable scheduling and allows for custom logic before invocation.

Answer

Use the Vertex AI Pipelines console to set a cron schedule directly on the pipeline.

Answer

Create a Cloud Build trigger that runs the pipeline on a schedule.

Answer

Use Cloud Tasks to create a recurring task that invokes the pipeline.

Question 18

You are building a Vertex AI pipeline using the KFP SDK v2. One component processes a large dataset and outputs a metrics artifact. You notice that the component is being cached even when the dataset changes, because the component code and image remain the same. How can you force the component to always re-execute when the dataset changes?

Accepted Answer

Set caching_strategy.max_cache_staleness = "0s" on the component.. In KFP SDK v2, caching is keyed on the component code, image digest, and all input values. If the dataset is passed as a URI string, changing the URI will invalidate the cache. If the dataset changes without a URI change, you can disable caching per component by setting caching_strategy.max_cache_staleness to 0 or using the disable_cache method.

Answer

Use the dsl.CacheKey annotation to explicitly set cache keys.

Answer

Change the component's image tag to :latest.

Answer

Add a random integer parameter to the component to vary the inputs.

Question 19

A team wants to implement continuous training for their ML model. The pipeline should be triggered when new training data arrives in a Cloud Storage bucket. Which combination of services should they use?

Accepted Answer

Cloud Storage → Cloud Functions → Vertex AI Pipelines. Event-driven triggers for Vertex AI pipelines are typically implemented using Cloud Storage notifications to Pub/Sub, which triggers a Cloud Function that creates a pipeline job. Cloud Build is for CI/CD, not for event-driven data triggers.

Answer

Cloud Storage → BigQuery → Vertex AI Pipelines

Answer

Cloud Storage → Cloud Build → Vertex AI Pipelines

Answer

Cloud Storage → Cloud Scheduler → Vertex AI Pipelines

Question 20

You are using KFP SDK v2 to define a pipeline. You need to pass a large dataset between components. What is the best practice for passing data?

Accepted Answer

Write the data to Cloud Storage and pass the GCS URI as an artifact.. In KFP SDK v2, passing large datasets between components is best done by writing the data to Cloud Storage and passing the GCS URI as an artifact. This approach leverages KFP's built-in artifact tracking, ensures data persistence across container restarts, and avoids memory or disk limitations of ephemeral containers. The artifact is automatically serialized and passed as an input/output parameter, enabling efficient, scalable data exchange.

Answer

Use the component's temporary directory to share data between containers.

Answer

Pass the data as a serialized Python object in memory.

Answer

Store the data in a BigQuery table and pass the table reference.