CCNA Collaborating Within and Across Teams to Manage Data and Models Questions

Vertex AI Metadata stores artefacts and executions in a lineage graph, recording which datasets, parameters and models each pipeline run consumed and produced. This directly satisfies the requirement to track lineage across ML pipeline executions.

Delta Lake's transaction log records every write as a numbered commit, so querying a prior version by timestamp or version number retrieves the exact historical snapshot. This directly satisfies the team's data versioning requirement on Dataproc, enabling reproducible ML training and rollback without duplicating datasets.

Vertex AI Model Registry stores model versions with metadata and supports aliases, so the deployment team can review history and assign a production alias before deploying to an endpoint. It is the intended handoff mechanism between experimentation and serving, satisfying both version tracking and alias requirements.

Model aliases are mutable, named pointers (for example 'champion', 'challenger') that reference a specific version within a registered model, so traffic can be switched between versions without redeploying or changing version IDs. This directly supports designating production and A/B testing versions.

A centralized feature store hosted in one shared project lets multiple projects consume identical feature definitions, with IAM policies granting cross-project access. This satisfies the requirement to share definitions organisation-wide while avoiding duplicated, divergent feature stores per project.

Vertex AI Metadata stores artefacts, executions and contexts in a lineage graph, so querying it returns every downstream artefact derived from a given dataset. This satisfies the stem's need to trace dependencies from a specific dataset artefact.

Autologging hooks into supported frameworks (for example scikit-learn, TensorFlow, XGBoost) and records parameters, metrics and artifacts to an Experiment run automatically, satisfying the minimal-code-change constraint. Manual logging via the SDK would require explicit calls in every training script.

Vertex AI Metadata stores pipeline resources as a lineage graph of executions, artifacts and contexts, so you can query upstream and downstream dependencies of any artifact. Cloud Logging records events but holds no typed lineage relationships between datasets, models and metrics.

Vertex AI Pipelines supports conditional execution using the condition parameter on tasks. By adding a condition that evaluates the model's accuracy metric, you can gate the ModelUploadOp so that only models meeting the threshold are uploaded to Model Registry. This enforces the quality gate directly in the pipeline without manual intervention.

Vertex AI Model Registry is the centralised catalogue that tracks model artefacts, versions and lineage across projects. It satisfies the governance requirement by giving one authoritative location to register, discover and manage models throughout their lifecycle.

The optimized online store is purpose-built for low-latency, high-throughput serving of small feature values, so migrating from the Bigtable online store addresses the peak-hour latency directly. Increasing node count or reducing feature size does not change the underlying serving architecture.

Vertex AI Model Registry is the central catalog where models become discoverable, versioned, and deployable to endpoints. Importing the joblib artifact with the appropriate pre-built scikit-learn serving container and attaching evaluation metrics gives other teams visibility and a one-click path to deployment. This is the correct first step for sharing a model across teams.

User-managed notebooks give a persistent Compute Engine instance with a customisable environment, so installed packages and data survive stopping and restarting. This satisfies the stem's requirement for persistence, unlike managed notebooks with ephemeral or containerised runtimes.

Vertex AI Feature Store provides a centralised repository where features are computed once and served identically to both training and prediction pipelines, directly eliminating training-serving skew. Sharing one feature definition across multiple models satisfies the consistency requirement, since online and batch serving draw from the same managed source rather than duplicated transformations.

Point-in-time correct retrieval reconstructs the feature values that existed at each training label's timestamp, preventing future data from leaking into training. This directly satisfies the stem's requirement that training features match those available at serving time, eliminating training-serving skew.

Vertex AI Feature Store provides a centralised repository where feature values are defined once and served consistently to both training jobs and online prediction, eliminating training-serving skew. This shared definition satisfies the consistency requirement while allowing multiple models to reuse the same features.

DVC stores a small .dvc metafile in Git containing the data's hash and path, while the actual dataset lives outside the repository. This lightweight pointer lets Git track dataset versions without bloating the repo, satisfying the requirement to version data alongside ML code.

Capturing the model resource ID with each prediction creates the link from a specific prediction to a specific registered model version. Combined with lineage in Vertex ML Metadata, an auditor can follow that model version back to its training artifacts and pipeline execution, completing the end-to-end trace the regulation demands.

Point-in-time correct retrieval with timestamp matching returns feature values as they existed at each training example's timestamp, preventing leakage from future user activity counts. Naive latest-value retrieval would expose post-event data, inflating offline metrics relative to production.

Vertex AI Feature Store monitoring computes drift and skew metrics natively against feature values, detecting distribution shifts over time without custom code. It satisfies the requirement to monitor features for drift directly within the feature store, providing scheduled drift detection and alerting on the stored feature data.

Vertex AI Feature Store provides a central registry where feature definitions are authored once and served consistently to both training and online inference, removing manual Cloud Storage coordination errors. It also supplies point-in-time correctness, which prevents training-serving skew.

Bigtable online store supports high write throughput and frequent feature updates, scaling to thousands of writes per second with low-latency reads. This satisfies the stated requirement for minute-level updates and high-throughput ingestion that the default online store cannot match.

Feature monitoring in Vertex AI Feature Store continuously computes drift metrics by comparing production feature distributions against a baseline snapshot, directly satisfying the requirement to monitor feature drift. It detects skew and drift per feature, emitting alerts without retraining, which is precisely the capability the stem requests.

Vertex AI Model Registry generates model cards from registered model metadata, including purpose, evaluation results and intended use, when the model is uploaded with those details. This automates documentation from the pipeline's existing metadata rather than requiring manual authoring.

Vertex AI Feature Registry and Feature Groups let the platform team define each feature once over a BigQuery source, and a Feature Online Store serves the same definitions online. Granting both teams read access means their separate pipelines consume identical feature definitions, which is exactly the consistency and single-source-of-truth requirement described.

Model Cards in Vertex AI automatically capture and display model purpose, training data, evaluation metrics, and intended use, directly satisfying the documentation requirement. Unlike generic metadata or manual reports, Model Cards generate this structured governance artefact from the model's own evaluation results, giving the data scientist the exact fields the stem demands.

BigQuery table snapshots create immutable, point-in-time copies of a table that persist independently of later writes, so a training job can always be reproduced from the exact snapshot it consumed. Recording the snapshot ID in pipeline metadata ties the artifact to the run, giving the traceability the team currently lacks without duplicating data or blocking the engineer's writes.

Vertex AI Experiments logs parameters, metrics, and artifacts for each run and provides comparison across runs within a experiment. This directly satisfies the need to track training metadata and evaluate multiple runs side by side.

Feature Store decouples feature engineering from consumption: one definition feeds both the low-latency online store for serving and the offline store for point-in-time historical retrieval, eliminating training-serving skew. Reusing a Cloud Storage copy cannot meet single-digit millisecond online latency.

Vertex AI Metadata provides the lineage tracking required, storing artefacts, executions and contexts as nodes within a managed metadata graph. It records relationships between pipeline components and their outputs, satisfying the stem's demand for tracing component, artefact and execution provenance across ML workflows.

Delta Lake provides ACID transactions and scalable metadata handling over Parquet files in Cloud Storage, letting concurrent Dataproc jobs read and write multi-terabyte datasets consistently. Its transaction log delivers the snapshot isolation and versioning the stem demands.

Git integration in Vertex AI Workbench lets users commit notebook code to a remote repository, track revisions and share branches with teammates. This satisfies the version-control and collaboration requirement without leaving the managed notebook environment.

This is correct because Vertex AI Model Registry supports importing scikit-learn models with a prebuilt serving container, and the imported model can be deployed directly to an Endpoint. The prebuilt scikit-learn container handles prediction requests without requiring custom inference code, and deployment can be done through the console or gcloud CLI. This matches the standard workflow for deploying a locally trained scikit-learn model.

A BigQuery table snapshot is an immutable, point-in-time copy of the table that persists even if the source table changes. By creating a snapshot and passing its name to the training component, you ensure the pipeline run always reads the same data. This provides reproducibility and aligns with Vertex AI Pipelines artifact tracking, as the snapshot can be recorded as an input artifact.

The Model Registry resource holds the model version, its metadata, and aliases such as 'production'. Sharing this resource name lets the operations team see exactly which version is aliased for production and deploy it, satisfying the alias and versioning part of the handoff.

dvc add computes the hash of the processed directory, writes a .dvc file and updates .gitignore, registering the new dataset version for tracking. dvc push only uploads already-tracked data to remote storage; it does not register changes.

The Vertex AI SDK's experiment logging functions (aiplatform.log_params and log_metrics) attach hyperparameters and metrics to a named experiment run, which is exactly what enables side-by-side comparison of runs in the Vertex AI Experiments console. Code versions are captured separately via Git.

Deploying both versions to one endpoint with traffic splitting lets the engineer route a defined percentage to the challenger while the champion serves the remainder, enabling gradual A/B comparison. Separate endpoints would not permit proportional traffic distribution between versions.

The offline store performs point-in-time joins, matching each training example's timestamp against feature values valid at that moment. This guarantees the model only sees historically accurate features, eliminating the temporal leakage the scenario requires avoiding during training.

Feature group monitoring computes drift statistics against a baseline distribution and triggers alerts via a configured notification channel when the drift threshold is exceeded. Enabling it at the feature group level with both parameters satisfies the automated-alert requirement without custom pipeline code.

Vertex AI Endpoints support deploying multiple model versions and splitting prediction traffic by percentage. Deploying the new version alongside the current one and starting with a small split enables a controlled canary rollout, and because the old version remains deployed, reverting is simply resetting the split back to the previous version.

Vertex AI Pipelines can orchestrate a workflow that triggers upon model registration, running evaluation steps such as fairness and bias checks. It allows integration with other services and custom code, enabling automated pre-deployment assessment. By using pipelines, the team can enforce that no model is deployed without passing the fairness evaluation.

BigQuery table snapshots capture a table's contents at a specified timestamp as a lightweight, queryable reference that shares storage with the base table rather than duplicating data. This satisfies the stem's reproducibility and no-full-copy constraints.

Configuring monitoring on the feature view applies drift detection directly to the served feature data, satisfying the daily Jensen-Shannon divergence threshold of 0.1. Feature-view-level monitoring evaluates the numerical feature against its baseline distribution, so alerts trigger precisely when divergence exceeds the specified limit.

Managed notebooks include native Git integration, exposing repository cloning, credential handling and commit or push controls through the interface. This satisfies version control and collaboration requirements without manual command-line setup, unlike unmanaged instances where git must be configured by hand.

Vertex ML Metadata records the dataset artifact as an input to the training execution, and that artifact can carry a URI or metadata identifying the BigQuery snapshot. Traversing from the model artifact backward to this input artifact gives the exact data reference needed for reproduction.

Point-in-time retrieval returns feature values as they existed at the supplied timestamp, joining each training row to its historical feature state. This directly prevents label leakage, satisfying the requirement that retrieved features exactly match values at a specific historical timestamp rather than current values.

Time travel queries Delta table snapshots by version or timestamp, letting the team reproduce an exact training dataset and restore an earlier state via RESTORE. This directly satisfies the versioning and rollback requirement, unlike schema evolution or Z-ordering, which address structure and query performance.

A centralised model registry provides governance and version control, tracking each model's lineage, approvals, and deployment stage. This gives teams a single authoritative source for model artefacts, satisfying auditability and reproducibility requirements across the MLOps lifecycle.

DVC stores large dataset files outside Git while committing small metafiles that capture content hashes and versions, so Git tracks dataset changes without bloating the repository. This pointer-based mechanism satisfies the requirement to version data alongside code using Git.

Vertex AI Pipelines lets teams author reusable components and pipeline templates with the Kubeflow Pipelines DSL, then share and parameterise them across projects. This directly satisfies the requirement for standardised, reusable pipeline templates across multiple teams, since components are versioned artefacts rather than one-off scripts.

Git integration in managed notebooks provides branch-based version control and conflict resolution for concurrent editors, while a custom container image pins library versions so every data scientist gets an identical runtime. This satisfies the stem's three constraints: version-controlled changes, conflict-free collaboration, and reproducible environments across the team.

DVC versions large datasets held in Google Cloud Storage by storing content hashes in small metafiles committed to Git, leaving the data in place. This delivers change tracking and reproducible checkouts of previous data states, exactly the capability the team requires.

Point-in-time correct join retrieves feature values as they existed at each training example's timestamp, preventing future information from leaking into training. This satisfies the stem's requirement that training data avoid data leakage when joining real-time and historical features.

Artifacts are the versioned, typed outputs and inputs that Vertex AI ML Metadata records, such as datasets, models and metrics. Naming them satisfies the lineage requirement because each Artifact node links to the Executions and Contexts that produced or consumed it, forming the traceable graph.

BigQuery table snapshots provide a point-in-time immutable copy of the training data. Storing the pipeline code in Git and passing the snapshot ID as a parameter ensures both data and code are versioned. This combination enables exact reproduction of the training run.

The versions update subcommand with --update-aliases assigns the 'champion' alias to a specific registered model version, enabling production deployment by alias rather than version number. Other commands set IAM policy or labels, which do not create the deployment alias the scenario requires.

queryArtifactLineageSubgraph traverses the metadata store's lineage graph in both directions from a given artifact, returning the executions and artifacts that produced or consumed it. This directly satisfies the requirement to trace a model artifact back to its source dataset and hyperparameters.

Vertex AI Metadata records and stores lineage artefacts, capturing which dataset, pipeline component and hyperparameter tuning job produced each model. This satisfies the lineage requirement, letting teams trace model provenance across pipeline components for reproducibility and audit.

Deployment requires the model in the registry first, then an endpoint, then deploying that model to the endpoint where traffic splits are configured. This sequence satisfies the canary traffic-splitting constraint, since traffic allocation is set at deploy time on the endpoint, not during upload.

Vertex AI Pipelines executes containerised components, and storing pre-built and custom components in a component registry lets teams share and version them centrally, enforcing governance and reuse across teams. Components are defined by YAML specs, so standard interfaces are preserved.

Vertex ML Metadata stores artifacts, executions, and events, so you can start from the model artifact and walk backward to the training execution, then to the dataset artifact and code artifact that were inputs. This provides an auditable, programmatic chain of provenance for the deployed model version.

Point-in-time correct retrieval joins each training label to feature values as they existed at that timestamp, preventing future data leaking into training. This aligns offline training vectors with the online serving values, directly eliminating the training-serving skew caused by mismatched feature snapshots.

Vertex AI Experiments records parameters, metrics, and artifacts from training runs, and each run is associated with a lineage context. It provides a searchable UI and API so teammates can compare runs and reproduce them without inspecting notebook code, which satisfies the requirement for organization-wide tracking.

In Vertex AI Feature Store, offline and online stores are separate. Batch ingestion writes to the offline store, and you must explicitly run an ingestion job to update the online store, or use streaming ingestion for real-time updates. If the data engineer only updated the offline store, the online store will not reflect the changes until a sync job is executed.

Vertex AI Metadata stores artefacts, executions and contexts in a managed ML metadata store, recording lineage as models train and deploy. It satisfies the lineage-tracking constraint directly, letting teams trace which datasets and training runs produced each model version. Evaluation metrics attach as metadata to those artefacts, supporting governance across teams.

The lineage query API accepts an execution ID and returns the subgraph of artifacts produced by that execution, directly answering which artifacts a given pipeline run generated. Listing executions alone returns run metadata without the produced artifact relationships.

Table snapshots capture the table state at a point in time and are retained as independent, read-only copies that keep working even after the source table is overwritten. Taking a snapshot before each training run gives the reviewer a stable, addressable dataset to reproduce the exact training input without duplicating full storage cost.

Delta Lake stores its data files as Apache Parquet, layering a transaction log over them to provide ACID guarantees. Parquet supplies the columnar, compressed storage; the Delta log records commits, enabling concurrent reads and writes without corrupting the underlying Parquet files.

Vertex AI Model Registry provides a centralised, versioned catalogue tracking each model's versions, deployment history and evaluation metrics, giving the organisation-wide governance and compliance visibility the stem requires. It integrates with Vertex AI Pipelines and endpoints, so lineage is captured automatically.

Model Cards generate structured documentation covering model details, intended use and evaluation results, directly satisfying the documentation requirement. They attach to registered models in Vertex AI Model Registry, pulling evaluation metrics automatically rather than requiring manual authoring.

Vertex AI Feature Store provides a centralised repository for engineered features with low-latency online serving, letting multiple projects reuse the same feature definitions. This satisfies both the sharing requirement and the online prediction latency constraint.

Vertex AI Model Registry provides central model versioning, lineage tracking and deployment management across teams, satisfying the governance requirement. It organises models into versioned lineages and integrates with endpoints, giving one governed catalogue for all teams' models.

BigQuery table snapshots provide a point-in-time, immutable copy of the table. By capturing the snapshot ID as a pipeline parameter, the training run is linked to the exact data version. This ensures reproducibility and auditability, as the snapshot can be restored or queried later. It directly addresses the need to record the data snapshot.

Vertex AI Model Registry stores model cards alongside each version, capturing training data, feature importance, evaluation metrics and intended use. This satisfies the automated documentation requirement by centralising governance artefacts with the model lineage rather than in separate manual documents.