Databricks-ML-Assoc Flashcards — Free Databricks Certified Machine Learning Associate Study Cards

Log the git commit hash using mlflow.set_tag(). / Capture the execution environment using log_model(conda_env=...).

Reproducibility in machine learning requires strict tracking of code versions, dependencies, and parameters. By logging the source code version (git hash) and the specific Python environment (conda.yaml or requirements.txt), the scientist ensures that any collaborator can recreate the exact training state later. These practices are fundamental to the Databricks ML lifecycle, ensuring that models can be retrained or audited for compliance during the deployment phase.

Invoke mlflow.autolog() at the beginning of the notebook cell prior to training.

Using mlflow.autolog() is the recommended practice for capturing model metadata, hyperparameters, and metrics automatically in Databricks. This approach minimizes boilerplate code and ensures consistency across experiments, reducing human error. It is vital for reproducibility and model governance within the Databricks environment, as it captures the framework-specific details required for later model registration and deployment without requiring manual tracking of every individual metric.

The model signature defining input and output schema

Databricks Model Serving requires a model artifact to be logged with a signature. The signature defines the expected input schema, which allows the serving endpoint to validate incoming requests. Without a signature, the model cannot be correctly parsed by the serving infrastructure, preventing the deployment from starting. This ensures that the production inference endpoint operates with expected data formats, maintaining reliability in downstream applications.

Execute mlflow.autolog() at the start of the notebook cell before running the model training code.

MLflow provides autologging capabilities that automatically capture parameters, metrics, and models when using popular machine learning libraries like Scikit-learn, TensorFlow, or PyTorch. By calling mlflow.autolog() before the training code, the framework instruments the library calls to record telemetry automatically. This approach minimizes boilerplate code and ensures consistency across experiments, which is essential for auditability and model reproducibility in collaborative Databricks environments, as it prevents human error in manual logging.

Implement a retraining pipeline that is triggered when performance metrics drop.

Addressing data drift involves monitoring incoming production data and comparing it against the training data distribution. By using Delta Lake's time-travel capabilities and MLflow's experiment tracking, teams can identify when and why model performance degrades. This proactive monitoring allows for timely retraining of the model with updated data, ensuring that production predictions remain accurate and aligned with the current real-world environment, which is vital for long-term model reliability.

A Databricks container services custom Docker image specified for the cluster.

An engineer-controlled, node-consistent runtime is exactly what container services provides: a custom Docker image is used by all nodes, so library and OS versions are fixed. MLflow then records the environment alongside the run, giving both control and traceability. Cluster libraries, init scripts, and Repos each address only part of the problem without delivering an immutable image.

It includes additional local files/modules needed for the model to execute correctly.

The 'code_path' parameter allows users to include additional local source code files or directories required by the model, such as custom preprocessing classes. By bundling these dependencies, the model remains self-contained and portable. This is essential when the inference code relies on custom logic defined in external modules, ensuring that those modules are correctly packaged and available whenever the model is loaded in a new runtime environment.

Update the served entity in the endpoint configuration to reference model version 5, allowing rollback by re-pointing to version 3

The endpoint serves whichever model version URI its configuration references, so updating the served entity to version 5 redirects live traffic through a managed rollout and keeps rollback simple by re-pointing to version 3. Deleting and recreating, batch overwrites, and registry stage changes do not achieve reversible live traffic redirection.

The client application is sending data that does not conform to the defined input schema.

The error explicitly indicates a schema mismatch between the client's payload and the model's expected input signature. This frequently occurs when downstream applications send data without proper type casting. Validating the input data before sending it to the serving endpoint is a critical step in production MLOps to prevent type-related failures and ensure robust service interaction between disparate software components in a distributed architecture.

Update the inference client payload to explicitly cast the 'user_age' field to an integer or long type before sending the request.

Databricks Model Serving strict signature validation enforces data types defined during model logging. When incoming payloads contain mismatched types, such as double instead of long, requests fail. The engineer must re-log the model with a correct signature or cast incoming payload data types to match the expected schema.

It provides managed, auto-scaling infrastructure with built-in model versioning.

Databricks Model Serving provides a managed, serverless infrastructure that scales automatically based on load and provides low-latency inference. It integrates directly with the MLflow Model Registry, ensuring that the model version deployed is exactly the one tested. This managed approach handles complex infrastructure concerns like auto-scaling, high availability, and authentication, reducing operational overhead compared to manual deployments on standalone servers, which require custom management of dependencies, security, and scaling.

The cluster library configuration is missing the required MLflow package.

The ClassNotFoundException indicates that the MLflow library is missing from the runtime environment of the job cluster. In Databricks, if a job is configured to use a custom or minimal environment, essential libraries must be explicitly installed. This error highlights the importance of dependency management in production ML workflows, ensuring that all necessary packages are present in the cluster environment to prevent execution failures during automated deployment or batch scoring jobs.

Use the publish_table method to sync features to an online store configured in the Feature Store.

To enable online serving, the feature table must be published to a supported online store like Amazon DynamoDB, Azure Cosmos DB, or Google Cloud Bigtable. This process decouples the feature retrieval from the complex logic of feature engineering pipelines, allowing real-time models to fetch pre-computed features in milliseconds rather than recalculating them during the inference request, which is critical for high-throughput production ML applications.

The endpoint is overloaded or incorrectly configured.

A 'Connection refused' error coupled with a request timeout typically indicates that the serving endpoint is either overloaded, configured incorrectly with insufficient resources, or facing network connectivity issues. Since the error persists despite retries, it points to a failure in the endpoint infrastructure or scaling capacity, which requires an investigation of the endpoint's resource allocation and the network configuration within the Databricks workspace to restore service reliability.

MLflow log_metric API.

MLflow's logging API allows users to log metrics at each step of the training loop. By calling `mlflow.log_metric()` during the epochs of a deep learning model, the scientist can visualize the training progress in the MLflow UI. This real-time feedback loop is essential for detecting issues like vanishing gradients or overfitting early in the development lifecycle, preventing wasted compute hours on non-converging or poor-performing models.

MLflow Tracking

MLflow Tracking is the dedicated component within Databricks for recording experiments. By logging parameters, metrics, and artifacts, data scientists can reproduce results and compare different model versions effectively. This is crucial for maintaining model lineage and ensuring reproducibility across distributed training jobs in production environments. MLflow is integrated natively into the Databricks platform, providing a seamless experience for tracking machine learning workflows from experimentation to final deployment.

MLflow Tracking UI, which lists runs within an experiment and supports sorting and filtering by metrics.

The MLflow Tracking UI is designed for experiment comparison. It presents runs in a sortable table with metrics and parameters, supports filtering, and allows drilling into individual runs. The Model Registry UI, Projects, and Jobs run history serve different purposes and do not offer the same run comparison and metric sorting features.

The model has not been registered in the Model Registry with the specified name and version.

This error indicates that the specified model version does not exist in the Registry, either because it was never registered, was deleted, or the name/version number was referenced incorrectly. In a Databricks context, verify that the model has been successfully logged and registered using mlflow.register_model. This is critical for deployment pipelines as it validates that the artifact exists before the serving infrastructure attempts to load the model file.

The runs table with column sorting and the filter box using metric.weighted_f1.

The MLflow UI runs table provides a sortable and filterable view of all runs in an experiment. By sorting on the 'weighted_f1' metric column or using a filter expression, the engineer can instantly locate the top-performing run. This is the standard way to compare runs and select the best model.

The learning rate was not adjusted for the distributed batch size.

In Horovod distributed training, the learning rate must often be scaled relative to the number of workers, because each worker processes a batch of data. A common mistake is failing to adjust the learning rate or optimizer settings to compensate for the aggregate batch size across the cluster. This results in ineffective gradient updates, slowing down convergence significantly compared to a single-node setup.

Model Serving with a served entity pointing at the registered model version.

Model Serving provides managed, autoscaling REST endpoints backed by registered model versions. Configuring a served entity that references a Unity Catalog model and version gives a secure URL and lets the team change the served version without rebuilding the endpoint, satisfying low-latency, secure, and updatable serving needs in one capability.