ML Workflows practice questions

Models can be registered from any local directory regardless of the tracking server.

Why it fails: Models must be logged to the MLflow tracking server to be registered. You cannot register a model from a local file system directly; it must be stored in the MLflow tracking store, which manages the metadata and artifact paths required for the registry to track lineage and versioning history.

Deleting a registered model version also deletes the raw training data used to build it.

Why it fails: Deleting a model version only removes the registry entry and its associated artifacts stored in DBFS or S3. It does not affect the external training datasets stored in tables or Delta Lake, as the registry is decoupled from the raw data management and storage used during the training process.

The Model Registry automatically triggers retraining when a model version is updated.

Why it fails: The Model Registry is a passive repository; it does not contain logic to trigger retraining pipelines. Retraining must be handled by external workflow orchestrators like Databricks Workflows or Airflow, which monitor data drifts or model performance and invoke the training jobs independently of the registry's state changes.

To increase the training speed of the model during the logging phase.

Why it fails: Signatures and input examples are metadata used for inference validation; they have no impact on the performance or speed of the training process. They are strictly used to improve the usability and safety of the model once it is deployed to a serving endpoint or a production scoring job.

To automatically convert the model into a different framework like ONNX.

Why it fails: While signatures are metadata, they do not trigger automatic model conversion. Framework conversion (e.g., to ONNX) requires explicit usage of the appropriate MLflow flavor or external conversion tools. The signature exists solely to define the interface of the model, not to transform its internal computational representation or library format.

To reduce the physical storage size of the logged model artifacts.

Why it fails: Signatures are small JSON files added to the model metadata; they do not impact the storage requirements of the model artifacts (like weights and parameters). The primary value is improved schema enforcement and usability for downstream consumers of the model, not storage optimization or artifact compression techniques.

It automatically scales the compute cluster when the feature store is queried.

Why it fails: Feature Store metadata is for lineage and lookup, not for infrastructure scaling. Cluster scaling is managed by the Databricks compute service, not the Feature Store API. The metadata facilitates data lookup and validation, but it does not have the ability to modify the underlying cluster's configuration or resource allocation.

It enables the model to be trained on data that hasn't been processed yet.

Why it fails: Feature stores require data to be pre-processed and stored. You cannot train on data that doesn't exist or hasn't been ingested. Metadata tracking does not provide magical data processing capabilities; it simply manages the association between models and the already calculated features stored in the feature tables.

It converts all features into a standard binary format for faster training.

Why it fails: Feature metadata does not perform data format conversion or compression. The features are stored in Delta tables and accessed as requested. The metadata is strictly for tracking and lineage, not for optimizing the physical storage format or computational representation of the features used in the training process.

Use Delta Live Tables to restrict access to the production data table.

Why it fails: Restricting access to data does not equate to reviewing or signing off on a model. A model could be flawed even if the data is secure. The model review process must focus on the model's performance, code quality, and compliance, which is a different domain than access control for data.

Enable 'Auto-Deploy' in the MLflow Model Registry settings.

Why it fails: Auto-deploy is the opposite of a human-signed-off workflow. It bypasses manual verification entirely, which directly contradicts the requirement for a human reviewer to sign off on production deployments. This setting would be used for high-velocity, automated pipelines, not for models requiring controlled, manual verification before release.

Set the MLflow experiment status to 'ReadOnly' until a review is complete.

Why it fails: Setting an experiment to read-only prevents any further logging of results, which would stop the model development process entirely. It does not provide a mechanism for approval or a clear path toward moving a model into a production-ready status. This is not a standard or effective way to manage workflows.

It defines the hardware specifications for the job cluster.

Why it fails: Cluster hardware specifications are defined within the 'job_cluster_key' configuration block, not the 'base_parameters' field. Parameters are designed for variable inputs into the application logic, such as hyperparameters or file paths, rather than infrastructure-level settings like instance types, node counts, or disk volumes for the underlying Spark cluster.

It restricts the notebook access to specific Databricks folders.

Why it fails: Access control and permissions for notebooks are managed via the Databricks Workspace Access Control Lists (ACLs) or unity catalog settings, not through job parameters. 'base_parameters' are intended for data or logic variables used during runtime, not for enforcing security boundaries or file system access constraints on the Databricks platform.

It automatically schedules the job to run every 10 minutes.

Why it fails: Job scheduling intervals are defined in the 'schedule' block of the job definition. Parameters in 'base_parameters' are strictly for data input and configuration of the notebook's internal variables; they have no influence over the timing, frequency, or cadence of the job execution cycle within the Databricks Workflow system.

It automatically generates synthetic training data for small datasets.

Why it fails: Feature lookups do not perform data augmentation or synthesis. Their primary purpose is to retrieve pre-computed feature values from the Feature Store, ensuring consistency between training and serving. Synthetic data generation is a separate process related to data science experimentation and does not involve the Feature Store lookup mechanism.

It allows the model to train directly on raw, unformatted JSON files.

Why it fails: Feature Stores are designed to work with structured, feature-engineered tables, not raw, unformatted JSON files. The purpose of the Feature Store is to provide high-quality, pre-computed features that have already undergone processing, cleaning, and formatting, ensuring the model input remains consistent regardless of the underlying raw data source.

It significantly reduces the latency of the training process itself.

Why it fails: While the Feature Store organizes data efficiently, the lookup mechanism is designed for point-in-time correctness and consistency, not for speeding up the computational process of training. Improving training speed usually requires hardware optimization, parallel processing, or algorithmic changes, rather than the feature management provided by the Feature Store lookup API.

The model artifact file size exceeds the allowed limit for production.

Why it fails: Model artifact size limitations are typically managed by the storage backend (e.g., S3 or ADLS) or specific model serving endpoints, not by the registry's stage transition policy. This error explicitly states a policy violation regarding the approval status, which is a logic-based governance constraint, not a storage capacity limit.

The model version 1 is not compatible with the production compute type.

Why it fails: Compute compatibility is not verified during the model registry stage transition. Compatibility issues are usually detected during the model deployment phase to a serving endpoint or during the evaluation stage. The error specifically references the registry's policy, which is about status and approval, not hardware or software environment specifications.

The user attempting to promote the model does not have Git access.

Why it fails: Git access is irrelevant to the internal state of the MLflow Model Registry. Model stage transitions are managed through the Databricks/MLflow API and workspace permissions, not external version control systems. The error relates to the registry-defined policy for promotion, not the user's interaction with external code repositories.

The Registry automatically retrains models when data drifts beyond a specific threshold.

Why it fails: Model Registry is a management tool, not an orchestration engine for retraining. While you can trigger retraining workflows via Databricks Workflows, the Registry itself does not monitor data drift or initiate compute tasks. It remains a repository for metadata and model versions rather than an active monitoring system.

You can only register models that were trained using Scikit-learn or PyTorch.

Why it fails: The Model Registry is framework-agnostic. It supports any ML model that can be packaged as a Python function, including XGBoost, TensorFlow, SpaCy, and even custom models created with arbitrary Python code. The flexibility allows teams to use the best framework for their specific business use case.

The Registry allows multiple models to share the same name without versioning.

Why it fails: Model names act as unique namespaces. Within a single model name, versions are strictly incremented. Allowing multiple models without versioning would cause collisions and prevent tracking the history of improvements, making it impossible to roll back to previous versions or compare performance between different iterations.

MLflow Projects.

Why it fails: MLflow Projects is a format for packaging data science code. While it helps with reproducibility, it is not a UI-based management tool for comparing run metrics or handling model versions. It focuses on the portability and execution of the training environment rather than the visualization of metrics.

MLflow Recipes.

Why it fails: MLflow Recipes provides a standardized framework for building ML pipelines. While useful for creating structured projects, it is an abstraction layer and not the central UI used for general experiment comparison or model lifecycle management. It is designed for pipeline architecture rather than general visualization.

MLflow Model Serving.

Why it fails: Model Serving is the infrastructure used to deploy models as REST endpoints. While you can view the status of a deployed model through it, it does not provide the capability to compare training runs or visualize experimental performance metrics across different iterations of a model.

The model was trained on a cluster with insufficient RAM.

Why it fails: Memory issues would manifest as 'Out of Memory' errors or cluster crashes, not as specific lookup failures for feature names. Feature lookup errors are strictly related to metadata or database connectivity problems within the Feature Store, independent of the available RAM on the compute cluster.

The input data provided to the inference request contains too many rows.

Why it fails: The number of rows does not affect the lookup of a feature name. Feature lookup errors indicate that the system cannot find the specific feature definition requested by the model code. Scaling issues might lead to timeouts, but they would not result in 'feature not found' errors.

The user does not have permission to view the model artifacts in the Registry.

Why it fails: Permission errors would generate access denied notifications or authorization exceptions. A 'feature not found' error specifically points to a configuration issue within the feature lookup logic or the registration of the model's feature dependencies, rather than an identity or access control limitation.

Running standard trials inside a for-loop on a single-node driver.

Why it fails: Running trials in a for-loop on the driver node does not leverage the distributed nature of the cluster. The driver becomes a bottleneck, and the training tasks are executed sequentially, failing to utilize the parallel processing capabilities available across the worker nodes in the Databricks cluster.

Using the MLflow autologging feature to optimize parameters automatically.

Why it fails: MLflow autologging records parameters and metrics during a run, but it does not perform hyperparameter optimization itself. It is a logging utility, not a search algorithm or a distributed execution framework, so it cannot be used to tune model parameters in a distributed manner.

Wrapping the training code in a pandas UDF for parallelization.

Why it fails: While pandas UDFs can parallelize data processing, they are not suited for Hyperopt-based hyperparameter tuning. Hyperopt requires a search algorithm to manage trial dependencies and objective function results, which manual UDF implementation would lack, leading to inefficient resource usage and complex, unmanageable code for optimization.

Call mlflow.log_artifact() on the conda.yaml file produced by mlflow.run.

Why it fails: The conda.yaml is typically generated by MLflow Projects at run time; manually logging it as an artifact does not ensure it reflects the actual environment used in this training Job. Artifact logging is passive storage, not environment capture. Reproduction requires MLflow to record the environment automatically as part of the run, not as an afterthought.

Use mlflow.set_experiment() to attach the environment metadata to the experiment.

Why it fails: set_experiment() only selects or creates the experiment under which runs are recorded. It does not capture package versions, Python version, or system dependencies. While useful for organizing runs, it has no mechanism to record or restore the training environment, so it does not satisfy the requirement for reproducible dependencies.

Call mlflow.log_artifact() on the requirements.txt file after training completes.

Why it fails: Logging a requirements.txt as an artifact records the file but does not guarantee that the training environment is captured or that the model can be recreated automatically. MLflow does not read that artifact to reconstruct the environment for scoring or reproduction. This approach also relies on the engineer manually maintaining the file, which can drift from the actual installed packages.

The feature table must be registered in the Databricks Model Registry.

Why it fails: Feature tables are registered in the Feature Store, not the Model Registry. The Model Registry is for models, not features. While a model that uses features may be registered, the feature table itself does not need to be in the Model Registry. This statement confuses the two registries and is incorrect.

The training dataset must be created using a Delta table that includes the primary key and timestamp columns.

Why it fails: Feature tables require a primary key and a timestamp column for point-in-time lookups, but the training dataset is created via create_training_set, which handles the joins and time travel automatically. You do not need to manually create a separate Delta table with these columns; the feature table itself must have them. This statement misstates the requirement.

The feature table must be stored in a Unity Catalog metastore.

Why it fails: While Unity Catalog is recommended for governance, it is not a strict requirement for using Feature Store with online serving. Feature Store can work with tables in the Hive metastore as well, though Unity Catalog provides better lineage and access control. This statement overstates the necessity and is therefore incorrect.

Use mlflow.start_run() with the nested=True parameter to suppress artifact logging.

Why it fails: The nested parameter in start_run() is used to create nested runs within a parent run; it does not suppress artifact logging. Artifacts are only logged when explicitly called or through autologging. This option incorrectly assumes nested runs change artifact behavior, which they do not.

Use mlflow.log_artifact() to log only the model file and a requirements.txt manually.

Why it fails: Manually logging artifacts and a requirements file does not capture the full code state or library versions automatically. It also does not prevent the dataset from being logged if the code still calls log_artifact on the dataset. This approach is error-prone and does not provide the reproducibility guarantees of MLflow's built-in environment capture.

Set the MLFLOW_TRACKING_URI to a Delta table location and log the dataset as a Delta table.

Why it fails: Setting the tracking URI to a Delta table location is not a standard MLflow configuration and does not control artifact logging. Logging the dataset as a Delta table would still record the dataset, which is what the engineer wants to avoid. This approach misuses MLflow tracking and does not solve the problem.

Configure a Model Registry webhook that listens for model version creation and automatically transitions to Production.

Why it fails: Webhooks in the Model Registry notify external systems of events but do not themselves perform stage transitions. They can trigger a job or HTTP endpoint, but the actual transition must still be executed by code calling the MLflow Client. A webhook alone will not move the model to Production, so this does not meet the requirement.

Set the model version's stage to Production directly in the MLmodel file before logging the model.

Why it fails: The MLmodel file does not contain stage information; stages are registry-level metadata assigned after model registration. Editing the MLmodel file cannot set a stage. Stage transitions must be done through the Model Registry API after the model version exists, so this approach is invalid.

Use the Databricks REST API endpoint /api/2.0/mlflow/model-versions/update to change the stage after validation.

Why it fails: The /api/2.0/mlflow/model-versions/update endpoint is used to update the model version description or tags, not its stage. Stage transitions require the transition-stage endpoint or the MLflow Client method. Using the update endpoint would not change the stage, so this is incorrect.

Store the training dataset in DBFS and log its path as a tag, but do not log the model itself to avoid large artifacts.

Why it fails: While logging the dataset path as a tag is useful, not logging the model means you cannot register it to the Model Registry. The requirement includes registering the best model, so the model artifact must be logged. Avoiding model logging defeats the purpose of the Model Registry integration.

Use mlflow.autolog() to automatically log all metrics and parameters, and rely on it to register the model automatically.

Why it fails: mlflow.autolog() logs metrics and parameters for supported libraries, but it does not automatically register the model to the Model Registry unless explicitly configured. Even then, it may not capture the Git commit hash or custom signature. Relying solely on autolog does not guarantee the required registration and reproducibility details.

Create a separate MLflow experiment for each run to isolate metrics and avoid confusion when comparing runs.

Why it fails: Creating a separate experiment per run fragments the tracking data and makes comparison difficult. MLflow experiments are meant to group related runs, so all runs for a model should typically be in the same experiment. This practice hinders reproducibility and model selection rather than helping.

Configure the online store to automatically generate features on the fly from raw data using SQL expressions defined in the model.

Why it fails: Feature Store does not generate features on the fly from raw data at inference; it serves precomputed feature values from the online store. Defining SQL expressions in the model is not a supported mechanism. Features must be materialized and published to the online store for low-latency retrieval.

Use a separate feature engineering notebook for online inference that reimplements the same transformations independently.

Why it fails: Reimplementing transformations in a separate notebook risks divergence between training and inference logic. Even minor differences in code can cause training-serving skew. Feature Store is designed to avoid this by centralizing feature definitions and serving them consistently.

Manually copy the feature values into a CSV file and include it as an artifact with the model.

Why it fails: Copying features to CSV does not ensure consistent computation at inference; it freezes values and cannot handle new entities. It also bypasses Feature Store's online serving and point-in-time correctness. This approach is not a recommended practice for maintaining training-serving consistency.

Enable autologging by calling mlflow.autolog() before starting the run.

Why it fails: Autologging automatically captures parameters, metrics, and models for supported libraries, but it does not configure the tracking URI. If the URI is not set to 'databricks', even autologged data would be stored locally. Autologging is irrelevant to the core issue of connecting to the Databricks-hosted tracking server.

Install the mlflow-skinny package instead of the full mlflow library.

Why it fails: mlflow-skinny is a lightweight version that excludes certain dependencies like SQL storage and the UI, but it still requires a tracking URI to log to a remote server. Installing it does not automatically configure the Databricks tracking server; you must explicitly set the URI. Using mlflow-skinny alone would not resolve the missing logs.

Create an MLflow experiment in the workspace and set it as the active experiment using mlflow.set_experiment().

Why it fails: Setting an active experiment determines where runs are grouped, but it does not define the tracking server. Without the correct tracking URI, runs are still logged to the local file store, not to the Databricks experiment. Thus, merely setting the experiment does not ensure logs appear in the UI.

Export the feature table to a CSV file and include it in the model artifact for the serving endpoint to read.

Why it fails: Exporting to CSV and including it in the model artifact does not provide real-time feature lookup or transformation. It also introduces staleness and scalability issues. Model Serving expects feature metadata to be logged with the model, not a static CSV file.

Log the feature table with the model using mlflow.log_artifact() and manually apply transformations in the serving endpoint.

Why it fails: Manually applying transformations in the serving endpoint is error-prone and does not guarantee consistency. Logging the feature table as an artifact does not automatically integrate with Model Serving for feature lookup. This approach lacks the automated feature retrieval and transformation that Feature Store provides.

Create a separate serving endpoint for each feature and use API calls to combine them at inference time.

Why it fails: Creating separate endpoints for each feature adds latency and complexity, and does not ensure transformation consistency. Model Serving is designed to handle feature lookup internally when the model is logged with feature metadata. This approach is inefficient and not recommended.

mlflow.log_dict({"f1_score": 0.87, "summary": "good"}, "eval.json")

Why it fails: mlflow.log_dict logs a dictionary as an artifact file, not as a metric or tag. While the file can be viewed, its contents are not directly surfaced in the MLflow UI's metrics and tags columns, making cross-run comparison cumbersome. This approach does not meet the goal of easily comparing the metric across runs in the experiment UI.

mlflow.log_metrics({"f1_score": 0.87})

Why it fails: mlflow.log_metrics only accepts a dictionary of numeric values, so attempting to pass a string summary would raise an error. This function is designed for batch-logging numeric metrics, not for combining numeric and string data. It cannot fulfill the requirement of logging both a metric and a text summary in one call, making it unsuitable for this scenario.

mlflow.log_params({"f1_score": 0.87, "summary": "good"})

Why it fails: mlflow.log_params logs parameters, which are typically configuration values, not evaluation metrics. While it can store strings and numbers, parameters are not intended for tracking model performance. Using it would misrepresent the metric as a parameter, making it harder to compare runs in the MLflow UI, which separates parameters and metrics.