ML Model Lifecycle And Operations practice questions

Configure pipeline run settings with 'continue_on_step_failure' set to True for the logging step.

Setting continue_on_step_failure to True on the logging step lets the pipeline proceed past its failure, while downstream steps still depend on the training step's successful completion. This satisfies both constraints: gating on training success and tolerating non-critical logging failure.

Set 'autoscale_enabled' to True and 'target_utilization' for GPU metrics.

Autoscaling on GPU utilisation lets AKS add replicas when graphics-processor demand rises, absorbing traffic spikes. Enabling autoscale_enabled with a GPU-based target_utilization satisfies the stated constraint, because CPU-driven scaling would not reflect the actual accelerator bottleneck governing inference throughput.

Managed Online Endpoints with multiple deployments

Managed Online Endpoints allow splitting traffic across multiple deployments in different regions.

Update the model asset by adding a 'Production' tag via the SDK or CLI.

Tagging the existing model asset with 'Production' via the SDK or CLI promotes it without creating a new version, satisfying the stem's constraint of no new asset version. Azure Machine Learning Registry tags are mutable metadata, so the same asset version moves between stages while its version number stays unchanged.

Within the 'experiment.start_logging()' call using the 'tags' parameter.

Passing the commit hash through the tags parameter on experiment.start_logging() attaches it as run metadata in Azure Machine Learning. This satisfies the reproducibility requirement by binding each tracked run to its exact source revision, so any run can be traced back to the code that produced it.

The 'get-logs' command for the online deployment.

The get-logs command retrieves container logs for a managed online deployment, capturing startup output where missing environment variables surface as errors. This directly satisfies the stem's need to identify why the container fails to start.

Azure Policy definitions.

Azure Policy definitions enforce governance constraints across Azure resources by evaluating deployments against rules and denying non-compliant ones. This directly satisfies the requirement that models must be registered before deployment, since policy can block deployment of unregistered models at the control plane.

Configure an Event Grid subscription to trigger a pipeline.

An Event Grid subscription on the drift monitor emits an event when drift is detected, triggering an Azure Machine Learning pipeline that retrains the model. This closes the loop automatically, satisfying the requirement to automate retraining after the monitor fires.

'process_count' or 'node_count' in the configuration.

PyTorch's distributed configuration requires an explicit process or node count so the launcher knows how many ranks to spawn across the cluster. Without it, multi-node training cannot partition work, making the setting mandatory for the distributed job.

'min_nodes' set to 0.

Setting 'min_nodes' to 0 lets the cluster scale down to zero nodes once no jobs are running, eliminating idle compute charges. Microsoft Entra ID handles authentication separately; the constraint here is automatic shutdown, which this autoscaling floor satisfies by deallocating all nodes after the queue empties.

'idle_seconds_before_scaledown'.

Setting idle_seconds_before_scaledown defines the inactivity period before the cluster scales down to zero nodes, directly satisfying the requirement to shut down automatically when no jobs are running. Without it, nodes remain provisioned and continue billing despite sitting idle.

Use an 'OutputDataBinding' to a registered Datastore.

Binding step output to a registered datastore writes artefacts to durable Azure storage rather than transient compute, so the scoring step can read training output after the originating node is released. This satisfies the cross-step persistence requirement that PipelineData alone does not guarantee.

The 'Experiments' tab, then 'Compare' button.

The 'Compare' button on the 'Experiments' tab directly satisfies the requirement to contrast training time across two runs. It renders a side-by-side metric chart, plotting the duration metric from each run's logged data on a shared axis, enabling accurate visual comparison without exporting logs or querying the run history programmatically.

Run 'az ml job get-logs' with the job ID.

The az ml job get-logs command retrieves the log output, including stdout and stderr, for a specified job or step using its job ID. This directly surfaces the failed step's stdout, satisfying the stem's debugging requirement.

Use a Managed Identity for the compute cluster.

Wait, Managed Identity is also a correct way to access services, but using a Datastore to manage the reference is the standard ML flow.

Use Key Vault to store secrets and a Datastore for the connection.

This is the secure standard pattern.

Via the Azure front-end load balancer/gateway service.

Managed online endpoints handle TLS termination at the Azure front-end gateway layer, so certificates are uploaded and bound there rather than inside the scoring container. This satisfies the SSL termination constraint, with the endpoint's TLS settings referencing the certificate for inbound HTTPS traffic.

'MedianStoppingPolicy'.

MedianStoppingPolicy compares each run's primary metric against the median of all runs at the same interval, terminating runs performing below that median. This halts poor performers early, satisfying the stated requirement to reduce compute costs during HyperDrive tuning.

'BanditPolicy'.

BanditPolicy uses a slack factor and evaluation interval, stopping runs whose primary metric falls outside the best-performing run's slack band. It aggressively terminates underperformers early, directly satisfying the requirement to cut compute costs during HyperDrive hyperparameter tuning.

The 'readiness_probe' timeout is too short.

Increasing this allows more time for the image to pull and load.

A target dataset to compare against.

A target dataset supplies the recent production data whose feature distribution is compared against the baseline. Without it, the monitor has nothing to evaluate for drift, so it is a required input alongside the baseline dataset.

A baseline dataset.

A baseline dataset provides the reference distribution captured from training or an earlier period. Drift is computed by comparing the target dataset against this baseline, making it a required input for configuring the Data Drift Monitor.

Azure DevOps pipelines.

Azure DevOps pipelines satisfy the post-deployment integration test requirement by triggering automated test stages immediately after the model deployment task completes. Native pipeline stages, gates and task chaining let you invoke the deployed endpoint, validate responses and fail the release automatically, which a standalone notebook or manual check cannot enforce.

Tags.

Tags are key-value metadata pairs attached to a run, stored alongside metrics and the run's properties. They satisfy the stem's requirement by capturing custom annotations such as model type or training dataset, enabling filtering and comparison of runs in Azure ML experiments without altering the run's logged outputs.

Metrics.

Metrics are recorded in the run object as logged values, capturing accuracy, loss and custom measurements produced during training. This satisfies the stem's requirement for information held within a run, since metrics attach directly to that run and are retrievable through the Azure ML SDK and studio.

Parameters.

Parameters are stored in the Run object as the resolved input values logged when the run was submitted, capturing the configuration the experiment actually executed with. This satisfies the stem's requirement for information contained in a Run object, alongside metrics and tags, distinguishing them from artefacts, which are separate outputs.