58 questions · ML Model Lifecycle And Operations · All types, answers revealed
You need to monitor the data drift of a model deployed in Azure ML. What is the first step you must take?
Data Drift Monitor is the specific feature for tracking distribution changes.
Why this answer
You must create a Data Drift Monitor object linked to your target dataset and baseline dataset to begin tracking.
Which THREE metrics can be logged during training to track performance in Azure ML?
Standard metric.
Why this answer
Accuracy, loss, and custom metrics are standard loggable items.
You are configuring a CI/CD pipeline for model deployment. Which THREE actions must you perform to ensure model traceability?
Versioning is essential for tracking changes over time.
Why this answer
Model registry registration, metadata tagging, and linking to the source code commit are standard MLOps practices for traceability.
You are deploying a model via a Managed Online Endpoint. You want to implement a Canary deployment where 10% of traffic is sent to a new model version. How do you configure this?
This is the correct way to shift traffic weights.
Why this answer
You use the 'update' command on the endpoint to set the 'traffic' parameter, distributing weights across deployment names.
Which TWO languages are natively supported for the Azure ML SDK?
Supported language.
Why this answer
Python and R are the primary languages with official SDK support.
Which THREE items are captured in the experiment lineage in Azure Machine Learning?
Code lineage.
Why this answer
Datasets, code, and environments are the pillars of reproducibility.
You want to perform hyperparameter tuning using the 'HyperDrive' service. You have a requirement to stop poor-performing runs early to save compute costs. Which policy should you use?
This is also valid, but Bandit is the most common answer for this scenario.
Why this answer
The 'BanditPolicy' is the standard early-termination policy that stops runs based on a slack factor/amount compared to the best-performing run.
Which TWO resources are created inside an Azure Machine Learning workspace?
ML resource.
Why this answer
Compute and Datastores are resources managed within the workspace.
You are running a distributed training job using the 'PyTorch' framework on Azure Machine Learning. You need to configure the 'DistributionConfiguration'. Which setting is mandatory for multi-node training?
This defines the parallel distribution parameters.
Why this answer
When using 'PyTorch' distribution, you must specify the 'process_count' or 'node_count' to correctly distribute the workload across the compute cluster.
Your team needs to share a model across different workspaces. What is the most efficient way to achieve this in Azure Machine Learning?
Registries provide cross-workspace asset management.
Why this answer
Azure Machine Learning Registries allow for the sharing of model assets, environments, and components across multiple workspaces.
Which TWO types of compute can be used for training in Azure Machine Learning?
Used for large, distributed training.
Why this answer
Compute Clusters and Compute Instances are the standard training targets.
You are debugging an Azure ML pipeline. You want to see the stdout of a specific step that failed. How do you access this?
This command retrieves the logs for the specified job run.
Why this answer
You can access the logs for each step by clicking on the 'Outputs + logs' tab of that specific step run in the Azure ML Studio interface.
What is the primary purpose of a 'Labeling Project' in Azure Machine Learning?
This is the core function of the tool.
Why this answer
Labeling projects are used to manage the process of annotating data (images, text) to create datasets for supervised learning.
Which Azure Machine Learning resource provides a pre-configured environment for development?
Designed for developer workstation experience.
Why this answer
A 'Compute Instance' is a managed, cloud-based development environment that comes pre-installed with the Azure ML SDK, Jupyter, and other tools.
What is the 'Workspace' in Azure Machine Learning?
This is the correct definition.
Why this answer
The Workspace is the top-level resource for Azure Machine Learning, providing a centralized place to manage all artifacts, computes, and jobs.
You are preparing a model for deployment. Which THREE items should you include in the model package?
The actual model.
Why this answer
Model file, environment requirements, and inference code.
What is the benefit of using 'Azure Machine Learning Datasets' (or Data Assets) over raw storage paths?
This is the primary benefit over raw URIs.
Why this answer
Data Assets allow for versioning, lineage tracking, and simplified usage across different compute targets.
You are managing model lifecycle security. Which THREE actions are recommended to secure your ML models?
Secures credentials.
Why this answer
Use RBAC, Key Vault for secrets, and private endpoints for network security.
Which service allows you to track ML models and their associated artifacts?
This is the correct component.
Why this answer
The 'Azure Machine Learning Model Registry' is specifically designed to store, version, and manage models.
You have an automated deployment pipeline. You want to run an integration test on the model after deployment. Which tool is best suited for this?
Standard CI/CD orchestration.
Why this answer
'Azure DevOps' (or GitHub Actions) is the industry-standard tool for orchestrating post-deployment testing as part of a CI/CD pipeline.
What is the purpose of a 'Datastore' in Azure Machine Learning?
This is the definition of a Datastore.
Why this answer
A Datastore is an abstraction layer over Azure storage services (Blob, Files, ADLS) used to manage connections and data access in ML tasks.
Which TWO actions can you perform in the Azure Machine Learning studio?
Core function.
Why this answer
Experiment management and data asset management are core UI functions.
You are troubleshooting a deployment. Which THREE logs are most helpful to check?
Shows infrastructure issues.
Why this answer
Application logs, system logs, and deployment status logs are crucial.
You are using MLflow to track experiments in Azure Machine Learning. You need to log a custom metric that is calculated every 100 iterations. Which MLflow function should you use?
This is the correct function to log numeric metrics.
Why this answer
The 'mlflow.log_metric' function is used to log key-value pairs of metrics, which can be called within the training loop.
You are orchestrating a multi-step ML pipeline in Azure Machine Learning. You need to ensure that a downstream step only executes if the upstream model training step finishes successfully, while allowing the pipeline to continue even if a non-critical logging step fails. Which configuration should you use?
This allows the pipeline to proceed if the specific step fails.
Why this answer
You should set the 'continue_on_step_failure' property to True for non-critical steps and ensure dependencies are defined via 'StepRun' output objects.
A team uses Azure Machine Learning to track experiments. You need to ensure that every run is associated with a specific git commit hash to ensure reproducibility. Where should this be configured?
This is the standard way to attach metadata to a run.
Why this answer
The 'run_configuration' or the 'Environment' object can be used to inject metadata, but standard practice is to use the 'tags' or 'properties' dictionary during the 'start_logging' or 'init' call.
You need to ensure that training data is encrypted at rest in the Blob Storage linked to your Azure Machine Learning workspace. How do you ensure this?
This is the standard platform security configuration.
Why this answer
Azure Storage accounts support Storage Service Encryption (SSE) by default. You can also use Customer-Managed Keys (CMK) for additional control.
You are implementing a retraining trigger for a demand forecasting model. You want to trigger a pipeline execution only when the drift metric for the 'Price' feature exceeds a predefined threshold. Which service should you integrate with Azure Machine Learning?
This is the native integration pattern for drift-based automation.
Why this answer
Azure Machine Learning Data Drift Monitors can be configured to emit events to Azure Event Grid, which can then trigger an Azure Logic App or Azure Function to start the pipeline.
You are configuring a 'Managed Online Endpoint' for a very large model (10GB+). The deployment is failing during the 'pulling image' phase. What is the most likely cause?
Increasing this allows more time for the image to pull and load.
Why this answer
Large images or model artifacts often cause timeout issues during container startup. You might need to increase the 'readiness_probe' timeout in the deployment configuration.
You are creating a 'Pipeline' and want to share a dataset across multiple steps. What is the most efficient way to access this data?
Mounting is efficient and avoids redundant downloads.
Why this answer
You should define the dataset as an 'Input' to the pipeline or use a 'Dataset' object that can be mounted by the compute for each step.
You are reviewing the 'Run History' in Azure Machine Learning. You want to compare the training time of two different experiments. Which UI feature should you use?
The built-in compare feature is designed for this.
Why this answer
The 'Charts' or 'Metrics' comparison view in the Azure ML Studio allows you to visualize and compare metrics across multiple runs.
You are deploying a model to an Azure Kubernetes Service (AKS) cluster. You need to ensure that the deployment handles traffic spikes by scaling based on GPU usage. Which setting must be enabled in the inference configuration?
This directly targets GPU-based scaling logic.
Why this answer
To scale based on GPU usage, you must define an 'autoscale' configuration within the 'InferenceConfig' that references 'target_utilization' for custom metrics like GPU.
Which THREE settings are part of the 'InferenceConfig' object in Azure ML?
Where the code lives.
Why this answer
Entry script, environment, and property settings are part of the inference config.
You are configuring a 'Managed Online Endpoint' for production. You want to ensure high availability. What should you configure?
Multiple instances allow for failover.
Why this answer
To ensure high availability, you should configure the 'instance_count' to be greater than 1, spanning multiple availability zones if supported.
You need to ensure that your model inference code has access to the latest secret keys without hardcoding them. What is the recommended integration?
This is the secure pattern for runtime secrets.
Why this answer
You should integrate your scoring script with Azure Key Vault using the workspace 'get_default_keyvault()' function to retrieve secrets at runtime.
You have an Azure Machine Learning pipeline that uses 'PipelineData' to pass information between steps. You want to share data between a training step and a scoring step. What is the recommended way to persist this data?
This ensures the data is persisted and accessible.
Why this answer
'PipelineData' allows intermediate data passing, but 'OutputDataBindings' are preferred for persisting artifacts that need to be accessed later, such as model files.
You are optimizing your Azure ML pipeline performance. Which THREE steps should you take to reduce execution time?
Saves I/O time.
Why this answer
Caching (reuse), parallelizing steps, and using efficient data access methods are key.
You are troubleshooting a model deployment failure where the container fails to start due to missing environment variables. Where do you find the logs to identify the cause?
This retrieves the specific container startup error logs.
Why this answer
The 'deployment logs' are accessible via the Azure ML Studio UI or the CLI command 'az ml online-deployment get-logs', which pulls from the container runtime.
You are implementing a custom container for model training. You need to push the image to the 'Azure Container Registry' (ACR) linked to your workspace. What is the correct command?
Standard Docker workflow for custom images.
Why this answer
The standard approach is 'docker build' followed by 'docker push' to the ACR URL, ensuring authentication via 'az acr login'.
You are building an Azure Machine Learning pipeline. You need to ensure that the pipeline components are reusable and versioned independently. What should you use?
Components are the fundamental units for building reusable and versioned ML pipelines.
Why this answer
Azure ML components allow for independent versioning and reusability across different pipelines.
You are moving a model from a local environment to Azure Machine Learning. Which file is required to define the entry script for the model inference?
This contains the inference logic.
Why this answer
The entry script (often named 'score.py') is required to define 'init()' and 'run()' functions for the deployment.
You are monitoring model drift. Which TWO features are required to configure a Data Drift Monitor?
Required for drift calculation.
Why this answer
You need a target dataset (baseline) and a comparison dataset (current).
You want to enforce a policy that all models must be registered before being deployed. Which feature should you use to implement this constraint?
Azure Policy allows governing resources based on properties.
Why this answer
Azure Policy for Azure Machine Learning can be used to restrict actions, such as preventing deployment of models that do not have a specific 'registered' status or tag.
You are configuring an 'Azure Machine Learning Compute Cluster' for a heavy training job. You want to ensure it shuts down automatically when no jobs are running. What setting should you configure?
Wait, min_nodes set to 0 allows the cluster to shrink to zero, but 'idle_seconds_before_scaledown' is the specific property that controls the timing.
Why this answer
The 'idle_seconds_before_scaledown' property in the compute configuration determines how long the cluster waits before removing idle nodes.
You are automating model registration using the Azure ML CLI. You need to ensure the registration only happens if the model accuracy is above 0.9. How do you implement this condition?
Pipeline orchestration is required for conditional steps.
Why this answer
You must include logic in your pipeline or script to evaluate the metric (e.g., via a 'PythonScriptStep') and only call the 'az ml model create' command if the condition is met.
You have an automated model training pipeline that is failing intermittently due to compute availability. What should you configure to ensure the pipeline is more resilient?
Retries handle transient failures.
Why this answer
You should configure 'retry' settings in the 'PipelineStep' definition to handle transient compute errors.
You are deploying a model that requires a high-memory compute for inference. You are using a 'Managed Online Endpoint'. Where do you specify the instance type for this deployment?
This is where compute resources are specified for the deployment.
Why this answer
The instance type is defined in the 'Deployment' configuration object, typically in the 'instance_type' field.
You are configuring a batch scoring job. You need to ensure that the job processes data in parallel to reduce completion time. What property should you adjust in the 'ParallelRunConfig'?
These define how many instances run the script concurrently.
Why this answer
The 'node_count' and 'process_count_per_node' are the key parameters in 'ParallelRunConfig' that dictate the degree of parallelism.
You are managing model versioning in Azure Machine Learning Registry. You need to promote a model from 'Staging' to 'Production' without creating a new asset version. Which command or action should you perform?
Tags are the standard way to track status without changing the asset version.
Why this answer
You should use the 'update' command on the existing model version asset to update its tags or properties to reflect the production status.
You are using Azure Machine Learning Pipelines. Which THREE triggers can be used to start a pipeline?
Standard time-based trigger.
Why this answer
REST endpoints, schedule, and event-based triggers (e.g., dataset changes) are supported.
You are defining an Azure Machine Learning environment for a training job. The environment requires a specific set of Python libraries. What is the best practice for defining these dependencies?
This ensures consistent environment creation.
Why this answer
Using a 'conda.yaml' file is the best practice for managing reproducible Python environments in Azure ML.
Which THREE types of information are found in a Run Object in Azure ML?
Metadata labels.
Why this answer
Metrics, parameters, and tags are all core components of a run record.
You need to ensure that your Azure ML models are deployed with high availability across multiple regions. Which deployment strategy should you configure?
Managed Online Endpoints allow splitting traffic across multiple deployments in different regions.
Why this answer
Managed Online Endpoints support multi-region traffic distribution by using managed deployments behind a single endpoint.
You need to implement a retraining trigger based on performance degradation. Which TWO metrics should you monitor to decide when to retrain?
Declining precision indicates the model is failing to identify classes correctly.
Why this answer
Accuracy and precision are key performance indicators that signify model decay.
You are configuring a 'Managed Online Endpoint' with SSL termination. Where do you manage the SSL certificates?
SSL/TLS termination happens at the infrastructure boundary.
Why this answer
The SSL certificates are managed at the Azure 'Front Door' or 'Application Gateway' level, or by using the built-in certificate management if using Azure-provided domains for endpoints.
Which THREE security features are essential for a production Azure ML deployment?
For credential management.
Why this answer
Managed Identities, VNet integration, and Key Vault are essential for secure enterprise ML.
Which THREE components are required to define a 'Managed Online Deployment'?
Defines runtime libraries.
Why this answer
An environment, a model artifact, and an inference script (score.py).
A model is exhibiting data drift. You have created a drift monitor. What is the next step to automate the retraining?
This bridges the monitor alert to the execution logic.
Why this answer
After creating a monitor, you set up an 'Event Grid' subscription to notify an Azure Function or Logic App to trigger the pipeline.
Ready to test yourself?
Try a timed practice session using only ML Model Lifecycle And Operations questions.