AI-300 ML Model Lifecycle And Operations — 20 Questions

Question 1

You have an Azure ML environment that needs to access a private Azure SQL database. How should you configure the connection?

Accepted Answer

Use a Managed Identity for the compute cluster.. You should use a 'Datastore' that stores the credentials in Azure Key Vault to securely access the SQL database from the compute environment.

Answer

Enable public access for the SQL database.

Answer

Hardcode the connection string in the script.

Question 2

You need to implement a retraining trigger based on performance degradation. Which TWO metrics should you monitor to decide when to retrain?

Accepted Answer

Model prediction precision. Accuracy and precision are key performance indicators that signify model decay.

Answer

CPU usage of the inference cluster

Answer

Network latency

Answer

Number of active users

Question 3

You are configuring a CI/CD pipeline for model deployment. Which THREE actions must you perform to ensure model traceability?

Accepted Answer

Register the model in the Azure ML Model Registry with versioning. Model registry registration, metadata tagging, and linking to the source code commit are standard MLOps practices for traceability.

Answer

Use a shared local folder for model artifacts

Answer

Delete all training logs after deployment

Question 4

You are building an Azure Machine Learning pipeline. You need to ensure that the pipeline components are reusable and versioned independently. What should you use?

Accepted Answer

Azure ML Components. Azure ML components allow for independent versioning and reusability across different pipelines.

Answer

Azure ML Datasets

Answer

Azure ML Pipelines Steps

Answer

Azure Container Registry images

Question 5

You are orchestrating a multi-step ML pipeline in Azure Machine Learning. You need to ensure that a downstream step only executes if the upstream model training step finishes successfully, while allowing the pipeline to continue even if a non-critical logging step fails. Which configuration should you use?

Accepted Answer

Configure pipeline run settings with 'continue_on_step_failure' set to True for the logging step.. You should set the 'continue_on_step_failure' property to True for non-critical steps and ensure dependencies are defined via 'StepRun' output objects.

Answer

Configure a 'WaitStep' to monitor the training job status.

Answer

Set the 'PipelineParameter' to execute only on success.

Answer

Use an 'Estimator' class with 'allow_reuse' set to False.

Question 6

You are deploying a model to an Azure Kubernetes Service (AKS) cluster. You need to ensure that the deployment handles traffic spikes by scaling based on GPU usage. Which setting must be enabled in the inference configuration?

Accepted Answer

Set 'autoscale_enabled' to True and 'target_utilization' for GPU metrics.. To scale based on GPU usage, you must define an 'autoscale' configuration within the 'InferenceConfig' that references 'target_utilization' for custom metrics like GPU.

Answer

Use 'enable_gpu' in the 'DeploymentConfig'.

Answer

Configure 'max_concurrent_requests'.

Answer

Enable 'cluster_purpose' as 'FastProd'.

Question 7

You need to monitor the data drift of a model deployed in Azure ML. What is the first step you must take?

Accepted Answer

Create a Data Drift Monitor. You must create a Data Drift Monitor object linked to your target dataset and baseline dataset to begin tracking.

Answer

Configure Application Insights

Answer

Run a manual evaluation script

Answer

Enable Azure Monitor logs

Question 8

You need to ensure that your Azure ML models are deployed with high availability across multiple regions. Which deployment strategy should you configure?

Accepted Answer

Managed Online Endpoints with multiple deployments. Managed Online Endpoints support multi-region traffic distribution by using managed deployments behind a single endpoint.

Answer

Kubernetes Online Endpoints

Answer

Batch Endpoints

Answer

Azure Container Instances (ACI)

Question 9

You are managing model versioning in Azure Machine Learning Registry. You need to promote a model from 'Staging' to 'Production' without creating a new asset version. Which command or action should you perform?

Accepted Answer

Update the model asset by adding a 'Production' tag via the SDK or CLI.. You should use the 'update' command on the existing model version asset to update its tags or properties to reflect the production status.

Answer

Change the 'run_id' in the model metadata.

Answer

Delete the old version and re-register as 'Production'.

Answer

Run 'az ml model create' with a new version number.

Question 10

Your team needs to share a model across different workspaces. What is the most efficient way to achieve this in Azure Machine Learning?

Accepted Answer

Use a shared Azure Machine Learning Registry.. Azure Machine Learning Registries allow for the sharing of model assets, environments, and components across multiple workspaces.

Answer

Export the model as a pickle file and upload to each workspace.

Answer

Duplicate the workspace storage account.

Answer

Re-train the model in every workspace.

Question 11

A team uses Azure Machine Learning to track experiments. You need to ensure that every run is associated with a specific git commit hash to ensure reproducibility. Where should this be configured?

Accepted Answer

Within the 'experiment.start_logging()' call using the 'tags' parameter.. The 'run_configuration' or the 'Environment' object can be used to inject metadata, but standard practice is to use the 'tags' or 'properties' dictionary during the 'start_logging' or 'init' call.

Answer

In the 'workspace.json' file.

Answer

By modifying the compute cluster configuration.

Answer

Inside the 'conda_dependencies.yml' file.

Question 12

You are implementing a retraining trigger for a demand forecasting model. You want to trigger a pipeline execution only when the drift metric for the 'Price' feature exceeds a predefined threshold. Which service should you integrate with Azure Machine Learning?

Accepted Answer

Azure Event Grid and Logic Apps.. Azure Machine Learning Data Drift Monitors can be configured to emit events to Azure Event Grid, which can then trigger an Azure Logic App or Azure Function to start the pipeline.

Answer

Azure Batch schedules.

Answer

Azure Data Factory triggers.

Answer

Azure Monitor alerts on logs.

Question 13

You are deploying a model via a Managed Online Endpoint. You want to implement a Canary deployment where 10% of traffic is sent to a new model version. How do you configure this?

Accepted Answer

Configure the 'traffic' parameter in the 'az ml online-endpoint update' command.. You use the 'update' command on the endpoint to set the 'traffic' parameter, distributing weights across deployment names.

Answer

Modify the 'inference_config.json' file to include weights.

Answer

Create two endpoints and use an Azure Load Balancer.

Answer

Set the 'replica_count' to 10% of total capacity.

Question 14

You are troubleshooting a model deployment failure where the container fails to start due to missing environment variables. Where do you find the logs to identify the cause?

Accepted Answer

The 'get-logs' command for the online deployment.. The 'deployment logs' are accessible via the Azure ML Studio UI or the CLI command 'az ml online-deployment get-logs', which pulls from the container runtime.

Answer

The Key Vault access logs.

Answer

The pipeline run history.

Answer

The workspace diagnostic logs in Azure Monitor.

Question 15

You want to enforce a policy that all models must be registered before being deployed. Which feature should you use to implement this constraint?

Accepted Answer

Azure Policy definitions.. Azure Policy for Azure Machine Learning can be used to restrict actions, such as preventing deployment of models that do not have a specific 'registered' status or tag.

Answer

Virtual Network peering.

Answer

Role-Based Access Control (RBAC).

Answer

Compute Instance quotas.

Question 16

A model is exhibiting data drift. You have created a drift monitor. What is the next step to automate the retraining?

Accepted Answer

Configure an Event Grid subscription to trigger a pipeline.. After creating a monitor, you set up an 'Event Grid' subscription to notify an Azure Function or Logic App to trigger the pipeline.

Answer

Update the dataset version.

Answer

Manually restart the compute cluster.

Answer

Create a 'RetrainingTrigger' in the model registry.

Question 17

You are configuring a batch scoring job. You need to ensure that the job processes data in parallel to reduce completion time. What property should you adjust in the 'ParallelRunConfig'?

Accepted Answer

Adjust 'process_count_per_node' and 'node_count'.. The 'node_count' and 'process_count_per_node' are the key parameters in 'ParallelRunConfig' that dictate the degree of parallelism.

Answer

Change the 'output_action' to 'append_row'.

Answer

Enable 'distributed_training'.

Answer

Set 'min_nodes' to 1.

Question 18

You are using MLflow to track experiments in Azure Machine Learning. You need to log a custom metric that is calculated every 100 iterations. Which MLflow function should you use?

Accepted Answer

'mlflow.log_metric()'.. The 'mlflow.log_metric' function is used to log key-value pairs of metrics, which can be called within the training loop.

Answer

'mlflow.set_tag()'.

Answer

'mlflow.log_artifact()'.

Answer

'mlflow.log_param()'.

Question 19

You are defining an Azure Machine Learning environment for a training job. The environment requires a specific set of Python libraries. What is the best practice for defining these dependencies?

Accepted Answer

Define dependencies in a 'conda.yaml' file.. Using a 'conda.yaml' file is the best practice for managing reproducible Python environments in Azure ML.

Answer

Hardcode pip install commands in the training script.

Answer

Install libraries via a startup script in the compute cluster.

Answer

Pre-install them on the virtual machine image.

Question 20

You are moving a model from a local environment to Azure Machine Learning. Which file is required to define the entry script for the model inference?

Accepted Answer

'score.py'.. The entry script (often named 'score.py') is required to define 'init()' and 'run()' functions for the deployment.

Answer

'inference_config.json'.

Answer

'model.pkl'.

Answer

'environment.yml'.