AI-300 Hard Questions — 25 Difficult Questions

Question 1

You need to implement a retraining trigger based on performance degradation. Which TWO metrics should you monitor to decide when to retrain?

Accepted Answer

Model prediction precision. Accuracy and precision are key performance indicators that signify model decay.

Answer

CPU usage of the inference cluster

Answer

Network latency

Answer

Number of active users

Question 2

You are managing model versioning in Azure Machine Learning Registry. You need to promote a model from 'Staging' to 'Production' without creating a new asset version. Which command or action should you perform?

Accepted Answer

Update the model asset by adding a 'Production' tag via the SDK or CLI.. You should use the 'update' command on the existing model version asset to update its tags or properties to reflect the production status.

Answer

Change the 'run_id' in the model metadata.

Answer

Delete the old version and re-register as 'Production'.

Answer

Run 'az ml model create' with a new version number.

Question 3

You are configuring a 'Managed Online Endpoint' for a very large model (10GB+). The deployment is failing during the 'pulling image' phase. What is the most likely cause?

Accepted Answer

The 'readiness_probe' timeout is too short.. Large images or model artifacts often cause timeout issues during container startup. You might need to increase the 'readiness_probe' timeout in the deployment configuration.

Answer

The instance count is too high.

Answer

The ACR is private.

Answer

The compute is too small.

Question 4

You are troubleshooting a model deployment failure where the container fails to start due to missing environment variables. Where do you find the logs to identify the cause?

Accepted Answer

The 'get-logs' command for the online deployment.. The 'deployment logs' are accessible via the Azure ML Studio UI or the CLI command 'az ml online-deployment get-logs', which pulls from the container runtime.

Answer

The Key Vault access logs.

Answer

The pipeline run history.

Answer

The workspace diagnostic logs in Azure Monitor.

Question 5

You are running a distributed training job using the 'PyTorch' framework on Azure Machine Learning. You need to configure the 'DistributionConfiguration'. Which setting is mandatory for multi-node training?

Accepted Answer

'process_count' or 'node_count' in the configuration.. When using 'PyTorch' distribution, you must specify the 'process_count' or 'node_count' to correctly distribute the workload across the compute cluster.

Answer

'framework' set to 'TensorFlow'.

Answer

'shm_size' set to 1GB.

Answer

'enable_gpu' set to False.

Question 6

Which THREE metrics can be logged during training to track performance in Azure ML?

Accepted Answer

Model accuracy.. Accuracy, loss, and custom metrics are standard loggable items.

Answer

Compute instance IP address.

Answer

Workspace subscription ID.

Question 7

To optimize the cost of your AI infrastructure, which THREE actions should you consider?

Accepted Answer

Deleting unused model deployments.. Choosing the right SKU, cleaning up unused deployments, and using reserved capacity are standard cost-optimization steps.

Answer

Deploying all models in every available region.

Answer

Always selecting the 'Premium' SKU.

Question 8

You are troubleshooting a failed agent deployment. Which TWO areas should you inspect first?

Accepted Answer

The endpoint availability status.. Deployment logs and endpoint connectivity are the primary sources for identifying deployment failures.

Answer

The Azure status page.

Answer

The user profile settings.

Answer

The billing usage report.

Question 9

A pipeline step fails because it cannot find a file in the datastore. What is the most likely cause?

Accepted Answer

Incorrect path definition in the component. The mount path or the relative path defined in the pipeline step component configuration is likely incorrect.

Answer

The workspace is too large

Answer

The compute cluster is off

Answer

The subscription is expired

Question 10

You are deploying a large model. During the deployment, you encounter a 'Resource Not Available' error. What is the most likely cause?

Accepted Answer

The region does not have capacity for the requested model SKU.. This error usually indicates that the region lacks sufficient capacity for the specific SKU requested.

Answer

The API key is invalid.

Answer

The project name is too long.

Answer

The subscription is not registered for the service.

Question 11

You are deploying a custom model in a container. To ensure the model infrastructure is highly available, what is the best practice?

Accepted Answer

Deploy to 'Azure Kubernetes Service' with 'Availability Zones' enabled.. Deploying across multiple Availability Zones ensures that the infrastructure remains operational even if a specific datacenter fails.

Answer

Disable the 'Load Balancer' to save costs.

Answer

Use a 'Standard' tier App Service plan.

Answer

Deploy the container on a single large VM.

Question 12

Which THREE metrics are critical for monitoring the health of a GenAI deployment?

Accepted Answer

Request Latency.. Latency, throughput (tokens per second), and error rates are the standard health metrics for AI deployments.

Answer

User email addresses.

Answer

CPU Usage of the host node.

Question 13

You are automating the deployment of your AI project. What is the recommended tool to manage infrastructure-as-code (IaC) for Azure AI Foundry?

Accepted Answer

Azure Bicep. Bicep is the native, recommended IaC tool for Azure, providing deep integration for AI Foundry resources.

Answer

Azure DevOps pipelines

Answer

Azure CLI alone

Answer

Python script with Azure SDK

Question 14

You are using MLflow to track experiments in Azure Machine Learning. You need to log a custom metric that is calculated every 100 iterations. Which MLflow function should you use?

Accepted Answer

'mlflow.log_metric()'.. The 'mlflow.log_metric' function is used to log key-value pairs of metrics, which can be called within the training loop.

Answer

'mlflow.set_tag()'.

Answer

'mlflow.log_artifact()'.

Answer

'mlflow.log_param()'.

Question 15

You are experiencing throttling on your AI endpoint. Which TWO steps should you take?

Accepted Answer

Check current usage against quota limits.. To resolve throttling, you need to either optimize usage or increase the available quota/capacity.

Answer

Disable all security features.

Answer

Change the model version to one that is faster but less smart.

Answer

Delete all existing deployments.

Question 16

You want to measure 'Relevance' in a RAG application. The relevance evaluator detects how well the response answers the user query. If the model provides a factually correct answer that does not address the prompt, which metric will capture this failure?

Accepted Answer

Relevance. Relevance specifically measures if the response directly addresses the user's intent.

Answer

Fluency

Answer

Coherence

Answer

Groundedness

Question 17

You are configuring a connection to a vector store. What is the best way to handle the secret credential?

Accepted Answer

Store in Azure Key Vault and reference in the connection.. Key Vault is the centralized store for all application secrets in Azure, which should be linked to the project connection.

Answer

Hardcode the key in the flow configuration.

Answer

Pass it as a parameter in the code.

Answer

Store it in an environment variable.

Question 18

You are deploying a model that requires a high-memory compute for inference. You are using a 'Managed Online Endpoint'. Where do you specify the instance type for this deployment?

Accepted Answer

In the 'Deployment' YAML file under 'instance_type'.. The instance type is defined in the 'Deployment' configuration object, typically in the 'instance_type' field.

Answer

In the 'Endpoint' YAML file.

Answer

In the 'Environment' definition.

Answer

In the 'Workspace' settings.

Question 19

You are building a RAG application and notice that the model sometimes hallucinates information not present in the retrieved documents. Which evaluation metric should you prioritize to mitigate this?

Accepted Answer

Groundedness. Groundedness specifically assesses whether the generated response is derived from the retrieved documents.

Answer

Relevance

Answer

Performance

Answer

Fluency

Question 20

You are optimizing the cost of your GenAI infrastructure. You have several agents running in Prompt Flow that are idle for large portions of the day. Which runtime configuration should be modified?

Accepted Answer

Set the 'automatic_shutdown' property on the Prompt Flow runtime compute instance.. Setting the 'idle_time_before_shutdown' or similar inactivity policies in the Prompt Flow runtime ensures compute resources are deallocated when not in use.

Answer

Change the VM SKU to a burstable instance type.

Answer

Implement a script to delete the runtime when the flow is finished.

Answer

Set the concurrency limit to zero.