204 questions total · 3pages · All types, answers revealed
Page 3 of 3
You are building a RAG application. Where should you store the indexed documents for optimal retrieval performance?
AI Search is the recommended service for RAG indexing.
Why this answer
Azure AI Search is the native indexer and retriever for AI Foundry RAG applications.
You are optimizing the cost of your GenAI infrastructure. You have several agents running in Prompt Flow that are idle for large portions of the day. Which runtime configuration should be modified?
Configuring the automatic shutdown property allows the compute instance to release resources when idle.
Why this answer
Setting the 'idle_time_before_shutdown' or similar inactivity policies in the Prompt Flow runtime ensures compute resources are deallocated when not in use.
You are deploying a GenAI app to production and want to track the 'Token Usage' and 'Latency' metrics per user session. Which service should you integrate with your application?
Application Insights allows custom tracking of request duration and metadata.
Why this answer
Application Insights is the standard observability tool for Azure, enabling custom event tracking for tokens and latency.
Which component in an Azure ML pipeline definition is responsible for specifying the runtime environment, including Python packages and Conda dependencies?
The Environment defines the software configuration, packages, and Docker image for the execution.
Why this answer
The Environment object defines the software stack, including dependencies and Docker images, needed for a job.
You are configuring a batch scoring job. You need to ensure that the job processes data in parallel to reduce completion time. What property should you adjust in the 'ParallelRunConfig'?
These define how many instances run the script concurrently.
Why this answer
The 'node_count' and 'process_count_per_node' are the key parameters in 'ParallelRunConfig' that dictate the degree of parallelism.
Which TWO of the following are valid ways to authenticate to an Azure AI Foundry project?
Used for programmatic access.
Why this answer
Entra ID (RBAC) and API Keys are the two primary methods for securing access to AI resources.
Which Azure AI Foundry feature allows you to evaluate your model's performance?
This is the built-in feature for model assessment.
Why this answer
Evaluation in Azure AI Foundry allows you to run metrics against your model outputs to assess quality.
A pipeline step fails because it cannot find a file in the datastore. What is the most likely cause?
Pipeline components must accurately reference paths within the mounted datastore.
Why this answer
The mount path or the relative path defined in the pipeline step component configuration is likely incorrect.
You notice that your fine-tuned model is 'forgetting' base capabilities after training on a small dataset. What strategy should you use to mitigate this?
Mixing datasets prevents the model from over-optimizing for the new data at the expense of old capabilities.
Why this answer
Rehearsal or mixing in base data (catastrophic forgetting prevention) preserves core capabilities.
You are configuring an agent orchestration system using Prompt Flow. You need to capture trace data for every step in the agent's reasoning process. Which infrastructure component must be enabled to visualize this in Azure AI Foundry?
Diagnostic settings are the standard mechanism to capture and store detailed execution traces from Prompt Flow.
Why this answer
Enabling the connection to an Azure Log Analytics workspace within the Prompt Flow project settings allows for the storage and visualization of trace data.
You are deploying a model as a real-time endpoint. You want to ensure the deployment can handle sudden spikes in traffic. Which scaling configuration should you enable?
Auto-scaling dynamically adjusts resources based on defined metrics like CPU utilization.
Why this answer
Auto-scaling allows the endpoint to dynamically adjust node count based on CPU/Memory usage.
What is the recommended approach for managing configuration settings for different environments (Dev, Test, Prod) in an AI application?
App Configuration enables centralized management of settings across environments.
Why this answer
Using environment variables or app configuration services is the standard way to inject configuration without hardcoding.
Which THREE of the following are valid compute targets for Azure Machine Learning training?
Valid training target.
Why this answer
Azure ML supports Compute Instances, Compute Clusters, and attached Kubernetes clusters for training.
Which TWO of the following are benefits of using Prompt Flow for orchestration?
Streamlines quality assurance.
Why this answer
Prompt Flow offers a visual interface for complex flows and integrated evaluation, which simplifies orchestration.
You are managing model versioning in Azure Machine Learning Registry. You need to promote a model from 'Staging' to 'Production' without creating a new asset version. Which command or action should you perform?
Tags are the standard way to track status without changing the asset version.
Why this answer
You should use the 'update' command on the existing model version asset to update its tags or properties to reflect the production status.
You are using Azure Machine Learning Pipelines. Which THREE triggers can be used to start a pipeline?
Standard time-based trigger.
Why this answer
REST endpoints, schedule, and event-based triggers (e.g., dataset changes) are supported.
You are building a RAG application and notice that the model sometimes hallucinates information not present in the retrieved documents. Which evaluation metric should you prioritize to mitigate this?
Groundedness verifies the response is based on the source context.
Why this answer
Groundedness specifically assesses whether the generated response is derived from the retrieved documents.
You are defining an Azure Machine Learning environment for a training job. The environment requires a specific set of Python libraries. What is the best practice for defining these dependencies?
This ensures consistent environment creation.
Why this answer
Using a 'conda.yaml' file is the best practice for managing reproducible Python environments in Azure ML.
Which cost-tracking tool in the Azure portal allows you to view usage by specific Azure OpenAI deployments?
Provides detailed breakdowns of spending across resources.
Why this answer
Azure Cost Analysis allows you to filter and group costs by resource and resource-specific tags or deployments.
You are debugging a prompt that is performing poorly on edge cases. You decide to use a 'Chain-of-Thought' approach. Why does this improve performance?
Intermediate reasoning steps guide the model toward correct final conclusions.
Why this answer
CoT forces the model to generate intermediate reasoning steps, which improves logic in complex tasks.
Which THREE types of information are found in a Run Object in Azure ML?
Metadata labels.
Why this answer
Metrics, parameters, and tags are all core components of a run record.
You want to measure 'Relevance' in a RAG application. The relevance evaluator detects how well the response answers the user query. If the model provides a factually correct answer that does not address the prompt, which metric will capture this failure?
Relevance measures the alignment of the answer to the question.
Why this answer
Relevance specifically measures if the response directly addresses the user's intent.
Which TWO of the following are effective ways to reduce hallucination?
This stops the model from speculating.
Why this answer
Giving the model an 'I don't know' option and providing relevant context are proven techniques.
Which TWO metrics are most useful for evaluating the 'User Experience' in a conversational agent?
An irrelevant answer is a poor user experience.
Why this answer
Latency and the quality of the answer (Relevance) are the primary drivers of user satisfaction.
Which THREE of the following represent lifecycle stages of an ML model in Azure ML?
The model generation phase.
Why this answer
Training, Registration, and Deployment are sequential stages in the model lifecycle.
You need to ensure that your Azure ML models are deployed with high availability across multiple regions. Which deployment strategy should you configure?
Managed Online Endpoints allow splitting traffic across multiple deployments in different regions.
Why this answer
Managed Online Endpoints support multi-region traffic distribution by using managed deployments behind a single endpoint.
You need to implement a retraining trigger based on performance degradation. Which TWO metrics should you monitor to decide when to retrain?
Declining precision indicates the model is failing to identify classes correctly.
Why this answer
Accuracy and precision are key performance indicators that signify model decay.
Which infrastructure artifact acts as the primary container for organizing and managing your models, deployments, and associated connections in Azure AI Foundry?
The Project is the specific container for model deployments, flows, and agent configurations.
Why this answer
An Azure AI project is the core resource that organizes models, deployments, connections, and flow assets within the Azure AI Foundry ecosystem.
A developer is troubleshooting an agent that is failing to connect to an external tool. Which THREE of the following infrastructure settings should be verified?
The Connections object manages the credentials/API keys for external services.
Why this answer
Connection failures often stem from incorrect credential management, network blocking, or misconfigured environment variables that define the external tool's access.
Which THREE actions are essential when managing a 'Golden Dataset' for LLM evaluation?
Versioning ensures reproducibility.
Why this answer
Curating high-quality data, versioning it, and periodically updating it are key management activities.
You need to detect 'jailbreak' attempts in your RAG application. You are implementing a custom evaluation pipeline. Which technique is most effective for identifying adversarial inputs designed to bypass system instructions?
Using a judge model to classify prompt intent is the recommended approach for detecting jailbreaks.
Why this answer
Adversarial evaluation using a 'jailbreak' detection model or prompt-based evaluation is the standard approach to identify bypass attempts.
Which THREE of the following are common reasons for a model deployment to fail on a Managed Online Endpoint?
Causes runtime import errors.
Why this answer
Invalid scoring scripts, missing dependencies in the environment, and insufficient resource limits are frequent causes.
You are onboarding a team to Azure AI Foundry. You need to assign the minimum permissions for a developer to deploy and test a model. Which role should you assign?
This role is specifically designed for developers working within AI Foundry projects.
Why this answer
The 'Azure AI Developer' role provides sufficient permissions to manage resources and perform model deployments within the project.
You are configuring a 'Managed Online Endpoint' with SSL termination. Where do you manage the SSL certificates?
SSL/TLS termination happens at the infrastructure boundary.
What is the primary function of a 'Prompt Template' in an Azure AI Prompt Flow?
Templates provide a repeatable structure for LLM interaction.
Why this answer
A prompt template allows users to define the structure of the prompt while injecting dynamic variables.
What is the primary benefit of 'Prompt Versioning' in an MLOps lifecycle?
Versioning ensures you can track which prompt produced which output.
Why this answer
Versioning prompts allows for tracking changes, reverting to previous versions, and comparing performance over time.
You are implementing an agentic workflow using Azure AI Foundry. The agent needs to access a secure external API. How should you store the API key securely within the project?
Using Key Vault with AI project connections is the secure, recommended practice.
Why this answer
Azure AI Foundry provides a 'Connections' feature that allows storing credentials as secrets, which can then be referenced by agents.
Your team wants to monitor the safety of model outputs. Which THREE steps should you take?
Visibility into filtered events.
Why this answer
Setting up content filters, monitoring logs, and configuring alerts are key to ensuring output safety.
You notice your model is outputting redundant information. Which parameter specifically targets the penalty for repeating tokens?
Frequency penalty penalizes tokens based on how many times they have already appeared.
Why this answer
The frequency_penalty parameter is explicitly designed to reduce the probability of tokens that have already appeared.
You want to automate the testing of your prompt flows as part of your CI/CD pipeline. Which tool should you use?
The CLI is the standard tool for integration into CI/CD.
Why this answer
The 'pf' (Prompt Flow) CLI is designed to run flows and evaluations programmatically in CI/CD pipelines.
Which THREE steps are necessary to successfully fine-tune an Azure OpenAI model?
JSONL is the standard format for fine-tuning data.
Why this answer
Data preparation, selecting the base model, and managing the fine-tuning job are fundamental steps.
Your team wants to perform 'Red Teaming' on your application. Which activity describes this process correctly?
Red teaming is specifically about finding vulnerabilities through adversarial simulation.
Why this answer
Red Teaming involves adversarial testing to find edge cases, safety flaws, and security vulnerabilities.
You notice your model deployment status is 'Succeeded' but inference requests are failing. Where should you first check for connectivity issues?
This is the primary location to verify endpoint health and URI.
Why this answer
The 'Endpoints' view in the AI Foundry project shows the connectivity status and endpoint URLs for the model.
Which THREE factors influence the cost of an Azure ML Compute Cluster?
Reduces cost during inactivity.
Why this answer
VM SKU, node count, and idle time settings directly impact spending.
You need to monitor the 'latency' and 'token usage' of your model deployments in real-time. Which tool provides the most granular view?
This allows querying specific events, latency, and token consumption.
Why this answer
Azure Monitor with custom log queries against the diagnostic logs of the AI resource provides the most granular view.
What is the primary function of the 'Prompt Flow' tool within Azure AI Foundry?
Prompt flow is dedicated to building and testing agentic workflows.
Why this answer
Prompt Flow is a development tool designed to streamline the entire development cycle of AI applications, from prototyping to deployment.
When implementing LLM-as-a-judge for evaluation, which TWO factors can influence the reliability of your results?
The judge's system prompt dictates evaluation logic.
Why this answer
The choice of judge model and the design of the system prompt for the judge significantly impact the result quality.
Which THREE security features are essential for a production Azure ML deployment?
For credential management.
Why this answer
Managed Identities, VNet integration, and Key Vault are essential for secure enterprise ML.
Your Azure ML training job fails with an 'Out of Memory' error on a GPU cluster. You need to ensure the job runs successfully without modifying the model code. What should you do?
Selecting a VM SKU with more RAM and GPU memory resolves OOM errors.
Why this answer
Scaling up to a VM size with more RAM or GPU memory is the most direct way to resolve OOM issues in training jobs.
You are reviewing the infrastructure architecture for a GenAI application that requires strict regulatory compliance. Which TWO of the following configurations are required to ensure data privacy and residency?
Ensuring data is not persisted by the AI service is a key compliance requirement.
Why this answer
Data residency and privacy in GenAI are best managed by ensuring data is not stored permanently in the AI service and by strictly controlling network traffic via Private Links.
Which THREE components are required to define a 'Managed Online Deployment'?
Defines runtime libraries.
Why this answer
An environment, a model artifact, and an inference script (score.py).
To optimize costs for an enterprise chatbot, you want to implement token usage monitoring. Which Azure service should you integrate to track token consumption per user?
Azure Monitor allows for custom queries on token usage metrics.
Why this answer
Azure Monitor logs can be used to track and analyze usage metrics exported from Azure OpenAI instances.
Which technique is most appropriate for optimizing RAG performance when the vector database returns too much noisy information?
Re-ranking improves precision of context retrieval.
Why this answer
Re-ranking filters top retrieved results to ensure high-quality context is passed to the LLM.
A model is exhibiting data drift. You have created a drift monitor. What is the next step to automate the retraining?
This bridges the monitor alert to the execution logic.
Why this answer
After creating a monitor, you set up an 'Event Grid' subscription to notify an Azure Function or Logic App to trigger the pipeline.
Page 3 of 3
Practice AI-300 by domain
Target a specific domain to shore up weak areas.