AI-300 Practice Test 7 — 10 Questions

Question 1

You are building a RAG application and notice that the model sometimes hallucinates information not present in the retrieved documents. Which evaluation metric should you prioritize to mitigate this?

Accepted Answer

Groundedness. Groundedness specifically assesses whether the generated response is derived from the retrieved documents.

Answer

Relevance

Answer

Performance

Answer

Fluency

Question 2

You want to evaluate how well your model adheres to specific brand guidelines. Which evaluation method is best suited for this?

Accepted Answer

Custom LLM-as-a-judge evaluation. Custom evaluation using a judge model (LLM-as-a-judge) configured with a rubric allows for checking specific style or brand compliance.

Answer

Token usage threshold alert

Answer

Default coherence metric

Answer

Automated regression testing of code

Question 3

You are optimizing the cost of your GenAI infrastructure. You have several agents running in Prompt Flow that are idle for large portions of the day. Which runtime configuration should be modified?

Accepted Answer

Set the 'automatic_shutdown' property on the Prompt Flow runtime compute instance.. Setting the 'idle_time_before_shutdown' or similar inactivity policies in the Prompt Flow runtime ensures compute resources are deallocated when not in use.

Answer

Change the VM SKU to a burstable instance type.

Answer

Implement a script to delete the runtime when the flow is finished.

Answer

Set the concurrency limit to zero.

Question 4

A developer is troubleshooting an agent that is failing to connect to an external tool. Which THREE of the following infrastructure settings should be verified?

Accepted Answer

The Azure AI Foundry 'Connections' configuration for the specific tool.. Connection failures often stem from incorrect credential management, network blocking, or misconfigured environment variables that define the external tool's access.

Answer

The Azure Resource Group location.

Answer

The model version of the LLM.

Question 5

Which TWO of the following strategies are commonly used to optimize for cost in Generative AI?

Accepted Answer

Selecting the smallest capable model for the task.. Caching and choosing smaller, task-appropriate models are the primary levers for cost management.

Answer

Increasing the context window size to 128k.

Answer

Enabling verbose logging.

Answer

Using the highest possible temperature.

Question 6

You are monitoring an Azure OpenAI deployment and need to identify if a model is outputting content that violates safety policies. Which Azure AI Content Safety feature should you enable to categorize harmful content?

Accepted Answer

Content Safety API text analysis. The Content Safety service provides classification categories such as Hate, Self-Harm, Sexual, and Violence to filter model outputs.

Answer

Azure Monitor Logs

Answer

Application Insights Profiler

Answer

Prompt Flow Guardrails

Question 7

You need to detect 'jailbreak' attempts in your RAG application. You are implementing a custom evaluation pipeline. Which technique is most effective for identifying adversarial inputs designed to bypass system instructions?

Accepted Answer

Adversarial input classification using a dedicated judge model. Adversarial evaluation using a 'jailbreak' detection model or prompt-based evaluation is the standard approach to identify bypass attempts.

Answer

Token usage tracking

Answer

Latency threshold monitoring

Answer

Semantic similarity scoring

Question 8

You are reviewing the 'Run History' in Azure Machine Learning. You want to compare the training time of two different experiments. Which UI feature should you use?

Accepted Answer

The 'Experiments' tab, then 'Compare' button.. The 'Charts' or 'Metrics' comparison view in the Azure ML Studio allows you to visualize and compare metrics across multiple runs.

Answer

The 'Models' registry list.

Answer

The 'Endpoints' dashboard.

Answer

The 'Notebooks' file browser.

Question 9

You are evaluating an LLM application using Prompt Flow. You want to measure the 'Groundedness' of the model response relative to the retrieved context. Which evaluator should you configure?

Accepted Answer

Groundedness evaluator. The Groundedness evaluator in Prompt Flow checks if the response is supported by the context provided.

Answer

Relevance evaluator

Answer

Coherence evaluator

Answer

Fluency evaluator

Question 10

You have an automated model training pipeline that is failing intermittently due to compute availability. What should you configure to ensure the pipeline is more resilient?

Accepted Answer

Configure 'retry' settings for the step.. You should configure 'retry' settings in the 'PipelineStep' definition to handle transient compute errors.

Answer

Increase the 'max_nodes' of the cluster.

Answer

Set the 'allow_reuse' parameter to True.

Answer

Use 'LowPriority' compute.