AI-300 Practice Test 15 — 15 Questions

Question 1

You are automating the deployment of your AI project. What is the recommended tool to manage infrastructure-as-code (IaC) for Azure AI Foundry?

Accepted Answer

Azure Bicep. Bicep is the native, recommended IaC tool for Azure, providing deep integration for AI Foundry resources.

Answer

Azure DevOps pipelines

Answer

Azure CLI alone

Answer

Python script with Azure SDK

Question 2

You have a large set of documents for a RAG system. How should you optimize retrieval speed?

Accepted Answer

Store embeddings in a dedicated vector database with indexing.. Storing embeddings in a dedicated vector database with an ANN index (HNSW, IVF) enables sub-linear nearest-neighbor search instead of scanning every document. This is the standard architecture for fast retrieval over large corpora in RAG systems. The index precomputes a graph or cluster structure so queries touch only a small fraction of vectors.

Answer

Process all documents in the prompt.

Answer

Increase the model's max_tokens.

Answer

Perform a brute-force search on all documents.

Question 3

You are optimizing your Azure ML pipeline performance. Which THREE steps should you take to reduce execution time?

Accepted Answer

Mount datasets instead of downloading them.. Option C is correct because mounting a dataset exposes it as a FUSE-based virtual filesystem (e.g., /mnt/azureml/...) so the compute target reads only the files it needs on demand, avoiding the full download-and-copy overhead that inflates startup and I/O time. Option D is correct because enabling step run reuse lets Azure ML skip re-executing a pipeline step when its code, inputs, parameters, and environment are unchanged, returning the cached output and eliminating redundant compute. Option E is correct because ParallelRunStep partitions a batch workload across multiple nodes/processes of the cluster and runs them concurrently, cutting wall-clock time for large-scale scoring or data processing. Option A is not appropriate because adding workspaces only multiplies management boundaries and does not speed up a single pipeline's execution. Option B is not appropriate because undersized clusters for every step cause throttling, longer runtimes, and possible out-of-memory failures rather than faster execution.

Answer

Increase the number of workspaces.

Answer

Use smaller compute clusters for everything.

Question 4

You are building a generative AI assistant for a financial advisory firm using Azure OpenAI. The assistant must answer questions based on a large repository of internal policy documents that are updated frequently. You need to reduce hallucination and ensure the model cites the most current policy versions. Which approach should you implement?

Accepted Answer

Use Retrieval-Augmented Generation (RAG) with Azure AI Search to retrieve relevant document chunks and include them in the prompt.. Retrieval-Augmented Generation with Azure AI Search retrieves up-to-date document chunks and injects them into the prompt, grounding the model's responses in current policies and reducing hallucination. This dynamic approach is ideal for frequently updated repositories, as the search index can be refreshed independently of model retraining. Other options either fail to provide current information or increase randomness.

Answer

Increase the temperature setting to encourage more creative and comprehensive answers.

Answer

Fine-tune the base model on the entire document repository weekly.

Answer

Deploy a larger model with a higher token limit to accommodate more context.

Question 5

You are the MLOps engineer for a customer support chatbot built on Azure OpenAI. Your team wants to continuously monitor the deployed model's responses for harmful content without storing the raw user prompts in your own database. You must implement a solution that evaluates every response in near real time, logs only the safety verdicts and severity scores, and raises an alert when severity exceeds a threshold. Which Azure service should you use to perform this evaluation?

Accepted Answer

Azure AI Content Safety. Azure AI Content Safety is purpose-built to analyze generative AI inputs and outputs for harmful categories, returning per-category severity scores in near real time. Because it evaluates content without requiring you to persist raw prompts, it satisfies both the privacy constraint and the monitoring requirement. Its severity scores can be routed to Azure Monitor to trigger alerts when thresholds are exceeded.

Answer

Azure Monitor Application Insights custom events

Answer

Azure Machine Learning model data collector

Answer

Azure AI Language Personally Identifiable Information detection

Question 6

You are monitoring an Azure OpenAI deployment used for a customer support chatbot. You notice that the model sometimes generates responses that include fabricated product features. You want to implement a technique that encourages the model to say 'I don't know' when it lacks information, rather than guessing. Which approach should you use?

Accepted Answer

Use a system message that instructs the model to only answer if it is confident and to otherwise respond with 'I don't know'.. A system message that instructs the model to only answer when confident and to otherwise respond with 'I don't know' directly leverages the model's instruction-following capability to reduce fabrication. This prompt engineering approach is effective for encouraging honesty and is easy to implement. Other options adjust randomness or filter harmful content but do not address the core issue of hallucination.

Answer

Enable content filtering to block harmful or inappropriate responses.

Answer

Increase the frequency_penalty parameter to discourage repetitive tokens.

Answer

Decrease the top_p parameter to 0.1 to make the output more focused.

Question 7

You are an MLOps engineer at a financial services company. Your team trains a fraud detection model weekly using an Azure Machine Learning pipeline. Compliance requires that every model registered in the workspace is traceable to the exact training run, dataset version, and code commit that produced it. You need to ensure this lineage is automatically captured for every registered model without manual documentation. What should you do?

Accepted Answer

Register the model using the output of the training pipeline step so Azure Machine Learning automatically records the run ID, dataset asset, and job context as model metadata.. Model lineage in Azure Machine Learning is captured automatically when a model is registered as the output of a job or pipeline step, because the service records the run ID, data assets, environment, and job context as model properties. Manual documentation, diagnostic logs, or naming conventions do not create that enforced association, so they cannot guarantee audit-ready traceability.

Answer

Store the training script and a CSV of metrics in the workspace's default datastore and reference the folder path in the model description field.

Answer

Create a Git tag in the repository after each pipeline run and name the registered model using the tag value so engineers can correlate them later.

Answer

Enable diagnostic settings on the Azure Machine Learning workspace and route all logs to a Log Analytics workspace for long-term retention.

Question 8

You are responsible for monitoring a deployed model on an Azure Machine Learning managed online endpoint. The model's input feature distribution has drifted from the training baseline, and you need to detect this automatically and act on it. Which two actions should you take? (Choose two.)

Accepted Answer

Create a model monitor in Azure Machine Learning that computes a drift metric against a reference dataset and set an alert when the threshold is exceeded.. Automatic drift detection requires two things working together: collecting live inference data from the deployment and configuring a model monitor that compares that data to a reference dataset and raises alerts past a threshold. Together they detect distribution change and notify the team. Scaling, timeout tuning, and version bumps do not create drift signals.

Answer

Set the deployment's request timeout to a lower value so that anomalous requests fail fast and are excluded from metrics.

Answer

Re-register the model with a higher version number so the monitoring system recognizes it as the current production model.

Answer

Increase the deployment's instance count so that the endpoint can absorb the additional inference traffic caused by drift.

Question 9

You are evaluating a retrieval-augmented generation (RAG) application that answers questions about internal HR policies. During a quality review, you discover that for some queries the model produces fluent, confident answers that are not supported by the retrieved policy documents. You need to add an automated evaluation that specifically detects whether the response is factually supported by the retrieved context. Which evaluator should you use?

Accepted Answer

Groundedness evaluator. Groundedness evaluation directly compares a model's response against the retrieved context to determine whether claims are supported. Because the observed problem is confident but unsupported answers, this evaluator targets the exact failure mode. Fluency, relevance, and coherence measure other quality dimensions and would not reliably flag responses that are well-written but not grounded in the source documents.

Answer

Relevance evaluator

Answer

Fluency evaluator

Answer

Coherence evaluator

Question 10

You are preparing an Azure Machine Learning training job that must read a large Parquet dataset stored in an Azure Data Lake Storage Gen2 account. The job runs on an Azure ML compute cluster, and you want the data to be streamed directly from the storage account without first downloading the full dataset to the compute node's local disk. You also need to avoid storing account keys in the job script. Which Azure ML datastore configuration should you use?

Accepted Answer

Register the storage account as an Azure ML datastore using credential-less (identity-based) authentication, then use the datastore URI with the azureml-fsspec or mltable input in the job.. The scenario requires both credential-less access and streaming reads. Registering the ADLS Gen2 account with identity-based authentication allows the compute cluster's managed identity to reach the data without embedding account keys. Combining that datastore with a URI-based input such as fsspec or mltable enables the training script to stream Parquet data directly from storage, avoiding a full local download and meeting both constraints.

Answer

Register the storage account as an Azure ML datastore using account key authentication, then mount the datastore in the job.

Answer

Download the dataset to the compute cluster's default datastore before the job starts, then read it from the local path in the training script.

Answer

Create a FileDataset that references the ADLS Gen2 path and pass it as a mounted input to the job.

Question 11

You manage a managed online endpoint in Azure Machine Learning that serves a recommendation model. The model was trained months ago, and you now need to deploy a new version trained on recent data with zero downtime. You must be able to send a small percentage of live traffic to the new version to validate it before full rollout. What should you do?

Accepted Answer

Add a second deployment to the same endpoint with the new model, then update the endpoint's traffic weights so the new deployment receives a small percentage of requests.. Managed online endpoints decouple the stable scoring URI from the deployments behind it. By adding a second deployment and adjusting traffic weights, you keep the current model serving while a small share of requests exercises the new model, achieving zero-downtime canary validation. Replacing, deleting, or duplicating endpoints breaks either availability or the controlled traffic split.

Answer

Delete the existing deployment, create a new deployment with the updated model, and set the endpoint's traffic allocation to 100 percent on the new deployment.

Answer

Create a new managed online endpoint for the updated model and have clients switch their base URL to the new endpoint after you finish testing.

Answer

Redeploy the existing deployment in place with the new model artifact and set the deployment's instance count to double during the update.

Question 12

You are optimizing a generative AI application that uses Azure OpenAI's GPT-4 model for summarizing long legal contracts. The summaries are often incomplete because the model truncates important details from the middle of the contract. You need to improve the completeness of the summaries without exceeding the model's context window. What should you do?

Accepted Answer

Use a chunking strategy with overlapping segments and summarize each chunk, then combine the summaries.. Chunking with overlapping segments and hierarchical summarization allows the model to process the entire contract in parts, preserving context at boundaries and combining partial summaries into a complete one. This approach works within the model's context window and avoids truncation. Increasing output tokens or lowering temperature does not solve input truncation, and switching to a larger model may still be insufficient and costly.

Answer

Switch to a model with a larger context window, such as GPT-4-32k, and pass the entire contract.

Answer

Increase the max_tokens parameter to allow longer summaries.

Answer

Lower the temperature to 0 to make the model more deterministic.

Question 13

You are designing a reusable Azure ML pipeline that must run a data validation step, a training step, and a registration step. The pipeline will be executed repeatedly with different input datasets, and you need each step to run in its own isolated environment with pinned dependencies. You also need the pipeline to be versioned and shareable across workspaces. What should you use to define the pipeline?

Accepted Answer

A YAML pipeline definition that references registered components with pinned environments, submitted with the Azure ML CLI v2.. Registered components in a YAML pipeline definition give each step its own environment, enabling dependency isolation. The pipeline can be registered and versioned as a pipeline asset, and because it references components and environments by name and version, it can be shared and rerun across workspaces that have those assets. This satisfies isolation, versioning, and shareability in one approach.

Answer

A pipeline that uses a single environment for all steps and passes the dataset as a pipeline parameter, with the pipeline registered as a model.

Answer

A pipeline defined with the designer drag-and-drop interface and exported as a JSON file stored in a Git repository.

Answer

A single PythonScriptStep-based pipeline defined inline in a notebook, with all dependencies installed at the start of the notebook session.

Question 14

You are configuring a real-time managed online endpoint in Azure Machine Learning. The model must be updated frequently with minimal downtime, and you need the ability to roll back quickly if a new version performs poorly. Which deployment strategy should you use?

Accepted Answer

Create a second deployment with the new model, shift traffic gradually to it, and keep the original deployment available for rollback.. Managed online endpoints allow multiple deployments to coexist behind one endpoint, with traffic allocation controlled by a percentage split. Deploying the new model as a separate deployment and shifting traffic gradually enables safe validation and minimal downtime. Keeping the previous deployment active provides a fast rollback path, because traffic can be redirected back without redeploying the model.

Answer

Use a batch endpoint and invoke it on a schedule to simulate real-time scoring.

Answer

Deploy the model to an Azure Kubernetes Service cluster outside Azure ML and manage traffic with an ingress controller.

Answer

Create a single deployment and update the model in place, then restart the deployment.

Question 15

You are preparing a red teaming exercise for a generative AI application built with Azure AI Foundry. The goal is to systematically probe the application for harmful outputs and safety weaknesses before release. Which two activities are appropriate for a red teaming exercise? (Choose two.)

Accepted Answer

Crafting adversarial prompts that attempt to elicit prohibited content across categories such as violence, hate, and self-harm.. Red teaming is an adversarial process that probes a generative AI application for safety weaknesses. Crafting adversarial prompts across harm categories and running automated adversarial prompt datasets through Azure AI Foundry risk and safety evaluations both directly test whether the application can be induced to produce harmful outputs. The other activities concern performance, access control, or text similarity and do not evaluate safety behavior.

Answer

Calculating the BLEU score between model responses and a reference translation dataset.

Answer

Measuring the average token latency of the model across a representative production workload.

Answer

Reviewing the application's role-based access control assignments for the deployment resource.