Select Two (Multi-Select) Questions

Automated CI/CD pipelines

Automated CI/CD pipelines provide repeatable, versioned deployment of models and scoring infrastructure, satisfying the stem's demand for a robust MLOps strategy. They integrate with Azure Machine Learning and Microsoft Entra ID, enabling automated retraining, testing and governed release, which manual processes cannot sustain reliably at scale.

Version control for code and models

Version control tracks code, model artefacts and configurations, enabling reproducibility and rollback. This satisfies the MLOps strategy requirement for traceability, ensuring every deployed model maps to a specific commit and dataset for audit and recovery.

Automated model monitoring

Automated model monitoring continuously tracks deployed models for data drift, performance degradation and infrastructure health, triggering alerts or retraining when thresholds breach. This closes the operational loop that a robust MLOps strategy requires, ensuring models remain accurate after deployment rather than degrading silently in production.

API Keys.

API keys authenticate directly against an Azure AI Foundry project endpoint, satisfying the requirement for a valid project-level authentication method. The key is scoped to the project resource, so callers present it without needing a separate identity token exchange. This makes it one of the two supported mechanisms alongside Microsoft Entra ID.

Microsoft Entra ID (RBAC).

Microsoft Entra ID with role-based access control authenticates callers through OAuth 2.0 tokens issued to a security principal, satisfying the stem's requirement for a valid Foundry project authentication method. Access is granted by assigning roles such as Azure AI Developer at the project or resource scope, so no account keys are embedded in code.

Model prediction precision

Precision measures the proportion of positive predictions that are actually correct, so a sustained drop signals the model is degrading on the positive class. Monitoring it alongside recall directly satisfies the stem's requirement to detect performance degradation and trigger retraining, rather than tracking infrastructure or data-drift signals alone.

Model prediction accuracy

Prediction accuracy directly quantifies performance degradation, so monitoring it reveals when the model's outputs drift below an acceptable threshold. This satisfies the stem's requirement for a metric that triggers retraining, since declining accuracy signals the deployed model no longer generalises to current data.

Relevance

Relevance measures whether responses actually address the user's question, a core driver of conversational satisfaction. Alongside coherence, it reflects perceived quality directly, making it a useful user experience metric rather than an internal system measure.

Latency

Latency directly measures the response delay a user perceives during each conversational turn, making it a core user-experience metric. It captures responsiveness, distinct from quality or accuracy metrics, and satisfies the stem's requirement to evaluate how the interaction feels to the end user.

Training

Training fits model parameters against prepared data to produce a candidate artefact, making it the first lifecycle stage before evaluation, registration and deployment. Azure ML captures it as a job whose outputs feed subsequent stages, satisfying the lifecycle-stage criterion in the stem.

Registration

Registration records a trained model in the Azure ML workspace or registry, assigning a name and version so it becomes a trackable, deployable asset. This versioning step sits between training and deployment, satisfying the lifecycle-stage criterion the question asks for.

Deployment

Deployment publishes a registered model to an online endpoint or batch endpoint, making it consumable for inference. It follows registration in the lifecycle and is a distinct managed stage, satisfying the criterion the question asks for.

The Azure AI Foundry 'Connections' configuration for the specific tool.

The Connections configuration in Azure AI Foundry stores the endpoint, authentication credentials and access keys the agent uses when invoking an external tool. A missing, stale or mis-scoped connection entry prevents the tool call from authenticating, so it must be verified first.

The Environment Variables passed to the Prompt Flow runtime.

Prompt Flow resolves tool connections through environment variables injected into the runtime, so missing or malformed values such as endpoint URLs or credentials cause connection failures. Verifying them confirms the agent can reach the external tool.

The Network Security Group (NSG) rules allowing outbound traffic from the runtime.

Outbound connectivity to an external tool traverses the subnet's Network Security Group, so a deny rule on the required port or destination blocks the agent's egress. Verifying NSG outbound rules confirms whether the runtime can reach the tool's endpoint, directly addressing the connection failure.

Implementing Private Link for network isolation.

Private Link gives the GenAI deployment a private endpoint, so inference traffic traverses the Microsoft backbone rather than the public internet. This satisfies the production requirement for network isolation, preventing data exposure during model prompts and responses.

Setting up auto-scaling for inference endpoints.

Auto-scaling adjusts inference endpoint capacity to match request volume, satisfying the production requirement for availability and performance under variable load. Without it, token throughput bottlenecks or idle capacity undermine the deployment's ability to serve concurrent generative requests reliably.

Enabling diagnostic logging for audit and monitoring.

Diagnostic logging captures request, response and operational telemetry from the GenAI deployment, satisfying the production requirement for audit and monitoring. It enables tracing of prompt activity, latency and failures, which is essential for incident investigation and compliance evidence.

Autoscale policies.

Autoscale policies define the rules that translate monitored metrics into scaling actions, specifying thresholds, minimum and maximum replica counts, and cooldown periods. They are the decision layer that automates orchestration scaling, satisfying the stem's requirement.

Monitoring metrics (e.g., latency, throughput).

Autoscaling requires telemetry to trigger it; latency and throughput metrics quantify agent workload, letting the orchestrator detect saturation and scale out. Without these signals no scaling decision can fire, satisfying the stem's requirement for a scaling component.

Register the model in the Azure ML Model Registry with versioning

Registering the model in the Azure ML Model Registry with versioning assigns each artefact an immutable identifier and lineage record. This satisfies the stem's traceability requirement, letting any deployed endpoint be mapped back to the exact registered model version used in the pipeline.

Add experiment tags to the model artifact

Experiment tags attach searchable metadata to the model artefact, linking it to the run, dataset and configuration that produced it. This satisfies the stem's traceability requirement by making the artefact's provenance queryable within the CI/CD pipeline.

Store the training script hash in the model properties

Storing the training script hash in the model properties creates a cryptographic link between the artefact and the exact code that produced it. This satisfies the stem's traceability requirement, proving which source revision generated the deployed model.

Prepare and upload a training dataset in JSONL format.

Fine-tuning requires supervised training examples, and Azure OpenAI accepts them as JSONL files with prompt-completion pairs uploaded to the resource. Without this formatted dataset, no training job can be submitted, making it a mandatory prerequisite step in the fine-tuning workflow.

Select a compatible base model for fine-tuning.

Not every model supports fine-tuning; only designated base models can be adapted. Selecting a compatible base model ensures the subsequent training job is accepted, satisfying the platform's compatibility constraint before dataset upload and job creation proceed.

Monitor the training job progress.

Fine-tuning runs as an asynchronous job, so tracking its status and metrics is required to confirm completion and detect failures before deploying the tuned model. Monitoring progress is therefore an essential step, not optional, in the end-to-end fine-tuning process.

The specific prompt instructions provided to the judge LLM

Prompt instructions directly shape the judge's scoring behaviour, so vague or leading rubrics produce inconsistent verdicts and bias scores toward surface features rather than the criteria being evaluated. Precise, anchored instructions with explicit scoring scales reduce this variance, satisfying the stem's reliability requirement for LLM-as-a-judge evaluation.

The capability of the judge LLM

A more capable judge model detects subtle errors, follows rubrics accurately and resists position or verbosity bias, directly raising scoring reliability. Weaker judges misapply criteria, producing inconsistent verdicts that undermine the evaluation's trustworthiness. Judge capability therefore satisfies the stem's reliability constraint alongside prompt design and bias mitigation.

Key Vault integration.

Key Vault integration stores secrets, keys and connection strings outside code and configuration files, letting the workspace and compute retrieve them through managed identities. This prevents credential exposure in scripts or logs, a baseline requirement for production deployments.

Managed Identity.

Managed Identity gives the Azure ML workspace and its compute an automatically rotated identity in Microsoft Entra ID, eliminating stored credentials in code or config. This satisfies the production requirement for secretless, auditable authentication to Azure resources such as Key Vault and Storage.

Virtual Network (VNet).

A Virtual Network isolates workspace compute, storage and endpoints from public internet exposure, with private endpoints and NSGs controlling traffic. This satisfies the production requirement that training data and model endpoints remain unreachable from untrusted networks.

Integrated testing and evaluation.

Prompt Flow provides built-in evaluation flows that score outputs against metrics such as groundedness and relevance, plus batch testing across datasets. This satisfies the stem's orchestration benefit by embedding repeatable quality measurement directly into the pipeline, rather than requiring separate tooling to validate prompt variants before deployment.

Visual design of complex flows.

Prompt Flow's visual designer lets developers compose LLM prompts, Python tools and conditional logic as a connected DAG, so complex multi-step orchestration can be built and debugged graphically rather than hand-coded, directly satisfying the orchestration benefit the question asks for.

A Virtual Network (VNet)

A virtual network is mandatory because the private endpoint must be placed in a subnet within it, letting traffic reach the workspace privately. Without a VNet, no private endpoint can exist, so the Private Link requirement cannot be satisfied.

Private Endpoint

A private endpoint provides the actual private IP connection into the workspace, mapping the Azure ML service to a subnet address. It is the core component that removes public exposure, satisfying the Private Link requirement alongside the VNet.

Model weights (pickle/onnx).

Serialised weights in pickle or ONNX format carry the learned parameters that define the trained model's behaviour. Including them satisfies the packaging requirement, since the inference script alone cannot produce predictions without the numeric weights loaded into the model architecture.

Inference script (score.py).

The score.py inference script defines init() and run() to load the model and handle scoring requests. Including it satisfies the packaging requirement, since the deployment endpoint needs this entry-point code to deserialise input, invoke the model and return predictions.

Environment file (conda.yaml).

The conda.yaml environment file pins the Python, CUDA and library versions the model was trained against. Including it lets the deployment target rebuild an identical runtime, preventing dependency drift that would otherwise cause scoring failures.

Consistent software dependencies

Azure ML Environments package dependencies into a reusable definition referenced across training and scoring. This satisfies the stem's benefit by guaranteeing identical library versions wherever the environment is used, eliminating drift between local experimentation, pipelines and deployed inference endpoints.

Versioning of software stacks

Azure ML Environments capture the exact software stack, including Python packages and base images, as immutable, versioned artefacts. This satisfies the stem's benefit by letting you pin a specific environment version per run, ensuring reproducibility and safe rollback when dependencies change.

Deleting unused model deployments.

Deleting unused model deployments directly eliminates idle compute charges, since provisioned throughput and hosted endpoints bill continuously regardless of traffic. This satisfies the cost-optimisation constraint by removing resources that consume quota and incur hourly fees without serving requests. Regular audits of deployment usage therefore reduce spend without affecting active inference workloads.

Purchasing reserved capacity for long-term needs.

Reserved capacity commits to a one- or three-year term, applying a discounted rate to predictable, steady-state compute. This directly satisfies the cost-optimisation goal for long-term needs, unlike pay-as-you-go pricing, which charges the undiscounted rate for the same sustained usage.

Right-sizing instances based on actual usage metrics.

Right-sizing matches instance SKU and count to measured utilisation, eliminating spend on over-provisioned vCPU and memory. It satisfies the cost-optimisation requirement by removing idle capacity that actual usage metrics reveal, rather than guessing at provisioning from peak or nominal demand.

Batch endpoint

Batch endpoints run asynchronous scoring jobs over large datasets, returning results to a Datastore rather than a synchronous response. They are one of the three Azure Machine Learning endpoint types, alongside online and serverless endpoints.

Web service (legacy)

Web service (legacy) endpoints are the older AKS or ACI-backed deployment target in Azure Machine Learning, predating managed online endpoints. It remains a recognised endpoint type, satisfying the stem's request for endpoint categories alongside managed online and batch endpoints.

Managed online endpoint

Managed online endpoints provide a turnkey, Microsoft-managed real-time inference target in Azure Machine Learning, handling scaling, authentication and infrastructure. It is a distinct endpoint type, satisfying the stem's request for endpoint categories alongside batch and legacy web service endpoints.

Prompt Flows.

Prompt Flows provide the orchestration component within an Azure AI Foundry project, letting you chain prompts, models, and tools into executable flows. They are a first-class project asset, satisfying the stem's requirement to identify genuine project components.

Connections.

Connections store authenticated links to external resources such as Azure OpenAI or storage, scoped to the Azure AI Foundry project. They are a native project component, satisfying the stem's requirement to identify genuine project components.

Models.

Models form a core component of an Azure AI Foundry project, providing the deployed foundation models that power inference, fine-tuning and evaluation. This satisfies the stem's requirement to identify project components, since every project consumes models through the model catalogue or custom deployments to run generative AI workloads.

Compute Cluster

Compute Cluster is a valid AML training target: it provisions multi-node CPU or GPU clusters that automatically scale up on job submission and scale down to zero nodes when idle, satisfying the stem's requirement for a scalable, managed training compute target.

Attached Kubernetes Cluster

Attached Kubernetes Cluster is a valid AML training target: you attach an existing Azure Kubernetes Service cluster to the workspace, letting training jobs run on your own orchestrated infrastructure while AML handles image deployment and job scheduling.

Compute Instance

Compute Instance is a valid Azure Machine Learning training target, satisfying the requirement for a managed, single-node development environment that can also run training jobs. It supports running scripts and notebooks directly, and can execute training workloads without requiring you to provision or manage underlying infrastructure.