Hard Difficulty Questions

Lower the temperature.

Why it fails: Temperature controls sampling randomness, not which source content the model treats as in scope, so internal notes would still be summarised. Lowering temperature suits tasks needing deterministic, repeatable output such as classification or structured extraction.

Switch to a different model.

Why it fails: Changing the model does not define which transcript content is in scope, so internal notes remain eligible for inclusion. Model substitution addresses capability or cost limits, such as when a task needs stronger reasoning than the current deployment provides.

Add a 'stop_sequence' to the API call.

Why it fails: A stop_sequence halts token generation when a specified string appears, so it cannot filter notes already woven into the model's output. It is genuinely useful for truncating responses at a known delimiter. Preventing internal notes requires prompt-level grounding constraints or content filtering on the source transcript.

Fixed-character chunking with zero overlap

Why it fails: Zero overlap guarantees that any sentence straddling a boundary is severed, so retrieval quality stays degraded. It is tempting because fixed-character chunking is simple and predictable for uniform embedding sizes, but resolving split semantics requires overlap or boundary-aware splitting, not fixed windows.

Paragraph-only splitting with maximum size limits of 10,000 tokens

Why it fails: A 10,000-token ceiling is far larger than typical embedding input limits, so chunks are effectively unsplit paragraphs and sentences still cross boundaries. Paragraph splitting is tempting for preserving topic coherence, but without sentence-aware boundaries and realistic token caps it cannot fix the reported splitting.

Random token slicing

Why it fails: Random token slicing cuts at arbitrary positions, so sentences are split more often, not less. It is tempting as a quick way to bound chunk length, but semantic integrity requires boundary detection such as sentence-aware or recursive splitting; random slicing ignores linguistic structure entirely.

The 'Load Balancer' settings in the VNET.

Why it fails: VNET load balancer settings distribute inbound traffic across backend instances; they do not monitor CPU or add replicas. It is tempting because load balancing and scaling are both availability concerns, and it would be correct for spreading requests across healthy targets, but CPU-based scale-out requires autoscale rules on the deployment.

Azure Front Door health probe settings.

Why it fails: Front Door health probes monitor endpoint availability and route traffic away from unhealthy backends; they do not trigger scaling. It is tempting because Front Door sits in front of endpoints and reacts to backend state, and would be correct for global load balancing and failover, but autoscale rules on the deployment handle CPU thresholds.

The 'Compute' instance restart policy.

Why it fails: A restart policy governs what happens when a compute instance fails or is recycled; it never evaluates CPU metrics or changes instance counts. It is tempting because it is a compute-level setting on the deployment, and would be correct for recovering crashed instances, but scaling on CPU load requires autoscale rules with metric thresholds.

Azure App Service routing rules.

Why it fails: App Service routing rules direct HTTP requests to app slots or instances, not to individual model versions within an Azure AI deployment, so percentage-based version routing cannot be expressed. It is tempting because App Service traffic splitting does exist, but it applies to deployment slots, not model versioning.

Azure Load Balancer backend pools.

Why it fails: Azure Load Balancer distributes TCP/UDP flows by IP and port, with no awareness of model versions, so it cannot split 10% of inference requests to a new version. It is tempting because load balancers do front traffic, but they suit network-layer balancing, not model-version traffic splitting.

Azure Traffic Manager DNS settings.

Why it fails: Traffic Manager distributes at DNS resolution, so clients resolve one endpoint and stay there; it cannot route 10% of inference requests to a new model version. It is tempting because it performs percentage-based distribution, but that suits regional endpoint failover, not per-request model versioning.

Using random sentence selection without indexing

Why it fails: Random sentence selection discards semantic structure, so retrieved context is irrelevant and accuracy collapses while latency stays unpredictable. It is tempting as a cheap way to shrink context, and sampling would suit exploratory summarisation, but high-accuracy RAG requires indexed, embedding-based retrieval with reranking.

Disabling vector search entirely to rely solely on random string matching

Why it fails: Random string matching discards semantic similarity, so retrieval returns irrelevant chunks and accuracy collapses; vector search is the retrieval backbone of RAG. It is tempting because exact keyword matching is cheap and deterministic, and it would be correct for precise identifier or code lookups, not open-ended semantic queries.

Deleting and recreating the entire resource group.

Why it fails: Recreating the resource group rebuilds the whole environment rather than inspecting the flow's input processing, so it cannot isolate the fault and may destroy diagnostic evidence. It is tempting as a clean-slate reset for corrupted deployments, but input-processing faults need inspection of the flow's input mappings, schema validation and run logs.

Increasing the model temperature to see if the error persists.

Why it fails: Temperature governs the language model's sampling randomness, not how inputs are parsed, mapped or validated before the model call, so it cannot surface an input-processing fault. It is tempting because raising it changes outputs, but that is the correct lever when diagnosing overly deterministic or repetitive model responses.

Increase the confidence threshold parameter in the analyzeDocument request body to 0.99.

Why it fails: The confidence threshold only filters which extracted fields the response returns; raising it to 0.99 suppresses low-confidence results rather than improving the model's accuracy. It is tempting as a quick quality gate, which suits downstream validation logic, not raising the underlying extraction confidence of a custom template model.

Convert all training PDFs to grayscale and reduce their resolution to 72 DPI to speed up processing.

Why it fails: Grayscale conversion and 72 DPI reduction strip colour and detail the custom template model relies on to match field layouts, degrading extraction confidence further. It is tempting because smaller files process faster, which suits latency or throughput optimisation, not accuracy on poorly extracted fields.

Switch the model type from custom template to a prebuilt business card model.

Why it fails: A prebuilt business card model extracts a fixed schema of contact fields and cannot target the custom fields the template model was trained on, so those fields lose extraction entirely. It is tempting because prebuilt models need no training, which suits standard documents such as business cards, not bespoke field extraction.

Disable the OCR engine parameter in the API call to bypass text recognition steps.

Why it fails: Disabling OCR removes the text recognition stage entirely, so no field values are extracted and confidence scores become meaningless. It is tempting because skipping OCR reduces latency and cost, which would be correct for pipelines handling documents where text is already available as embedded digital content.

Decrease the model training iterations configuration setting to prevent overfitting on handwriting.

Why it fails: Training iterations govern convergence on the labelled dataset, not recognition of hand-written glyphs; lowering them degrades overall accuracy. It is tempting because iteration tuning addresses overfitting on small training sets, which would be the right lever if validation loss diverged from training loss.

Switch from a Custom Neural model to a Custom Template model.

Why it fails: Custom Template models rely on fixed positional anchors, so they cannot absorb the handwriting variation that Neural models handle through learned features. It is tempting because Template models suit stable, identical layouts, which would be the correct choice for standardised forms with consistent field positions.

Use a single Azure OpenAI resource for all tenants and track usage using custom HTTP headers.

Why it fails: A single Azure OpenAI resource pools all tenants into one quota and one billing boundary; custom headers are not metered by Azure, so per-tenant token counts cannot be enforced. This approach is tempting for simplicity, and would be correct where only aggregate usage matters and no per-tenant quota isolation is required.

Deploy all tenants in the F0 free pricing tier.

Why it fails: The F0 free tier caps monthly transactions and provides no production quota guarantees, so it cannot isolate per-tenant token usage or cost. It is tempting because it avoids spend during development, and would be correct for prototyping or evaluation, not for a multi-tenant production workload.

Rely on Azure Cost Management resource group tags alone without resource separation.

Why it fails: Tags alone cannot enforce per-tenant quota limits, because Azure OpenAI token quotas are allocated at the resource level, not by tag. Tagging is tempting for chargeback reporting, and would suffice if the requirement were only cost visibility, but strict quota isolation demands separate resources.

List entity type

Why it fails: A list entity matches only exact dictionary entries, so it cannot recognise serial numbers it has not seen. It is tempting because lists are simple to author, but they suit closed sets such as product categories; pattern-based extraction of variable serial numbers needs a regular expression entity.

Learned entity type

Why it fails: Learned entities infer boundaries from labelled examples rather than a fixed pattern, so they cannot reliably enforce the exact serial-number format. They suit entities defined by context and vocabulary, such as product names; a regular expression entity is required when the stem specifies a precise character pattern.

Prebuilt entity type

Why it fails: Prebuilt entities cover common concepts such as dates, quantities and URLs, not bespoke serial-number formats. They are tempting because they need no training data, but they cannot be constrained to two uppercase letters, four digits, a hyphen and three digits; that requires a regular expression entity.

Chain-of-thought prompting

Why it fails: Chain-of-thought elicits intermediate reasoning steps, which can rationalise fabricated details instead of preventing them. It is tempting because chain-of-thought genuinely lifts accuracy on multi-step arithmetic, logic and planning tasks where the model already possesses the underlying knowledge.

Increasing the Top_P parameter

Why it fails: Raising Top_P widens nucleus sampling, admitting lower-probability tokens and increasing invented content rather than suppressing it. It is tempting because Top_P tuning legitimately controls output diversity and creativity for brainstorming or varied generation, where broader sampling is the goal.

Few-shot prompting with varied examples

Why it fails: Few-shot examples shape output format and style; they cannot supply facts the model never held, so grounding remains absent. It is tempting because few-shot prompting genuinely improves task accuracy and consistency when the required knowledge already sits in the model or context.

Azure AI Content Safety

Why it fails: Content Safety classifies and filters harmful text or images; it exposes no schema describing an external REST API's operations, parameters or authentication for the agent to invoke. It is tempting because it is an Azure AI Foundry service configured alongside agents, and it would be correct where the requirement is moderating prompts and completions.

LLM output parser

Why it fails: An output parser converts the model's raw text into structured data; it does not declare the API's endpoint, parameters and callable operations that the agent needs as a tool definition. It is tempting because parsing model output is part of agent pipelines, and it would be right where the requirement is extracting structured fields from an LLM response.

Flow input schema

Why it fails: A flow input schema describes the parameters a prompt flow accepts at invocation; it cannot describe the external REST API's operations, so the agent has no callable tool contract. It is tempting because schemas do define interfaces, and it would be correct where the requirement is parameterising a prompt flow's inputs.

1,000,000 characters

Why it fails: Azure AI Language synchronous document inputs are capped at 5,120 characters per document, so 1,000,000 exceeds the limit and the request would be rejected. The larger figure is tempting because it resembles asynchronous or batch limits, but synchronous single-document calls enforce the smaller 5,120-character ceiling.

50,000 characters

Why it fails: 50,000 exceeds the synchronous single document limit.

1,024 characters

Why it fails: 1,024 is too low; the limit is higher.

Listen for incoming webhook notifications on an Azure Event Grid subscription attached to the text analytics resource.

Why it fails: Azure AI Language LROs expose status through the Operation-Location polling URL; no Event Grid subscription is created for the /analyze job. It is tempting because Event Grid delivers push notifications for resource events, but it is correct only where an event source publishes to a subscribed topic.

Query the Azure Monitor metrics API for job completion status.

Why it fails: Azure Monitor exposes resource metrics and diagnostic logs, not the per-job operation status or result payload of an LRO. It is tempting because Azure Monitor tracks service health and latency, but it is correct only for monitoring resource-level telemetry, not for retrieving a specific job's output.

Repeatedly send synchronous POST requests to the same document endpoint until a 200 OK is returned.

Why it fails: The /analyze endpoint returns an Operation-Location header; status and results are retrieved by polling that operation URL with GET, not by resubmitting documents. It is tempting because polling is the right pattern, but re-POSTing re-runs the job and returns 202 Accepted, never the completed summary payload.

Check the 'Activity Log'.

Why it fails: The Activity Log records Azure resource control-plane operations, such as deployments and configuration changes, not per-client inference requests, so it cannot attribute 429s to a calling application. It is tempting because it does surface operational events, and it would be the right choice when auditing who modified or deleted a resource.

Increase the resource quota in the 'Quotas' blade.

Why it fails: Raising the quota removes the symptom without identifying the offending caller, and may not even be permitted. Quota increases are the right action once the workload is confirmed legitimate and sustained demand genuinely exceeds the current limit.

Restart the AI service.

Why it fails: Restarting the AI service clears in-memory state but cannot attribute 429 responses to a specific client; the service resumes and the same client continues throttling. It is tempting as a quick remediation for transient faults, and would be correct for recovering a hung or memory-leaking service, not for identifying the offending caller via logs or metrics.

Model Training Jobs

Why it fails: Model Training Jobs are a separate Azure Machine Learning workload, not a node type in a Prompt Flow DAG, whose components are inputs, tools, prompts, LLM nodes and outputs. It is tempting because training jobs do appear in ML pipelines, and would be correct when authoring a training pipeline rather than a prompt orchestration graph.

Data Labeling Projects

Why it fails: Data Labeling Projects belong to Azure Machine Learning's labelling workflow for annotating datasets, not to Prompt Flow's DAG execution model. Prompt Flow nodes are tools, LLM prompts, Python code and flow inputs/outputs. It is tempting because labelling feeds prompt engineering, but it is a separate service.

Add the endpoint IP to the local hosts file of every Azure AI Vision instance.

Why it fails: Editing hosts files does not scale and bypasses Azure Private DNS zone resolution, which is what private endpoints require. It is tempting as a quick manual override for a single machine, and would suffice only for isolated troubleshooting on one host, not for all clients in the virtual network.

Configure public DNS records on your corporate DNS server to point to the public IP.

Why it fails: Pointing corporate DNS at the public IP routes traffic over the public internet, defeating the private endpoint's purpose. It is tempting because custom DNS forwarding is a genuine hybrid pattern, and would be correct when on-premises clients must resolve Azure private endpoints through a DNS forwarder to 168.63.129.16.

Modify the routing table to drop all DNS traffic.

Why it fails: Dropping DNS traffic breaks name resolution entirely rather than directing it to the private endpoint's private IP. It is tempting because routing tables do control traffic flow, and custom routes are correct when forcing traffic through a network virtual appliance or firewall, not for resolving private endpoint names.

All documents from the same printer

Why it fails: Documents may originate from any source; the training set needs varied layouts and labelled fields, not uniform hardware. Same-printer origin would suit isolating scanner-specific artefacts during troubleshooting, but it narrows layout diversity and harms model generalisation.

At least 1,000 pages per model

Why it fails: Page volume is not a training requirement; Azure AI Document Intelligence custom models accept small labelled sets, and quality of labelled examples matters instead. The threshold applies to bulk document processing quotas, so it would fit capacity planning rather than training-set design.

Store unencrypted user authentication tokens directly inside the embedding vector metadata.

Why it fails: Embedding metadata is returned with search results and readable by any client querying the index, so storing raw tokens there exposes credentials across permission tiers. Tokens belong in a secure identity store. Vector metadata is intended for filterable attributes such as tenant or document identifiers used in security trimming.

Grant every user application admin privileges to bypass retrieval latency checks.

Why it fails: Admin privileges grant configuration and data access far beyond a user's permission tier, breaking the tiered isolation the scenario requires. Retrieval latency is governed by index design and query patterns, not privileges. Application admin rights suit platform operators managing the search service, not end users querying a RAG pipeline.

Disable managed identities and store plaintext connection strings in the application source code.

Why it fails: Disabling managed identities and embedding plaintext connection strings directly contradicts the compliance requirement, since credentials in source code are exposed to anyone with repository access and cannot be rotated centrally. It is tempting because connection strings do authenticate to Azure AI Search, but Microsoft Entra ID with managed identities is the supported credential-free mechanism for this scenario.

A binary classification indicating whether the review is spam or genuine.

Why it fails: Opinion mining returns aspect-level sentiment and confidence scores, not spam classification. Spam detection is a separate content-moderation task, so a binary genuine/spam verdict would only be correct if the requirement were filtering abusive or unwanted text rather than extracting opinions.

A single neutral sentiment score for the entire document.

Why it fails: Opinion mining splits the sentence into aspect-sentiment pairs, yielding mixed results rather than one neutral score. A single document-level score is what plain sentiment analysis returns; opinion mining is chosen precisely when per-aspect granularity, not an aggregate label, is required.

A translated version of the review in English.

Why it fails: Translation is handled by Azure AI Translator, not opinion mining.