Troubleshooting Scenario Questions

The kbId is not published.

Why it fails: The error is more likely due to the endpoint.

The skill is not deployed.

Why it fails: The configuration file suggests the skill is configured but not connecting.

The endpointKey is incorrect.

Why it fails: While possible, the more fundamental issue is the modelUrl.

Increase the number of intents to split the flight-related queries further.

Why it fails: Adding intents fragments training utterances across more labels, reducing examples per intent and worsening the confusion between booking and status. It is tempting because more granular intents appear to separate concerns, and would be correct when distinct user goals genuinely exist that the current schema cannot represent.

Reduce the confidence score threshold for intent detection.

Why it fails: Lowering the threshold changes when a predicted intent is accepted, not how the model separates booking from status when flight numbers appear; overlapping utterances remain misclassified. It is tempting because it surfaces low-confidence matches, and would be correct when valid utterances are being rejected as None despite adequate training.

Use the prebuilt domain for flight booking to improve accuracy.

Why it fails: The prebuilt flight booking domain supplies its own intents and entities, which does not resolve ambiguity between two custom intents sharing flight-number phrasing. It is tempting because prebuilt domains accelerate development, and would be correct when building a generic flight-booking bot without bespoke intent definitions.

The indexer is not configured with the PDF parser; set the parsing mode

Why it fails: PDF parsing is already handled by the blob indexer's built-in document extraction; parsing mode governs text versus layout extraction, not decryption. It tempts because parser misconfiguration does cause indexing failures, but the error explicitly names encryption, so the fix is supplying decryption credentials.

The file format is unsupported; convert to PDF/A

Why it fails: PDF is a supported format, and PDF/A conversion does not remove password protection, so the encrypted content still fails. It tempts because unsupported formats do break indexers, but the error names encryption; the actual fix is decrypting the file or configuring the indexer with an encryption key.

The file is too large; split it into smaller parts

Why it fails: File size triggers a different error about document limits, not encryption; splitting the PDF leaves the encryption intact. It tempts because oversized blobs genuinely fail indexing, but the stated cause is encryption, which requires removing the password or granting the indexer a decryption key.

The Power Automate flow has a timeout and returns default true.

Why it fails: Power Automate timeouts fail or return an error, not a default true value; nothing in the flow defines such a fallback. It is tempting because timeouts are a familiar integration failure, and would be correct if the flow's error-handling branch explicitly assigned true to inStock on timeout.

The agent's response is based on a different variable that defaults to true.

Why it fails: A variable defaulting to true would produce consistently wrong answers, not the intermittent behaviour described. It is tempting because variable misbinding is a genuine topic-authoring error, and would be correct if the response node referenced an unrelated variable initialised to true instead of the flow's inStock output.

The agent's topic is not parsing the JSON output correctly.

Why it fails: Copilot Studio topics do parse flow JSON, and a boolean mismatch would fail consistently rather than intermittently. It is tempting because malformed JSON handling is a common integration fault, and would be correct if the flow returned a string like 'false' that the topic never converted to a boolean.

Keep page-level chunking but add 50% overlap between chunks.

Why it fails: Page-level chunks split clauses across boundaries, so overlap only duplicates fragments rather than preserving semantic units; retrieval still matches partial clauses. Overlap suits fixed-size token chunking where sentences straddle windows. Here, semantic or hierarchical chunking by clause or section keeps each clause intact for embedding.

Use a different embedding model, such as text-embedding-3-large.

Why it fails: Swapping embedding models leaves the page-level chunking unchanged, so clause-level retrieval still fails; the stem identifies chunking as the suspected cause. It is tempting because a stronger embedding model improves semantic matching, and would be correct if retrieval failed despite sound chunk boundaries.

Increase the chunk size to 5 pages per chunk and reduce overlap.

Why it fails: Larger five-page chunks dilute each embedding with unrelated clauses, worsening retrieval of specific clauses; the problem is granularity, not size. It is tempting because bigger chunks preserve surrounding context, and would suit summarisation tasks where broad passages matter.

Increase the max token limit

Why it fails: Raising the max token limit only allows longer prompts and completions; it does not reduce the count of sequential external tool calls causing the delay. It is tempting because token limits govern model throughput, and increasing them helps when responses are truncated, not when orchestration latency dominates.

Add more tools to distribute the load

Why it fails: Adding tools increases the number of functions the agent may select and invoke, worsening sequential call latency rather than reducing it. It is tempting because distributing load across services helps throughput in parallel architectures, but the stem requires cutting the number of sequential tool calls, not spreading them.

Reduce the thread history length

Why it fails: Trimming thread history reduces prompt tokens and cost, but the sequential tool calls themselves still execute one after another, so wall-clock latency is unchanged. Tempting because shorter prompts feel faster, it would be correct only if latency stemmed from context size rather than serialised tool invocation.

Modify the system_prompt to include the model version

Why it fails: The system prompt only shapes model behaviour at inference; it cannot make an undeployed model available in the resource. The failure is a missing deployment. Editing prompts is correct when tuning responses, not when resolving model availability errors.

Change the connection_type to 'Weak'

Why it fails: Connection type governs authentication and network reachability to the resource, not which models are deployed. A 'Weak' connection cannot supply a missing model. Connection type is the right setting when the agent cannot authenticate or reach the endpoint.

Change the provider to 'AzureAI'

Why it fails: Changing the provider to 'AzureAI' alters which service endpoint the agent calls; it does not deploy the missing model into the Azure OpenAI resource. The fix is deploying a supported model there. Provider selection matters when routing to a different AI service entirely.

Run 'az search indexer run --name myindexer' to trigger a new run.

Why it fails: Running again may produce the same warnings; you need to see the existing warnings.

Ignore the warnings because they are not errors.

Why it fails: Warnings may indicate data quality issues that should be investigated.

Run 'az search indexer reset --name myindexer' to reset the indexer.

Why it fails: Resetting re-runs the indexer but does not show warnings; warnings are per execution.

Remove the stop parameter.

Why it fails: Removing stop only matters if a stop sequence truncated the output; incomplete mid-sentence responses typically stem from max_tokens being reached, so deleting stop changes nothing when no stop sequence is configured. Stop sequences are legitimately used to halt generation at a chosen delimiter, which is a different scenario.

Increase temperature to 1.0.

Why it fails: Temperature controls randomness of token selection, not response length, so raising it to 1.0 cannot extend a truncated completion and may add incoherence. It tempts because temperature is the most familiar sampling knob, yet it is the right adjustment when outputs feel repetitive or need creative variation, not when max_tokens cuts them off.

Increase top_p to 1.0.

Why it fails: Top_p does not limit length.

Use the pre-built 'products' model from Computer Vision.

Why it fails: No pre-built 'products' model exists in Azure Computer Vision; the service offers generic domain models such as landmarks and celebrities, not retail SKUs. Custom object detection requires labelled training images of the new product line, which is the actual first step. Pre-built models suit generic image classification scenarios, not bespoke inventory detection.

Increase the confidence threshold in the API call.

Why it fails: Raising the confidence threshold makes detection stricter, suppressing borderline predictions and worsening recall of the new products. Threshold tuning suits filtering false positives from an already-trained model. The failure stems from the model never having learned the new product line, so retraining with labelled images is required first.

Recreate the Computer Vision resource in a different region.

Why it fails: Region relocation changes only where the resource is hosted; it does not alter model training data or inference behaviour, so the new products remain undetected. Recreating a resource in another region suits latency, data-residency or regional service-availability requirements, not a model that lacks training examples for added product lines.

The dates exceed the maximum entity length

Why it fails: Date strings are short, well within entity length limits, so truncation cannot explain the misses. Length limits are tempting because they do cause extraction failures, but only for long spans; here the training data simply lacks the dd/mm/yyyy format the model must learn.

The language detection is incorrectly identifying the locale

Why it fails: Custom Entity Recognition is language-agnostic regarding date formatting; locale detection does not cause a model trained on mm/dd/yyyy to miss dd/mm/yyyy strings. Locale settings matter for built-in prebuilt entities, not custom extraction, so this misattributes the failure to the wrong component.

The model is overfitting to invoice numbers

Why it fails: Overfitting would degrade all labels, yet invoice numbers extract correctly, so the model is not over-committed to that entity. Overfitting is tempting whenever one label performs well, but the real gap is that training examples never showed the dd/mm/yyyy pattern the model must recognise.

Use the prebuilt-layout model to extract clauses instead

Why it fails: Prebuilt models do not extract custom clauses.

Increase the OCR confidence threshold in the analysis request

Why it fails: OCR confidence threshold does not affect layout adaptation.

Label 15 more contracts with the original layout and retrain the model

Why it fails: This does not address the new layout variation.

Use Azure AI Translator to translate utterances into multiple languages

Why it fails: Translation multiplies utterances across languages but preserves the same semantic overlap between ProductInquiry and ReturnRequest, so the confusion boundary is unchanged. It is tempting because it expands training data cheaply, and translation would be correct if the problem were low coverage of non-English customer queries rather than ambiguous intent separation.

Increase the confidence threshold in the bot configuration

Why it fails: Raising the confidence threshold only changes when the bot defers or escalates; it cannot separate ProductInquiry from ReturnRequest, since both intents still score similarly on the same utterances. Threshold tuning suits filtering low-confidence matches once the model already classifies intents accurately.

Create a new custom language project and retrain from scratch

Why it fails: Rebuilding a custom language project discards the 500 labelled utterances per intent that already exist, so the confusion between ProductInquiry and ReturnRequest remains unaddressed. Creating a fresh project suits a new domain or language where no labelled data exists, not refining an existing model that simply needs more boundary examples.

Use a different OCR API from Azure Cognitive Services.

Why it fails: Switching OCR APIs does not address low-contrast input; the Read API already handles it, and alternative Cognitive Services OCR endpoints offer no contrast-specific preprocessing. It is tempting when accuracy plateaus, but the correct fix is image preprocessing or the Read model's enhanced contrast handling.

Train a custom OCR model using Azure Custom Vision.

Why it fails: Custom Vision trains image classification and object detection models, not OCR text extractors, so it cannot produce a recognition model for this task. It is tempting because custom training improves accuracy on domain-specific visuals, and would be correct for labelling bespoke objects or product categories rather than reading characters.

Increase the confidence threshold for text detection.

Why it fails: Raising the confidence threshold discards detections the model already made, reducing recall rather than recovering text it never recognised in low-contrast regions. It is tempting because thresholds tune output precision, and would be correct when too many false positives appear in otherwise legible images.