Implement agentic AI solutions practice questions

Use a single prompt that includes the entire conversation history and the latest user question, then generate SQL.

Why it fails: Conversation history grows quickly and may exceed token limits.

Use a chain-of-thought prompt to generate SQL queries directly from user input, without maintaining conversation history.

Why it fails: Without memory, the agent cannot handle multi-turn conversations.

Use embeddings-based retrieval to find relevant past interactions and include them in the prompt.

Why it fails: Retrieval is useful for knowledge but not for structured query generation with context.

Microsoft Bot Framework SDK with a custom connector.

Why it fails: Bot Framework does not inherently provide compliance.

Azure Cognitive Search with custom analyzers.

Why it fails: Cognitive Search is not a platform for building agents.

Azure OpenAI Service with data processing enabled.

Why it fails: Azure OpenAI Service may not be HIPAA compliant in all regions.

Use a custom model fine-tuned on the knowledge base and disable content filtering.

Why it fails: Fine-tuning does not guarantee adherence to a specific knowledge base and disabling filtering is risky.

Use a system message that says 'If you don't know, say you don't know' and rely on the model's training.

Why it fails: The model may still generate incorrect information despite the instruction.

Use prompt engineering with a system message that instructs the model to only answer from the knowledge base, with no additional filtering.

Why it fails: Prompt engineering is not sufficient to prevent hallucinations.

Increase the training data for location entities.

Why it fails: This does not address the date issue.

Add a prebuilt entity component for date.

Why it fails: Prebuilt components may not be suitable for custom models.

Use a different intent classification model.

Why it fails: Entity extraction is separate from intent classification.

Free (F0): Pay-as-you-go pricing for moderate workloads.

Why it fails: Incorrect: This description applies to Standard S0, not Free.

Standard (S0): Designed for high-scale enterprise workloads with higher throughput.

Why it fails: Incorrect: This describes the Premium tier, not Standard S0.

Premium: Provides limited transactions per month at no cost.

Why it fails: Incorrect: This describes the Free tier, not Premium.

Configure the agent to use the code interpreter tool with the 'auto' execution mode to allow the model to decide when to run code.

Why it fails: While the code interpreter tool can be configured, the execution mode is not a parameter that the team sets; the model automatically decides when to invoke the tool based on the conversation. There is no 'auto' execution mode setting for the code interpreter in Azure AI Foundry Agent Service. This option describes a non-existent configuration.

Ensure the agent's managed identity has the Storage Blob Data Contributor role on the Azure Storage account used by the code interpreter.

Why it fails: The code interpreter runs in a sandboxed environment managed by Azure AI Foundry, and file access is handled through the service's internal mechanisms. The agent's managed identity does not need direct access to the storage account. Assigning this role is unnecessary and could introduce security risks by granting excessive permissions.

Upload the CSV files to the agent's knowledge base and configure the agent to retrieve them using vector search.

Why it fails: Uploading CSV files to a knowledge base and using vector search is not suitable for code interpreter analysis. The code interpreter requires direct file access via file IDs, not vector search. Knowledge bases are for semantic retrieval, not for executing code against raw files. This approach would not allow the agent to run Python code on the CSV data.

Use a Power Automate flow to call the API with a stored connection, and invoke the flow from the agent as a tool.

Why it fails: Power Automate connections can store credentials, but invoking a flow from an Azure AI Foundry agent is not a native tool integration and would require custom connector development. This increases rather than minimizes effort. The flow also introduces a separate execution context that may not provide the low-latency, direct tool call expected for live flight data.

Store the OAuth 2.0 token in the agent's system message and update it hourly by using a Logic App.

Why it fails: Placing a token in the system message exposes it in the conversation context and potentially in logs. The system message is not designed for secret rotation, and an hourly Logic App update would require custom code, increasing effort. This approach also risks the token appearing in the chat transcript if the model references it.

Create an Azure API Management instance that proxies the API and injects a subscription key, then call the proxy from the agent.

Why it fails: API Management can secure APIs, but the scenario requires OAuth 2.0 token refresh every hour. A subscription key does not satisfy that requirement and would not automatically refresh. Proxying adds infrastructure and development effort, and the agent would still need to handle the OAuth flow unless additional policies are written.

Add the API credentials to Azure Key Vault and reference the vault URI in the agent's system prompt

Why it fails: Key Vault is the right place to keep secrets, but referencing a vault URI in the system prompt does not grant the agent a way to authenticate or retrieve the secret. The prompt is model-visible text and could leak the URI. This approach neither issues short-lived tokens nor removes the need for a stored credential, so it fails the stated requirement.

Store the API's client secret in the agent's environment variables and read it inside a function tool

Why it fails: Embedding a client secret in the agent's environment variables creates a long-lived credential that persists in the agent configuration, which directly violates the security team's requirement. Anyone with read access to the agent definition could extract it, and rotation would require redeploying the agent. The scenario requires short-lived, scoped credentials, not a static secret stored alongside the agent.

Configure the agent's function tool to request a token from the Microsoft Graph token endpoint for the API's scope

Why it fails: The Microsoft Graph token endpoint issues tokens for Microsoft Graph, not for an arbitrary internal REST API. Requesting a token for a non-Graph audience will fail or produce a token the API rejects. The correct pattern is to obtain a token for the API's own resource scope, typically through a Foundry connection backed by a managed identity.

Increase the temperature setting of the underlying model to encourage more creative responses.

Why it fails: Increasing temperature makes the model more random and creative, which would worsen the problem of fabricated tracking numbers. The goal is to ground responses in factual API data, not to introduce variability. Higher temperature would make hallucinations more likely, not less, and is inappropriate for a scenario requiring precise, verifiable outputs.

Add a post-processing step that validates tracking numbers against a regular expression and removes invalid ones.

Why it fails: A regular expression can validate format but cannot confirm that a tracking number exists in the tracking API. Fabricated numbers might still match the pattern, so invalid entries could pass through. This approach also adds complexity and does not prevent the model from generating false information in the first place, leaving the root cause unaddressed.

Reduce the model's max tokens setting so that the agent cannot generate long responses.

Why it fails: Limiting max tokens controls response length, not factual accuracy. A short response can still contain a fabricated tracking number. This setting does not influence whether the model uses the API data or invents values, and it may truncate legitimate responses. It fails to address the grounding requirement and could degrade the user experience.

Azure Logic Apps

Why it fails: Logic Apps orchestrates workflows and connectors; it provides no conversational agent runtime, language understanding, or channel publishing for web and mobile. It is tempting because Logic Apps is the right choice for automating order and return back-end processes that the agent would call.

Azure API Management

Why it fails: API Management publishes, secures and throttles existing APIs; it neither hosts the agent's language reasoning nor connects to website and mobile channels. It is tempting because it fronts backend order and returns APIs, and would be correct for exposing those APIs to partners with keys, quotas and policies.

Azure Functions

Why it fails: Azure Functions runs event-driven code, not conversational agent sessions with order-tracking and returns skills; it cannot host orchestration or channel integration for web and mobile. It is tempting because Functions suits serverless API backends and webhook handlers, which would be correct for executing a single return action rather than deploying the agent itself.

Azure Cognitive Search and Azure AI Language.

Why it fails: Cognitive Search indexes and retrieves documents, and Azure AI Language offers sentiment, entities and moderation-adjacent features, but neither provides the dedicated toxicity and bias scoring the agent requires. It is tempting because AI Language includes content moderation, which would suit filtering static text rather than monitoring live OpenAI generations.

Azure Bot Service and Azure Logic Apps.

Why it fails: Bot Service handles conversational channels and Logic Apps orchestrates workflows; neither analyses text for harmful or biased content. It is tempting because both commonly front and connect agents, and Logic Apps would suit routing flagged outputs to reviewers once a safety service had already classified them.

Azure Machine Learning and Azure Functions.

Why it fails: Azure Machine Learning trains and hosts models, and Azure Functions runs event-driven code; neither performs toxicity or bias classification on generated text. It is tempting because both are core Azure AI building blocks, and Machine Learning would suit evaluating model fairness offline rather than monitoring live agent output.

Fine-tune the model on a dataset of correct function calls.

Why it fails: Fine-tuning alters model weights for style or domain adaptation, not the schema-bound function selection the model performs at inference. It cannot prevent invented names, since function definitions are supplied per request. Fine-tuning suits teaching consistent output formatting or domain vocabulary when prompt engineering has plateaued, but here the fix is tightening function descriptions and parameters.

Reduce the number of functions to only the most common ones.

Why it fails: Removing functions reduces coverage of required tasks like CRM updates rather than improving selection accuracy; the model still lacks clear descriptions distinguishing remaining functions. Trimming is right when the schema exceeds token limits or contains genuinely unused definitions.

Set the temperature parameter to 0 for deterministic output.

Why it fails: Temperature controls sampling randomness, not schema adherence; at 0 the model still invents names when prompts are ambiguous. It is tempting because determinism sounds like reliability, and low temperature suits tasks needing reproducible phrasing, but function-name accuracy depends on clear descriptions and strict schema enforcement.

Use a different OCR API from Azure Cognitive Services.

Why it fails: Switching OCR APIs does not address low-contrast input; the Read API already handles it, and alternative Cognitive Services OCR endpoints offer no contrast-specific preprocessing. It is tempting when accuracy plateaus, but the correct fix is image preprocessing or the Read model's enhanced contrast handling.

Train a custom OCR model using Azure Custom Vision.

Why it fails: Custom Vision trains image classification and object detection models, not OCR text extractors, so it cannot produce a recognition model for this task. It is tempting because custom training improves accuracy on domain-specific visuals, and would be correct for labelling bespoke objects or product categories rather than reading characters.

Increase the confidence threshold for text detection.

Why it fails: Raising the confidence threshold discards detections the model already made, reducing recall rather than recovering text it never recognised in low-contrast regions. It is tempting because thresholds tune output precision, and would be correct when too many false positives appear in otherwise legible images.

Ensure the model is large enough to memorize the entire document library.

Why it fails: Context windows are finite regardless of model size, so no model memorises an entire document library, and memorisation provides no citation or update path. It tempts because larger models handle more context, yet that would matter only when prompts fit within the window, not when retrieval must select relevant passages.

Fine-tune the Azure OpenAI model on the document library.

Why it fails: Fine-tuning bakes document content into weights, which cannot be updated per query and gives no citations, so it fails the retrieval requirement. It tempts because fine-tuning customises model behaviour, yet it would be correct when the goal is adapting tone or output format, not grounding answers in a changing library.

Train a custom language model from scratch.

Why it fails: Training a model from scratch requires massive labelled corpora and compute, and produces no retrieval mechanism, so the agent still cannot ground answers in the library. It tempts because bespoke models promise domain mastery, yet it would be right only when no pretrained foundation model fits the language or task.

The temperature parameter is set too high.

Why it fails: Temperature controls randomness of generated text, not whether the model emits valid function-call arguments or the runtime executes them. It tempts because high values cause erratic output, but function execution failures usually trace to schema definitions, tool registration or prompt instructions, which temperature does not govern.

The model version is outdated.

Why it fails: Model version does not govern function-calling execution; the schema, tool definitions and response handling do. An outdated model may still invoke functions correctly. Upgrading is relevant when a newer model adds capabilities such as parallel tool calls, not when calls fail to execute.

Deploy the bot code to an Azure Function.

Why it fails: Azure Functions is one hosting option, not a required step; bots can run on App Service, containers or other compute. It is tempting because Functions suits event-driven bots and is a popular serverless choice for lightweight Teams integrations.

Write the bot in C# using Azure SDK for .NET.

Why it fails: Azure Bot Service supports any language or SDK, including Python, JavaScript and REST, so mandating C# and the .NET SDK is not a required step. It is tempting because C# with the Bot Framework SDK is a common, well-documented approach for Teams bots.

Switch to a different language model without fine-tuning.

Why it fails: Swapping models without fine-tuning leaves the same generic sentiment weights that misread domain-specific negative phrasing, so accuracy on your feedback stays unchanged. It appeals because Azure AI Language offers several prebuilt and custom model types, and a different base model would be the right move only if the current one lacked support for your language or text format.

Increase the number of decimal places in the sentiment score.

Why it fails: Sentiment scores are model outputs whose precision is fixed by the service; adding decimal places merely reformats the same value and cannot correct misclassification. It is tempting because the scores look imprecise, but the correct improvement is supplying domain-specific training data or adjusting the model.

Increase the confidence threshold for positive sentiment.

Why it fails: Raising the positive threshold reclassifies borderline positive text as neutral or negative; it never corrects a genuinely negative sentence scored positive, so the underlying misclassification persists. Threshold tuning is the right lever when you need to trade precision against recall for a chosen class, not when the model's polarity judgement itself is wrong.

Keep page-level chunking but add 50% overlap between chunks.

Why it fails: Page-level chunks split clauses across boundaries, so overlap only duplicates fragments rather than preserving semantic units; retrieval still matches partial clauses. Overlap suits fixed-size token chunking where sentences straddle windows. Here, semantic or hierarchical chunking by clause or section keeps each clause intact for embedding.

Use a different embedding model, such as text-embedding-3-large.

Why it fails: Swapping embedding models leaves the page-level chunking unchanged, so clause-level retrieval still fails; the stem identifies chunking as the suspected cause. It is tempting because a stronger embedding model improves semantic matching, and would be correct if retrieval failed despite sound chunk boundaries.

Increase the chunk size to 5 pages per chunk and reduce overlap.

Why it fails: Larger five-page chunks dilute each embedding with unrelated clauses, worsening retrieval of specific clauses; the problem is granularity, not size. It is tempting because bigger chunks preserve surrounding context, and would suit summarisation tasks where broad passages matter.

Create an Azure Bot Service bot, then create and publish a QnA Maker knowledge base, then connect the bot to the QnA Maker knowledge base, then test the bot.

Why it fails: This is incorrect because the QnA Maker knowledge base must exist before creating the bot, as the bot requires the knowledge base endpoint during creation to be properly configured.

Create and publish a QnA Maker knowledge base, then create an Azure Bot Service bot, then test the bot, then connect the bot to the QnA Maker knowledge base.

Why it fails: This is incorrect because the bot cannot be tested without first connecting it to the QnA Maker knowledge base; the connection must be established before testing.

Create an Azure Bot Service bot, then create and publish a QnA Maker knowledge base, then test the bot, then connect the bot to the QnA Maker knowledge base.

Why it fails: This is incorrect because both the QnA Maker knowledge base must be created before the bot, and the bot must be connected before testing.

Configure the agent to use a managed identity and assign it the necessary permissions to call the REST API directly.

Why it fails: Managed identity can authenticate to Azure resources that support Microsoft Entra ID, but the external REST API requires an OAuth 2.0 token from a third-party identity provider. Managed identity cannot directly obtain tokens for arbitrary external APIs. This approach would fail unless the external API supports federated identity, which is not stated.

Store the OAuth token in the agent's knowledge base and have the agent retrieve it when needed.

Why it fails: Storing tokens in a knowledge base is not secure and does not provide a mechanism for automatic token refresh. The agent might retrieve an expired token, causing API calls to fail. Knowledge bases are designed for documents, not secrets, and this approach would expose sensitive credentials to anyone with access to the knowledge base.

Embed the OAuth token in the agent's system prompt so the agent can include it in API calls.

Why it fails: Embedding the token in the system prompt is insecure because the token would be visible in logs and could be exposed to users. Additionally, tokens expire, so the agent would need manual updates. This approach does not support automatic token refresh and violates security best practices for handling credentials in Azure AI solutions.