AI-300 Generative AI Optimization — Timed 20 Questions

Question 1

You are developing a RAG pipeline and notice that the model often hallucinates when retrieving documents with low relevance. Which prompt engineering technique should you implement to improve grounding?

20 minute time limit — choose an answer to begin.

Accepted Answer

Implement Chain-of-Thought reasoning with negative constraints.. Few-shot prompting with explicit negative constraints helps the model understand the boundaries of its knowledge base.

Answer

Disable the system message.

Answer

Increase the temperature parameter to 1.0.

Answer

Switch from GPT-4o to GPT-3.5 Turbo.

Question 2

Which cost-tracking tool in the Azure portal allows you to view usage by specific Azure OpenAI deployments?

Accepted Answer

Azure Cost Analysis.. Azure Cost Analysis is the native cost management tool in the Azure portal that lets you slice and dice spending by resource, resource group, tag, and — critically — by service and meter, which includes Azure OpenAI deployments. Because Azure OpenAI usage is metered per deployment (by model, tokens, and PTU/standard tier), Cost Analysis can group costs by the deployment resource or by the 'Azure OpenAI' service meter. This makes it the correct tool for viewing usage tied to specific deployments.

Answer

Azure Network Watcher.

Answer

Azure Advisor.

Answer

Azure Policy.

Question 3

You are optimizing prompt latency by reducing tokens. Which of the following is the most effective way to reduce input token count for a recurring task?

Accepted Answer

Moving static task instructions to the system message.. Moving static task instructions to the system message is the most effective way to reduce input token count for a recurring task because the system message is cached and reused across requests in Azure OpenAI, so the static instructions are not re-billed as input tokens on every call. This leverages prompt caching, which stores the system message prefix and reduces both latency and cost for repeated tasks.

Answer

Changing the model to GPT-4o.

Answer

Removing the API key from the request.

Answer

Using a higher temperature.

Question 4

Which THREE parameters directly affect the output structure or style of an LLM response?

Accepted Answer

temperature. Temperature (C) is correct because it scales the logits before softmax, directly controlling the randomness/creativity of token selection and thus the stylistic character of the response. frequency_penalty (D) is correct because it penalizes tokens proportionally to how often they have already appeared, directly shaping word choice and reducing repetition in the output. presence_penalty (E) is correct because it applies a flat penalty to any token that has already appeared at least once, directly influencing whether the model introduces new topics or reuses existing tokens, which alters the response's style. By contrast, top_k (A) is a sampling parameter that restricts the candidate token pool but is not among the marked correct answers here, and max_tokens (B) only caps the response length rather than shaping its structure or style, so neither belongs to the set of correct options.

Answer

top_k

Answer

max_tokens

Question 5

You are fine-tuning a model on Azure OpenAI and notice the training loss curve is fluctuating significantly. What is the most likely cause?

Accepted Answer

The learning rate is too high.. A fluctuating training loss curve is typically caused by a learning rate that is too high. When the learning rate is too high, the optimization algorithm takes large steps and may overshoot the minimum, causing the loss to oscillate or even diverge. Reducing the learning rate often stabilizes training.

Answer

The model's temperature is set too high.

Answer

The training dataset size is too large.

Answer

The model lacks enough epochs.

Question 6

Which parameter in the Azure OpenAI API should be adjusted to make the model's output more deterministic and repeatable?

Accepted Answer

temperature. Temperature controls the randomness of the model's output by scaling the logits before the softmax function. A lower temperature (e.g., 0 or near 0) makes the probability distribution sharper, so the model almost always picks the highest-probability token, resulting in deterministic and repeatable responses. Setting temperature to 0 is the standard way to achieve consistent outputs for the same input in Azure OpenAI.

Answer

max_tokens

Answer

presence_penalty

Answer

frequency_penalty

Question 7

To optimize costs for an enterprise chatbot, you want to implement token usage monitoring. Which Azure service should you integrate to track token consumption per user?

Accepted Answer

Azure Monitor with Log Analytics.. Azure Monitor with Log Analytics is the correct service to track token consumption per user because Azure OpenAI emits metrics and diagnostic logs (including token usage) that can be routed to a Log Analytics workspace for querying and per-user aggregation. You enable diagnostic settings on the Azure OpenAI resource to send logs and metrics to Log Analytics, then use Kusto Query Language (KQL) to aggregate tokens by user or custom dimension. This provides centralized monitoring, alerting, and cost analysis.

Answer

Azure Data Factory.

Answer

Azure Cognitive Search.

Answer

Azure SQL Database.

Question 8

You are optimizing a long-context application. Which technique is most effective for reducing context window costs in Azure OpenAI?

Accepted Answer

Summarizing conversation history before passing to the next prompt.. Long-context prompts are billed per input token, so the most direct cost lever is reducing the number of tokens sent. Summarizing conversation history replaces a growing transcript with a compact synopsis, cutting input tokens substantially while preserving semantic continuity. This is the standard pattern for multi-turn chat applications where history would otherwise balloon the context window.

Answer

Using a smaller batch size.

Answer

Disabling streaming responses.

Answer

Increasing the frequency penalty.

Question 9

Your Azure OpenAI deployment is experiencing high latency during peak hours. You observe that input tokens are consistently high. Which strategy is most effective for reducing latency while maintaining quality?

Accepted Answer

Implement prompt caching for static instructions.. Prompt caching stores the KV cache for static prompt prefixes (system instructions, few-shot examples) so the model doesn't recompute attention over those tokens on every request. When input tokens are consistently high and dominated by static content, caching eliminates redundant prefill computation, directly reducing time-to-first-token and overall latency without changing output quality.

Answer

Increase the top_p parameter.

Answer

Enable dynamic scaling in Azure App Service.

Answer

Decrease the number of parallel requests.

Answer

Use a higher model version (e.g., o1-preview).

Question 10

Which feature in Azure OpenAI allows you to reserve throughput for a consistent user experience during high demand?

Accepted Answer

Provisioned Throughput Units (PTUs).. Provisioned Throughput Units (PTUs) let you reserve a fixed amount of model processing capacity on an Azure OpenAI deployment, guaranteeing consistent latency and throughput even under high demand. Unlike standard (pay-as-you-go) deployments that share a regional pool and can be throttled, PTUs give you dedicated capacity measured in units of tokens-per-minute equivalent. This is the feature designed specifically for predictable, high-volume workloads.

Answer

Regional failover.

Answer

Auto-scaling groups.

Answer

Token throttling.

Question 11

You notice that your fine-tuned model is 'forgetting' base capabilities after training on a small dataset. What strategy should you use to mitigate this?

Accepted Answer

Include a portion of the original training data during fine-tuning.. Catastrophic forgetting occurs when a model fine-tuned on a small dataset loses its general capabilities. To mitigate this, you should include a portion of the original training data (or a diverse set of general data) during fine-tuning so the model retains its base knowledge while learning the new task. This is a common technique in continual learning.

Answer

Increase the fine-tuning learning rate.

Answer

Change the model architecture to GPT-4.

Answer

Reduce the batch size to 1.

Question 12

You are configuring a chat application. What is the benefit of enabling streaming in the Azure OpenAI API?

Accepted Answer

Reduces time-to-first-token (TTFT) perception for the end-user.. Streaming improves perceived latency by delivering tokens as they are generated.

Answer

Increases the model's accuracy.

Answer

Removes the need for a system prompt.

Answer

Decreases the total number of tokens consumed.

Question 13

You are debugging a prompt that is performing poorly on edge cases. You decide to use a 'Chain-of-Thought' approach. Why does this improve performance?

Accepted Answer

It enables the model to break down complex logic into sequential steps.. Chain-of-Thought (CoT) prompting instructs the model to produce intermediate reasoning steps before the final answer, which improves accuracy on multi-step logic, arithmetic, and edge cases by giving the model more compute and context to work through ambiguity. This sequential decomposition is why CoT helps on problems where a direct answer would skip critical reasoning.

Answer

It bypasses the safety alignment layer.

Answer

It forces the model to use more input tokens, reducing costs.

Answer

It reduces the context window size.

Question 14

Which technique is most appropriate for optimizing RAG performance when the vector database returns too much noisy information?

Accepted Answer

Implementing a re-ranking stage.. A re-ranking stage applies a more expensive cross-encoder or semantic ranker to the initial candidate set returned by the vector database, reordering results by true relevance to the query. This filters out semantically similar but contextually irrelevant chunks that vector similarity alone cannot distinguish, directly addressing the noisy-retrieval problem. It is the standard second-stage pattern in production RAG pipelines.

Answer

Changing the embedding model to a smaller one.

Answer

Increasing the number of chunks retrieved.

Answer

Reducing the temperature to 0.

Question 15

You want to evaluate your prompt engineering changes quantitatively. Which method is most reliable for comparing two prompt versions?

Accepted Answer

Running the prompts against a benchmark evaluation dataset.. Running both prompt versions against a benchmark evaluation dataset produces quantitative, reproducible metrics (accuracy, F1, BLEU, groundedness, etc.) that allow direct statistical comparison. This removes subjectivity and isolates the effect of the prompt change, which is the only reliable way to determine whether one prompt outperforms another.

Answer

Monitoring the total request count.

Answer

Asking developers for subjective feedback.

Answer

Checking if the model is running on GPT-4.

Question 16

An application is hitting rate limits on the Azure OpenAI service. Which action is the most standard approach for handling this in production?

Accepted Answer

Implement exponential backoff in the client application.. The standard production approach for handling rate limits (HTTP 429) from Azure OpenAI is to implement retry logic with exponential backoff in the client application. Azure OpenAI returns a 429 status code with a Retry-After header when the tokens-per-minute (TPM) or requests-per-minute (RPM) quota is exceeded, and the client should wait and retry with progressively longer delays. This is the documented Microsoft-recommended pattern and is also built into the Azure OpenAI SDKs via retry policies.

Answer

Delete the deployment and recreate it.

Answer

Disable the rate limit in the portal.

Answer

Increase the temperature of the model.

Question 17

You are optimizing a model for a specific domain language. Which fine-tuning approach minimizes cost while maximizing domain adaptation?

Accepted Answer

Using LoRA (Low-Rank Adaptation).. LoRA (Low-Rank Adaptation) freezes the pretrained model weights and injects small trainable low-rank matrices into attention layers, so only a tiny fraction of parameters are updated. This dramatically reduces GPU memory, compute, and storage costs compared to full fine-tuning while still adapting the model to domain-specific language. It is the standard cost-efficient fine-tuning technique for domain adaptation.

Answer

Training the model from scratch.

Answer

Prompt engineering alone.

Answer

Full parameter fine-tuning.

Question 18

You notice your model is outputting redundant information. Which parameter specifically targets the penalty for repeating tokens?

Accepted Answer

frequency_penalty. The frequency_penalty parameter applies a penalty proportional to how many times a token has already appeared in the generated text, directly discouraging repeated tokens and reducing redundancy. Its value ranges from -2.0 to 2.0, with positive values increasing the penalty for frequent tokens.

Answer

max_tokens

Answer

presence_penalty

Answer

top_p

Question 19

You have a large set of documents for a RAG system. How should you optimize retrieval speed?

Accepted Answer

Store embeddings in a dedicated vector database with indexing.. Storing embeddings in a dedicated vector database with an ANN index (HNSW, IVF) enables sub-linear nearest-neighbor search instead of scanning every document. This is the standard architecture for fast retrieval over large corpora in RAG systems. The index precomputes a graph or cluster structure so queries touch only a small fraction of vectors.

Answer

Process all documents in the prompt.

Answer

Increase the model's max_tokens.

Answer

Perform a brute-force search on all documents.

Question 20

A user wants to restrict the model's output to valid JSON format. What is the most effective way to ensure this?

Accepted Answer

Enable JSON mode/Response Format in the API configuration.. Enabling JSON mode (Response Format) in the API configuration is the most effective way to constrain model output to valid JSON because the model is instructed and, in supported models, constrained to emit syntactically valid JSON. This is a first-class API feature (response_format: { type: 'json_object' }) that enforces structure at generation time. Prompt instructions alone are probabilistic, and post-processing only fixes output after the fact.

Answer

Increase the temperature.

Answer

Add 'output must be JSON' to the prompt.

Answer

Post-process the output with a regex script.