AI-300 Generative AI Optimization — 25 Questions

Question 1

You are developing a RAG pipeline and notice that the model often hallucinates when retrieving documents with low relevance. Which prompt engineering technique should you implement to improve grounding?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Implement Chain-of-Thought reasoning with negative constraints.. Few-shot prompting with explicit negative constraints helps the model understand the boundaries of its knowledge base.

Answer

Disable the system message.

Answer

Increase the temperature parameter to 1.0.

Answer

Switch from GPT-4o to GPT-3.5 Turbo.

Question 2

You are fine-tuning a model on Azure OpenAI and notice the training loss curve is fluctuating significantly. What is the most likely cause?

Accepted Answer

The learning rate is too high.. High learning rate in fine-tuning often causes divergence and loss fluctuations.

Answer

The model's temperature is set too high.

Answer

The training dataset size is too large.

Answer

The model lacks enough epochs.

Question 3

Which parameter in the Azure OpenAI API should be adjusted to make the model's output more deterministic and repeatable?

Accepted Answer

temperature. Lowering the temperature parameter reduces randomness, making outputs more deterministic.

Answer

max_tokens

Answer

presence_penalty

Answer

frequency_penalty

Question 4

To optimize costs for an enterprise chatbot, you want to implement token usage monitoring. Which Azure service should you integrate to track token consumption per user?

Accepted Answer

Azure Monitor with Log Analytics.. Azure Monitor logs can be used to track and analyze usage metrics exported from Azure OpenAI instances.

Answer

Azure Data Factory.

Answer

Azure Cognitive Search.

Answer

Azure SQL Database.

Question 5

You are optimizing a long-context application. Which technique is most effective for reducing context window costs in Azure OpenAI?

Accepted Answer

Summarizing conversation history before passing to the next prompt.. Summarizing previous turns in a conversation history reduces the number of tokens sent in each request, lowering costs.

Answer

Using a smaller batch size.

Answer

Disabling streaming responses.

Answer

Increasing the frequency penalty.

Question 6

Your Azure OpenAI deployment is experiencing high latency during peak hours. You observe that input tokens are consistently high. Which strategy is most effective for reducing latency while maintaining quality?

Accepted Answer

Implement prompt caching for static instructions.. Prompt caching and reduction of redundant input tokens are primary methods for reducing latency in high-traffic deployments.

Answer

Increase the top_p parameter.

Answer

Enable dynamic scaling in Azure App Service.

Answer

Decrease the number of parallel requests.

Answer

Use a higher model version (e.g., o1-preview).

Question 7

Which feature in Azure OpenAI allows you to reserve throughput for a consistent user experience during high demand?

Accepted Answer

Provisioned Throughput Units (PTUs).. Provisioned Throughput Units (PTUs) provide dedicated capacity for predictable performance.

Answer

Regional failover.

Answer

Auto-scaling groups.

Answer

Token throttling.

Question 8

You notice that your fine-tuned model is 'forgetting' base capabilities after training on a small dataset. What strategy should you use to mitigate this?

Accepted Answer

Include a portion of the original training data during fine-tuning.. Rehearsal or mixing in base data (catastrophic forgetting prevention) preserves core capabilities.

Answer

Increase the fine-tuning learning rate.

Answer

Change the model architecture to GPT-4.

Answer

Reduce the batch size to 1.

Question 9

You are configuring a chat application. What is the benefit of enabling streaming in the Azure OpenAI API?

Accepted Answer

Reduces time-to-first-token (TTFT) perception for the end-user.. Streaming improves perceived latency by delivering tokens as they are generated.

Answer

Increases the model's accuracy.

Answer

Removes the need for a system prompt.

Answer

Decreases the total number of tokens consumed.

Question 10

You are debugging a prompt that is performing poorly on edge cases. You decide to use a 'Chain-of-Thought' approach. Why does this improve performance?

Accepted Answer

It enables the model to break down complex logic into sequential steps.. CoT forces the model to generate intermediate reasoning steps, which improves logic in complex tasks.

Answer

It bypasses the safety alignment layer.

Answer

It forces the model to use more input tokens, reducing costs.

Answer

It reduces the context window size.

Question 11

Which technique is most appropriate for optimizing RAG performance when the vector database returns too much noisy information?

Accepted Answer

Implementing a re-ranking stage.. Re-ranking filters top retrieved results to ensure high-quality context is passed to the LLM.

Answer

Changing the embedding model to a smaller one.

Answer

Increasing the number of chunks retrieved.

Answer

Reducing the temperature to 0.

Question 12

You want to evaluate your prompt engineering changes quantitatively. Which method is most reliable for comparing two prompt versions?

Accepted Answer

Running the prompts against a benchmark evaluation dataset.. A/B testing with a ground-truth dataset allows for objective measurement of performance changes.

Answer

Monitoring the total request count.

Answer

Asking developers for subjective feedback.

Answer

Checking if the model is running on GPT-4.

Question 13

An application is hitting rate limits on the Azure OpenAI service. Which action is the most standard approach for handling this in production?

Accepted Answer

Implement exponential backoff in the client application.. Implementing exponential backoff is the standard architectural pattern for handling rate-limited API responses.

Answer

Delete the deployment and recreate it.

Answer

Disable the rate limit in the portal.

Answer

Increase the temperature of the model.

Question 14

You are optimizing a model for a specific domain language. Which fine-tuning approach minimizes cost while maximizing domain adaptation?

Accepted Answer

Using LoRA (Low-Rank Adaptation).. Low-Rank Adaptation (LoRA) allows for efficient fine-tuning by updating only a small subset of model parameters.

Answer

Training the model from scratch.

Answer

Prompt engineering alone.

Answer

Full parameter fine-tuning.

Question 15

You notice your model is outputting redundant information. Which parameter specifically targets the penalty for repeating tokens?

Accepted Answer

frequency_penalty. The frequency_penalty parameter is explicitly designed to reduce the probability of tokens that have already appeared.

Answer

max_tokens

Answer

presence_penalty

Answer

top_p

Question 16

You have a large set of documents for a RAG system. How should you optimize retrieval speed?

Accepted Answer

Store embeddings in a dedicated vector database with indexing.. Using a managed vector store with optimized indexing is essential for large-scale retrieval speed.

Answer

Process all documents in the prompt.

Answer

Increase the model's max_tokens.

Answer

Perform a brute-force search on all documents.

Question 17

A user wants to restrict the model's output to valid JSON format. What is the most effective way to ensure this?

Accepted Answer

Enable JSON mode/Response Format in the API configuration.. Using the 'Response Format' feature in Azure OpenAI enforces structured output, reducing parsing errors.

Answer

Increase the temperature.

Answer

Add 'output must be JSON' to the prompt.

Answer

Post-process the output with a regex script.

Question 18

Which THREE factors contribute to increased latency in an LLM application?

Accepted Answer

The size/complexity of the chosen model.. Model size, input/output token count, and network transit time are the main drivers of latency.

Answer

The number of system messages used.

Answer

The use of JSON output mode.

Question 19

Which TWO techniques should you use to improve the accuracy of a RAG pipeline?

Accepted Answer

Upgrading to a more advanced embedding model.. Chunking strategy and better embedding models directly impact the quality of retrieved data.

Answer

Reducing the number of documents in the vector store.

Answer

Increasing the temperature to 2.0.

Answer

Using only one single long chunk for all documents.

Question 20

Which THREE steps are necessary to successfully fine-tune an Azure OpenAI model?

Accepted Answer

Prepare and upload a training dataset in JSONL format.. Data preparation, selecting the base model, and managing the fine-tuning job are fundamental steps.

Answer

Manually update the model weights in the code.

Answer

Disable the input filter in the portal.