Fundamentals of Generative AI practice questions

Change the accelerator to TPU

Why wrong: TPU still has memory limits; reducing batch size is more direct.

Use a larger model

Why wrong: Larger models require more memory.

Increase the batch size

Why wrong: Increasing batch size increases memory usage.

Switch from Vertex AI Search to a different vector database like Pinecone.

Why wrong: Switching to a different vector database like Pinecone does not address the core issue of low-relevance retrieval; the quality of similarity search depends on the embedding and threshold, not the database vendor.

Reduce the model temperature to 0.0 to make outputs more deterministic.

Why wrong: Reducing temperature to 0.0 increases determinism but does not fix hallucinations caused by irrelevant or incorrect context; the model may still confidently generate wrong answers based on poor retrieval.

Increase the top-K retrieval value to 10 to provide more context to the model.

Why wrong: Increasing top-K to 10 adds more chunks, but if those additional chunks have low similarity, they introduce irrelevant or conflicting information, potentially worsening hallucinations instead of improving accuracy.

Create a training pipeline

Why wrong: Incorrect. Creating a training pipeline requires commands like `gcloud ai pipelines run` or `gcloud ai training jobs submit`, not `gcloud ai models upload`.

Deploy a model to an endpoint

Why wrong: Incorrect. Deploying a model to an endpoint uses `gcloud ai endpoints deploy-model`, not `gcloud ai models upload`.

Train a model

Why wrong: Incorrect. Training a model uses `gcloud ai training jobs submit` or custom training jobs, not `gcloud ai models upload`.

BigQuery ML

Why wrong: BigQuery ML is for SQL-based ML on tabular data.

Cloud Console Compute Engine

Why wrong: Compute Engine provides VMs, not a model playground.

Vertex AI Model Registry

Why wrong: Model Registry is for versioning, not experimentation.

The endpoint is misconfigured and will cause conflicts

Why wrong: Vertex AI supports multiple models per endpoint; this is not a misconfiguration.

The models are from different frameworks

Why wrong: Different frameworks do not prevent co-deployment on the same endpoint, but the exhibit does not indicate this.

It is a batch prediction endpoint

Why wrong: Batch prediction uses a single model; multiple models indicate online prediction with traffic splitting.

Too few training steps

Why wrong: Number of steps does not cause OOM.

Batch size too large

Why wrong: While possible, the machine type is the primary consideration; batch size can be adjusted.

Dataset too small

Why wrong: Dataset size does not cause OOM during training.

Increased model accuracy

Why wrong: Increased accuracy is a benefit, not a risk.

Include a system prompt that instructs the model to never generate code using internal APIs.

Why wrong: Prompt instructions are not reliably followed, especially under adversarial inputs.

Use a retriever to fetch policy documents and prepend them to each prompt.

Why wrong: RAG does not guarantee the model will adhere to policy during generation.

Train a separate classifier to rerank model outputs and reject non-compliant generations.

Why wrong: A classifier can filter outputs but is less integrated than fine-tuning; it may miss subtle violations.

Fine-tune a small model exclusively on legal contracts from a single jurisdiction and use it for generation.

Why wrong: Small model may lack capacity; also fails to adapt to multiple jurisdictions.

Implement retrieval-augmented generation (RAG) with a vector database of all relevant laws.

Why wrong: RAG provides references but does not enforce that the generated text is compliant.

Use a large instruction-tuned model with carefully engineered prompts describing jurisdiction details.

Why wrong: Prompting is not sufficient for guaranteeing legal accuracy.

Enable model versioning to automatically roll back to a previous version

Why wrong: Versioning helps with deployment stability but does not correct factual errors in responses.

Fine-tune the model on a dataset of product images and descriptions

Why wrong: Fine-tuning on images does not directly improve factual text accuracy; it may not address the root cause of hallucinations.

Increase the temperature parameter to 0.9

Why wrong: Higher temperature increases randomness, which would likely increase factual errors.

Deploy the model on Spot VMs to reduce infrastructure costs

Why wrong: Spot VMs can be preempted, making them unsuitable for production customer-facing applications.

Store prompts in plain text files for easy version control

Why wrong: Prompts may contain sensitive information; they should be stored securely, not in plain text.

Use a single large model for all query types to maintain consistency

Why wrong: A single large model may be overkill and expensive; routing queries to smaller models can be more cost-effective.

Use the Codey API for text generation.

Why wrong: Codey is for code generation, not grounding documentation.

Use the base model without any grounding to maximize flexibility.

Why wrong: Lack of grounding increases hallucination risk.

Fine-tune the model on the documentation and deploy.

Why wrong: Fine-tuning alone does not guarantee real-time grounding.

The max_output_tokens is too low; increase it to allow more diverse output.

Why wrong: Max tokens controls length, not repetition.

The model is overfitted; switch to a smaller model.

Why wrong: Overfitting is unlikely in pre-trained models; repetition is a decoding issue.

The temperature is too low; increase temperature to add randomness.

Why wrong: Low temperature makes output more deterministic, increasing repetition.

Vertex AI Grounding with their compliance database.

Why wrong: Grounding improves accuracy but does not block specific phrases.

Prompt engineering with instructions to avoid guarantees.

Why wrong: Prompt engineering is not foolproof; safety filters are more robust.

Reinforcement learning from human feedback (RLHF) on the model.

Why wrong: RLHF is costly and not a real-time filter.

Increase the max_output_tokens parameter.

Why wrong: Length does not affect factual accuracy.

Increase the top_p parameter.

Why wrong: Higher top_p increases diversity, which can reduce accuracy.

Add a post-processing step to verify facts using a database.

Why wrong: This is not a parameter adjustment.

Increase the number of training epochs.

Why wrong: More epochs exacerbate overfitting.

Increase the learning rate.

Why wrong: Higher learning rate may cause divergence, not solve overfitting.

Switch to a smaller batch size.

Why wrong: Batch size affects convergence but not directly overfitting.

An embedding model (e.g., textembedding-gecko@001).

Why wrong: Embedding models produce vector representations, not text.

A chat model (e.g., chat-bison@001).

Why wrong: Chat models are optimized for multi-turn dialogue.

A code generation model (e.g., code-bison@001).

Why wrong: Code models are for code generation.

Use a system prompt instructing the model to avoid PII.

Why wrong: Prompt instructions are not always followed reliably.

Fine-tune the model on a dataset that excludes PII.

Why wrong: Fine-tuning does not guarantee PII filtering.

Manually review each output before sending.

Why wrong: Manual review is not scalable or real-time.

They are deterministic by design.

Why wrong: They use probabilistic generation.

They always produce the same output for the same input.

Why wrong: They are stochastic; output can vary.

They require no data for training.

Why wrong: Training requires large datasets.

Maximize model throughput to handle high volumes.

Why wrong: Throughput is secondary to safety.

Use the cheapest model to reduce costs.

Why wrong: Cost should not compromise quality or safety.