Vertex AI and Generative AI

Vertex AI is Google Cloud's unified machine learning (ML) platform. It integrates data preparation, model training, deployment, and monitoring into a single service. For the GCDL exam, you need to understand that Vertex AI abstracts away much of the infrastructure complexity, allowing organizations to build, deploy, and scale ML models faster. Generative AI refers to models that generate new content – text, images, code, audio, video – based on prompts. Vertex AI provides access to Google's foundation models (PaLM 2, Gemini, Imagen) and tools to customize them.

Vertex AI Studio: A web-based interface for prototyping and testing generative AI models. It includes a prompt library, model comparison tools, and safety settings. The exam tests that Studio is used for early experimentation, not production deployment.

Model Garden: A curated repository of foundation models, including Google's own (PaLM 2 for text, Gemini for multimodal, Codey for code, Imagen for images) and third-party models (e.g., Anthropic's Claude, open-source models). Model Garden provides model cards with details on capabilities, limitations, and pricing. The exam emphasizes that Model Garden is the central place to discover and select models.

Generative AI Studio (part of Vertex AI Studio): Specifically for generative AI workflows. It includes: - Prompt Designer: For text prompts with parameters like temperature (0-1), top-k, top-p, and max output tokens. - Tuning: For supervised fine-tuning of foundation models using your own labeled data. The exam tests that fine-tuning is for adapting a model to a specific task (e.g., summarization of medical documents) and requires a dataset of at least 100 examples. - Safety Filters: Configurable thresholds to block harmful content based on categories like hate speech, harassment, sexually explicit, and dangerous content. Each category has a slider from 1 (most restrictive) to 5 (least restrictive). The exam expects you to know the default is 2 for most categories and that you can adjust per use case.

AutoML: For training custom models without writing code. It automates architecture search, hyperparameter tuning, and feature engineering. AutoML supports tabular, image, video, and text data. The exam tests that AutoML is appropriate when you have labeled data but lack ML expertise, and that it is not for generative AI (which uses foundation models).

Custom Training: For experienced ML engineers who want full control using TensorFlow, PyTorch, or scikit-learn. Vertex AI provides managed notebooks, distributed training, and hyperparameter tuning jobs. The exam tests that custom training is the most flexible but requires more effort.

The workflow for generative AI on Vertex AI is: 1. Select a foundation model from Model Garden. For example, for text generation, you might choose PaLM 2 for Bison (text-bison@002) or Gemini Pro (gemini-pro). 2. Prototype in Generative AI Studio – write prompts, adjust parameters, test with sample inputs. 3. Optional: Tune the model using your own dataset. This creates a tuned model version that you can deploy. 4. Deploy the model to an endpoint for serving predictions. Vertex AI handles scaling, load balancing, and monitoring. 5. Integrate via the Vertex AI SDK or REST API.

Important exam details: - Temperature: Controls randomness. Low temperature (0.1) makes output more deterministic; high (0.9) makes it more creative. Default is 0.0 for PaLM 2 text models. - Max output tokens: Limits response length. Default is 256 for text-bison@002, max 1024. - Top-k and top-p: Token sampling strategies. Top-k picks from the k most likely tokens; top-p picks from tokens with cumulative probability p. Default top-p is 0.95. - Context window: The number of tokens the model can consider. For Gemini Pro, it's 32,768 tokens (about 25,000 words). For PaLM 2 Bison, it's 8,192 tokens.

Vertex AI enforces safety filters at the API level. The exam tests that you can configure thresholds per request or per project. Categories and default thresholds: - Hate speech: Default threshold 2 - Harassment: Default threshold 2 - Sexually explicit: Default threshold 2 - Dangerous content: Default threshold 2

Thresholds range from 1 (most restrictive) to 5 (least restrictive). If the model's response exceeds the threshold, it returns a safety error. You can also set blocklists for specific words or phrases.

The exam tests that you understand these integrations, especially that BigQuery is used for large-scale data analysis and Vertex AI for ML modeling.

A large e-commerce company wants to build a chatbot to handle customer inquiries about orders, returns, and product information. They have a large dataset of historical support tickets. Using Vertex AI, they select the PaLM 2 for Chat model (chat-bison@002) from Model Garden. They prototype in Generative AI Studio, crafting prompts that include examples of good responses. They fine-tune the model on their support ticket data (10,000 examples) to improve domain-specific accuracy. After tuning, they deploy the model to an endpoint with autoscaling (min 2, max 10 nodes) to handle variable traffic. They integrate with their existing web app via the Vertex AI API. Safety filters are set to threshold 2 for all categories to prevent offensive responses. A common misconfiguration is setting temperature too high (e.g., 0.9), causing the chatbot to give creative but incorrect answers. The correct range is 0.1-0.3 for factual responses. The exam might ask about the appropriate model (chat-bison vs text-bison) or the need for fine-tuning.

A healthcare provider needs to summarize lengthy medical reports into concise summaries for doctors. They use Vertex AI's PaLM 2 for Text (text-bison@002). They cannot fine-tune due to data privacy concerns (PHI), so they rely on prompt engineering with few-shot examples. They set temperature to 0.0 for deterministic output. Max output tokens is set to 500 to limit summary length. Safety filters are critical: they set the 'dangerous content' threshold to 1 (most restrictive) because medical advice must be accurate. They deploy the model to a private endpoint with VPC-SC (Service Controls) for data isolation. The exam tests that fine-tuning is not always possible due to data sensitivity, and that prompt engineering with safety filters is a valid alternative.

A marketing agency uses Vertex AI's Imagen to generate product images. They select Imagen from Model Garden. In Generative AI Studio, they write prompts like 'A modern office chair with a sleek design, photorealistic, 4K'. They adjust parameters like aspect ratio (1:1, 4:3, 16:9) and number of images (1-8). Safety filters are set to threshold 3 for 'sexually explicit' to allow some artistic nudity but block explicit content. They deploy the model via an endpoint and integrate with their content management system. A common mistake is using a text model for image generation – the exam tests that you must use Imagen for images.