Generative AI Leader Techniques to Improve Generative AI Model Output — All Questions With Answers

Refer to the exhibit.

```
# Model configuration before change
model = GenerativeModel("text-bison@002")
response = model.generate(
    prompt="Summarize the following article: ...",
    temperature=0.7,
    top_k=40,
    top_p=0.95
)
# After change
model = GenerativeModel("text-bison@002")
response = model.generate(
    prompt="Summarize the following article: ...",
    temperature=0.2,
    top_k=10,
    top_p=0.85
)
```

Refer to the exhibit.

```
# Vertex AI Endpoint configuration
{
  "model": "gemini-1.5-pro",
  "endpoint": "projects/my-project/locations/us-central1/endpoints/123456789",
  "deployedModel": {
    "modelVersion": "1",
    "minReplicaCount": 1,
    "maxReplicaCount": 5,
    "autoscalingMetricSpecs": [
      {
        "metricName": "custom.googleapis.com|genai|request_count",
        "target": 100
      }
    ]
  }
}
```

gcloud ai tuning-jobs create \
  --project=my-project \
  --region=us-central1 \
  --model=gemini-1.5-pro-001 \
  --tuned-model-display-name=test-tune \
  --training-data=gs://my-bucket/data.jsonl \
  --model-serving-regions=us-east1

{
  "dedicatedEndpoints": 1,
  "machineType": "n1-standard-2",
  "minReplicaCount": 1,
  "maxReplicaCount": 5,
  "scaleTarget": 0.5
}

User: Translate the following sentence to French: Hello, how are you?
Model: Bonjour, comment vas-tu?
Expected: Bonjour, comment allez-vous? (formal)

Model ID: 1234567890
Display Name: qa-chat-v1
State: DEPLOYING
Errors: []
Training Pipeline: projects/123/locations/us-central1/trainingPipelines/abc
Evaluation Metrics: {}

{
  "bindings": [
    {
      "role": "roles/aiplatform.user",
      "members": ["user:dev@example.com"]
    },
    {
      "role": "roles/aiplatform.modelUser",
      "members": ["user:dev@example.com"]
    }
  ]
}

{
  "instances": [{"prompt": "Write a poem about AI."}],
  "parameters": {
    "temperature": 0.0,
    "maxOutputTokens": 256,
    "topP": 1.0,
    "topK": 40
  }
}

Refer to the exhibit.
```
{
  "model": "gemini-1.5-pro",
  "parameters": {
    "temperature": 0.9,
    "topK": 40,
    "topP": 0.95,
    "maxOutputTokens": 256,
    "safetySettings": [
      {"category": "HARM_CATEGORY_DANGEROUS_CONTENT", "threshold": "BLOCK_MEDIUM_AND_ABOVE"}
    ]
  }
}
```

Refer to the exhibit.
```
Error: Model output quality has degraded over time.
Cloud Monitoring metrics show:
- Prediction latency stable
- Error rate less than 1%
- Input token count per request increasing 10% weekly
```

Refer to the exhibit.
```
System Instruction: You are a helpful assistant.
Prompt: Tell me about the Eiffel Tower.
Response: The Eiffel Tower is located in Paris, France. It is 330 meters tall.
```

Refer to the exhibit. The following is a partial output from 'gcloud ai models list' command:
---
MODEL_ID: 123456789 
DISPLAY_NAME: my-summary-model
MODEL_REGISTRY: vertex-ai
SUPPORT_ENGINE: False
GROUNDING_CONFIG: NONE
---

Refer to the exhibit. The following is a configuration block from a Vertex AI PaLM API request:
{
  "instances": [{"context": "You are a helpful assistant.", "messages": [{"author": "user", "content": "Explain quantum computing"}]}],
  "parameters": {
    "temperature": 0.9,
    "maxOutputTokens": 1000,
    "topK": 40,
    "topP": 0.95,
    "candidateCount": 1
  }
}

Refer to the exhibit. The following is an error log snippet from a Vertex AI generative AI deployment:
ERROR: Response blocked due to safety filter: Blocked categories: [VIOLENCE, SEXUAL]. Input tokens: 150. Output tokens: 0.
A developer is building a chatbot for a medical application that discusses sensitive health topics. The chatbot consistently gets its outputs blocked.

Refer to the exhibit.

```json
{
  "model": "publishers/google/models/chat-bison@001",
  "endpoint": "us-central1-aiplatform.googleapis.com",
  "parameters": {
    "temperature": 0.9,
    "topP": 0.95,
    "maxOutputTokens": 256,
    "groundingConfig": {
      "sources": []
    }
  },
  "deployment": "production"
}
```