Select Two (Multi-Select) Questions

Pre-compute embeddings for all documents

Pre-computed embeddings avoid real-time embedding calls during query.

Use approximate nearest neighbor search

ANN is faster than exact search, reducing vector search latency.

Use a smaller embedding model

Smaller models have lower inference latency for embedding creation.

Ability to generate plausible-sounding text

Correct: Fluency can mask inaccuracies.

Lack of real-world grounding

Correct: Without grounding, models may invent facts.

Gaps in training data coverage

Correct: Missing information leads to guesswork.

OCI Object Storage bucket

Object Storage is a common source for large datasets.

Local file uploaded through the OCI Console

The console allows uploading smaller files directly.

The softmax function is used in the attention mechanism to normalize attention scores.

Softmax converts attention scores into probabilities.

Positional encodings are necessary because the model is not recurrent.

Without recurrence, positional info must be added via encodings.

The self-attention layer allows the model to weigh the importance of different tokens.

Self-attention computes attention weights that determine token importance.

Self-attention mechanism

Correct: Core component of Transformers.

Feed-forward neural network

Correct: Essential part of Transformer blocks.

Temperature scaling

Temperature scaling smooths token probabilities and can improve the quality-diversity trade-off.

Top-k sampling

Top-k sampling limits choices to the k most likely tokens, improving quality.

Beam search

Beam search explores multiple possible sequences to find the best output.

Using the OCI Console chat interface

The console provides a chat UI to interact with models.

Using the OCI Generative AI REST API

The REST API is the primary method for programmatic access.

OCI Logging for audit

Logging is critical for monitoring and compliance.

Dedicated AI cluster

Essential for hosting the model with predictable performance.

OCI IAM policies for access control

IAM policies are fundamental for securing the deployment.

Top_p

Top_p controls the nucleus of tokens considered; lower values make output more focused.

Temperature

Higher temperature increases randomness, lower makes output more deterministic.

Test the prompt with a variety of input examples

Testing ensures robustness across different inputs.

Define the desired output format and constraints

Clear objectives guide prompt design.

Iteratively refine the prompt based on model responses

Iteration is key to effective prompt engineering.

Size of available dataset

Fine-tuning requires a sufficiently large dataset; prompt engineering can work with few examples.

Available budget for compute resources

Fine-tuning incurs training costs; prompt engineering is typically cheaper.

Need for domain-specific terminology

Fine-tuning can embed domain knowledge; prompt engineering may require careful crafting.

Llama 3

Meta's Llama 3 is available in OCI GenAI.

Cohere Command

Cohere Command is a text generation model available in OCI GenAI.

Cohere Embed

Cohere Embed is an embedding model available for vector generation.

Incorrect endpoint URL

A wrong URL may cause the request to be malformed or routed incorrectly, resulting in a 400.

Missing required parameters in the request body

Required fields like 'modelId' or 'prompt' are omitted.

Exceeding the model's maximum token limit

The input or output exceeds the model's token limit, causing a bad request.

Set up OCI IAM policies to prevent data egress from the region for the model's resources

Correct: IAM policies can restrict access to resources from outside the region.

Deploy the dedicated AI cluster in the region that meets data residency requirements

Correct: The cluster's location determines where data is processed.

OCI Logging for inference requests and responses

Correct: Logging allows auditing and debugging of inference calls.

OCI Monitoring metrics for endpoint latency and error rates

Correct: Metrics drive performance dashboards and alerts.

OCI Application Performance Monitoring (APM) for tracing inference requests

Correct: APM provides distributed tracing to diagnose performance bottlenecks.

OCIGenAIEmbeddings

This wraps OCI's embedding models.

Vector store (e.g., FAISS, OracleVS)

Stores embeddings for retrieval.

An inference script (e.g., score.py) to handle prediction requests

Required to define how the model is called.

A model artifact containing the model files

Required for the model to be deployed.

A deployment configuration specifying resources and environment

Defines compute shape, replicas, etc.

Create a model artifact (e.g., pickle, ONNX) with inference code

Model must be packaged with dependencies for serving.

Upload the model artifact to an OCI Object Storage bucket

Model Deployment loads the artifact from Object Storage.

Create a model deployment using the OCI Data Science Model Deployment service

This creates the endpoint.

Use preemptible instances for non-critical inference workloads

Preemptible instances are cheaper and suitable for fault-tolerant tasks.

Use autoscaling to adjust nodes based on demand

Matches capacity to actual usage, reducing cost.

Number of concurrent requests

Dedicated clusters are better for high concurrency due to reserved capacity.

Latency requirements

Dedicated clusters offer lower and more predictable latency.