Hard Difficulty Questions

Switch to a different base model

Why wrong: Base model may not be the root cause if fine-tuning data is flawed.

Increase the cluster size

Why wrong: Cluster size affects performance, not accuracy.

Use a serverless endpoint

Why wrong: Endpoint type does not fix accuracy issues.

Batch size

Why wrong: Batch size affects memory but not per-token compute cost.

Quantization bits

Why wrong: Quantization reduces cost, not increases.

Dataset size

Why wrong: Dataset size affects total training time but not per-step cost.

Use early stopping based on training loss to avoid overfitting.

Why wrong: Early stopping on training loss may not address OOD issues.

Reduce the model size to prevent overfitting to the training data.

Why wrong: Smaller model has less capacity to learn generalizable features.

Increase the learning rate during fine-tuning to adapt faster to new patterns.

Why wrong: Higher learning rate can cause instability and catastrophic forgetting.

The CLI is not configured with OCI credentials

Why wrong: Missing credentials would give an authentication error, not NotAuthorizedOrNotFound.

The model ID is incorrectly formatted

Why wrong: An invalid format would likely cause a validation error, not a 404/403.

The model is in a different region than iad

Why wrong: Region mismatch would give a different error.

Switch to cohere.command-light model for faster inference and add a post-processing step using a BERT-based NER model to validate entities.

Why wrong: A lighter model may be faster but likely less accurate; post-processing NER helps but does not prevent hallucinations at generation time.

Increase max_tokens to 4096 and use chunked processing with overlapping context windows to provide more context.

Why wrong: Chunking with overlap may reduce hallucinations by providing more context, but increasing max_tokens increases latency and cost; the improvement might be marginal.

Enable the safety filter with strict content moderation and set up OCI Logging to audit all generations.

Why wrong: Safety filters block harmful content but do not reduce hallucinations about medical facts; auditing only detects issues after the fact.

Increase the temperature parameter of the generation model.

Why wrong: Higher temperature increases randomness, potentially worsening hallucinations.

Increase the number of retrieved chunks (k) to provide more context.

Why wrong: More context can include irrelevant or contradictory information.

Use a larger generative model with more parameters.

Why wrong: Larger models may still hallucinate; size alone does not guarantee faithful output.

The space type 'cosinesimil' is not supported; it should be 'cosine'.

Why wrong: 'cosinesimil' is valid.

The dimension 768 does not match the embedding model's output dimension.

Why wrong: 768 is a common dimension.

The mapping uses 'knn_vector' type with 'faiss' engine, which is incompatible.

Why wrong: faiss is a supported engine.

Increase the model deployment endpoint timeout setting from 60 seconds to 300 seconds in the OCI console.

Why wrong: Option C is wrong because increasing timeout only masks the symptom without addressing the root cause (insufficient capacity).

Upgrade the model deployment shape to VM.GPU.A100.4 and keep a single replica.

Why wrong: Option A is wrong because upgrading to a larger GPU (A100) increases compute power per request, but with only one replica, concurrency remains a bottleneck; scaling out is more effective for high concurrency.

Move the model deployment to the US East (Ashburn) region to leverage lower-cost GPU capacity and reduce latency.

Why wrong: Option B is wrong because moving to a different region increases data residency risk and may add latency.

Train separate models for each team

Why wrong: This does not enforce access control; teams could still access each other's models.

Encrypt model artifacts with different keys for each team

Why wrong: Encryption protects data at rest, but does not prevent authorized users from accessing the data.

Use network security lists to isolate traffic

Why wrong: Network security lists control traffic at the network level, not API-level access to models.

Allocate a dedicated compartment for generative AI resources to apply specific IAM policies

Why wrong: While good practice, it is more about access control than directly meeting data privacy regulations.

Store all inference inputs and outputs in a public bucket for transparency

Why wrong: Public buckets expose data, violating privacy.

The metric name is misspelled

Why wrong: A misspelled metric name would return no data for any model.

The aggregation interval is too short

Why wrong: A short interval would still produce data points for all models; it would not cause missing points for specific models.

The compartmentId is incorrect

Why wrong: An incorrect compartmentId would return data for a different compartment, potentially missing all models, not just some.

Required response time

Why wrong: Both approaches have similar response times; not a differentiator.

Internet connectivity

Why wrong: Both approaches require internet connectivity to access the service.

OCI Compute with custom Docker container and Prometheus monitoring

Why wrong: This requires manual setup for authentication, scaling, and monitoring, increasing operational overhead.

OCI Functions with API Gateway for authentication

Why wrong: OCI Functions have limited memory (max 2GB) and no native GPU support, unsuitable for a 20GB model.

OCI Data Flow with OCI Data Catalog for model registry

Why wrong: Data Flow is designed for batch Spark jobs, not real-time inference.

A local peering gateway

Why wrong: Local peering gateway is for VCN connectivity, not IAM.

A dynamic group

Why wrong: Dynamic groups are used for resources, not users.

Expose the model endpoint to the internet for ease of access

Why wrong: Public exposure increases security risk and is not recommended for internal apps.

Disable audit logging to minimize storage costs

Why wrong: Audit logging is essential for security compliance and monitoring.

The learning rate is too high.

Why wrong: A high learning rate typically causes divergent behavior and both losses would increase or oscillate, not the pattern described.

The validation set is not representative.

Why wrong: An unrepresentative validation set could show poor performance even with good generalization, but the training loss decrease is consistent with overfitting, not just a mismatch.

The training data is too small.

Why wrong: Small data can lead to overfitting, but the immediate symptom of training loss decreasing while validation does not improve is directly indicative of overfitting, not necessarily the root cause.

Top_p = 0.1, frequency_penalty = 0.5

Why wrong: Top_p = 0.1 severely limits the pool of possible tokens, often making output repetitive despite the penalty.

Top_p = 0.9, frequency_penalty = 0.0

Why wrong: Without any frequency penalty, the model has no incentive to avoid repetition.

Top_p = 1.0, frequency_penalty = 0.0

Why wrong: Top_p = 1.0 includes all tokens (no nucleus sampling) and no penalty, so repetition is likely.

Low top-p

Why wrong: Low top-p reduces diversity but does not directly cause repetition of user messages.

High temperature

Why wrong: High temperature increases randomness but not necessarily repetition.

High frequency penalty

Why wrong: High frequency penalty reduces repetition, not causes it.

Dedicated fine-tuned endpoint per tenant.

Why wrong: Dedicated endpoints are costly and do not leverage shared infrastructure.

On-premises deployment of open-source models.

Why wrong: On-premises deployment shifts operational burden and may not provide the same scalability or integration with OCI.

Single large fine-tuned model with conditional logic.

Why wrong: A single model with conditional logic is prone to prompt injection and lacks proper data isolation.

Use a general-purpose LLM like Llama 2.

Why wrong: Incorrect: General models are less optimized for code.

Use a multimodal model.

Why wrong: Incorrect: Multimodal models handle images and text, not code-specific.

Use an embedding model for text.

Why wrong: Incorrect: Embedding models are for similarity, not generation.