A developer is using OCI Generative AI to build a question-answering system over a large corpus of technical manuals. The developer uses the Cohere Embed model to generate embeddings and stores them in an OCI OpenSearch cluster. Queries are slow and the team needs to reduce latency. Which approach is BEST for improving search speed while maintaining acceptable accuracy?
Fewer neighbors means less distance computation and faster retrieval.
Why this answer
Reducing the k value in the nearest neighbor search directly decreases the number of vectors that must be compared during query time, which lowers latency. In approximate nearest neighbor (ANN) search, a smaller k means fewer candidates are evaluated, speeding up retrieval while still maintaining acceptable accuracy if the original k was unnecessarily high. This is the most effective tuning knob for latency in vector search systems like OCI OpenSearch with Cohere embeddings.
Exam trap
The trap here is that candidates often confuse reducing k with reducing accuracy, but in practice, many RAG systems use a k value larger than necessary, and reducing it to a reasonable minimum (e.g., from 20 to 5) can dramatically improve speed without noticeable quality loss.
How to eliminate wrong answers
Option A is wrong because increasing the embedding dimension increases the computational cost of distance calculations and memory usage, which would worsen latency, not improve it. Option B is wrong because exact nearest neighbor search (k-NN) requires scanning all vectors, which is O(n) and significantly slower than approximate methods, especially on large corpora. Option D is wrong because increasing the index refresh interval reduces write overhead but does not affect query latency; it only delays the visibility of new documents.