What techniques are used for contextual retrieval?

Contextual retrieval in Retrieval-Augmented Generation (RAG) refers to the methods used to fetch relevant information from a knowledge base that is semantically aligned with the user's query. The goal is to provide highly pertinent and diverse context to the Language Model (LLM) to generate accurate and informed responses, reducing hallucinations and improving factual consistency. Various techniques are employed, often in combination, to achieve robust contextual retrieval.

Core Retrieval Mechanisms

1. Vector Search (Semantic Search)

This is a primary method where both the user query and the chunks in the knowledge base are converted into high-dimensional numerical vectors (embeddings) using a deep learning model. Retrieval then involves finding document chunks whose embeddings are 'closest' or most similar to the query embedding in the vector space, typically using similarity metrics like cosine similarity. This technique excels at capturing semantic meaning and synonyms.

• Embedding Models: Sentence-BERT, OpenAI Embeddings, Cohere Embeddings, Instructor models.
• Vector Databases/Libraries: FAISS, Annoy, Pinecone, Weaviate, Chroma, Milvus, Qdrant.

2. Keyword Search (Lexical Search)

Also known as lexical search, this technique focuses on finding exact or fuzzy matches of keywords between the query and the documents. It's highly effective for specific factual queries where keywords are clearly present in the relevant documents.

• Algorithms: BM25 (Okapi BM25), TF-IDF (Term Frequency-Inverse Document Frequency).
• Platforms: Elasticsearch, Solr, Lucene.

3. Hybrid Search

Combining vector and keyword search to leverage the strengths of both. Vector search handles semantic understanding, while keyword search ensures precise term matching. The results from both methods are often fused using techniques like Reciprocal Rank Fusion (RRF) to produce a more comprehensive and robust set of retrieved documents.

Contextual Enhancement Techniques

4. Reranking

After an initial set of top-K documents is retrieved by a primary retriever (e.g., vector search), a more sophisticated reranking model re-evaluates and reorders these documents. Rerankers typically use a more powerful (and often slower) model to assess the relevance of each retrieved chunk in relation to the original query, improving the precision of the final context fed to the LLM.

• Models: Cross-encoder models (e.g., BERT-based models fine-tuned for relevance), specialized rerankers like Cohere Rerank, BGE-reranker.

5. Query Expansion / Rewriting

Modifying or augmenting the user's original query to improve retrieval effectiveness. This can involve adding synonyms, related terms, or rephrasing the query to better match potential document content, especially for ambiguous or short queries.

• Techniques: Synonym expansion, adding related terms, using an LLM to generate alternative phrasings or sub-queries, HyDE (Hypothetical Document Embedding) where an LLM generates a hypothetical answer and its embedding is used for retrieval.

6. Advanced Chunking Strategies

The way documents are split into smaller 'chunks' significantly impacts retrieval. Effective chunking ensures that relevant information is contained within a single retrievable unit and minimizes noise.

• Fixed-size/Overlap: Standard chunking with fixed token counts and some overlap.
• Semantic Chunking: Chunking based on semantic boundaries (e.g., paragraphs, sections, or using embedding similarity to define boundaries).
• Parent-Child/Small-to-Large: Retrieve small, precise chunks first, then expand to larger 'parent' chunks for richer context if needed.
• Sentence Window Retrieval: Embed and retrieve individual sentences, then expand to include surrounding sentences during generation to provide more context.

7. Metadata Filtering

Using structured metadata associated with document chunks (e.g., author, date, document type, topic tags) to narrow down the search space. This can be combined with vector search to perform pre-filtering or post-filtering, ensuring only context relevant to specific criteria is considered.

8. Multi-Stage / Multi-Hop Retrieval

Involves multiple steps of retrieval, potentially refining the query or search space based on intermediate results. For instance, a first stage might retrieve broad documents, and a second stage might focus on specific sections within those documents based on the initial query or an LLM-generated follow-up query.