How does document retrieval work in a Naive RAG system?

In a Naive Retrieval-Augmented Generation (RAG) system, document retrieval is the foundational step responsible for fetching relevant information from a knowledge base to inform the language model's generation process. It directly influences the quality and factual accuracy of the generated response by providing context.

Overview of Naive RAG Retrieval

At its core, document retrieval in a Naive RAG system involves finding pieces of information from a predefined corpus that are most relevant to a given user query. This process typically relies on embedding techniques to represent both the query and the documents in a high-dimensional vector space, followed by a similarity search to identify the closest matches.

Key Steps in Document Retrieval

1. Knowledge Base Preparation (Indexing)

Before any query can be processed, the entire knowledge base (e.g., documents, articles, web pages) must be prepared. This usually involves:

• Chunking: Breaking down long documents into smaller, manageable segments or 'chunks' to ensure relevance and fit within context window limits.
• Embedding: Each chunk is then converted into a numerical vector (an embedding) using an embedding model (e.g., Transformer-based models like BERT, OpenAI Embeddings). These embeddings capture the semantic meaning of the text.
• Indexing: These vectors are stored in a vector database or an in-memory index, making them searchable efficiently.

2. User Query Embedding

When a user submits a query, it undergoes a similar embedding process. The same embedding model used for the document chunks is applied to the user query, transforming it into a query vector in the same high-dimensional space as the document embeddings.

3. Similarity Search

The heart of retrieval is finding document chunks whose embeddings are 'closest' to the user query embedding. This is typically achieved using similarity metrics to measure the distance or angle between vectors. The closer the vectors, the more semantically similar the query and the document chunk are considered to be.

• Cosine Similarity: Measures the cosine of the angle between two vectors, indicating their directional similarity.
• Dot Product: Another common metric, closely related to cosine similarity, especially when vectors are normalized.
• Vector Database Algorithms: Efficient algorithms and data structures (e.g., FAISS, HNSW) are used to perform fast similarity searches over millions or billions of vectors.

4. Top-K Selection

After calculating similarity scores for all document chunks against the query, the system selects the 'Top-K' most similar chunks. 'K' is a predefined parameter determining how many relevant pieces of information will be passed to the Large Language Model (LLM). These selected chunks constitute the 'retrieved context'.

Characteristics and Limitations of Naive RAG Retrieval

• Semantic Matching Focus: Relies purely on the semantic similarity captured by embeddings. If the embedding model fails to capture nuances or requires specific keyword matching, relevance can suffer.
• No Re-ranking: In a naive system, retrieved documents are typically passed directly to the LLM without further re-ranking or filtering based on their content in relation to the query or each other.
• Context Window Constraints: The value of 'K' is often limited by the LLM's context window size, meaning only a limited number of chunks can be retrieved, potentially omitting other relevant but lower-ranked information.
• Susceptibility to Noise: Highly similar but factually incorrect or irrelevant documents can still be retrieved if their embeddings are close, leading to hallucinations or incorrect answers from the LLM.
• Simplicity and Efficiency: Naive RAG retrieval is straightforward to implement and relatively efficient for initial use cases but lacks the sophisticated improvements found in advanced RAG architectures (e.g., query rewriting, advanced chunking, re-ranking).