How are embeddings generated for hypothetical documents?

In the Hypothetical Document Embedding (HyDE) technique, a key step for enhancing Retrieval Augmented Generation (RAG) is the creation and subsequent embedding of a hypothetical answer. This process aims to generate a richer, more context-aware query representation for improved retrieval.

The Role of the Language Model in Document Generation

The first step in HyDE involves using a large language model (LLM) to generate a hypothetical, relevant document based solely on the user's query. The LLM is prompted to produce a document that would likely contain the answer to the user's question, even if it might not be factually accurate itself. This hypothetical document serves as a 'bridge' between the short, potentially ambiguous user query and the detailed, vector-encoded documents in the knowledge base.

Embedding the Hypothetical Document

Once the hypothetical document is generated, it is then passed to a standard embedding model. This is the same embedding model that was used to create the vector representations of the actual documents in the knowledge base. The process is identical to how any other text document would be embedded:

• Input: The LLM-generated hypothetical document serves as the input text.
• Embedding Model: This text is fed into a pre-trained embedding model (e.g., Sentence Transformers, OpenAI Embeddings, Cohere Embeddings, etc.).
• Vector Output: The embedding model processes the text and outputs a fixed-size numerical vector (an embedding). This vector captures the semantic meaning of the hypothetical document.

Utilizing the Hypothetical Embedding

The resulting embedding of the hypothetical document is then used as the query vector for performing a similarity search against the vector database containing the embeddings of the real documents. The intuition is that a hypothetical document, being more verbose and semantically rich than the original short query, will produce an embedding that is closer in vector space to the relevant actual documents, thereby improving retrieval accuracy.