Item — HN Reader

> "Has somebody experience with Apache Lucene / Solr or Elasticsearch?"

I've been working on a RAG with Solr, and quickly hit some of the issues you describe when dealing with real-world messy data and user input, e.g. using all-MiniLM-L6-v2 and cosine similarity, "Can you summarize Immanuel Kant's biography?" matched a chunk containing just the word "Biography" rather than one which started "Immanuel Kant, born in 1724...", and "How high is Ben Nevis?" matched a chunk of text about someone called Benjamin rather than a chunk about mountains containing the words "Ben Nevis" and its height[0]. Switching embedding model has helped, but still not convinced that vector search alone is the silver bullet some claim it is. Still lots more to try though, e.g. hybrid search[1], query expansion[2], knowledge graphs etc.

[0] https://www.michael-lewis.com/posts/vector-search-and-retrie...

[1] https://sease.io/2023/12/hybrid-search-with-apache-solr.html

[2] https://news.ycombinator.com/item?id=38706913

2 comments

Exactly in the same place as you with Elastic Search (8.11). Went down the vector path to get better matches for adjectives, verbs and negations ( "room with no skylight" vs. "room with skylights" & "room with a large skylight"). Different dataset obviously, but I think I get slightly better results than your examples and it might be worth looking for a different sentence transformer (I tried a few and settled on roberta-base-nli-stsb-mean-tokens).

was reading through open llama, looks like way to get pertinent results is via different ranking algorithm and score based on convergence. then shove that back into the LLM

If you know that your search queries will be actual questions (like in the example you listed), you can possibly use the HyDE[0] to create a hypothetical answer which will usually have an embedding that's closer to the RAG chunks you are looking for.

It has the downside that an LLM (rather than just a embedding model) is used in the query path, but it has helped me multiple times in the past to strongly reduce problems with RAG like the ones you outlined, where it likes to latch onto individual words.

[0]: https://arxiv.org/abs/2212.10496

Thanks, sounds interesting, not-dissimilar from some of the query expansion techniques. But in my case (open source, zero budget) I'm doing (slow) CPU inference, so an LLM in the query chain isn't really viable. As it is there is a near-instant "Source: [url]" returned by the vector search, followed by the LLM-generated "answer" (quite some time) later. So I think next steps will be "traditional" techniques such as query re-ranking and hybrid search, in line with the original "Build a search engine, not a vector DB" article.