Alternatives to Vector DB?
A while back I was looking for a vector database that would work across Windows / Mac / Linux platforms. Some of the options required specific processors like Intel. I am curious if there are any alternatives to a Vector DB that can run cross platform and are easy to setup?
There's Postgres with pgvector extension.
You can try Milvus, Weaviate, or Qdrant. They all support multiple platforms and are relatively easy to set up.
Postgres with pgvector extension. Have been using it in production for months and it works great.
We used Qdrant in production - it's a solid vector db offering and highly recommend. However we are moving everything to Postgres with pgvector for simplicity i.e fewer moving parts. It was a PITA keeping data synced between pgsql <> qdrant.
SQLite via libsql is my go to for lightweight stuff.
Has vector embedding columns out of the box.
Qdrant runs on Linux/Mac/Windows and on x86/ARM processors
if you have small data requirements (less than 100mb) I just run FAISS for embeddings and store the rows of data and vectors in JSON.
I then have a small python script to run a vector similarity algo like cosine similarity or whatever. It's not the fanciest or most efficient, but it works surprisingly well.
I use it to search/rank my own blog posts (~300+) for relevancy. So the entire thing is only like 10mb. Probably will get super slow for really large dataasets
Elasticsearch is a good bet, if you need to use multiple filters with your queries, and when you grow above the acceptable size of an in-memory database.
If you strictly want things that are not vector databases, you can choose to categorize each item with multi-hierarchical categories. To query, just filter for items belonging to the categories that interest you. As a bonus, this approach is a lot more interpretable than using embeddings.
Brute force it. Gemm routines can give you a best dot product among 300k vectors well under a second
postgres pgvector is a decent implementation by now which is likely available in most setups that postgres will run on.
You can also use faiss if you want it all in memory at all times and have the RAM to support it.
What's your use case and the volume of vectors you want to look up?
There’s a bazillion python libraries and regular databases / search engines with vector indices that run cross platform. Postgres, hnswlib, Elasticsearch, etc.
Check out lanceDB, it looks awesome for this.
Running everything on windows plaform is not easy... maybe just run with a container?
The correct answer is duckdb with the vec extension :-)
two years ago annoy library from spotify was good enough for small RAG, stored everything in single file...
Can I ask what the use case is?
Always fascinated to hear why people are using vector DBs, especially outside AI embeddings
Starting in v7.0, ravendb has ve tor search and ai integration
Run on windows , linux and mac