What are vector databases? Why are they hot right now?

So What are Vector Dbs?

Vector Dbs are Databases that have native support to store and retrieve vector data. In Deep Learning everything is a vector -> Your inputs, weights, and your learned semantic space and embeddings. Vector Dbs can be used to store and retrieve these artifacts efficiently.

What is the right way of storing and retrieving these vectors? We typically want ‘Similar’ vectors to be bunched together. We define similarity as the distance between two vectors. Some common distance metrics are Euclidean distance, Manhattan distance and, cosine similarity. You want fast retrieval of vectors similar to the one you are interested in. Here is a Milvus tutorial on how to do that.

It all started with Billion-scale similarity search with GPUs paper from Meta.

The underlying algorithms of vector dbs involving building efficient indices using heuristics. Different implementations choose different trade-offs between accuracy and speed.

Pinecone recently raised hundred millions dollars in their latest funding round. Pinecone’s blog is great for additional reading.

What changed after ChatGPT? And why are they blowing up now?

LLMs for many are just APIs that they can integrate into their workflows. If you are an organization that is buying an API subscription to the foundational LLM models, you do not get to re-train the models, or feed all the facts and policies that are specific to your organization. The products on top of LLMs we have today do not have a persistence layer. That’s where vector databases can help. You could augment your prompts with extra information from your documents to produce customized outputs. You could store older responses to particular prompts and re-use them. These are also being used in Auto-gpt like projects to replace the role of prompt engineers where it’s necessary to prompt the LLMs multiple times to complete a task.

This blog talks about one such use case.