When pgvector is not enough anymore
I started saas.pet's RAG with pgvector in a free-tier Supabase Postgres. It worked fine for 10K document chunks: search latency was 80ms, indexing took 30 seconds. At 100K chunks, latency jumped to 500ms and indexing took 8 minutes. At 500K, queries timed out. Milvus on the same machine (2 vCPU, 8GB RAM): 300K vectors, search latency 12ms, indexing 45 seconds. The difference is that Milvus builds specialized indices (IVF_FLAT, HNSW) that pgvector's basic IVFFlat cannot match.
Self-hosted setup and the Docker Compose trap
Milvus Standalone via Docker Compose is the documented quickstart. It works but pulls 6 containers (Milvus, Etcd, MinIO, Attu UI, and 2 sidecars) totaling 3GB of images. For production, use Milvus Lite (pip install pymilvus, embedded mode, zero extra services) for <1M vectors. The Python SDK is clean: `from pymilvus import Collection; collection = Collection('docs'); collection.search(embeddings, 'vector', param={'metric_type': 'L2'}, limit=10)`. The Attu web UI gives you a table view of your collections, which is more useful than it sounds for debugging.
Performance benchmarks on modest hardware
Test setup: 2 vCPU, 8GB RAM, 100GB SSD. Dataset: 1M vectors (768-dim from text-embedding-3-small). Index: IVF_FLAT with nlist=1024. Insert: 8,000 vectors/second. Search (top 10): 12ms average, 25ms p99. Recall@10: 97.8% (tradeoff between speed and accuracy via nprobe parameter). Memory: 3.2GB for the index. For comparison, Pinecone's p1 pod (roughly equivalent) costs $70/month. Milvus self-hosted on a $20/month VPS: 3x cheaper and 2x faster for my workload.
The cloud vs self-hosted tradeoff
Zilliz Cloud (managed Milvus): starts at $0 for a free tier with 500K vectors. Paid starts at $33/month for 5M vectors. The benefit: zero ops, auto-scaling, 99.9% SLA. Self-hosted Milvus: zero software cost, you pay only for the server. The downside: you manage Etcd, MinIO, backups, and upgrades. For production RAG where downtime costs money, use Zilliz Cloud. For internal tools and prototypes where you can tolerate a few hours of downtime, self-host.
Milvus vs Pinecone vs Weaviate vs pgvector
Pinecone: best DX, zero ops, most expensive at scale ($70+/month for 1M vectors). Best for teams that want zero infra management. Weaviate: built-in vectorization and hybrid search (BM25 + vector), good DX, mid-range pricing. Best for projects that need both keyword and semantic search. Milvus: fastest at scale, cheapest when self-hosted, highest ops burden. Best for performance-sensitive RAG at 1M+ vectors. pgvector: simplest, uses existing Postgres, good to 100K vectors. Best for projects already on Postgres.
