Posts tagged with #rag from the Contra Collective team.
Where the latency actually goes in a fully local RAG pipeline on M5 Max: embed, retrieve, rerank, and generate as one budget. Stage by stage timings, the cost of loading three models on one box, and the orchestration choices that decide whether local RAG feels instant or slow in 2026.
BGE reranker v2, Cohere Rerank 3 (hosted), and Qwen3 reranker compared on M5 Max for local RAG. Latency per 100 candidates, NDCG, recall, and cost per million queries that decide whether reranking belongs on device or behind an API.
Local embedding models on M5 Max. nomic-embed-text-v2, bge-m3, and Qwen3-Embedding-8B compared on throughput, retrieval quality, memory footprint, and the production tradeoffs that matter for RAG on Apple Silicon.
Your retrieval-augmented generation pipeline is only as good as the framework underneath it. Pick the wrong one and you are refactoring six months later while your competitors ship.