Adviser CLI
An experiment in local-first AI and document retrieval, exploring hybrid search, local LLM execution, multimodal document retrieval, and MCP integration with AI coding assistants.
Keeping Sensitive Context in the Terminal
Developers frequently need intelligent assistance when querying local documentation, proprietary codebases, or internal project records. However, pasting confidential files into public cloud AI interfaces creates serious security and intellectual property risks, while context switching out of the terminal drains engineering focus. I built Adviser CLI as a terminal-native, zero-infrastructure Retrieval-Augmented Generation (RAG) assistant. It allows developers to index and search private text and markdown documents locally, combining hybrid search relevance with local model execution on their own hardware.
Hybrid Search with Reciprocal Rank Fusion
Relying solely on vector embeddings for retrieval often leads to frustrating misses when developers search for exact function names, specific error codes, or technical jargon. Conversely, keyword searches fail when queries use natural language synonyms. To solve this, I built a hybrid retrieval pipeline. During document ingestion, Adviser chunks markdown files concurrently, writing dense vector embeddings to ChromaDB using the bge-small-en-v1.5 model while simultaneously constructing an in-memory BM25 index. At query time, the system retrieves candidates from both sources and fuses their rankings using Reciprocal Rank Fusion (RRF), delivering consistently accurate context.
Hardware Profiling and Priority Cascades
Hardware capabilities vary widely across developer setups, and attempting to run heavy local models on modest machines leads to system freezes. During setup initialization, Adviser profiles local CPU cores, RAM, and available VRAM, recommending quantized Ollama model sizes that fit safely within memory boundaries. To handle API instability, I implemented a session-aware circuit breaker. When cloud endpoints experience throttling or rate limits, the request router automatically cools down failing providers and cascades queries down a priority chain—transitioning from frontier APIs to fast Groq inference, and finally to local Ollama models offline.
IDE Integration via Model Context Protocol
While an interactive terminal prompt is useful, developers spend most of their time inside editors like Cursor, Windsurf, or Claude Code. I extended Adviser CLI with an automated Model Context Protocol (MCP) server integration. Running adviser mcp-install launches an interactive terminal wizard that configures the stdio server directly inside the editor's configuration. To prevent concurrency locks when an IDE queries the vector index while the CLI is ingesting files, I configured SQLite with Write-Ahead Logging (WAL mode) and shared-cache read connections, enabling uninterrupted parallel retrieval.
What Local RAG Systems Taught Me
Developing Adviser CLI reinforced that retrieval precision is determined far more by chunking strategies and ranking algorithms than by raw LLM parameter counts. Providing a smaller 8B parameter model with well-ranked, noise-free context chunks consistently yields better answers than feeding large context windows with unranked text dumps. Building this tool taught me how to write high-performance Python terminal interfaces with Rich and prompt_toolkit, manage local vector databases cleanly, and design modular multi-provider fallbacks that keep developers productive offline.