zvec-grep: Local-First Search for AI Agents

Alibaba’s Zvec team has open-sourced zg, short for zvec-grep, a local-first search layer designed for both developers and AI agents. The project addresses a common failure mode in modern engineering workflows: the information needed to answer a question is present somewhere in a workspace, but its wording, location, or relationship to other files is not obvious.

Direct answer: zg combines exact ripgrep search, BM25 full-text retrieval, and vector search behind one CLI and MCP interface. It can discover relevant material by meaning, rank candidates, and then verify exact source text—all while keeping indexing, retrieval, local embeddings, and workspace indexes on the machine by default.

The result is not another hosted “chat with your code” service. It is a local retrieval substrate that gives humans and agents a shared way to search code, documentation, configuration, research notes, and structured text.

Official project visuals

Official project visual from the zvec-grep repository. It demonstrates the intended install, indexing, and agent-search flow.

Official benchmark visual from the project repository. Benchmark results should be read with the published task, model, prompt, and environment details rather than treated as universal guarantees.

Why ripgrep alone is no longer enough

rg remains excellent when the developer knows the exact symbol, filename, error message, configuration key, or quotation to find. It is fast, exhaustive, and easy to verify. The difficulty begins when the question is expressed in natural language but the implementation uses different vocabulary.

A request such as “where are theme preferences restored?” may point to code named hydratePreferences. “How does the access request process work?” may require a document discussing account authorization and approval. Exact matching cannot reliably bridge that vocabulary gap without repeated guesses.

An agent then tends to follow an expensive loop:

1. Guess keywords.
2. Run a search.
3. Open several files.
4. Discover a new term.
5. Search again.
6. Assemble the answer from scattered fragments.

That loop consumes tool calls, latency, and context tokens. Worse, an agent can stop after finding a plausible but incomplete match. zg is designed to reduce this detour without throwing away the precision of exact search.

What is zg?

zg is a single local-first interface for three complementary retrieval modes:

• Vector search discovers passages and code entities related by meaning.
• BM25/full-text search ranks lexical matches using the words that actually appear in the workspace.
• Managed ripgrep performs exact text, symbol, path, and regular-expression lookup without requiring an index or embedding model.

For indexed retrieval, zg can fuse lexical and vector candidates using Reciprocal Rank Fusion (RRF). The application can therefore start with an intent-level question, narrow the result set through ranking, and finish with exact evidence from a known file and line range.

This is a more realistic retrieval loop for engineering work than choosing between “semantic search everywhere” and “grep everywhere.” Meaning is useful for discovery; exact matching is useful for proof.

The architecture: one engine, two entry points

Developers use the zg CLI. Agents normally connect through a local Streamable HTTP MCP server configured by zg install. Both routes reach the same local search engine and workspace index.

The normal workspace index is stored under <workspace>/.zvec-grep/. Global configuration and daemon state live under ~/.zvec-grep/. The server listens on loopback, which keeps the default execution boundary local to the machine.

The project’s architecture documentation distinguishes three CLI execution modes:

• Auto: choose the appropriate local runtime.
• Server: use a persistent process that coordinates indexes, refreshes, MCP access, and model loading.
• Direct: run the engine without relying on a long-lived server.

Managed zg query --rg can work without an index or server, which preserves the practical advantages of ripgrep for exact lookups.

Search code, docs, notes, and structured data

zg is not limited to source files. Its structure-aware extraction is intended for mixed workspaces containing:

• C, C++, Go, Java, JavaScript, TypeScript, Python, and Rust code
• Markdown documentation with heading structure
• JSON, YAML, TOML, CSV, HTML, XML, and plain text
• Configuration files, research material, manuals, notes, and knowledge-base exports

For code, indexing can preserve symbols, signatures, and breadcrumbs. For prose, it can organize content into sections and bounded chunks. Search results retain file paths and source locations so an agent can request only the evidence it needs instead of loading an entire repository into context.

The current release also allows raster images to be explicitly included when a compatible multimodal embedding model is selected. That capability is different from automatically understanding every binary asset: teams should test extraction and retrieval quality against their own documents.

Built for agents through MCP

The public preview exposes a deliberately narrow default MCP toolset. The main search tool, zvec_grep_search, is intended for workspace-grounded questions where the target wording or location is unknown, or where semantic, fuzzy, relationship, chronology, causality, comparison, or cross-file synthesis is required.

Agents can still use native grep or rg when an exact identifier or pattern is sufficient. For mixed tasks, the recommended behavior is:

1. Use semantic or hybrid retrieval to discover the relevant area.
2. Use exact search for a symbol, phrase, path, or regular expression.
3. Read the compact source-linked result.
4. Stop when the evidence is sufficient.

This matters because an MCP integration should not turn every simple lookup into a vector-search operation. The right tool depends on the question.

The local endpoint is normally:

1http://127.0.0.1:7999/mcp

The default agent toolset exposes search rather than silently giving agents authority to create, rebuild, or delete persistent indexes. Administrative operations are available only through the optional full toolset, making the trust boundary explicit.

Installation and first search

zg requires Node.js 22 or newer. The public preview can be installed from npm:

1npm install -g @zvec/[email protected]

From a workspace, create the first local index with an embedding model:

1cd your-workspace
2zg index --embedding local/potion-code-16m-v2

Then search directly from the terminal:

1zg query --human "where is authentication handled?"

To connect a supported agent, run:

1zg install

The installer is designed to detect and configure integrations for tools such as Codex, Claude Code, Qwen Code, OpenCode, and Cursor. An explicit target can be selected when needed:

1zg install --target codex --yes

The exact integration behavior can evolve during the preview period, so teams should verify the generated configuration and run a real query before depending on it in automation.

Local-first does not mean remote-disabled

Privacy is one of zg’s clearest design choices. By default, workspace scanning, extraction, index storage, retrieval, and local embedding inference happen on-device. Local models are cached under ~/.zvec-grep/models, and the index stays with the workspace.

Remote embedding providers are supported, but they are not treated as an invisible optimization. Selecting a remote provider can send query text or workspace content outside the machine. zg separates provider credentials from permission to transfer data and requires explicit remote-embedding authorization.

That distinction is important in an agent environment. A developer may have permission to use a provider but still lack approval to send proprietary source code, customer records, unreleased research, or regulated documents to it.

The project documents separate controls for:

• Local MCP transport and optional Bearer authentication
• Remote embedding provider credentials
• Workspace-level authorization for data transfer
• One-off --allow-remote authorization for a CLI command

A sensible production policy is to begin with local embeddings, inspect retrieval quality, and approve remote embeddings only for a clearly defined workspace and provider.

Embedding models and trade-offs

The embedding model affects language coverage, memory, index size, input length, and indexing speed. The project catalog includes small Model2Vec models, ONNX models, GGUF models, and remote Qwen models.

Useful starting points include:

• local/potion-code-16m-v2 for a fast first index of a code repository
• local/potion-retrieval-32m for fast English document retrieval
• local/potion-multilingual-128m for multilingual document search
• local/jina-embeddings-v2-base-code for code-oriented long-context retrieval
• local/gte-modernbert-base or local/nomic-embed-text-v1.5 for long English documents
• qwen/qwen3.7-text-embedding when a managed remote model is explicitly approved

The smallest model that covers the required language and input length is usually the best first experiment. A larger model is not automatically better if its extra quality does not improve the team’s real queries.

Changing an embedding model requires an explicit rebuild because vector spaces are incompatible, even when two models use the same number of dimensions:

1zg index --rebuild --embedding local/jina-embeddings-v2-base-code

What the benchmarks show—and what they do not

The repository includes paired A/B evaluations for two different workloads:

• SWE-QA-Bench: repository-level, cross-file software-engineering questions
• BrowseComp-Plus: multi-document retrieval over a fixed corpus

The comparisons hold the task, agent or model, prompt, environment, and resource limits constant while changing zg access and usage guidance. They report answer quality, input tokens, tool calls, execution time, and traces, with index preparation measured separately.

The project’s published summary reports lower tool-call and token usage in these evaluated workflows, alongside maintained or improved answer quality. The exact results depend on the benchmark protocol, the selected embedding model, and the baseline agent. These are useful engineering signals, not a guarantee that every repository will see the same percentage improvement.

A team evaluating zg should build a small private benchmark of its own:

1. Select representative repository, incident, and documentation questions.
2. Freeze the model, prompt, tool permissions, and success criteria.
3. Compare baseline search with zg-assisted search.
4. Measure accepted-answer rate, tool calls, input tokens, latency, and human correction time.
5. Keep failed traces and stale-index cases in the evaluation set.

The key metric is often cost per accepted result rather than raw retrieval latency.

Where zg fits in a production agent stack

zg is most valuable when an agent must work across an unfamiliar codebase or a large local knowledge base. It can provide the retrieval layer, but it does not replace the rest of the agent system.

A production setup still needs:

• Clear workspace boundaries and ignore rules
• Least-privilege tool permissions
• Secrets kept outside prompts and command output
• Freshness checks after major file changes
• Source-linked evidence in the agent’s final answer
• Timeouts, budgets, retries, and stop conditions
• Human review for code changes or sensitive conclusions
• Monitoring for remote-embedding authorization and index failures

An indexed result can be relevant and still be stale. A semantic match can be conceptually close and still not answer the exact question. The strongest workflow combines retrieval with verification rather than treating the first ranked result as truth.

Preview status and roadmap

The v0.2.0 release is a public preview. The core indexing, retrieval, CLI, server, and agent integrations are ready for real-world testing, but CLI and MCP contracts, configuration defaults, index compatibility, and installation behavior may continue to evolve before a stable release.

The project’s roadmap points toward graph search, improved reranking and explainability, broader PDF/Word/PowerPoint extraction, better image OCR and cross-modal understanding, smaller local runtimes, and support for more constrained devices. It also calls out operational work such as upgrades, incremental indexing, concurrent access, self-recovery, diagnostics, and index compatibility.

That roadmap is credible because it focuses on the difficult parts of retrieval infrastructure: freshness, explainability, formats, and operational behavior—not only another search algorithm.

Bottom line

zg is an interesting open-source answer to a practical problem in AI-assisted development: agents often know what they want conceptually but do not know the exact words used by a codebase or document collection.

By unifying vector discovery, BM25 ranking, and exact ripgrep verification, zvec-grep gives humans and agents one retrieval surface without requiring a hosted index. Its local-first defaults are particularly relevant for private repositories and internal knowledge bases, while explicit authorization keeps remote embeddings from becoming an accidental data-exfiltration path.

The right way to adopt it is not to replace every search command immediately. Index one representative workspace, connect one agent, compare real tasks, inspect the evidence paths, and measure whether fewer detours lead to more accepted results.

FAQ

What is zvec-grep?

zvec-grep, exposed through the zg command, is a local-first search layer that combines ripgrep, BM25 full-text search, and vector search for human and AI-agent workflows.

Does zg upload my code?

Not by default. Local scanning, indexing, retrieval, and local embedding inference stay on-device. Remote embedding providers can receive query or workspace content only after the relevant authorization is granted.

Does zg replace ripgrep?

No. It manages ripgrep as one of its retrieval paths. Use semantic or hybrid search when the wording or location is unknown, then use exact ripgrep lookup when you need exhaustive verification.

Which operating systems are supported?

The project README lists macOS, Linux, and Windows support. It requires Node.js 22 or newer.

Which agents can use zg?

The v0.2.0 public preview documents managed integrations for Codex, Claude Code, Qwen Code, OpenCode, and Cursor, with installation handled through zg install.

Is zvec-grep production-ready?

It is a public preview rather than a stable release. It is suitable for real-world testing and internal evaluation, but teams should expect CLI, MCP, configuration, and index-compatibility behavior to evolve.

Sources and official image references

• Qwen Developers announcement on X
• zvec-ai/zvec-grep GitHub repository
• zvec-grep v0.2.0 public preview release
• zvec-grep architecture documentation
• zvec-grep MCP guide
• zvec-grep embedding models guide
• zvec-grep benchmark documentation
• Official tour asset
• Official benchmark asset

Related reading

Continue exploring related AI engineering and developer tooling topics:

• Cloudflare AI Search: A Practical Guide to Agent Retrieval Infrastructure
• Perplexity Portable Computer: Local-First AI Agent Guide
• Google Cloud Agent Registry: A Practical MCP Governance Guide