Local Vector RAG Engine

Zero-cloud semantic search across personal markdown archives.

Testing browser-native vector embeddings via WebGPU and WASM quantized models to query private local notes without network transit.

01 // CORE HYPOTHESIS

TESTABLE PREMISE

Small quantized embedding models (e.g., bge-micro) running via WebGPU in the browser can deliver sub-50ms semantic search over 10,000 personal notes with zero server costs and absolute privacy.

02 // WHAT WE'RE LEARNING

Cold-start model download (approx 25MB) requires transparent progress UX to prevent user dropoff.

WebGPU shader compilation varies across mobile browser runtimes; fallback to CPU WASM is mandatory.

Cosine similarity on typed Float32Arrays is surprisingly fast when vectorized with SIMD.

03 // NEXT HORIZON & CRITERIA

IMMEDIATE NEXT STEP

Benchmark memory consumption on mobile Safari and add streaming token response.

CONDITIONS TO EARN PRODUCT STATUS:

Sustained query latency under 75ms on consumer laptops

Stable offline caching via CacheStorage API

Clear integration path as a standalone developer utility

RESEARCH KEYWORDS:WebGPUQuantized ModelsLocal AIWASM

TRANSPARENT R&D LEDGER // ZENITHDISTRICT LABS