🎙️ VoxRAG — Multilingual Voice-Enabled RAG

VoxRAG is a voice- and text-enabled Retrieval-Augmented Generation system built on MSMARCO-XI. Speak or type in English, हिन्दी, मराठी, বাংলা, മലയാളം, ગુજરાતી, অসমীয়া — the pipeline transcribes, retrieves grounded passages from ~3.2M (32 lakh) embedded records, and streams a cited answer back, targeting <200 ms retrieval / time-to-first-token.

📦 Ingestion deep-dive: how the corpus was minified, sharded across 5 T4 workers, chunked, checkpointed and indexed is documented in ingestion/README.md.

---

🏗️ Architecture

🔄 Query Workflow

---

✨ Key Features

• Voice + text input with per-language STT (Sarvam saaras:v3, ElevenLabs fallback)
• Token-aware semantic chunking (400-token chunks, 50-token overlap) — never splits mid-word in Indic scripts
• Parent–child retrieval — embed small chunks, expand a context window at query time
• halfvec(384) storage — FP16 vectors, half the bandwidth of float32
• IVFFlat ANN index — chosen to fit a 4 GB RAM instance (see below)
• Streaming grounded generation on Groq openai/gpt-oss-120b
• 4-tier guardrails — input safety, topic scope, hallucination scan, grounding check
• Live telemetry — rolling-window P50 / P70 / P100 for STT, embed, retrieve, TTFT

---

🧰 Tech Stack

---

📂 Project Structure

textCopy
.
├── README.md                      # This file
├── analytics.py                   # Rolling-window P50/P70/P100 latency tracker
├── app.py                         # Streamlit frontend ("Quiet Studio")
├── config.py                      # Pydantic settings + language map
├── latency_logs.jsonl             # Telemetry log (rolling window source)
├── packages.txt                   # System deps (ffmpeg)
├── pyproject.toml / uv.lock       # uv-managed dependencies
├── ingestion/
│   ├── README.md                  # 👈 Ingestion deep-dive (read this!)
│   ├── download_corpus.py         # Pulls minified MSMARCO-XI JSONL from Kaggle
│   ├── multi_embed_marco_chunk.py # Distributed multi-T4 embedding worker
│   └── semantic_chunker.py        # Token-aware chunking (E5 offset_mapping)
├── pipeline/
│   ├── orchestrator.py            # Synchronous harness: retries, timing, events
│   ├── stt.py                     # Sarvam / ElevenLabs clients
│   ├── retriever.py               # pgvector parent–child retrieval
│   ├── llm.py                     # Groq streaming client
│   └── guardrails.py              # Input filter + citation verifier
└── src/voice_rag_project/         # Package stub

---

🚀 Quick Start

Prerequisites

• Python 3.12+, PostgreSQL 15+ with pgvector ≥ 0.7 (for halfvec)
• uv and ffmpeg

1 · Install

bashCopy
sudo apt-get update && sudo apt-get install -y ffmpeg   # system deps
uv sync                                                  # python deps

2 · Configure

Create .env:

envCopy
DATABASE_URL=postgresql://user:password@host:5432/voxrags_db
GROQ_API_KEY=gsk_...
SARVAM_API_KEY=...
ELEVENLABS_API_KEY=...        # optional fallback

STT_PROVIDER=sarvam
VECTOR_BASE=halfvec
TABLE_NAME=corpus_embeddings_chunks
GROQ_MODEL=openai/gpt-oss-120b

3 · Database & Index

sqlCopy
CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE IF NOT EXISTS corpus_embeddings_chunks (
    source_file text NOT NULL,
    line_no     bigint NOT NULL,
    chunk_no    integer NOT NULL,
    id text, query_id text,
    chunk_start integer, chunk_end integer,
    text text,
    embedding halfvec(384) NOT NULL,
    updated_at timestamptz NOT NULL DEFAULT now(),
    PRIMARY KEY (source_file, line_no, chunk_no)
);

-- ANN index: IVFFlat, sized for a 4 GB RAM instance
SET maintenance_work_mem = '2GB';
CREATE INDEX corpus_embeddings_chunks_embedding_idx
    ON corpus_embeddings_chunks
    USING ivfflat (embedding halfvec_cosine_ops)
    WITH (lists = 1800);          -- ≈ sqrt(row count)

ANALYZE corpus_embeddings_chunks;

Why IVFFlat and not HNSW? Building HNSW over ~3.2 M halfvec(384) rows needs 10–12 GB RAM; this project runs on a 4 GB instance. IVFFlat builds in minutes within that budget and still delivers sub-50 ms retrieval.

4 · Ingest the corpus

Full details, iteration history and worker sharding live in ingestion/README.md.

bashCopy
uv run python ingestion/download_corpus.py                          # Kaggle JSONL
FILE_START=1    FILE_END=500  uv run python ingestion/multi_embed_marco_chunk.py   # worker 1
FILE_START=501  FILE_END=1000 uv run python ingestion/multi_embed_marco_chunk.py   # worker 2

5 · Run the app

bashCopy
uv run streamlit run app.py

---

🎛️ The UI

• Dual input — record voice or type; language selector for forced STT/generation language
• Answer core — streamed, cited answers with source expander
• Telemetry deck — per-stage latency bars + rolling P50/P70/P100 (last 20 queries, immune to cold-start skew)
• Guardrail matrix — live LEDs for grounding, topic, hallucination, safety
• Reset control — clear latency history to drop cold-start outliers

---

✂️ Chunking, Retrieval & Latency

Chunking. semantic_chunker.py uses the E5 tokenizer's offset_mapping for token-aware boundaries (400 tokens + 50 overlap), preserving metadata (source_file, line_no, chunk_no, char offsets) on every chunk.

Retrieval. The query embeds with the query: prefix; pgvector returns the nearest chunks; a self-join expands ±2 chunks around each match so the LLM sees continuous context, not isolated fragments.

Latency budget. "200 ms" targets retrieval + TTFT, not full generation (physically >500 ms for a paragraph). FP16 halfvec + IVFFlat + Groq streaming + a synchronous (no-asyncio) orchestrator keep the perceived response instant.

---

🛡️ Guardrails

1. Input safety — regex blocklist stops unsafe/PII queries before any compute is spent.
2. Topic scope — off-corpus queries are refused gracefully.
3. Hallucination scan — citations [1]…[n] in the answer are verified against the actual retrieved chunk IDs; invented sources flip the matrix to Review.
4. Grounding — the prompt hard-constrains the model to the provided context; "I don't know" is a first-class answer.

---

🙏 Credits

• Corpus: MSMARCO-XI (AI4Bharat) · minified mirrors by amankroot on Kaggle
• STT: Sarvam AI · LLM serving: Groq · Embeddings: intfloat/multilingual-e5-small