RAG Knowledge Assistant

RAG Knowledge Assistant is an N8N-based Retrieval-Augmented Generation proof-of-concept that splits functionality into two sub-flows: a Load Data flow that ingests PDF/CSV files via a web form, converts them into vector embeddings, and stores them in an in-memory vector store; and a Retriever flow that accepts chat messages, retrieves relevant document chunks, and uses a large language model to generate grounded answers. The prototype demonstrates how domain-specific documents can be made queryable through natural language without fine-tuning or rebuilding an LLM from scratch.

Organizations possess large amounts of unstructured knowledge (PDFs, CSVs, SOPs, reports), but employees often struggle to find precise answers quickly. Generic LLMs can answer broad questions, but they:

A practical solution is needed to let users chat with their own documents in a cost-effective, low-code manner without sending sensitive data to third-party APIs.

• —Hallucinate on internal data: Models produce outdated or fabricated answers when they lack access to proprietary documents.
• —Cannot access private repositories: Cloud LLMs have no visibility into internal file shares or knowledge bases.
• —Require expensive fine-tuning: Adapting a model to a specific domain is costly and operationally complex.

Enable any team to turn static document repositories into an interactive knowledge assistant:

"Upload any document, ask a question in plain English, and get an accurate answer grounded in your own data — without writing code, training a model, or sending files to external services."

Longer-term, the architecture supports:

• —Non-technical users uploading files and immediately asking questions in plain English.
• —AI responses grounded in actual source documents, improving accuracy and trust.
• —RAG pipelines assembled visually using N8N's drag-and-drop nodes, reducing dependency on specialized ML engineering.
• —Integration with enterprise tools (CRM, Slack, email) for automated knowledge workflows.

• —Embedding model selection: The workflow includes both `text-embedding-ada-002` (OpenAI) and HuggingFace Inference embedding nodes. The active connection uses HuggingFace, indicating research into open-source alternatives that avoid per-token costs and keep data processing local.
• —LLM options and cost optimization: Both `gpt-4o-mini` (OpenAI) and `meta-llama/llama-4-scout-17b-16e-instruct` (Groq) are wired in. The active connection is to the Groq model, prioritizing faster inference and lower cost through open-weight models.
• —Vector store trade-offs: The in-memory vector store (`vectorStoreInMemory`) with a shared `memoryKey` enables cross-flow access between the ingestion and retrieval pipelines. This is ideal for prototyping but not persistent across restarts — production would require Pinecone, Qdrant, or pgvector.
• —Agent pattern: An AI Agent node uses the vector store as a tool named `knowledge_base`, allowing the LLM to autonomously decide when retrieval is needed versus answering from parametric knowledge.

The workflow is a functional N8N prototype with two distinct flows:

Load Data Flow: 1. Form Trigger — Accepts .pdf / .csv uploads via a web form. 2. Default Data Loader — Parses binary documents into text chunks. 3. Embeddings (HuggingFace) — Converts chunks into vector embeddings. 4. Insert Data to Store — Loads vectors into the shared in-memory vector store.

Retriever Flow: 5. Chat Trigger — Receives user questions via a public chat interface. 6. AI Agent — Decides when to call the `knowledge_base` retrieval tool. 7. Query Data Tool — Retrieves relevant chunks from the vector store. 8. LLM (Groq Llama 4 Scout) — Generates the final grounded answer.

The workflow originated from N8N's `rag-starter-template` and demonstrates the complete RAG lifecycle from document upload to conversational Q&A.

• —Faster information retrieval: Users ask questions instead of manually searching through documents.
• —Reduced hallucination risk: Responses are grounded in retrieved source chunks rather than model parametric knowledge alone.
• —Lower cost than fine-tuning: RAG keeps the base model frozen and only injects relevant context at inference time.
• —No-code extensibility: N8N allows business users to adapt the pipeline — add CRM, Slack, email notifications — without writing code.
• —Vendor flexibility: Having both OpenAI and Groq/HuggingFace nodes enables price/performance comparison and avoids vendor lock-in.

• —Embedding consistency is critical. Insert and retrieve operations must use the same embedding model and settings; mixing embeddings will silently break retrieval quality.
• —In-memory storage is a prototype limitation. The `vectorStoreInMemory` is not persistent across restarts. A production version must use a dedicated vector database.
• —Model selection is modular. Keeping multiple LLM/embedding providers side-by-side in the workflow makes it easy to A/B test latency, cost, and quality without re-architecting.
• —File type scope is narrow. Currently restricted to .pdf and .csv. Expanding to .docx, .txt, .md, and web pages would significantly increase utility.
• —Chunking configuration matters. The default loader is used without custom chunk size or overlap; production RAG should tune these parameters for better retrieval precision.
• —Access control is needed. The `public: true` chat trigger enables easy sharing but requires attention to authentication, authorization, and rate limiting before any multi-user deployment.