Digital Engines logoDigital Engines
←Back to Journal
AI·October 13, 2026·12 min read

RAG explained: build an AI that answers from your own data with Postgres and pgvector

DE
Digital Engines
Design & Technology Studio
Knowledge workspace representing document search and retrieval
ESSAY · 12 min read

Language models are trained on public data. They don't know your policies, products or internal documents, and when they guess they can sound confident and be wrong. Retrieval-Augmented Generation (RAG) fixes this by finding the most relevant pieces of your own content and giving them to the model as context. It's the technology behind most 'chat with your docs' and smart support assistants we build.

1. How RAG works

RAG has two phases. Ingestion happens once (and whenever content changes); retrieval happens on every question.

  • Ingest: split documents into small chunks, convert each chunk into an embedding (a list of numbers that captures meaning) and store it.
  • Retrieve: convert the user's question into an embedding and find the chunks with the closest meaning.
  • Generate: send the question plus the best chunks to the model, with instructions to answer only from them.
  • Cite: return the sources alongside the answer so users can verify it.
“A model doesn't know your business. RAG hands it the right pages at the right moment, and asks it to answer only from them.”
— Strategic Takeaway

2. Set up the vector store in Postgres

pgvector is a Postgres extension supported by Supabase, Neon, PlanetScale Postgres and most managed providers. It lets you store embeddings next to your normal data and search them with SQL.

migrations/001_documents.sqlsql
CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE documents (
  id         BIGSERIAL PRIMARY KEY,
  source     TEXT NOT NULL,          -- e.g. file name or URL, used for citations
  content    TEXT NOT NULL,
  embedding  VECTOR(1536) NOT NULL,  -- text-embedding-3-small returns 1536 dimensions
  created_at TIMESTAMPTZ DEFAULT now()
);

-- HNSW index for fast similarity search
CREATE INDEX ON documents USING hnsw (embedding vector_cosine_ops);
RAG explained: build an AI that answers from your own data with Postgres and pgvector visual illustration
FIG. 01 — AI Study

3. Ingest: chunk and embed your content

Chunks of roughly 500–1,000 characters with a small overlap work well for most documents. Overlap prevents an important sentence from being cut in half between two chunks.

Installbash
npm install openai @neondatabase/serverless
scripts/ingest.tsts
import OpenAI from "openai";
import { neon } from "@neondatabase/serverless";

const openai = new OpenAI();
const sql = neon(process.env.DATABASE_URL!);

function chunkText(text: string, size = 800, overlap = 150): string[] {
  const chunks: string[] = [];
  for (let start = 0; start < text.length; start += size - overlap) {
    chunks.push(text.slice(start, start + size));
  }
  return chunks;
}

export async function ingestDocument(source: string, text: string) {
  const chunks = chunkText(text);

  // Embed many chunks in one request: faster and cheaper
  const { data } = await openai.embeddings.create({
    model: "text-embedding-3-small",
    input: chunks,
  });

  for (let i = 0; i < chunks.length; i++) {
    await sql.query(
      "INSERT INTO documents (source, content, embedding) VALUES ($1, $2, $3::vector)",
      [source, chunks[i], JSON.stringify(data[i].embedding)]
    );
  }
  console.log(`Ingested ${chunks.length} chunks from ${source}`);
}

4. Retrieve and answer

At question time, we embed the question, fetch the five closest chunks with the pgvector cosine distance operator (<=>), and pass them to the model with strict instructions.

lib/ask.tsts
import OpenAI from "openai";
import { neon } from "@neondatabase/serverless";

const openai = new OpenAI();
const sql = neon(process.env.DATABASE_URL!);

export async function ask(question: string) {
  // 1. Embed the question
  const { data } = await openai.embeddings.create({
    model: "text-embedding-3-small",
    input: question,
  });

  // 2. Find the most similar chunks
  const matches = await sql.query(
    `SELECT source, content, 1 - (embedding <=> $1::vector) AS similarity
       FROM documents
      ORDER BY embedding <=> $1::vector
      LIMIT 5`,
    [JSON.stringify(data[0].embedding)]
  );

  const context = matches
    .map((m, i) => `[${i + 1}] (${m.source})\n${m.content}`)
    .join("\n\n");

  // 3. Answer using only the retrieved context
  const response = await openai.responses.create({
    model: process.env.OPENAI_MODEL!,
    instructions:
      "Answer the question using ONLY the context provided. Cite sources like [1]. " +
      "If the answer is not in the context, say you don't know and suggest contacting support.",
    input: `Context:\n${context}\n\nQuestion: ${question}`,
  });

  return {
    answer: response.output_text,
    sources: matches.map((m) => m.source),
  };
}

5. Making RAG production-ready

A working demo takes a day. A reliable assistant needs a few more steps.

  • Re-ingest automatically when content changes (CMS webhook or scheduled job).
  • Store metadata (product, language, access level) and filter by it in the SQL query, so users only see what they're allowed to.
  • Combine vector search with keyword search (hybrid search) for product codes, names and exact terms.
  • Set a minimum similarity threshold and fall back to 'I don't know' below it.
  • Log questions with no good answer. They show exactly which content is missing.
  • Evaluate answers against a set of real questions before every major change.

6. What it costs

Embeddings are inexpensive. text-embedding-3-small costs $0.02 per million tokens (checked November 2026), so embedding hundreds of pages typically costs cents. Most of the running cost comes from the answer-generation model and database hosting, which depends on the model you choose and how many questions users ask.

Core Takeaways

Principles for deliberate execution

  • 1RAG = retrieve relevant chunks of your data, then let the model answer using only those chunks.
  • 2You don't need a separate vector database: Postgres with pgvector handles most projects.
  • 3Chunking and good metadata matter more than the choice of model.
  • 4Always return sources, and tell the model to say 'I don't know' when the context has no answer.
Tagged:#RAG#Embeddings#pgvector#Vector Search#OpenAI
DE
Written by
Digital Engines

Digital Engines engineers SaaS platforms, high-performance websites, mobile apps and AI systems for ambitious teams worldwide. These guides come from the tools we use on real client projects every day.

Get in touch →
Continue reading

More from the journal

All essays →