Practical RAG implementation with Next.js, Supabase pgvector and OpenAI. Index documents, semantic search, augmented answers. Real code, no abstractions.

Index documents, run semantic search, and assemble LLM-augmented answers using Next.js + Supabase pgvector + OpenAI.

Implement RAG in a Next.js SaaS

Enable pgvector in Supabase and create the embeddings table

Turn on the pgvector extension in your Supabase project and create a table with columns for the text chunk, the embedding (vector 1536), and metadata.

Chunk documents and generate embeddings

Split each document into 400-800 token chunks with overlap, generate embeddings using text-embedding-3-small from OpenAI, and insert into Supabase.

Semantic search with cosine similarity

Create a Postgres RPC that, given a query embedding, returns the top-k most similar chunks using pgvector's <=> operator.

Augment the prompt with retrieved context

In the Next.js handler, embed the user question, retrieve top-5 chunks, and inject them into the system prompt before calling the LLM.

Monitor costs and metrics

Track tokens per request, cache hit rate, and latency. Log the augmented prompt for debugging when an answer is bad.

RAG step by step for your SaaS: from PDF to chatbot in 2 hours

Integrate OpenAI in Next.js without the bill spiraling: model selection, caching, streaming, per-tenant rate limiting, and budget caps.

Wrap the SDK with telemetry, pick the right model per call, cache repeated prompts, stream responses, per-tenant rate limiting, and budget caps.

Integrate the OpenAI API in a Next.js SaaS with cost discipline

Wrap the SDK with telemetry

Centralize all OpenAI calls behind one module that logs model, tokens, latency, tenant_id, and estimated cost per request.

Pick the model dynamically per task type

Route every call to the cheapest model that can do the job: gpt-4o-mini for classification/extraction, gpt-4o for rich generation, o3-mini for reasoning.

Cache repeated prompts in Upstash Redis

Deterministic hash of prompt + parameters as key. Cache hit saves 100% of cost and latency. TTL 7-30 days depending on case.

Stream responses for better UX

Use Server-Sent Events to return tokens as they're generated. UX improves 5x and reduces client-side timeouts.

Per-tenant rate limiting

Limit requests per minute and monthly tokens per tenant. Sliding window in Redis. Return 429 with Retry-After header.

Budget caps with kill switch

Track per-tenant monthly accumulated spend. If it exceeds the cap, return an informative error instead of continuing to charge.

Articles about Next.js

RAG step by step for your SaaS: from PDF to chatbot in 2 hours

How to integrate the OpenAI API in a Next.js SaaS without blowing up costs