Recipe 06, RAG document Q&A

Goal: answer a user's question by retrieving relevant chunks from a corpus of PDFs and grounding an LLM on them.

Difficulty: Advanced · Time: ~60 minutes (one-time index) + ongoing queries · Connectors: OpenAI / Anthropic, a vector database (Pinecone / Weaviate / pgvector), File Operations

This recipe has two workflows:

1. Indexing workflow, runs once (or whenever your corpus changes). Ingests documents into the vector DB.
2. Query workflow, runs per question. Retrieves and answers.

Prerequisites

• LLM provider credential (Anthropic or OpenAI).
• Embedding provider credential (OpenAI text-embedding-3-small is a sensible default).
• Vector database, Pinecone or pgvector is easiest. Create an index named flero-docs with the right dimension count (1536 for OpenAI small).
• A folder of PDF documents accessible to Flero (S3, local disk, or uploaded via file ops).

Workflow 1, Indexing

┌──────────┐    ┌────────────┐    ┌───────────────┐    ┌────────────┐    ┌──────────────┐
│  Start   │───▶│ File Ops:  │───▶│   PDF Ops:    │───▶│  Document  │───▶│  Embeddings  │
│ (manual) │    │  list files│    │ extract text  │    │   Chunker  │    │              │
└──────────┘    └────────────┘    └───────────────┘    └────────────┘    └──────┬───────┘
                                                                                ▼
                                                                       ┌─────────────────┐
                                                                       │  Vector DB:     │
                                                                       │  upsert chunks  │
                                                                       └─────────────────┘

Step-by-step

1. Start, manual trigger. Run when the corpus changes.
2. File Operations: list, list every PDF in your source folder. Outputs an array.
3. Loop over the list.
4. Inside the loop: File Operations: read the PDF.
5. PDF Operations: extract text → outputs the text plus per-page metadata.
6. Document Chunker, chunk size 1 000 chars, overlap 200, strategy paragraph. Carry forward metadata fields: filename, pageNumber.
7. Embeddings, provider OpenAI, model text-embedding-3-small, input from the chunker. Outputs a vector per chunk.
8. Vector DB: upsert, index flero-docs, with the chunk text + metadata + the embedding.

Run once. Wait. Verify the vector DB shows N chunks.

Workflow 2, Query

┌─────────────┐    ┌──────────────┐    ┌───────────────┐    ┌───────────┐    ┌────────────┐    ┌──────────┐    ┌────────────┐
│   Webhook   │───▶│  Embeddings  │───▶│  Vector DB:   │───▶│   RAG     │───▶│  Transform │───▶│ AI Agent │───▶│ Guardrails │
│   trigger   │    │  (embed Q)   │    │  query top-10 │    │ Reranker  │    │  (format   │    │ (answer  │    │ (PII filter│
└─────────────┘    └──────────────┘    └───────────────┘    │  top-3    │    │  context)  │    │  with    │    │  + topic)  │
                                                            └───────────┘    └────────────┘    │  context)│    └─────┬──────┘
                                                                                               └──────────┘          │
                                                                                                                     ▼
                                                                                                              ┌────────────┐
                                                                                                              │ Terminal:  │
                                                                                                              │  return    │
                                                                                                              │  answer    │
                                                                                                              └────────────┘

Step-by-step

1. Webhook trigger, POST { "question": "..." }.
2. Embeddings, embed $trigger.body.question with the same model used for indexing.
3. Vector DB: query, index flero-docs, top K = 10, optional metadata filter (e.g. tenant == $trigger.body.tenant).
4. RAG Reranker, narrow to top 3.
5. Transform, build a context block:

   map:
     context: |
       {{ $node["RAG Reranker"].json.results[0].text }}
       [Source: {{ $node["RAG Reranker"].json.results[0].metadata.filename }}, page {{ $node["RAG Reranker"].json.results[0].metadata.pageNumber }}]
   
       {{ $node["RAG Reranker"].json.results[1].text }}
       [Source: ...]
   
       {{ $node["RAG Reranker"].json.results[2].text }}
       [Source: ...]

6. AI Agent:
   • System instructions: "Answer the user's question using only the provided context. If the answer isn't in the context, say 'I don't know'. Always cite the source (filename + page) for each claim."
   • Prompt:

     Context:
     {{ $node["Transform"].json.context }}
     
     Question: {{ $trigger.body.question }}

   • Temperature: 0.2
   • Max tokens: 700
7. AI Guardrails, PII filter, topic guard.
8. Terminal, return { answer: $node["Guardrails"].json.output, sources: <list of filenames cited> }.

Set workflow's response mode (on the Webhook Trigger) to On completion so the answer goes back to the caller synchronously.

Try it

curl -X POST <query-workflow webhook URL> \
  -H "Content-Type: application/json" \
  -d '{"question": "What is our refund policy for products bought during a sale?"}'

Response (if the corpus has the answer):

{
  "answer": "Products bought during a promotional sale carry a 14-day refund window... [Source: policies.pdf, page 7]",
  "sources": ["policies.pdf"]
}

If the corpus doesn't cover the question, the answer should be "I don't know", that's the test that the system isn't hallucinating.

Variations

Per-user / per-tenant isolation

Add a tenant field to chunk metadata at index time. Filter on it at query time so different customers' documents don't bleed into each others' answers.

Hybrid search

Some vector DBs (Weaviate, pgvector with TS) support BM25 + vector. Configure the Vector DB node to use hybrid mode for better recall.

Multi-modal

Include images in the index, embed image captions, store image references in metadata. Allow the AI Agent to reference image filenames in its answers.

Streaming responses

For chat-style UI, set the AI Agent to stream: true and front this workflow with the Chat trigger instead of a webhook.

Re-index automatically

Hook the indexing workflow to a Schedule trigger (nightly) and add a "files changed since last index" filter.

Troubleshooting

Next

• RAG nodes
• AI & LLM nodes
• Chat trigger

Found something out of date? This page lives in the Flero docs content set.