Recipe 06: RAG document Q&A

Goal: answer a question from your own documents. Flero finds the passages that match the question and has an AI model answer from those passages only.

Difficulty: Advanced · Time: about 60 minutes · Connectors: OpenAI (embeddings), an AI provider for answers

This recipe has two workflows:

1. Indexing: splits a document into chunks, embeds them and stores them. Run it once per document.
2. Query: embeds a question, finds the closest chunks and answers from them.

Prerequisites

• An OpenAI credential for embeddings. This recipe uses text-embedding-3-small, which makes 1,536-number vectors.
• An AI provider credential for the answers (Anthropic, OpenAI or Google).
• Comfort with the Code node. Two small JavaScript steps reshape data between nodes.

Step 0: create the collection

Create the place the vectors will live, once:

1. Make a scratch workflow with a manual Start node and one Vector Database node.
2. Configure it: Operation create_collection, Collection docs, Dimensions 1536, Metric cosine.
3. Run it once.

The Vector Database node keeps collections as files on the Flero worker.

Workflow 1: Indexing

1. API/Webhook Trigger: POST, path /demo/rag-index. The body is { "source": "refund-policy", "text": "…the full document text…" }.

2. Document Chunker:

   • Input: {{ $trigger.payload.text }}
   • Strategy: recursive
   • Chunk size: 1000, Overlap: 200

   It outputs chunks, a list of { content, index, … }.

3. Code node named Prepare chunks, language JavaScript:

   const chunks = {{ $node["Document Chunker"].output.chunks }};
   const source = {{ $trigger.payload.source }};
   return [{
     texts: chunks.map(c => c.content),
     ids: chunks.map(c => source + '-' + c.index),
     metadata: chunks.map(c => ({ source: source, content: c.content })),
   }];

   The {{ }} values arrive as real data, not pasted text. Storing content in the metadata lets the query workflow read the passage back.

4. Embeddings:

   • Provider: OpenAI, Model: text-embedding-3-small, Credential: your OpenAI credential
   • Input: {{ $node["Prepare chunks"].output.texts }}

   It outputs embeddings, one vector per text.

5. Vector Database:

   • Operation: store, Collection: docs
   • Vectors: {{ $node["Embeddings"].output.embeddings }}
   • Ids: {{ $node["Prepare chunks"].output.ids }}
   • Metadata: {{ $node["Prepare chunks"].output.metadata }}

Storing the same id again replaces the old vector, so re-indexing a changed document is safe.

Workflow 2: Query

1. API/Webhook Trigger: POST, path /demo/rag-ask, Response Mode Wait, so the caller gets the result. The body is { "question": "…" }.

2. Embeddings named Embed question: the same provider and model as indexing, Input {{ $trigger.payload.question }}.

3. Vector Database named Search:

   • Operation: query, Collection: docs
   • Query vector: {{ $node["Embed question"].output.embeddings[0] }}
   • Top K: 3

   It outputs results, each with id, score and metadata.

4. Code node named Build context:

   const results = {{ $node["Search"].output.results }};
   const context = results
     .map(r => r.metadata.content + '\n[Source: ' + r.metadata.source + ']')
     .join('\n\n');
   return [{ context: context }];

5. AI Agent:

   • System instructions: "Answer the question using only the context. If the answer is not in the context, say 'I don't know'. Cite the source for each claim."
   • Prompt:

     Context:
     {{ $node["Build context"].output.context }}
     
     Question: {{ $trigger.payload.question }}

   • Temperature: 0.2

   The answer is in {{ $node["AI Agent"].output.content }}.

Activate both workflows.

Try it

Index a document:

curl -X POST <indexing webhook URL> \
  -H "Content-Type: application/json" \
  -d '{"source": "refund-policy", "text": "Products bought in a sale can be refunded within 14 days. ..."}'

Then ask:

curl -X POST <query webhook URL> \
  -H "Content-Type: application/json" \
  -d '{"question": "How long do I have to return a sale item?"}'

The response has results[0].status and the run's data. Open the run in Logs to read the AI Agent's answer and its cited source.

Ask something the document does not cover. The answer should be "I don't know". That is the check that the model is not making things up.

Variations

Rerank before answering

Search with Top K 10, then add a RAG Reranker between the Code node and the AI Agent. Pass it documents as a list of { id, content, score } and the query, and keep the best 3.

Separate tenants

Add a tenant key to each chunk's metadata at index time. At query time, set the Vector Database Filter to { "tenant": "acme" }.

Check the answer

Add an AI Guardrails node after the AI Agent with operation hallucination_check, and pass the context as Source documents.

Troubleshooting

Next

• RAG nodes
• AI & LLM nodes
• Recipes

Found something out of date? This page lives in the Flero docs content set.