blog/rag-pipeline-tutorial

[giscus[bot]]

blog/rag-pipeline-tutorial

Step by step tutorial: how to build a production-ready RAG pipeline.

https://zackproser.com/blog/rag-pipeline-tutorial

[scott91e1]

Thanks for publishing the tutorial, very helpful. I have a question about the project I am working on. Claude suggested that for JSON RAG that this would be an acceptable format:

{
  "table_metadata": {
    "title": "Product Inventory",
    "last_updated": "2025-01-03"
  },
  "columns": ["product_id", "name", "quantity", "price"],
  "data": [
    {"product_id": "P001", "name": "Widget A", "quantity": 100, "price": 19.99},
    {"product_id": "P002", "name": "Widget B", "quantity": 150, "price": 24.99}
  ]
}

Would it be possible to ingest and query these types of documents into your solution?

[zackproser]

@scott91e1

Thank you so much for the feedback. I'm really glad to hear the tutorial was helpful!

Would you mind telling me how you found it?

In terms of your project goals - yes, this general RAG pipeline could be adopted to handle many different types of documents - and depending upon your requirements that may potentially involve:

• Some tweaks to the data processing Jupyter Notebook to ingest / chunk / upsert documents of that format
• Modifications to the prompt you send to the LLM along with the retrieved context / related items - mostly to specify if you have a specific output format you're looking for - there's also now zod and Structured Outputs from OpenAI which would likely be relevant as well.
• Modifications to the UI code to display your related products, if they have a significantly different format than the related posts example objects in the companion site for this tutorial

Feel free to send me more info about what you're trying to accomplish end to end and I'll see if there's any other helpful examples or info I can share.