Skip to main content

Prompt Iteration

Prompt Iteration is a popular way to optimize your RAG System. Refining your prompts can significantly improve your system's performance at low cost and effort. RAG Workbench has a specific tool designed for refining the prompts. It allows you to experiment with your prompt using relevant metadata (ex. retrieved context) and try different models as well.

Pre-requisite: Setup Tracing​

Debugging prompts is enabled when you have specified add_query_event() with the necessary metatdata to your traced application using the LastMile AI Tracing SDK. Refer to the Tracing documentation to set up tracing.

1. Launch RAG Workbench UI​

Start by launching RAG Workbench UI from your terminal with this CLI command:

rag-debug launch

2. 'Traces' Page​

Go to the 'Traces' page within the RAG Workbench UI that displays all your traces.

Click on the 'Debug Icon' associated with a specific trace.

click_debug_trace

3. Select 'Resolve-Prompt' Span​

In the trace view, you will see all spans within the trace. Select the 'Resolve-Prompt' span since this is the span where we logged add_query_event.

resolve prompt

4. Click 'Debug Prompts'​

debug prompt

5. Iterate on Prompt​

You now have a prompt playground with access to information such as the retrieved context, model, prompt template, and metadata.

debugger view

More Resources​

Here are other helpful guides: