We showcase the importance of good query contextualization in document retrieval Q&A scenarios with various industry examples. We present approaches to improve contextualization by prompt engineering with TextGrad and show how crucial these improvements are for delivering industry-grade chatbot solutions and customer satisfaction.
Background: When performing document retrieval in a Q&A/chatbot scenario, just using the last Q&A message, namely the current human inquiry, as a target might not yield good results. Often the context needed for retrieving relevant documents is spread out over several previous messages. Query contextualization helps by transforming a message history into a singular retrieval query including the relevant context. Main threats to the quality of the retrieval are missing context, a direct answer to the question, and follow-up questions back to the user in the contextualized query. These threats can be contained by a good choice of the contextualization model and a well-designed system prompt.
