langchain/docs/use_cases/evaluation/qa_generation.ipynb

QA Generation

This notebook shows how to use the QAGenerationChain to come up with question-answer pairs over a specific document. This is important because often times you may not have data to evaluate your question-answer system over, so this is a cheap and lightweight way to generate it!

In [1]:

from langchain.document_loaders import TextLoader

In [2]:

loader = TextLoader("../../modules/state_of_the_union.txt")

In [3]:

doc = loader.load()[0]

In [4]:

from langchain.chat_models import ChatOpenAI
from langchain.chains import QAGenerationChain
chain = QAGenerationChain.from_llm(ChatOpenAI(temperature = 0))

In [8]:

qa = chain.run(doc.page_content)

In [10]:

qa[1]

Out [10]:

{'question': 'What is the U.S. Department of Justice doing to combat the crimes of Russian oligarchs?',
 'answer': 'The U.S. Department of Justice is assembling a dedicated task force to go after the crimes of Russian oligarchs.'}

In [ ]: