0 Fork 0

mirror of https://github.com/langchain-ai/langchain.git synced 2026-10-14 05:39:23 +03:00

Files

T

lonestrikerandHien Ngo 6f36f0f930 Add oobabooga/text-generation-webui support as a llm (#5997 )

Add oobabooga/text-generation-webui support as an LLM. Currently,
supports using text-generation-webui's non-streaming API interface.
Allows users who already have text-gen running to use the same models
with langchain.

#### Before submitting

Simple usage, similar to existing LLM supported:

```
from langchain.llms import TextGen
llm = TextGen(model_url = "http://localhost:5000")
```
#### Who can review?

 @hwchase17 - project lead

---------

Co-authored-by: Hien Ngo <Hien.Ngo@adia.ae>

2023-06-17 09:42:15 -07:00

2.4 KiB

Raw Blame History

TextGen

GitHub:oobabooga/text-generation-webui A gradio web UI for running Large Language Models like LLaMA, llama.cpp, GPT-J, Pythia, OPT, and GALACTICA.

This example goes over how to use LangChain to interact with LLM models via the text-generation-webui API integration.

Please ensure that you have text-generation-webui configured and an LLM installed. Recommended installation via the one-click installer appropriate for your OS.

Once text-generation-webui is installed and confirmed working via the web interface, please enable the api option either through the web model configuration tab, or by adding the run-time arg --api to your start command.

Set model_url and run the example

In [1]:

model_url = "http://localhost:5000"

In [ ]:

import langchain
from langchain import PromptTemplate, LLMChain
from langchain.llms import TextGen

langchain.debug = True

template = """Question: {question}

Answer: Let's think step by step."""


prompt = PromptTemplate(template=template, input_variables=["question"])
llm = TextGen(model_url=model_url)
llm_chain = LLMChain(prompt=prompt, llm=llm)
question = "What NFL team won the Super Bowl in the year Justin Bieber was born?"

llm_chain.run(question)

2.4 KiB Raw Blame History

TextGen

Set model_url and run the example

2.4 KiB

Raw Blame History