langchain/docs/modules/models/llms/integrations/gpt4all.ipynb

GPT4All

GitHub:nomic-ai/gpt4all an ecosystem of open-source chatbots trained on a massive collections of clean assistant data including code, stories and dialogue.

This example goes over how to use LangChain to interact with GPT4All models.

In [1]:

%pip install pyllamacpp > /dev/null

Note: you may need to restart the kernel to use updated packages.

In [1]:

from langchain import PromptTemplate, LLMChain
from langchain.llms import GPT4All
from langchain.callbacks.base import CallbackManager
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler

In [2]:

template = """Question: {question}

Answer: Let's think step by step."""

prompt = PromptTemplate(template=template, input_variables=["question"])

Specify Model

To run locally, download a compatible ggml-formatted model. For more info, visit https://github.com/nomic-ai/pyllamacpp

For full installation instructions go here.

The GPT4All Chat installer needs to decompress a 3GB LLM model during the installation process!

Note that new models are uploaded regularly - check the link above for the most recent .bin URL

In [ ]:

local_path = './models/gpt4all-lora-quantized-ggml.bin'  # replace with your desired local file path

Uncomment the below block to download a model. You may want to update url to a new version.

In [ ]:

# import requests

# from pathlib import Path
# from tqdm import tqdm

# Path(local_path).parent.mkdir(parents=True, exist_ok=True)

# # Example model. Check https://github.com/nomic-ai/pyllamacpp for the latest models.
# url = 'https://the-eye.eu/public/AI/models/nomic-ai/gpt4all/gpt4all-lora-quantized-ggml.bin'

# # send a GET request to the URL to download the file. Stream since it's large
# response = requests.get(url, stream=True)

# # open the file in binary mode and write the contents of the response to it in chunks
# # This is a large file, so be prepared to wait.
# with open(local_path, 'wb') as f:
#     for chunk in tqdm(response.iter_content(chunk_size=8192)):
#         if chunk:
#             f.write(chunk)

In [ ]:

# Callbacks support token-wise streaming
callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])
# Verbose is required to pass to the callback manager
llm = GPT4All(model=local_path, callback_manager=callback_manager, verbose=True)

In [ ]:

llm_chain = LLMChain(prompt=prompt, llm=llm)

In [ ]:

question = "What NFL team won the Super Bowl in the year Justin Bieber was born?"

llm_chain.run(question)