langchain/docs/modules/models/llms/integrations/deepinfra_example.ipynb

DeepInfra LLM Example

This notebook goes over how to use Langchain with DeepInfra.

Imports

In [ ]:

import os
from langchain.llms import DeepInfra
from langchain import PromptTemplate, LLMChain

Set the Environment API Key

Make sure to get your API key from DeepInfra. You are given a 1 hour free of serverless GPU compute to test different models. You can print your token with deepctl auth token

In [ ]:

os.environ["DEEPINFRA_API_TOKEN"] = "YOUR_KEY_HERE"

Create the DeepInfra instance

Make sure to deploy your model first via deepctl deploy create -m google/flat-t5-xl (for example)

In [ ]:

llm = DeepInfra(model_id="DEPLOYED MODEL ID")

Create a Prompt Template

We will create a prompt template for Question and Answer.

In [ ]:

template = """Question: {question}

Answer: Let's think step by step."""

prompt = PromptTemplate(template=template, input_variables=["question"])

Initiate the LLMChain

In [ ]:

llm_chain = LLMChain(prompt=prompt, llm=llm)

Run the LLMChain

Provide a question and run the LLMChain.

In [ ]:

question = "What NFL team won the Super Bowl in 2015?"

llm_chain.run(question)