langchain/docs/examples/model_laboratory.ipynb

Model Laboratory

This example goes over basic functionality of how to use the ModelLaboratory to test out and try different models.

In [1]:

from langchain import LLMChain, OpenAI, Cohere, HuggingFaceHub, Prompt
from langchain.model_laboratory import ModelLaboratory

In [2]:

llms = [
    OpenAI(temperature=0), 
    Cohere(model="command-xlarge-20221108", max_tokens=20, temperature=0), 
    HuggingFaceHub(repo_id="google/flan-t5-xl", model_kwargs={"temperature":1})
]

In [3]:

model_lab = ModelLaboratory(llms)

In [4]:

model_lab.compare("What color is a flamingo?")

Input:
What color is a flamingo?

OpenAI
Params: {'model': 'text-davinci-002', 'temperature': 0.0, 'max_tokens': 256, 'top_p': 1, 'frequency_penalty': 0, 'presence_penalty': 0, 'n': 1, 'best_of': 1}


Flamingos are pink.

Cohere
Params: {'model': 'command-xlarge-20221108', 'max_tokens': 20, 'temperature': 0.0, 'k': 0, 'p': 1, 'frequency_penalty': 0, 'presence_penalty': 0}


Pink

HuggingFaceHub
Params: {'repo_id': 'google/flan-t5-xl', 'temperature': 1}
pink

In [5]:

prompt = Prompt(template="What is the capital of {state}?", input_variables=["state"])
model_lab_with_prompt = ModelLaboratory(llms, prompt=prompt)

In [6]:

model_lab_with_prompt.compare("New York")

Input:
New York

OpenAI
Params: {'model': 'text-davinci-002', 'temperature': 0.0, 'max_tokens': 256, 'top_p': 1, 'frequency_penalty': 0, 'presence_penalty': 0, 'n': 1, 'best_of': 1}


The capital of New York is Albany.

Cohere
Params: {'model': 'command-xlarge-20221108', 'max_tokens': 20, 'temperature': 0.0, 'k': 0, 'p': 1, 'frequency_penalty': 0, 'presence_penalty': 0}


The capital of New York is Albany.

HuggingFaceHub
Params: {'repo_id': 'google/flan-t5-xl', 'temperature': 1}
st john s

In [ ]: