The Chat API provides methods for interacting with Goodfire’s language models in a chat format. The base chat interface is OpenAI-compatible. It supports both streaming and non-streaming completions, as well as logits computation. Once you have a model variant, you can use it to create chat completions.

​
Examples

​
Basic Chat Completion

# Initialize the client
client = goodfire.Client(
    '{YOUR_API_KEY}',
)

# Create a non-streaming completion
response =  client.chat.completions.create(
    messages=[
        {"role": "user", "content": "What is the capital of France?"}
    ],
    model="meta-llama/Llama-3.1-8B-Instruct"
)

print(response.choices[0].message["content"])

​
Streaming Chat Completion

# Stream the response tokens
for chunk in client.chat.completions.create(
    messages=[
        {"role": "user", "content": "Write a short poem"}
    ],
    model="meta-llama/Llama-3.1-8B-Instruct",
    stream=True
):
    print(chunk.choices[0].delta.content, end="")

​
Using with Model Variants

# Create a variant with feature modifications
variant = goodfire.Variant("meta-llama/Llama-3.1-8B-Instruct")
pirate_features = client.features.search(
    "talk like a pirate",
    model=variant,
    top_k=1
)
variant.set(pirate_features[0], 0.5)

# Use the variant in chat completion
for token in client.chat.completions.create(
    messages=[
        {"role": "user", "content": "Tell me about the ocean."}
    ],
    model=variant,
    stream=True,
    max_completion_tokens=100,
):
    print(token.choices[0].delta.content, end="")

​
Computing Token Probabilities

logits =  client.chat.logits(
    messages=[
        {"role": "user", "content": "The capital of France is"}
    ],
    model="meta-llama/Llama-3.1-8B-Instruct",
    filter_vocabulary=["Paris", "London", "Berlin"]
)
print(logits.logits)

​
Methods

​
create()

Create a chat completion with the model. Parameters:
​
messages
list[ChatMessage]
required
List of messages in the conversation. Each message should have role (“user”, “assistant”, or “system”) and content fields.
​
model
Union[str, VariantInterface]
required
Model identifier or variant to use for completion
​
stream
bool
default:"False"
Whether to stream the response tokens
​
max_completion_tokens
Optional[int]
default:"2048"
Maximum number of tokens to generate
​
top_p
float
default:"0.9"
Nucleus sampling parameter
​
temperature
float
default:"0.6"
Sampling temperature
​
stop
Optional[Union[str, list[str]]]
Sequences where the API will stop generating further tokens
​
seed
Optional[int]
default:"42"
Random seed for reproducible outputs
​
system_prompt
str
System prompt to prepend to the conversation
Returns:
  • If stream=False: ChatCompletion object
  • If stream=True: Iterator of StreamingChatCompletionChunk objects
Examples: Non-streaming completion:
Basic Chat Completion
response =  client.chat.completions.create(
    messages=[
        {"role": "user", "content": "What is the capital of France?"}
    ],
    model="meta-llama/Llama-3.1-8B-Instruct"
)
print(response.choices[0].message["content"])
Streaming completion:
Streaming Chat Completion
for chunk in client.chat.completions.create(
    messages=[
        {"role": "user", "content": "Write a short poem"}
    ],
    model="meta-llama/Llama-3.1-8B-Instruct",
    stream=True
):
    print(chunk.choices[0].delta.content, end="")

​
logits()

Compute token probabilities for the next token in a conversation. Parameters:
​
messages
list[ChatMessage]
required
List of messages in the conversation
​
model
Union[str, VariantInterface]
required
Model identifier or variant to use
​
top_k
Optional[int]
Limit response to top K most likely tokens
​
filter_vocabulary
Optional[list[str]]
List of tokens to compute probabilities for
Returns: LogitsResponse containing token probabilities Example:
Token Probabilities
logits =  client.chat.logits(
    messages=[
        {"role": "user", "content": "The capital of France is"}
    ],
    model="meta-llama/Llama-3.1-8B-Instruct",
    filter_vocabulary=["Paris", "London", "Berlin"]
)
print(logits.logits)

​
Response Objects

​
ChatCompletion

Response from a non-streaming chat completion. Properties:
​
id
str
Unique identifier for the completion
​
object
str
Object type identifier
​
created
Optional[int]
Unix timestamp of when the completion was created
​
model
str
ID of the model used
​
system_fingerprint
str
System fingerprint for the completion
​
choices
list[ChatCompletionChoice]
List of completion choices

​
StreamingChatCompletionChunk

Individual chunk from a streaming chat completion. Properties:
​
id
str
Unique identifier for the completion
​
object
str
Object type identifier
​
created
Optional[int]
Unix timestamp of when the chunk was created
​
model
str
ID of the model used
​
system_fingerprint
str
System fingerprint for the completion
​
choices
list[StreamingChoice]
List of completion choices in this chunk

​
LogitsResponse

Response from a logits computation request. Properties:
​
logits
dict[str, float]
Dictionary mapping tokens to their probabilities