Chat Completions

The Chat API provides methods for interacting with Goodfire’s language models in a chat format. The base chat interface is OpenAI-compatible. It supports both streaming and non-streaming completions, as well as logits computation.

Once you have a model variant, you can use it to create chat completions.

​

Examples

​

Basic Chat Completion

​

Streaming Chat Completion

​

Using with Model Variants

​

Computing Token Probabilities

​

Methods

​

create()

Create a chat completion with the model.

Parameters:

Returns:

• If stream=False: ChatCompletion object
• If stream=True: Iterator of StreamingChatCompletionChunk objects

Examples:

Non-streaming completion:

response =  client.chat.completions.create(
    messages=[
        {"role": "user", "content": "What is the capital of France?"}
    ],
    model="meta-llama/Llama-3.3-70B-Instruct"
)
print(response.choices[0].message["content"])

Streaming completion:

for chunk in client.chat.completions.create(
    messages=[
        {"role": "user", "content": "Write a short poem"}
    ],
    model="meta-llama/Llama-3.3-70B-Instruct",
    stream=True
):
    print(chunk.choices[0].delta.content, end="")

​

logits()

Compute token probabilities for the next token in a conversation.

Parameters:

Returns: LogitsResponse containing token probabilities

Example:

logits =  client.chat.logits(
    messages=[
        {"role": "user", "content": "The capital of France is"}
    ],
    model="meta-llama/Llama-3.3-70B-Instruct",
    filter_vocabulary=["Paris", "London", "Berlin"]
)
print(logits.logits)  

​

Response Objects

​

ChatCompletion

Response from a non-streaming chat completion.

Properties:

​

StreamingChatCompletionChunk

Individual chunk from a streaming chat completion.

Properties:

​

LogitsResponse

Response from a logits computation request.

Properties: