Features

​

Accessing Features

Before using the Features API, you’ll need a model variant:

variant = Variant("meta-llama/Llama-3.3-70B-Instruct")

You can then access features through the client’s features interface. For example, to search for features:

# Search for features related to a concept
features = client.features.search(
    "angry",
    model=variant,
    top_k=5
)

# Print the found features
print(features)

Or inspect feature activations in text:

# Analyze how features activate in text
inspector = client.features.inspect(
    [
        {"role": "user", "content": "What do you think about pirates and whales"},
        {"role": "assistant", "content": "I think pirates are cool and whales are cool"}
    ],
    model=variant
)

# Get top activated features
for activation in inspector.top(k=5):
    print(f"{activation.feature.label}: {activation.activation}")

The Features API provides methods for working with interpretable features of language models. Features represent learned patterns in model behavior that can be analyzed and modified.

​

Methods

​

neighbors()

Get the nearest neighbors of a feature or group of features.

Parameters:

Returns: FeatureGroup

Example:

# Get neighbors of a feature
neighbors =  client.features.neighbors(
    feature,
    model="meta-llama/Llama-3.3-70B-Instruct",
    top_k=10
)

# Print neighbor labels
for neighbor in neighbors:
    print(neighbor.label)

​

search()

Search for features based on semantic similarity to a query string.

Parameters:

Returns: FeatureGroup - Collection of matching features

Example:

# Search for features related to writing style
features = client.features.search(
    "formal writing style",
    model="meta-llama/Llama-3.3-70B-Instruct",
    top_k=10
)

# Print features
for feature in features:
    print(feature.label)

​

inspect()

Analyzes how features are activated across the input messages.

Parameters:

Returns: ContextInspector - An inspector object that provides methods for analyzing and visualizing how features are activated in the given context.

Example:

# Analyze how features activate in text
inspector = client.features.inspect(
    [
        {"role": "user", "content": "What do you think about pirates and whales"},
        {"role": "assistant", "content": "I think pirates are cool and whales are cool"}
    ],
    model=variant
)

# Get top activated features
for activation in inspector.top(k=5):
    print(f"{activation.feature.label}: {activation.activation}")

​

contrast()

Identify features that differentiate between two conversation datasets.

Parameters:

Returns: tuple[FeatureGroup, FeatureGroup] - Two FeatureGroups containing:

• Features steering towards dataset_1
• Features steering towards dataset_2

Example:

# Compare formal vs informal conversations
dataset_1 = [[
    {"role": "user", "content": "Hi how are you?"},
    {"role": "assistant", "content": "I'm doing well..."}
]]
dataset_2 = [[
    {"role": "user", "content": "Hi how are you?"},
    {"role": "assistant", "content": "Arr my spirits be high..."}
]]

formal_features, informal_features =  client.features.contrast(
    dataset_1=dataset_1,
    dataset_2=dataset_2,
    model="meta-llama/Llama-3.3-70B-Instruct",
    top_k=5
)

​

rerank()

Rerank a set of features based on a query.

Parameters:

Returns: FeatureGroup

Example:

# Rerank features based on relevance to "writing style"
reranked = client.features.rerank(
    features=formal_features,
    query="writing style",
    model="meta-llama/Llama-3.3-70B-Instruct",
    top_k=10
)

​

activations()

Retrieves feature activation values for each token in the input messages.

Parameters:

Returns: NDArray[np.float64] - Sparse activation matrix of shape [n_tokens, n_features] where each element represents the activation strength of a feature at a specific token. Most values are zero due to sparsity.

Example:

# Get activation matrix for a conversation
matrix =  client.features.activations(
    messages=[{"role": "user", "content": "Hello world"}],
    model="meta-llama/Llama-3.3-70B-Instruct"
)

​

lookup()

Retrieves details for a list of features by their indices.

Parameters:

Returns: dict[int, Feature] - Mapping of feature index to Feature object

​

list()

Retrieves details for a list of features by their UUIDs.

Parameters:

Returns: FeatureGroup - Collection of Feature objects

​

Classes

​

Feature

A class representing a human-interpretable “feature” - a model’s conceptual neural unit. Features can be combined into groups and compared using standard operators.

​

FeatureGroup

A collection of Feature instances with group operations.

​

ConditionalGroup

Groups multiple conditions with logical operators.

​

FeatureActivation

Represents the activation of a feature.

​

ContextInspector

Analyzes feature activations in text.

# Get top 5 activated features
top_features = inspector.top(k=5)