Google Colab

The fact that most individual neurons are uninterpretable presents a serious roadblock to a mechanistic understanding of language models. We demonstrate a method for decomposing groups of neurons into interpretable features with the potential to move past that roadblock.

Introduction

This library provides utilities for generating and scoring text explanations of sparse autoencoder (SAE) features. The explainer and scorer models can be run locally or acessed using API calls via OpenRouter.

Note that we're still actively cleaning up the codebase and scripts.

Installation

Install this library as a local editable install... See more

model-agnostic meta-learning, or MAML, it trains a model using two machine-learning processes, one nested inside the other.[...] setting his AI the challenge of learning more than one task led it to come up with its own version of a sparse model that outperformed human-designed ones.

IMO @AnthropicAI is very close to making a breakthrough in productizable interpretability. For ~4 years all we've had to really control LLMs is temperature/top_p and logit bias. We recently got `seed` and constrained structured output, with `interactive=false` on the way. But now Claude Sonnet can let people clamp up/down 34 million features rang... See more

Model Explorer is a powerful graph visualization tool that helps one understand, debug, and optimize ML models. It specializes in visualizing large graphs in an intuitive, hierarchical format, but works well for smaller models as well.

Graph visualization plays a pivotal role in the machine learning (ML) development process. Visual representations o... See more