Paving the way to efficient architectures: StripedHyena-7B, open source models offering a glimpse into a world beyond Transformers

One of the focus areas at Together Research is new architectures for long context, improved training, and inference performance over the Transformer architecture. Spinning out of a research program from our team and academic collaborators, with roots in signal processing-inspired sequence models, we are excited to introduce the StripedHyena models.... See more