A Mamba-based state space foundation model for metagenomic contextualization of proteins.
Nammu is a 167M-parameter, mixed-modality foundation model built on the Mamba state space architecture. Because it scales linearly with sequence length rather than quadratically like transformers, it can read long metagenomic contigs — spanning many coding and non-coding regions — at a 20K-token context. It outperforms the comparable gLM2 transformer on both taxonomy and function prediction, despite training on fewer tokens.
Since metagenomics — the culture-free sequencing of microbial community genomes — took off around 2011, sequence data has grown more than 500-fold across ecosystems. Annotation has not kept pace: over half of all sequences still lack a confident functional assignment, leaving large gaps in our understanding of microbial processes central to planetary and human health.
Genomic language models have started to close this gap, but most state-of-the-art models are transformers. The quadratic cost of attention caps how long a sequence they can read, so they capture only a fraction of an assembled metagenomic sequence and cannot reason over broad context across multiple coding and non-coding regions.
Nammu is a bidirectional encoder trained on the OpenMetaGenomic (OMG) corpus at a 20K context length using a two-stage curriculum: first on 64M protein sequences, then on 32M mixed-modality metagenomic contigs. This lets a single model reason jointly over proteins and the genomic context they sit in.
Against gLM2 — a mixed-modality transformer trained on the same OMG corpus with 37% more tokens — Nammu wins at every taxonomic level on taxonomy inference over a marine dataset from the Critical Assessment of Metagenome Interpretation (CAMI). On function, Nammu also outperforms gLM2 (150M) at KEGG Orthology prediction in deep-sea metagenome-assembled genomes.