Research

A mixed-modality, Mamba-based foundation model for metagenomic contextualization of proteins. Where most state-of-the-art genomic language models rely on transformers — whose quadratic attention caps sequence length and captures only a fraction of assembled metagenomic sequences — Nammu uses a state space architecture that scales linearly with length, enabling modeling of long contigs spanning multiple coding and non-coding regions. With 167M parameters, it is a bidirectional encoder trained on the OpenMetaGenomic (OMG) corpus at a 20K context length using a curriculum strategy: first on 64M protein sequences, then on 32M mixed-modality contigs. Nammu outperforms the gLM2 transformer at every taxonomic level on the CAMI marine benchmark and on KEGG Orthology prediction in deep-sea metagenome-assembled genomes.

A protein language model for predicting and classifying microbial proteins involved in biogeochemical cycling — methane, sulfur, nitrogen, and phosphorus transformations. Built on the ESM-2 transformer architecture and fine-tuned on four curated metabolic pathway databases (MCycDB, NCycDB, PCycDB, SCycDB), BioGeoFormer covers 610 unique gene families across 37 metabolic pathways. We trained 8 models at clustered identity splits ranging from 20–90% dissimilarity; the 70% split model is most effective at identifying remote homologues. BioGeoFormer complements classical homology-based tools for metagenome and genome mining.

Biological databases encode rich, structured knowledge about proteins, pathways, and interactions. I am developing graph neural network frameworks that can leverage this structured knowledge to improve model performance and generalization, particularly for tasks where experimental labels are expensive or limited.

A from-scratch, modular pipeline for training transformer models in JAX. It implements the full stack — tokenizer and dataset loading, the transformer architecture, a trainer with a learning-rate scheduler and loss functions, and evaluation metrics — as a clean, configurable codebase for experimenting with language model training.

Multi-cancer risk prediction from individual genotype data using The Cancer Genome Atlas. A standard feed-forward network failed to generalize; the best-performing model was an encoder-based architecture that jointly learns a latent representation and a classifier. Analysis revealed deep entanglement between cancer types in the latent space, motivating further interpretability work on which genomic loci drive classification.

M.Sc. capstone project at the University of Hawaiʻi at Mānoa (2021). I tested whether a classifier trained on genomic data encodes class similarity in its learned representations — experimenting on both simulated and real data from The Cancer Genome Atlas. By analyzing the learned parameters directly, I explored whether neural networks can surface biologically meaningful structure beyond their primary classification objective.