Beyond
next-token prediction.
Research note · v1.0
Abstract
We research how foundation models are pretrained. Most learn by predicting the next token. We work on other training objectives, and on representations that stay useful when you move to a new task or type of data.
Selected writing
Recent notes from the lab.
Improving synthetic data generation bounds via constrained decoding.
Interlocking specialized models: routing and merging domain experts for compound AI systems.
Adversarial robustness in domain-specific models: red-teaming beyond the generic benchmark.
Product
What the research ships as.
A system of domain-specialized models, each trained to be an expert in its vertical.
Milestones
Since 2025- 2025.Q4
Domain annotation service
- 2026.Q1
Accepted to NVIDIA Innovation Labs
- 2026.Q2
Research
In progress - 2026.Q3
Kestrel
Classifier model
Team
Two founders. USA, Japan.
Philip Abao
Architecture & training infrastructure.
Soraya Johnson
Data & evaluation.