Most models write one word at a time. Ours starts from blanks and fills in the words it is surest of, pass after pass, on a state-space backbone instead of attention.
An independent research lab in Abu Dhabi.
Why is the sky blue?
The answer starts as thirty-two blanks.
Illustrative. Words stand in for tokens. The bar under each is the model's confidence when it committed.
DIMBA pairs masked diffusion with a bidirectional Mamba-2 backbone. It reads the whole masked sequence from both directions, commits the tokens it is most sure of, and repeats until no masks remain.
Every other masked diffusion text model we know of sits on a transformer.
A fixed-size state per layer. Long context costs linear time, and nothing piles up in a cache.
Many positions fill in parallel. The model can reconstruct missing text and resists repetitive loops.
The number of passes is a setting. More compute for a better answer, less for a faster one, same weights.
hr-diffuse-1-nano, a 288M-parameter DIMBA model distilled from SmolLM-135M, lost the factual test to its teacher and led every baseline on infill. The losses are published with the wins.
| Model | Factual answers40 questions, higher is better | Repetitive loopslower is better | Infill recoveryhigher is better |
|---|---|---|---|
| hr-diffuse-1-nano | 15.0% | 7.5% | 14.0% |
| SmolLM-135M, the teacher | 82.5% | 37.5% | 2.9% |
| SmolLM-135M-Instruct | 60.0% | 2.5% | 0.0% |
| GPT-2 | 20.0% | 90.0% | 0.0% |
| Pythia-160M | 10.0% | 15.0% | 1.7% |
From the July 2026 technical report. SmolLM-135M-Instruct loops less than nano, so no general claim about repetition.
self-correction methods tried. None worked at this scale.
critic precision at flagging a wrong token. Chance is 10%.
of real errors ranked above a correct token. Chance is 50%.
What the first model gets right, what it gets wrong, and what we would do with more compute.
Read the report or download the model on Hugging Face
An earlier preprint describes the first DIMBA design, which used latent diffusion. That design has been replaced and the preprint is kept only for the record.
An independent lab in Abu Dhabi, led by Faris Allafi, architect of DIMBA and one of the youngest published researchers working on post-transformer architectures.
The current generation of language models is defined by an architectural choice, not a law of nature. Better dynamics are worth building openly, with technical depth as the currency.
Formerly operating as DimbaLabs.
