Our DeMo paper has been accepted to ICLR 2026!
Developed by @bloc97_ and @theemozilla, DeMo helped kickstart industry research into communication-efficient optimizers. Today, it has become the core building block for Nous' low-bandwidth distributed training network.
This year's ICLR class was particularly tough, with an acceptance rate of 28% across more than 19,000 submissions. We're thankful to the area chairs and reviewers for their overwhelming and difficult work.
If you're in Rio in April, keep an eye out for our team (and some Nous swag)
http:/arxiv.org/pdf/2411.19870
