Heterogeneous Low-Bandwidth Pre-Training of LLMs.

Amir Sarfi

Yazan Obeidi Amir Sarfi Joel Lidin Paul Janson Eugene Belilovsky Heterogeneous Low-Bandwidth Pre-Training of LLMs. 2026 January abs/2601.02360 CoRR https://doi.org/10.48550/arXiv.2601.02360 db/journals/corr/corr2601.html#abs-2601-02360 streams/journals/corr

Joel Lidin Amir Sarfi Erfan Miahi Quentin Anthony Shivam Chauhan Evangelos Pappas Benjamin Thérien Eugene Belilovsky Samuel Dare Covenant-72B: Pre-Training a 72B LLM with Trustless Peers Over-the-Internet. 2026 March abs/2603.08163 CoRR https://doi.org/10.48550/arXiv.2603.08163 db/journals/corr/corr2603.html#abs-2603-08163 streams/journals/corr

Joel Lidin Amir Sarfi Evangelos Pappas Samuel Dare Eugene Belilovsky Jacob Steeves Incentivizing Permissionless Distributed Learning of LLMs. 12-18 2025 DAI https://doi.org/10.1145/3772429.3772431 conf/dai2/2025 db/conf/dai2/dai2025.html#LidinSPDBS25

Joel Lidin Amir Sarfi Evangelos Pappas Samuel Dare Eugene Belilovsky Jacob Steeves Incentivizing Permissionless Distributed Learning of LLMs. 2025 May abs/2505.21684 CoRR https://doi.org/10.48550/arXiv.2505.21684 db/journals/corr/corr2505.html#abs-2505-21684 streams/journals/corr

Amir Sarfi Benjamin Thérien Joel Lidin Eugene Belilovsky Communication Efficient LLM Pre-training with SparseLoCo. 2025 August abs/2508.15706 CoRR https://doi.org/10.48550/arXiv.2508.15706 db/journals/corr/corr2508.html#abs-2508-15706 streams/journals/corr

Quentin Anthony Eugene Belilovsky Shivam Chauhan Samuel Dare Paul Janson Joel Lidin Erfan Miahi Yazan Obeidi Evangelos Pappas Jacob Steeves Benjamin Thérien