An Engineering Journey Training Large Language Models at Scale on Alps: The Apertus Experience.

Josh Romero

Jonathan Coles Stefano Schuppli Lukas Drescher Fawzi Mohamed Elia Palme Henrique Mendonça Miguel Gila Mark Klein 0002 Maxime Martinasso Joost VandeVondele Torsten Hoefler Thomas C. Schulthess Josh Romero Igor Gorodetsky Ryan Hankins Isa Wazirzada Martin Jaggi Antoine Bosselut Imanol Schlag Antoni-Joan Solergibert i Llaquet Alejandro Hernández-Cano Theofilos Ioannis Manitaras Nicholas John Browning An Engineering Journey Training Large Language Models at Scale on Alps: The Apertus Experience. 2026 April abs/2604.12973 CoRR https://doi.org/10.48550/arXiv.2604.12973 db/journals/corr/corr2604.html#abs-2604-12973 streams/journals/corr

Abigail C. Dommer Lorenzo Casalino Fiona L. Kearns Mia A. Rosenfeld Nicholas Wauer Surl-Hee Ahn John Russo 0003 A. Sofia F. Oliveira Clare Morris Anthony T. Bogetti Anda Trifan Alexander Brace Terra Sztain Austin Clyde Heng Ma S. Chakra Chennubhotla Hyungro Lee Matteo Turilli Syma Khalid Teresa Tamayo-Mendoza Matthew Welborn Anders S. Christensen Daniel G. A. Smith Zhuoran Qiao 0001 Sai Krishna Sirumalla Michael O'Connor Frederick R. Manby Anima Anandkumar David J. Hardy James C. Phillips Abraham C. Stern Josh Romero David Clark Mitchell Dorrell Tom Maiden Lei Huang 0019 John D. McCalpin Christopher J. Woods Alan Gray Matt Williams Bryan Barker Harinda Rajapaksha Richard Pitts Tom Gibbs John E. Stone Daniel M. Zuckerman Adrian J. Mulholland Thomas F. Miller III Shantenu Jha Arvind Ramanathan Lillian T. Chong Rommie E. Amaro #COVIDisAirborne: AI-enabled multiscale computational microscopy of delta SARS-CoV-2 in a respiratory aerosol. 28-44 2023 37 Int. J. High Perform. Comput. Appl. 1 https://doi.org/10.1177/10943420221128233 db/journals/ijhpca/ijhpca37.html#DommerCKRWAROMB23

Maxim Zvyagin Alexander Brace Kyle Hippe Yuntian Deng Bin Zhang Cindy Orozco Bohorquez Austin Clyde Bharat Kale Danilo Perez-Rivera Heng Ma Carla M. Mann Michael W. Irvin Defne G. Ozgulbas Natalia Vassilieva J. Gregory Pauloski Logan T. Ward Valérie Hayot-Sasson Murali Emani Sam Foreman Zhen Xie Diangen Lin Maulik Shukla Weili Nie Josh Romero Christian Dallago Arash Vahdat Chaowei Xiao Thomas Gibbs Ian T. Foster James J. Davis 0002 Michael E. Papka Thomas S. Brettin Rick Stevens Anima Anandkumar Venkatram Vishwanath Arvind Ramanathan GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamics. 683-705 2023 November 37 Int. J. High Perform. Comput. Appl. 6 https://doi.org/10.1177/10943420231201154 db/journals/ijhpca/ijhpca37.html#ZvyaginBHDZBCKPMMIOVPWHEFXLSNRDV23

Shuaiwen Leon Song Bonnie Kruft Minjia Zhang Conglong Li Shiyang Chen 0004 Chengming Zhang 0006 Masahiro Tanaka Xiaoxia Wu Jeff Rasley Ammar Ahmad Awan Connor Holmes Martin Cai Adam Ghanem Zhongzhu Zhou Yuxiong He Pete Luferenko Divya Kumar Jonathan A. Weyn Ruixiong Zhang Sylwester Klocek Volodymyr Vragov Mohammed AlQuraishi Gustaf Ahdritz Christina Floristean Cristina Negri Rao Kotamarthi Venkatram Vishwanath Arvind Ramanathan Sam Foreman Kyle Hippe Troy Arcomano Romit Maulik Maxim Zvyagin Alexander Brace Bin Zhang Cindy Orozco Bohorquez Austin Clyde Bharat Kale Danilo Perez-Rivera Heng Ma Carla M. Mann Michael W. Irvin J. Gregory Pauloski Logan T. Ward Valérie Hayot-Sasson Murali Emani Zhen Xie Diangen Lin Maulik Shukla Ian T. Foster James J. Davis 0002 Michael E. Papka Thomas S. Brettin Prasanna Balaprakash Gina Tourassi John Gounley Heidi A. Hanson Thomas E. Potok Massimiliano Lupo Pasini Kate Evans Dan Lu 0001 Dalton D. Lunga Junqi Yin Sajal Dash Feiyi Wang Mallikarjun Shankar Isaac Lyngaas Xiao Wang 0004 Guojing Cong Pei Zhang Ming Fan Siyan Liu Adolfy Hoisie Shinjae Yoo Yihui Ren 0001 William Tang 0002 Kyle Felker Alexey Svyatkovskiy Hang Liu 0001 Ashwin M. Aji Angela Dalton Michael J. Schulte Karl W. Schulz Yuntian Deng Weili Nie Josh Romero Christian Dallago Arash Vahdat Chaowei Xiao Thomas Gibbs Anima Anandkumar Rick Stevens DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies. 2023 abs/2310.04610 CoRR https://doi.org/10.48550/arXiv.2310.04610 db/journals/corr/corr2310.html#abs-2310-04610

Gustaf Ahdritz Surl-Hee Ahn Ashwin M. Aji Mohammed AlQuraishi Rommie E. Amaro Anima Anandkumar Troy Arcomano Ammar Ahmad Awan Prasanna Balaprakash Bryan Barker Anthony T. Bogetti Cindy Orozco Bohorquez Antoine Bosselut Alex BraceAlexander Brace Thomas S. Brettin Nicholas John Browning Martin Cai Lorenzo Casalino Shiyang Chen 0004 S. Chakra Chennubhotla Lillian T. Chong Anders S. Christensen David Clark Austin Clyde Jonathan Coles Guojing Cong Christian Dallago Angela Dalton Sajal Dash James J. Davis 0002 Yuntian Deng Abigail C. Dommer Mitchell Dorrell Lukas Drescher Murali Emani Kate Evans Ming Fan Kyle Felker Christina Floristean Sam Foreman Ian T. Foster Adam Ghanem Thomas Gibbs Tom Gibbs Miguel Gila Defne GorgunDefne G. Ozgulbas Igor Gorodetsky John Gounley Alan Gray Ryan Hankins Heidi A. Hanson David J. Hardy Valérie Hayot-Sasson Yuxiong He Alejandro Hernández-Cano Kyle Hippe Torsten Hoefler Adolfy Hoisie Connor Holmes Lei Huang 0019 Michael W. Irvin Martin Jaggi Shantenu Jha Bharat Kale Fiona L. Kearns Syma Khalid Mark Klein 0002 Sylwester Klocek Rao Kotamarthi Bonnie Kruft Divya Kumar Hyungro Lee Conglong Li Diangen Lin Hang Liu 0001 Siyan Liu Antoni-Joan Solergibert i Llaquet Dan Lu 0001 Pete Luferenko Dalton D. Lunga Isaac Lyngaas Heng Ma Tom Maiden Frederick R. Manby Theofilos Ioannis Manitaras Carla M. Mann Maxime Martinasso Romit Maulik John D. McCalpin Henrique Mendonça Thomas F. Miller III Fawzi Mohamed Clare Morris Adrian J. Mulholland Cristina Negri Weili Nie Michael O'Connor A. Sofia F. Oliveira Elia Palme Michael E. Papka Massimiliano Lupo Pasini J. Gregory Pauloski Danilo Perez-Rivera James C. Phillips Richard Pitts Thomas E. Potok Jarren Zhuoran QiaoZhuoran Qiao 0001 Harinda Rajapaksha Arvind Ramanathan Jeff Rasley Yihui Ren 0001 Mia A. Rosenfeld John Russo 0003 Imanol Schlag Michael J. Schulte Thomas C. Schulthess Karl W. Schulz Stefano Schuppli Mallikarjun Shankar Maulik Shukla Sai Krishna Sirumalla Daniel G. A. Smith Shuaiwen SongShuaiwen Leon Song Abraham C. Stern Rick L. StevensRick Stevens John E. Stone Alexey Svyatkovskiy Terra Sztain Teresa Tamayo-Mendoza Masahiro Tanaka William Tang 0002 Gina Tourassi Anda Trifan Matteo Turilli Arash Vahdat Joost VandeVondele Natalia Vassilieva Venkatram Vishwanath Volodymyr Vragov Feiyi Wang Xiao Wang 0004 Logan T. Ward Nicholas Wauer Isa Wazirzada Matthew Welborn Jonathan A. Weyn Matt Williams Christopher J. Woods Xiaoxia Wu Chaowei Xiao Zhen Xie Junqi Yin Shinjae Yoo Bin Zhang Chengming Zhang 0006 Minjia Zhang Pei Zhang Ruixiong Zhang Zhongzhu Zhou Daniel M. Zuckerman Maxim Zvyagin