Abstract: The explosion of machine learning (ML) model size has led to its execution on distributed clusters at a large scale. Many works have tried to optimize the process of producing collective ...