Parallelizing Machine Learning– Functionally
A Framework and Abstractions for Parallel Graph Processing
Abstract:
Implementing machine learning algorithms for large data, such as the Web graph and social networks, is challenging. Even though much research has focused on making sequential algorithms more scalable, their running times continue to be prohibitively long. Meanwhile, parallelization remains a formidable challenge for this class of problems, despite frameworks like MapReduce which hide much of the associated complexity. We present a framework for implementing parallel and distributed machine learning algorithms on large graphs, flexibly, through the use of functional programming abstractions. Our aim is a system that allows researchers and practitioners to quickly and easily implement (and experiment with) their algorithms in a parallel or distributed setting. We introduce functional combinators for the flexible composition of parallel, aggregation, and sequential steps. To the best of our knowledge, our system is the first to avoid inversion of control in a (bulk) synchronous parallel model.
I am particularly interested in the authors’ claim that:
While also based on graphs, Pregel is a closed system that was designed to solve large-scale “graph processing” problems, which are usually simpler in nature than typical real-world ML problems. In an effort to capitalize on Pregel’s strengths while focusing on a framework more aptly-suited to ML problems, we introduce a more flexible programming model, based on high-level functional abstractions.
Mostly because identifying where we are researching because our algorithms work versus areas where algorithms await discovery is important.
But, in part so that we know where it is appropriate to apply our usual algorithms and where those are likely to break down.