Think globally and solve locally: secondary memory-based network learning for automated multi-species function prediction by Marco Mesiti, Matteo Re, and Giorgio Valentini.
Abstract:
Background: Network-based learning algorithms for Automated Function Prediction (AFP) are negatively a ffected by the limited coverage of experimental data and limited a priori known functional annotations. As a consequence their application to model organisms is often restricted to well characterized biological processes and pathways, and their e ffectiveness with poorly annotated species is relatively limited. A possible solution to this problem might consist in the construction of big networks including multiple species, but this in turn poses challenging computational problems, due to the scalability limitations of existing algorithms and the main memory requirements induced by the construction of big networks. Distributed computation or the usage of big computers could in principle respond to these issues, but raises further algorithmic problems and require resources not satis able with simple off -the-shelf computers.
Results: We propose a novel framework for scalable network-based learning of multi-species protein functions based on both a local implementation of existing algorithms and the adoption of innovative technologies: we solve “locally” the AFP problem, by designing “vertex-centric” implementations of network-based algorithms, but we do not give up thinking “globally” by exploiting the overall topology of the network. This is made possible by the adoption of secondary memory-based technologies that allow the efficient use of the large memory available on disks, thus overcoming the main memory limitations of modern off -the-shelf computers. This approach has been applied to the analysis of a large multi-species network including more than 300 species of bacteria and to a network with more than 200,000 proteins belonging to 13 Eukaryotic species. To our knowledge this is the fi rst work where secondary-memory based network analysis has been applied to multi-species function prediction using biological networks with hundreds of thousands of proteins.
Conclusions: The combination of these algorithmic and technological approaches makes feasible the analysis of large multi-species networks using ordinary computers with limited speed and primary memory, and in perspective could enable the analysis of huge networks (e.g. the whole proteomes available in SwissProt), using well-equipped stand-alone machines.
The biomolecular network material may be deep wading but you will find GraphChi making a significant difference in this use case.
What I found particularly interesting in Table 7 (page 20) was the low impact that additional RAM has on GraphChi.
I take that to mean that GraphChi can run efficiently on low-end boxes (4 GB RAM).
Yes?
I first saw this in a tweet by Aapo Kyrola.