Making Machine Learning Algorithms More Robust

  • by
Making Machine Learning Algorithms More Robust

Making Machine Learning Algorithms More Robust – In the past few years, we are witnessing exponential growth in the amount of data that’s stored and processed on the internet. With this data, there is an urgent need to use it effectively to make more intelligent decisions.

Machine learning algorithms enable us to do that. But many commonly used methods, which are at the core of machine learning today, such as stochastic gradient descent, for example, are inherently sequential. So the goal of my group’s research is to come up with ways to parallelize these algorithms and, in particular, make them robust to unpredictable fluctuations, failures, or errors that can occur in computing nodes, especially in today’s cloud infrastructure.

So, one of the impacts of my group’s research is that by making algorithms more robust to unreliable nodes, we can democratize machine learning. So, somebody does not need to have access to an expensive supercomputer to be able to learn from their data. They can use cheap servers that are entered from the cloud, or even lightweight mobile, or IoT devices.

My group takes a unique approach of bringing concepts from information theory and applied probability to this problem of designing system-aware machine learning algorithms. And using these tools, we are able to give theoretical guarantees on the performance of these algorithms.

And one of our recent results on communication efficient SGD, and also on erasure codes for distributed matrix computation, is able to give an order of magnitude boost in the speed to perform training and inference on machine learning models. This is a particularly exciting time to work at the intersection of machine learning and computing systems, because the space is simultaneously evolving in both academia, as well as industry.

And that is why we are actively collaborating with partners outside CMU, including IBM Research and Lawrence Livermore National Lab, in order to transfer the insights of our research to practical implementation. Carnegie Mellon attracts some of the brightest and the best students from all over the world. And beyond the research impact, the opportunity to work with these students, to teach and mentor them is, I think, the best part of my job as a faculty member at CMU.



Useful links:

reference – Making Machine Learning Algorithms More Robust

Share this post ...

Leave a Reply

Your email address will not be published. Required fields are marked *