
Research
Background
Petuum is built on years of ground-breaking work on machine learning and distributed computing.
Major Petuum innovations include the parameter server architecture and theory, the stale-synchronous parallel bridging model, elastic resource scheduling technology, and managed network communication technology in high-performance distributed systems for ML and DL – as well as new ML and DL models and algorithms for actionable, human-level understanding and learning from images, videos and natural language.
Publications
Overview
Petuum: A New Platform for Distributed Machine Learning on Big Data
Eric P. Xing, Qirong Ho, Wei Dai, Jin Kyu Kim, Jinliang Wei, Seunghak Lee, Xun Zheng, Pengtao Xie, Abhimanu Kumar, Yaoliang Yu
IEEE Transactions on Big Data, Volume 1, No. 2, Pages 49-67, 2015
Strategies and Principles of Distributed Machine Learning on Big Data
Eric P. Xing, Qirong Ho, Pengtao Xie, Wei Dai
Engineering, Volume 2, No. 2, Pages 179-95, 2016
Systems
Orpheus: Efficient Distributed Machine Learning via System and Algorithm Co-design.
Pengtao Xie, Jin-Kyu Kim, Qirong Ho, Yaoliang Yu and Eric P.Xing
Symposium of Cloud Computing (SoCC 2018)
Poseidon: An Efficient Communication Interface for Distributed Deep Learning on GPU Clusters
Hao Zhang, Zeyu Zheng, Wei Dai, Qirong Ho, Eric P. Xing
USENIX Annual Technical Conference (ATC 2017)
STRADS: A Distributed Framework for Scheduled Model Parallel Machine Learning
Jin Kyu Kim, Qirong Ho, Seunghak Lee, Xun Zheng, Wei Dai, Garth A. Gibson, Eric P. Xing
European Conference on Computer Systems (Eurosys 2016)
Managed Communication and Consistency for Fast Data-Parallel Iterative Analytics
Jinliang Wei, Wei Dai, Aurick Qiao, Henggang Cui, Qirong Ho, Gregory R Ganger, Phillip B. Gibbons, Garth A. Gibson, Eric P. Xing
ACM Symposium on Cloud Computing (SOCC 2015), Best paper award!
More Effective Distributed ML via a Stale Synchronous Parallel Parameter Server
Qirong Ho, James Cipar, Henggang Cui, Jin Kyu Kim, Seunghak Lee, Phillip. B. Gib
Neural Information Processing Systems (NIPS 2013), Oral (top 5%)
On Model Parallelization and Scheduling Strategies for Distributed Machine Learning
Seunghak Lee, Jin Kyu Kim, Xun Zheng, Qirong Ho, Garth A. Gibson, Eric P. Xing
Neural Information Processing Systems (NIPS 2014)
LightLDA: Big Topic Models on Modest Compute Clusters
Jinhui Yuan, Fei Gao, Qirong Ho, Wei Dai, Jinliang Wei, Xun Zheng, Eric P. Xing, Tie-Yan Liu, Wei-Ying Ma
International World Wide Web Conference (WWW 2015)
Litz: An Elastic Framework for High-Performance Distributed Machine Learning
Aurick Qiao, Abutalib Aghayev, Weiren Yu, Haoyang Chen, Qirong Ho, Garth A. Gibson, Eric P. Xing
USENIX Annual Technical Conference (ATC 2018)
Toward Understanding the Impact of Staleness in Distributed Machine Learning
Wei Dai, Yi Zhou, Nanqing Dong, Hao Zhang, Eric Xing
International Conference on Learning Representations (ICLR 2019)
Fault Tolerance in Iterative-Convergent Machine Learning
Aurick Qiao, Bryon Aragam, Bingjing Zhang, Eric P. Xing
International Conference on Machine Learning (ICML 2019)
Cavs: An Efficient Runtime System for Dynamic Neural Networks
Shizhen Xu, Hao Zhang, Graham Neubig, Wei Dai, Jin Kyu Kim, Zhijie Deng, Qirong Ho, Guangwen Yang, Eric P Xing
2018 USENIX Annual Technical Conference (USENIX ATC 18)
Theory
On Convergence of Model Parallel Proximal Gradient Algorithm for Stale Synchronous Parallel System
Yi Zhou, Yaoliang Yu, Wei Dai, Yingbin Liang, Eric P. Xing
Artificial Intelligence and Statistics (AISTATS 2016)
Analysis of High-Performance Distributed ML at Scale through Parameter Server Consistency Models
Wei Dai, Abhimanu Kumar, Jinliang Wei, Qirong Ho, Garth A. Gibson, Eric P. Xing
AAAI Conference on Artificial Intelligence (AAAI 2015)
Lighter-Communication Distributed Machine Learning via Sufficient Factor Broadcasting
Pengtao Xie, Jin Kyu Kim, Yi Zhou, Qirong Ho, Abhimanu Kumar, Yaoliang Yu, Eric P. Xing
Uncertainty in Artificial Intelligence (UAI 2016)
On Unifying Deep Generative Models
Zhiting Hu, Zichao Yang, Ruslan Salakhutdinov, Eric P. Xing
International Conference on Learning Representations (ICLR 2018)
Healthcare
Petuum Healthcare Research
Click here to read about our research
Petuum Healthcare Research is dedicated to developing state-of-the-art deep learning and machine learning techniques to aid clinical workflow and support the clinical decision-making process.
Other Applications
Harnessing Deep Neural Networks with Logic Rules
Zhiting Hu, Xuezhe Ma, Zhengzhong Liu, Eduard Hovy and Eric P. Xing
Annual Meeting of the Association for Computational Linguistics (ACL 2016) | Outstanding Paper Award!
Learning Answer-Entailing Structures for Machine Comprehension
Mrinmaya Sachan, Avinava Dubey, Matthew Richardson, Eric P. Xing
Annual Meeting of the Association for Computational Linguistics (ACL 2015) | Honorable Mention Recipient
ZM-Net: Real-time Zero-shot Image Manipulation Network
Hao Wang, Xiaodan Liang, Hao Zhang, Dit-Yan Yeung, Eric P. Xing
Currently under submission
Diversity-Promoting Bayesian Learning of Latent Variable Models
Pengtao Xie, Jun Zhu, Eric P. Xing
International Conference on Machine Learning (ICML 2016)
Science Question Answering using Instructional Materials
Mrinmaya Sachan and Eric P. Xing
Annual Meeting of the Association for Computational Linguistics (ACL 2016)
Learning Concept Taxonomies from Multi-modal Data
Hao Zhang, Zhiting Hu, Yuntian Deng, Mrinmaya Sachan, Zhicheng Yan, Eric P. Xing
Annual Meeting of the Association for Computational Linguistics (ACL 2016)
Grounding Topic Models with Knowledge Bases
Zhiting Hu, Gang Luo, Mrinmaya Sachan, Zaiqing Nie and Eric P. Xing
International Joint Conference on Artificial Intelligence (IJCAI 2016)
Deep Neural Networks with Massive Learned Knowledge
Zhiting Hu, Zichao Yang, Ruslan Salakhutdinov, Eric P. Xing
Conference on Empirical Methods on Natural Language Processing (EMNLP 2016)
A Constituent-Centric Neural Architecture for Reading Comprehension.
Pengtao Xie and Eric P. Xing
Annual Meeting of the Association for Computational Linguistics (ACL 2017)
Uncorrelation and Evenness: A New Diversity-Promoting Regularizer
Pengtao Xie, Aarti Singh and Eric P. Xing
International Conference on Machine Learning (ICML 2017)
Learning Latent Space Models with Angular Constraints
Pengtao Xie, Yuntian Deng, Yi Zhou, Abhimanu Kumar, Yaoliang Yu, James Zou, Eric P. Xing
International Conference on Machine Learning (ICML 2017)
Toward Controlled Generation of Text
Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, Eric P. Xing
International Conference on Machine Learning (ICML 2017)
Improved Variational Autoencoders for Text Modeling using Dilated Convolutions
Zichao Yang, Zhiting Hu, Ruslan Salakhutdinov, Taylor Berg-Kirkpatrick
International Conference on Machine Learning (ICML 2017)
Near-Orthogonality Regularization in Kernel Methods
Pengtao Xie, Barnabas Poczos, Eric P. Xing
Conference on Uncertainty in Artificial Intelligence (UAI 2017)
Efficient Correlated Topic Modeling with Topic Embedding
Zhiting Hu, Junxian He, Taylor Berg-Kirkpatrick, Ying Huang, Eric P. Xing
SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2017)
Deep Determinantal Point Process for Large-Scale Multi-Label Classification
Pengtao Xie, Ruslan Salakhutdinov, Luntian Mou and Eric P. Xing
International Conference on Computer Vision (ICCV 2017)
Recurrent Topic-Transition GAN for Visual Paragraph Generation
Xiaodan Liang, Zhiting Hu, Hao Zhang, Chuang Gan, Eric P. Xing
International Conference on Computer Vision (ICCV 2017)
Nonparametric Variational Auto-encoders for Hierarchical Representation Learning
Prasoon Goyal, Zhiting Hu, Xiaodan Liang, Chenyu Wang, Eric P. Xing
International Conference on Computer Vision (ICCV 2017)
Structured Generative Adversarial Networks
Hao Zhang, Zhijie Deng, Xiaodan Liang, Luona Yang, Shizhen Xu, Jun Zhu, and Eric P. Xing
Conference on Neural Information Processing Systems (NeurIPS 2017)
Orthogonality-Promoting Distance Metric Learning: Convex Relaxation and Theoretical Analysis
Pengtao Xie, Wei Wu, Yichen Zhu, Eric P. Xing
International Conference on Machine Learning (ICML 2018)
Nonoverlap-Promoting Variable Selection
Pengtao Xie, Hongbao Zhang, Yichen Zhu, Eric P. Xing
International Conference on Machine Learning (ICML 2018)
Generative Semantic Manipulation with Mask-Contrasting GAN
Xiaodan Liang, Hao Zhang, Eric P. Xing
European Conference on Computer Vision (ECCV 2018)
Symbolic Graph Reasoning Meets Convolutions
Xiadon Liang, Zhiting Hu, Hao Zhang, Liang Lin, Eric P. Xing
Conference on Neural Information Processing Systems (NeurIPS 2018)
Deep Generative Models with Learnable Knowledge Constraints
Zhiting Hu, Zichao Yang, Ruslan Salakhutdinov, Xiaodan Liang, Lianhui Qin, Haoye Dong, Eric P. Xing
Conference on Neural Information Processing Systems (NeurIPS 2018)
Unsupervised Text Style Transfer using Language Models as Discriminators
Zichao Yang, Zhiting Hu, Chris Dyer, Eric P. Xing, Taylor Berg-Kirkpatrick
Conference on Neural Information Processing Systems (NeurIPS 2018)
AutoLoss: Learning Discrete Schedules for Alternate Optimization
Haowen Xu, Hao Zhang, Zhiting Hu, Xiaodan Liang, Ruslan Salakhutdinov, Eric Xing
International Conference on Learning Representations (ICLR 2019)
Connecting the Dots Between MLE and RL for Sequence Generation
Bowen Tan, Zhiting Hu, Zichao Yang, Ruslan Salakhutdinov, Eric P. Xing
International Conference on Machine Learning (ICML 2019)
Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation
Zhiting Hu, Haoran Shi, Zichao Yang, Bowen Tan, Tiancheng Zhao, Junxian He, Wentao Wang, Xingjiang Yu, Lianhui Qin, Di Wang, Xuezhe Ma, Hector Liu, Xiaodan Liang, Wanrong Zhu, Devendra Singh Sachan, Eric P. Xing
Toward Unsupervised Text Content Manipulation
Zhiting Hu, Wentao Wang, Zichao Yang, Haoran Shi, Frank Xu, Eric Xing
Presentations
A Statistical Machine Learning Perspective of Deep Learning: Algorithm, Theory, Scalable Computing
Eric P. Xing
International Summer School on Deep Learning, Bilbao, Spain
System and Algorithm Co-Design, Theory and Practice, for Distributed Machine Learning
Eric P. Xing
Simons Institute for the Theory of Computing
Distinguished Lecture: Strategies & Principles for Distributed Machine Learning
[Slides]
Eric P. Xing
Allen Institute for Artificial Intelligence
High Efficiency Systems for Distributed AI and ML at Scale
Qirong Ho
Strata+Hadoop World in Singapore, 2016
A New Look at the System, Algorithm and Theory Foundations of Distributed Machine Learning
Eric P. Xing and Qirong Ho
KDD 2015 Tutorial
We have also given tutorials and talks at these venues:
ACML 2015, IJCAI 2015, Data Science Summit in San Francisco 2015, WSDM Winter School 2015, WWW 2015, Big Data Technology Conference in China 2014, ParLearning Workshop at IPDPS 2015