Research

Background

Petuum is built on years of ground-breaking work on machine learning and distributed computing.

Major Petuum innovations include the parameter server architecture and theory, the stale-synchronous parallel bridging model, elastic resource scheduling technology, and managed network communication technology in high-performance distributed systems for ML and DL – as well as new ML and DL models and algorithms for actionable, human-level understanding and learning from images, videos and natural language.

Publications

Overview

Petuum: A New Platform for Distributed Machine Learning on Big Data

Eric P. Xing, Qirong Ho, Wei Dai, Jin Kyu Kim, Jinliang Wei, Seunghak Lee, Xun Zheng, Pengtao Xie, Abhimanu Kumar, Yaoliang Yu
IEEE Transactions on Big Data, Volume 1, No. 2, Pages 49-67, 2015

Strategies and Principles of Distributed Machine Learning on Big Data

Eric P. Xing, Qirong Ho, Pengtao Xie, Wei Dai
Engineering, Volume 2, No. 2, Pages 179-95, 2016

Systems

Orpheus: Efficient Distributed Machine Learning via System and Algorithm Co-design.

Pengtao Xie, Jin-Kyu Kim, Qirong Ho, Yaoliang Yu and Eric P.Xing
Symposium of Cloud Computing (SoCC 2018)

Poseidon: An Efficient Communication Interface for Distributed Deep Learning on GPU Clusters

Hao Zhang, Zeyu Zheng, Wei Dai, Qirong Ho, Eric P. Xing
USENIX Annual Technical Conference (ATC 2017)

STRADS: A Distributed Framework for Scheduled Model Parallel Machine Learning

Jin Kyu Kim, Qirong Ho, Seunghak Lee, Xun Zheng, Wei Dai, Garth A. Gibson, Eric P. Xing
European Conference on Computer Systems (Eurosys 2016)

Managed Communication and Consistency for Fast Data-Parallel Iterative Analytics

Jinliang Wei, Wei Dai, Aurick Qiao, Henggang Cui, Qirong Ho, Gregory R Ganger, Phillip B. Gibbons, Garth A. Gibson, Eric P. Xing
ACM Symposium on Cloud Computing (SOCC 2015), Best paper award!

More Effective Distributed ML via a Stale Synchronous Parallel Parameter Server

Qirong Ho, James Cipar, Henggang Cui, Jin Kyu Kim, Seunghak Lee, Phillip. B. Gib
Neural Information Processing Systems (NIPS 2013), Oral (top 5%)

On Model Parallelization and Scheduling Strategies for Distributed Machine Learning

Seunghak Lee, Jin Kyu Kim, Xun Zheng, Qirong Ho, Garth A. Gibson, Eric P. Xing
Neural Information Processing Systems (NIPS 2014)

LightLDA: Big Topic Models on Modest Compute Clusters

Jinhui Yuan, Fei Gao, Qirong Ho, Wei Dai, Jinliang Wei, Xun Zheng, Eric P. Xing, Tie-Yan Liu, Wei-Ying Ma
International World Wide Web Conference (WWW 2015)

Litz: An Elastic Framework for High-Performance Distributed Machine Learning

Aurick Qiao, Abutalib Aghayev, Weiren Yu, Haoyang Chen, Qirong Ho, Garth A. Gibson, Eric P. Xing
USENIX Annual Technical Conference (ATC 2018)

Toward Understanding the Impact of Staleness in Distributed Machine Learning

Wei Dai, Yi Zhou, Nanqing Dong, Hao Zhang, Eric Xing
International Conference on Learning Representations (ICLR 2019)

Fault Tolerance in Iterative-Convergent Machine Learning

Aurick Qiao, Bryon Aragam, Bingjing Zhang, Eric P. Xing
International Conference on Machine Learning (ICML 2019)

Cavs: An Efficient Runtime System for Dynamic Neural Networks

Shizhen Xu, Hao Zhang, Graham Neubig, Wei Dai, Jin Kyu Kim, Zhijie Deng, Qirong Ho, Guangwen Yang, Eric P Xing
2018 USENIX Annual Technical Conference (USENIX ATC 18)

Theory

On Convergence of Model Parallel Proximal Gradient Algorithm for Stale Synchronous Parallel System

Yi Zhou, Yaoliang Yu, Wei Dai, Yingbin Liang, Eric P. Xing
Artificial Intelligence and Statistics (AISTATS 2016)

Analysis of High-Performance Distributed ML at Scale through Parameter Server Consistency Models

Wei Dai, Abhimanu Kumar, Jinliang Wei, Qirong Ho, Garth A. Gibson, Eric P. Xing
AAAI Conference on Artificial Intelligence (AAAI 2015)

Lighter-Communication Distributed Machine Learning via Sufficient Factor Broadcasting

Pengtao Xie, Jin Kyu Kim, Yi Zhou, Qirong Ho, Abhimanu Kumar, Yaoliang Yu, Eric P. Xing
Uncertainty in Artificial Intelligence (UAI 2016)

On Unifying Deep Generative Models

Zhiting Hu, Zichao Yang, Ruslan Salakhutdinov, Eric P. Xing
International Conference on Learning Representations (ICLR 2018)

Healthcare

Petuum Healthcare Research

Click here to read about our research

Petuum Healthcare Research is dedicated to developing state-of-the-art deep learning and machine learning techniques to aid clinical workflow and support the clinical decision-making process.

Other Applications

Harnessing Deep Neural Networks with Logic Rules

Zhiting Hu, Xuezhe Ma, Zhengzhong Liu, Eduard Hovy and Eric P. Xing
Annual Meeting of the Association for Computational Linguistics (ACL 2016) | Outstanding Paper Award!

Learning Answer-Entailing Structures for Machine Comprehension

Mrinmaya Sachan, Avinava Dubey, Matthew Richardson, Eric P. Xing
Annual Meeting of the Association for Computational Linguistics (ACL 2015) | Honorable Mention Recipient

ZM-Net: Real-time Zero-shot Image Manipulation Network

Hao Wang, Xiaodan Liang, Hao Zhang, Dit-Yan Yeung, Eric P. Xing
Currently under submission

Diversity-Promoting Bayesian Learning of Latent Variable Models

Pengtao Xie, Jun Zhu, Eric P. Xing
International Conference on Machine Learning (ICML 2016)

Science Question Answering using Instructional Materials

Mrinmaya Sachan and Eric P. Xing
Annual Meeting of the Association for Computational Linguistics (ACL 2016)

Learning Concept Taxonomies from Multi-modal Data

Hao Zhang, Zhiting Hu, Yuntian Deng, Mrinmaya Sachan, Zhicheng Yan, Eric P. Xing
Annual Meeting of the Association for Computational Linguistics (ACL 2016)

Grounding Topic Models with Knowledge Bases

Zhiting Hu, Gang Luo, Mrinmaya Sachan, Zaiqing Nie and Eric P. Xing
International Joint Conference on Artificial Intelligence (IJCAI 2016)

Deep Neural Networks with Massive Learned Knowledge

Zhiting Hu, Zichao Yang, Ruslan Salakhutdinov, Eric P. Xing
Conference on Empirical Methods on Natural Language Processing (EMNLP 2016)

A Constituent-Centric Neural Architecture for Reading Comprehension.

Pengtao Xie and Eric P. Xing
Annual Meeting of the Association for Computational Linguistics (ACL 2017)

Uncorrelation and Evenness: A New Diversity-Promoting Regularizer

Pengtao Xie, Aarti Singh and Eric P. Xing
International Conference on Machine Learning (ICML 2017)

Learning Latent Space Models with Angular Constraints

Pengtao Xie, Yuntian Deng, Yi Zhou, Abhimanu Kumar, Yaoliang Yu, James Zou, Eric P. Xing
International Conference on Machine Learning (ICML 2017)

Toward Controlled Generation of Text

Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, Eric P. Xing
International Conference on Machine Learning (ICML 2017)

Improved Variational Autoencoders for Text Modeling using Dilated Convolutions

Zichao Yang, Zhiting Hu, Ruslan Salakhutdinov, Taylor Berg-Kirkpatrick
International Conference on Machine Learning (ICML 2017)

Near-Orthogonality Regularization in Kernel Methods

Pengtao Xie, Barnabas Poczos, Eric P. Xing
Conference on Uncertainty in Artificial Intelligence (UAI 2017)

Efficient Correlated Topic Modeling with Topic Embedding

Zhiting Hu, Junxian He, Taylor Berg-Kirkpatrick, Ying Huang, Eric P. Xing
SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2017)

Deep Determinantal Point Process for Large-Scale Multi-Label Classification

Pengtao Xie, Ruslan Salakhutdinov, Luntian Mou and Eric P. Xing
International Conference on Computer Vision (ICCV 2017)

Recurrent Topic-Transition GAN for Visual Paragraph Generation

Xiaodan Liang, Zhiting Hu, Hao Zhang, Chuang Gan, Eric P. Xing
International Conference on Computer Vision (ICCV 2017)

Nonparametric Variational Auto-encoders for Hierarchical Representation Learning

Prasoon Goyal, Zhiting Hu, Xiaodan Liang, Chenyu Wang, Eric P. Xing
International Conference on Computer Vision (ICCV 2017)

Structured Generative Adversarial Networks

Hao Zhang, Zhijie Deng, Xiaodan Liang, Luona Yang, Shizhen Xu, Jun Zhu, and Eric P. Xing
Conference on Neural Information Processing Systems (NeurIPS 2017)

Orthogonality-Promoting Distance Metric Learning: Convex Relaxation and Theoretical Analysis

Pengtao Xie, Wei Wu, Yichen Zhu, Eric P. Xing
International Conference on Machine Learning (ICML 2018)

Nonoverlap-Promoting Variable Selection

Pengtao Xie, Hongbao Zhang, Yichen Zhu, Eric P. Xing
International Conference on Machine Learning (ICML 2018)

Generative Semantic Manipulation with Mask-Contrasting GAN

Xiaodan Liang, Hao Zhang, Eric P. Xing
European Conference on Computer Vision (ECCV 2018)

Symbolic Graph Reasoning Meets Convolutions

Xiadon Liang, Zhiting Hu, Hao Zhang, Liang Lin, Eric P. Xing
Conference on Neural Information Processing Systems (NeurIPS 2018)

Deep Generative Models with Learnable Knowledge Constraints

Zhiting Hu, Zichao Yang, Ruslan Salakhutdinov, Xiaodan Liang, Lianhui Qin, Haoye Dong, Eric P. Xing
Conference on Neural Information Processing Systems (NeurIPS 2018)

Unsupervised Text Style Transfer using Language Models as Discriminators

Zichao Yang, Zhiting Hu, Chris Dyer, Eric P. Xing, Taylor Berg-Kirkpatrick
Conference on Neural Information Processing Systems (NeurIPS 2018)

AutoLoss: Learning Discrete Schedules for Alternate Optimization

Haowen Xu, Hao Zhang, Zhiting Hu, Xiaodan Liang, Ruslan Salakhutdinov, Eric Xing
International Conference on Learning Representations (ICLR 2019)

Connecting the Dots Between MLE and RL for Sequence Generation

Bowen Tan, Zhiting Hu, Zichao Yang, Ruslan Salakhutdinov, Eric P. Xing
International Conference on Machine Learning (ICML 2019)

Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation

Zhiting Hu, Haoran Shi, Zichao Yang, Bowen Tan, Tiancheng Zhao, Junxian He, Wentao Wang, Xingjiang Yu, Lianhui Qin, Di Wang, Xuezhe Ma, Hector Liu, Xiaodan Liang, Wanrong Zhu, Devendra Singh Sachan, Eric P. Xing

Toward Unsupervised Text Content Manipulation

Zhiting Hu, Wentao Wang, Zichao Yang, Haoran Shi, Frank Xu, Eric Xing

We have also given tutorials and talks at these venues:

ACML 2015, IJCAI 2015, Data Science Summit in San Francisco 2015, WSDM Winter School 2015, WWW 2015, Big Data Technology Conference in China 2014, ParLearning Workshop at IPDPS 2015