About

Sheng Zha 查晟

I lead the pretraining research team at Amazon AGI that supports Amazon Nova, focusing on principled scaling, model architecture, optimization, and novel pretraining objectives.

My work centers on the co-evolution of algorithms and systems: the intersection that makes AI more capable, efficient, and accessible. I built my team from zero, beginning with distributed training and shared representations, and grew it into a group supporting foundation models used across AWS.

Before that, I helped shape the open-source AI ecosystem as VP and PMC Chair of Apache MXNet, co-authored the Gluon interface, founded GluonNLP, served on the ONNX Steering Committee, and co-founded the Python Data API Standards Consortium.

I believe AI should amplify human agency and ingenuity. That principle guides how I approach both research and leadership.

Selected work

Amazon Nova and foundation model stack

Amazon AGI, 2024–present. Leading research on principled scaling, model architecture, optimization, and pretraining objectives.

Foundation models for AWS AI services

AWS, 2018–2024. Built the team and models from zero, supporting Amazon Q, Titan, Lex, Comprehend, and Kendra.

Distributed training infrastructure

AWS, 2018–2023. Contributed core technology for scalable, fault-resilient training infrastructure including SageMaker HyperPod.

Apache MXNet

VP and PMC Chair, 2016–2023. Co-authored the Gluon API and led project maintenance, releases, and community engagement.

GluonNLP

Founded, 2018–2023. Created a deep-learning NLP toolkit that reproduced BERT with record-setting training speeds.

ONNX

Served on the Steering Committee for the open standard for machine-learning model interoperability.

Python Data API Standards

Founding member of the consortium standardizing array APIs across the Python data ecosystem.

ML platform for fraud detection

Amazon TRMS, 2013–2015. Designed scalable machine-learning and graph systems for fraud and abuse detection.

Selected publications
DEM: Distribution Edited Model for Training with Mixed Data DistributionsDifferentially Private Bias-term Fine-tuning of Foundation ModelsHyTrel: Hypergraph-enhanced Tabular Data Representation LearningZero Redundancy Distributed Learning with Differential PrivacyMeta-learning via Language Model In-context TuningAutomatic Clipping: Differentially Private Deep Learning Made Easier and StrongerDifferentially Private Optimization on Large Model at Small CostThe Amazon Nova Family of Models: Technical Report and Model CardLarge Language Models of Code Fail at Completing Code with Potential BugsSequence-level Large Language Model Training with Contrastive Preference OptimizationGluonCV and GluonNLPView all publications on Google Scholar →
Talks and interviews
大模型训练的一线实践Accelerate the Bridging of ML and DL with NVIDIA-Accelerated Apache MXNet 2.0Apache MXNet 2.0 at NVIDIA GTCApache MXNet 2.0: Bridging the Gap between DL and MLDive into Deep Learning for Natural Language ProcessingFrom Shallow to Deep Language RepresentationsDeep Learning Lectures
Education

MS in Computer Science, University of Maryland. BS, Shanghai Jiao Tong University.