Yujia Li

Research Scientist

Google DeepMind

News

Our latest large language model Gemini is released!
Our tech report about the second version of AlphaCode is released! This model is based on Gemini and better than 85% of human competitors in competitive programming.
Our paper on position embedding for directed graphs is published at ICML 2023.
Our paper on optimizing sorting algorithms is published on Nature.

Publications

[Google Scholar]
Gemini: A Family of Highly Capable Multimodal Models

Gemini Team, Google
arXiv:2312.11805, 2023
[Gemini]
AlphaCode 2 Technical Report

AlphaCode Team, Google DeepMind
Technical Report, 2023
Large Language Models as Analogical Reasoners

Michihiro Yasunaga, Xinyun Chen, Yujia Li, Panupong Pasupat, Jure Leskovec, Percy Liang, Ed H. Chi, Denny Zhou
arXiv:2310.01714, 2023
Transformers Meet Directed Graphs

Simon Geisler, Yujia Li, Daniel J Mankowitz, Ali Taylan Cemgil, Stephan Günnemann, Cosmin Paduraru
International Conference on Machine Learning (ICML), 2023
Faster sorting algorithms discovered using deep reinforcement learning

Daniel J. Mankowitz, Andrea Michi, Anton Zhernov, Marco Gelmi, Marco Selvi, Cosmin Paduraru, Edouard Leurent, Shariq Iqbal, Jean-Baptiste Lespiau, Alex Ahern, Thomas Köppe, Kevin Millikin, Stephen Gaffney, Sophie Elster, Jackson Broshear, Chris Gamble, Kieran Milan, Robert Tung, Minjae Hwang, Taylan Cemgil, Mohammadamin Barekatain, Yujia Li, Amol Mandhane, Thomas Hubert, Julian Schrittwieser, Demis Hassabis, Pushmeet Kohli, Martin Riedmiller, Oriol Vinyals, David Silver
Nature, 2023
[blog]
Competition-Level Code Generation with AlphaCode

Yujia Li*, David Choi*, Junyoung Chung*, Nate Kushman*, Julian Schrittwieser*, Rémi Leblond*, Tom Eccles*, James Keeling*, Felix Gimeno*, Agustin Dal Lago*, Thomas Hubert*, Peter Choy*, Cyprien de Masson d’Autume*, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, James Molloy, Daniel J. Mankowitz, Esme Sutherland Robson, Pushmeet Kohli, Nando de Freitas, Koray Kavukcuoglu, Oriol Vinyals (*denotes joint first authors)
Science, 2022
arXiv:2203.07814, 2022
[blog][dataset][examples][arXiv]
Scaling Language Models: Methods, Analysis & Insights from Training Gopher

Jack W. Rae and many others
arXiv:2112.11446, 2021
[blog]
WikiGraphs: A Wikipedia Text - Knowledge Graph Paired Dataset

Luyu Wang*, Yujia Li*, Ozlem Aslan, Oriol Vinyals (*denotes equal contribution)
TextGraphs-15: Graph-based Methods for Natural Language Processing (NAACL 2021 Workshop), 2021
ETA Prediction with Graph Neural Networks in Google Maps

Austin Derrow-Pinion, Jennifer She, David Wong, Oliver Lange, Todd Hester, Luis Perez, Marc Nunkesser, Seongjae Lee, Xueying Guo, Brett Wiltshire, Peter W. Battaglia, Vishal Gupta, Ang Li, Zhongwen Xu, Alvaro Sanchez-Gonzalez, Yujia Li, Petar Veličković
The 30th ACM International Conference on Information & Knowledge Management (CIKM), 2021
Computer-Aided Design as Language

Yaroslav Ganin, Sergey Bartunov, Yujia Li, Ethan Keller, Stefano Saliceti
Neural Information Processing Systems (NeurIPS), 2021
Solving Mixed Integer Programs Using Neural Networks

Vinod Nair, Sergey Bartunov, Felix Gimeno, Ingrid von Glehn, Pawel Lichocki, Ivan Lobov, Brendan O'Donoghue, Nicolas Sonnerat, Christian Tjandraatmadja, Pengming Wang, Ravichandra Addanki, Tharindi Hapuarachchi, Thomas Keck, James Keeling, Pushmeet Kohli, Ira Ktena, Yujia Li, Oriol Vinyals, Yori Zwols
arXiv:2012.13349, 2020
Strong Generalization and Efficiency in Neural Programs

Yujia Li, Felix Gimeno, Pushmeet Kohli, Oriol Vinyals
arXiv:2007.03629, 2020
Scalable Deep Generative Modeling for Sparse Graphs

Hanjun Dai, Azade Nazi, Yujia Li, Bo Dai, Dale Schuurmans
International Conference on Machine Learning (ICML), 2020
[code]
REGAL: Transfer Learning for Fast Optimization of Computation Graphs

Aditya Paliwal, Felix Gimeno, Vinod Nair, Yujia Li, Miles Lubin, Pushmeet Kohli, Oriol Vinyals
International Conference on Learning Representations (ICLR), 2020
[data]
Graph Convolutional Transformer: Learning the Graphical Structure of Electronic Health Records

Edward Choi, Zhen Xu, Yujia Li, Michael W. Dusenberry, Gerardo Flores, Yuan Xue, Andrew M. Dai
AAAI Conference on Artificial Intelligence (AAAI), 2020
ICML workshop on Learning and Reasoning with Graph-Structured Representations, 2019
Prioritized Unit Propagation with Periodic Resetting is (Almost) All You Need for Random SAT Solving

Xujie Si*, Yujia Li*, Vinod Nair*, Felix Gimeno (*denotes equal contribution)
arXiv:1912.05906, 2019
Learning Transferable Graph Exploration

Hanjun Dai, Yujia Li, Chenglong Wang, Rishabh Singh, Po-Sen Huang, Pushmeet Kohli
Neural Information Processing Systems (NeurIPS), 2019
Efficient Graph Generation with Graph Recurrent Attention Networks

Renjie Liao, Yujia Li, Yang Song, Shenlong Wang, Charlie Nash, William L. Hamilton, David Duvenaud, Raquel Urtasun, Richard S. Zemel
Neural Information Processing Systems (NeurIPS), 2019
[code]
Fast Training of Sparse Graph Neural Networks on Dense Hardware

Matej Balog, Bart van Merriënboer, Subhodeep Moitra, Yujia Li, Daniel Tarlow
arXiv:1906.11786, 2019
Graph Matching Networks for Learning the Similarity of Graph Structured Objects

Yujia Li, Chenjie Gu, Thomas Dullien, Oriol Vinyals, Pushmeet Kohli
International Conference on Machine Learning (ICML), 2019Long Oral Presentation
[slides][poster][code]
Compositional Imitation Learning: Explaining and executing one task at a time

Thomas Kipf, Yujia Li, Hanjun Dai, Vinicius Zambaldi, Edward Grefenstette, Pushmeet Kohli, Peter Battaglia
International Conference on Machine Learning (ICML), 2019Long Oral Presentation
A previous version appeared in the NeurIPS Learning by Instruction workshop, 2018
Deep Reinforcement Learing with Relational Inductive Biases

Vinicius Zambaldi, David Raposo, Adam Santoro, Victor Bapst, Yujia Li, Igor Babuschkin, Karl Tuyls, David Reichert, Timothy Lillicrap, Edward Lockhart, Murray Shanahan, Victoria Langston, Razvan Pascanu, Matthew Botvinick, Oriol Vinyals, Peter Battaglia
International Conference on Learning Representations (ICLR), 2019
[openreview]
Relational inductive biases, deep learning, and graph networks

Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, Caglar Gulcehre, Francis Song, Andrew Ballard, Justin Gilmer, George Dahl, Ashish Vaswani, Kelsey Allen, Charles Nash, Victoria Langston, Chris Dyer, Nicolas Heess, Daan Wierstra, Pushmeet Kohli, Matt Botvinick, Oriol Vinyals, Yujia Li, Razvan Pascanu
arXiv:1806.01261, 2018
Learning Deep Generative Models of Graphs

Yujia Li, Oriol Vinyals, Chris Dyer, Razvan Pascanu, Peter Battaglia
arXiv:1803.03324, 2018
Invited to ICLR Workshop Track, 2018
[slides]
Learning Model-Based Planning from Scratch

Razvan Pascanu*, Yujia Li*, Oriol Vinyals, Nicolas Heess, Lars Buesing, Sébastien Racanière, David Reichert, Théophane Weber, Daan Wierstra, Peter Battaglia (*denotes equal contribution)
arXiv:1707.06170, 2017
Dualing GANs

Yujia Li, Alexander Schwing, Kuan-Chieh Wang, Richard Zemel
Neural Information Processing Systems (NIPS), 2017Spotlight Presentation
[poster][teaser slides]
Imagination-Augmented Agents for Deep Reinforcement Learning

Théophane Weber*, Sébastien Racanière*, David P. Reichert*, Lars Buesing, Arthur Guez, Danilo Jimenez Rezende, Adria Puigdomènech Badia, Oriol Vinyals, Nicolas Heess, Yujia Li, Razvan Pascanu, Peter Battaglia, David Silver, Daan Wierstra (*denotes equal contribution)
Neural Information Processing Systems (NIPS), 2017Oral Presentation
Understanding the Effective Receptive Field in Deep Convolutional Neural Networks

Wenjie Luo*, Yujia Li*, Raquel Urtasun, Richard Zemel (*denotes equal contribution)
Neural Information Processing Systems (NIPS), 2016
[paper][poster]
Gated Graph Sequence Neural Networks

Yujia Li, Daniel Tarlow, Marc Brockschmidt, Richard Zemel
International Conference on Learning Representations (ICLR), 2016
[slides][poster][code][talk]
The Variational Fair Auto Encoder

Christos Louizos, Kevin Swersky, Yujia Li, Max Welling, Richard Zemel
International Conference on Learning Representations (ICLR), 2016Oral Presentation
Generative Moment Matching Networks

Yujia Li, Kevin Swersky, Richard Zemel
International Conference on Machine Learning (ICML), 2015
[code][project page]
Feedback-Based Handwriting Recognition from Inertial Sensor Data for Wearable Devices

Yujia Li, Kaisheng Yao, Geoffrey Zweig
The 40th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015
Learning Unbiased Features

Yujia Li, Kevin Swersky, Richard Zemel
NIPS workshop on Transfer and Multi-Task Learning, 2014
[slides][poster]
High Order Regularization for Semi-Supervised Learning of Structured Output Problems

Yujia Li, Richard Zemel
International Conference on Machine Learning (ICML), 2014
[slides][poster][data][code][video]
Mean Field Networks

Yujia Li, Richard Zemel
ICML workshop on Learning Tractable Probabilistic Models, 2014
[slides][poster]
Exploring Compositional High Order Pattern Potentials for Structured Output Learning

Yujia Li, Daniel Tarlow, Richard Zemel
The 26th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013Oral Presentation
[supplementary material][slides][poster][data][code][video]
Celebrity Recommendation with Collaborative Social Topic Regression

Xuetao Ding, Xiaoming Jin, Yujia Li, Lianghao Li
Proceedings of the 23rd International Joint Conference on Artificial Intelligence (IJCAI), 2013

Thesis

Building More Expressive Structured Models, Ph.D. Thesis
Exploring Compositional High Order Pattern Potentials for Structured Output Learning, M.Sc. Thesis

Education / Work Experience

2023.5 - present Senior Staff Research Scientist, Google DeepMind.
2020.11 - 2023.5 Staff Research Scientist, DeepMind.
2018.5 - 2020.11 Senior Research Scientist, DeepMind.
2016.11 - 2018.5 Research Scientist, DeepMind.
2013.2 - 2017.2 Doctor of Philosophy, University of Toronto.
2015.6 - 2015.9 Research Intern, Microsoft Research Cambridge.
2014.6 - 2014.9 Research Intern, Microsoft Research Redmond.
2011.9 - 2013.1 Master of Science, University of Toronto.
2011.6 - 2011.8 R&D Intern, Baidu, Inc..
2007.8 - 2011.7 Bachelor of Engineering, Tsinghua University.

Community Service

Area Chair: NeurIPS (2020, 2021, 2022), ICLR (2021, 2022, 2023).
Action Editor: TMLR.
Reviewer: NeurIPS (2015-2019), ICML (2016-2020, 2022), ICLR (2017, 2019, 2020), UAI (2015, 2016, 2018), CVPR (2017, 2019), ECCV (2016), JMLR, IJCV, TPAMI.

Teaching Experience

Courses TA'ed at University of Toronto:

Winter 2015 CSC 412/2506Probabilistic Graphical Models
Fall 2014 CSC 411Machine Learning and Data Mining
Winter 2014 CSC 108Introduction to Computer Programming
Fall 2013 CSC 263Data Structures
Winter 2013 CSC 108Introduction to Computer Programming
Fall 2012 CSC 411Machine Learning and Data Mining
Winter 2012 CSC 190Computer Algorithms, Data Structures and Languages
Fall 2011 CSC 165Mathematical Expression and Reasoning for Computer Science

Courses TA'ed at Tsinghua University:

Spring 2011Advanced Data Structures
Spring 2011Machine Learning and Knowledge Discovery

Honors and Awards

2019CVPR outstanding reviewer
2016ICLR travel award
2015Microsoft Ph.D. fellowship (US and Canada) finalist
2015ICML travel grant
2013CVPR travel grant
2013University of Toronto School of Graduate Studies conference grant
2008-2010University-wide comprehensive merit scholarship, three times - including Kai-Feng Scholarship, which is the highest amount among all scholarships and awarded to only 30 undergraduate students in Tsinghua University every year.
20082nd Prize - Chinese National College Physics Contest
20061st Prize - Chinese Physics Olympiad (CPhO) in Provinces
20052nd Prize - Chinese National Olympiad in Informatics in Provinces (NOIP)

Other Stuff

My name in Chinese: