UC Berkeley to lead $10M NSF/Simons Foundation program to investigate theoretical underpinnings of deep learning

August 25, 2020

NSF and Simons Foundation

Peter Bartlett (top center) will lead the $10M project funding by the National Science Foundation and Simons Foundation. Co-PIs include Alexander Rakhlin of MIT (top left),  Andrea Montanari, Stanford (top right),  Nathan Srebro, Toyota Technological Institute at Chicago (bottom left), and Bin Yu of UC Berkeley (bottom right)

UC Berkeley tackles AI and deep learning as part of two $10M NSF/Simons Foundation programs

Berkeley, Calif.--The National Science Foundation (NSF) and Simons Foundation today (Aug. 25) awarded $10 million to a UC Berkeley-led program to gain a theoretical understanding of deep learning. Berkeley staff are also involved in a second project funded at $10 million.

The first project, the Collaboration on the Theoretical Foundations of Deep Learning, is led by UC Berkeley researchers Peter Bartlett, principal investigator, and Bin Yu, co-principal investigator. Funded for five years, the project aims to gain a theoretical understanding of deep learning, which is making significant impacts across industry, commerce, science, and society.

Both Bartlett and Yu have appointments in the university’s Departments of Statistics and  Electrical Engineering and Computer Sciences (EECS). Other participating institutions are Stanford University, the Massachusetts Institute of Technology, UC Irvine, UC San Diego, Toyota Technological Institute at Chicago, EPFL in Lausanne, Switzerland, and the Hebrew University in Jerusalem.

Although deep learning is a widely used artificial intelligence approach for teaching computers to learn from data, its theoretical foundations are poorly understood, a challenge that the project will address. Understanding the mechanisms that underpin the practical success of deep learning will allow researchers to address its limitations, including its sensitivity to data manipulation.

“Our excitement over receiving this award is that we will be digging into the theoretical foundations of deep learning,” said Bartlett, who is also Associate Director of the Simons Institute for the Theory of Computing. “The recent success in machine learning has been driven by a spirit of craftsmanship by people who find ways to make this technology successful. But much of this work contradicts a lot of our classical understanding of statistical methodology, and there are many things we don’t understand about how and why these systems work.” 

The second project funded under the NSF-Simons program, Transferable, Hierarchical, Expressive, Optimal, Robust, Interpretable NETworks (THEORINET), includes Yi Ma, a professor in EECS, as Berkeley's principal investigator. THEORINET will be led by Johns Hopkins University Prof. René Vidal. Berkeley's S. Shankar Sastry, a professor in EECS, bioengineering, and mechanical engineering, and Prof. Jacob Steinhardt of statistics are co-investigators for the project which will analyze properties of deep neural networks, develop accelerated optimization algorithms, and uncover new architecture for improved neural network performance.

All five of the Berkeley researchers are affiliated with BAIR, the Berkeley AI Research Lab. The BAIR Lab brings together UC Berkeley researchers across the areas of computer vision, machine learning, natural language processing, planning, and robotics. BAIR includes over 45 faculty and more than 200 graduate students and postdocs.

Both awards are part of a partnership between the NSF and the Simons Foundation Division of Mathematics and Physical Sciences to fund research in the Mathematical and Scientific Foundations of Deep Learning. Deep Learning is part of a broader family of machine learning methods based on artificial neural networks that digest large amounts of raw data inputs and train AI systems with limited human supervision.

Most of the research and education activities will be hosted by the Simons Institute for the Theory of Computing, UC Berkeley's global center for collaboration in theoretical computer science, in the form of structured programs of varying themes.

Yu, who has been using deep learning to advance research in neuroscience said that the approach is also used to analyze medical images, study cardiovascular disease and improve precision medicine, and these high-stake applications call for an understanding of deep learning theoretically.

“This is really cutting-edge research at the frontier of data science theory and practice,” said Yu, who is also in Berkeley’s Division of Computing, Data Science, and Society (CDSS). “It’s leading us intellectually where we want to go with deep learning. Theory can help improve our practice, and that practice then drives theory.”

Yu said the project is a good fit for CDSS because although deep learning offers great potential for solving problems, it’s critical to understand how it works because it could cause harm if not used properly, especially in areas such as health and medicine.

CDSS fosters cross-campus partnerships, bringing together researchers in areas ranging from economics to social welfare, climate studies to public policy, computer science, electrical engineering, and statistics to biomedicine to apply tools like deep learning to solving societal problems. The new project builds on work by the EECS Department, which is jointly housed in CDSS and Berkeley’s College of Engineering, and the Statistics Department, which is in both CDSS and the Division of Mathematical and Physical Sciences.

Bartlett said that the Simons Institute, as a fully embedded research center within the university with its distinguished international network of scientists and scholars, will enhance the breadth and depth of the research. 

In addition to research, Berkeley’s Data Science Education Program under CDSS educates students in data science, giving them the tools to look at data confidently, critically, and ethically in whatever field they choose for a career. To contribute to that mission, the newly funded project will organize workshops open to the public, sponsor an annual summer school for grad students, postdocs, and faculty, and train a diverse cohort of postdocs in the theoretical basis of deep learning.

“I’m really pleased with the strength of our team; in our online meetings to develop the proposal, it was already clear that this is a group that’s going to work really well together,” Bartlett said. "We're also excited by the fact that we will have close ties to the Berkeley team working on the other project.

Other participants in the Berkeley-led project are Emmanuel Abbe, EPFL; Mikhail Belkin, UC San Diego; Amit Daniely, Hebrew University; Andrea Montanari, Stanford University; Elchanan Mossel, Alexander Rakhlin and Nike Sun, MIT; Nathan Srebro, Toyota Technological Institute at Chicago; and Roman Vershynin, UC Irvine.

The project will also benefit by gaining perspectives from two industry advisors; one from Google and the second from Mobileye, an Israeli subsidiary of Intel that develops self-driving vehicles and advanced driver-assistance systems.

Read the NSF news release.

 

About CDSS

The Division of Computing, Data Science, and Society launched in July 2019 to leverage Berkeley’s preeminence in research and excellence across disciplines to propel data science discovery, education, and impact. Core to the Division is an understanding of how the digital revolution affects equality, equity, and opportunity—and the capacity to respond to related challenges.

The Division’s dynamic structure connects Data Science Education, the School of Information; the departments of Electrical Engineering and Computer Sciences and Statistics; and includes the Berkeley Institute for Data Science and the Data Science Commons. It’s designed to meet the opportunities and demands of a world increasingly informed and shaped by data, machine learning, and artificial intelligence in virtually every arena, from health to business to politics; from our cities to our climate to the cosmos.

 

About the Simons Institute for the Theory of Computing

The Simons Institute for the Theory of Computing based at UC Berkeley is the world's leading venue for collaborative research in theoretical computer science. The Institute brings together distinguished researchers in theoretical computer science and related fields, as well as the next generation of outstanding young scholars, to explore the foundations of computer science, and other scientific and social disciplines through a computational lens. Research is facilitated through curated, semester-long programs organized and attended by international cohorts of 60-70 scientists, scholars and practitioners.