Deep Learning for Computational Imaging

Computational techniques for image reconstruction problems enable imaging technologies including high-resolution microscopy, astronomy and seismology, computed tomography, and magnetic resonance imaging. Until recently, methods for solving such inverse problems were derived by experts without any learning. Now, the best performing image reconstruction methods are based on deep learning. This textbook gives the first comprehensive introduction to deep learning based image reconstruction methods. This book first introduces important inverse problems in imaging, including denoising and reconstructing an image from few and noisy measurements, and explains what makes those problems hard and interesting. Then, the book briefly discusses traditional optimization and sparsity based reconstruction methods, as well as optimization techniques as a basis for training and deriving deep neural networks for image reconstruction. The main part of the book is about how to solve image reconstruction problems with deep learning techniques: The book first disuses supervised deep learning approaches that map a measurement to an image as well as network architectures for imaging including convolutional neural networks and transformers. Then, reconstruction approaches based on generative models such as variational autoencoders and diffusion models are discussed, and how un-trained neural networks and implicit neural representations enable signal and image reconstruction. The book ends with a discussion on the robustness of deep learning based reconstruction as well as a discussion on the important topic of evaluating models and datasets, which are a critical ingredient of deep learning based imaging.

Reinhard Heckel is a Professor of Machine Learning (Tenured Associate Professor) at the Department of Computer Engineering at the Technical University of Munich (TUM), and adjunct faculty at Rice University, where he was an assistant professor of Electrical and Computer Engineering from 2017-2019. Before that, he was a postdoctoral researcher in the Berkeley Artificial Intelligence Research Lab at UC Berkeley, and before that a researcher at IBM Research Zurich. He completed his PhD in 2014 at ETH Zurich and was a visiting PhD student at Stanfords University's Statistics Department. Reinhard's work is centered on machine learning, artificial intelligence, and information processing, with a focus on developing algorithms and foundations for deep learning, particularly for medical imaging, on establishing mathematical and empirical underpinnings for machine learning, and on the utilization of DNA as a digital information technology.