Advanced Structured Prediction

An overview of recent work in the field of structured prediction, the building of predictive machine learning models for interrelated and dependent outputs. The goal of structured prediction is to build machine learning models that predict relational information that itself has structure, such as being composed of multiple interrelated parts. These models, which reflect prior knowledge, task-specific relations, and constraints, are used in fields including computer vision, speech recognition, natural language processing, and computational biology. They can carry out such tasks as predicting a natural language sentence, or segmenting an image into meaningful components. These models are expressive and powerful, but exact computation is often intractable. A broad research effort in recent years has aimed at designing structured prediction models and approximate inference and learning procedures that are computationally efficient. This volume offers an overview of this recent research in order to make the work accessible to a broader research community. The chapters, by leading researchers in the field, cover a range of topics, including research trends, the linear programming relaxation approach, innovations in probabilistic modeling, recent theoretical progress, and resource-aware learning. Contributors Jonas Behr, Yutian Chen, Fernando De La Torre, Justin Domke, Peter V. Gehler, Andrew E. Gelfand, Sébastien Giguère, Amir Globerson, Fred A. Hamprecht, Minh Hoai, Tommi Jaakkola, Jeremy Jancsary, Joseph Keshet, Marius Kloft, Vladimir Kolmogorov, Christoph H. Lampert, François Laviolette, Xinghua Lou, Mario Marchand, André F. T. Martins, Ofer Meshi, Sebastian Nowozin, George Papandreou, Daniel Průša, Gunnar Rätsch, Amélie Rolland, Bogdan Savchynskyy, Stefan Schmidt, Thomas Schoenemann, Gabriele Schweikert, Ben Taskar, Sinisa Todorovic, Max Welling, David Weiss, Thomáš Werner, Alan Yuille, Stanislav Živný

Gathering leading specialists in the field of structure prediction, this book provides a unique view of this complex and rapidly developing field, reflecting the numerous viewpoints of the different authors. A summary of the major achievements over the last few years and of the challenges still remaining makes this monograph very timely.

While crystal structure prediction (CSP) remains a longstanding challenge, we introduce ParetoCSP, a novel algorithm for CSP, which combines a multi-objective genetic algorithm (GA) with a neural network inter-atomic potential model to find energetically optimal crystal structures given chemical compositions. We enhance the updated multi-objective GA (NSGA-III) by incorporating the genotypic age as an independent optimization criterion and employ the M3GNet universal inter-atomic potential to guide the GA search. Compared to GN-OA, a state-of-the-art neural potential-based CSP algorithm, ParetoCSP demonstrated significantly better predictive capabilities, outperforming by a factor of 2.562 across 55 diverse benchmark structures, as evaluated by seven performance metrics. Trajectory analysis of the traversed structures of all algorithms shows that ParetoCSP generated more valid structures than other algorithms, which helped guide the GA to search more effectively for the optimal structures. Our implementation code is available at https://github.com/sadmanomee/ParetoCSP.

State-of-the-art algorithms and theory in a novel domain of machine learning, prediction when the output has structure.

Processing, Analyzing and Learning of Images, Shapes, and Forms: Part 2, Volume 20, surveys the contemporary developments relating to the analysis and learning of images, shapes and forms, covering mathematical models and quick computational techniques. Chapter cover Alternating Diffusion: A Geometric Approach for Sensor Fusion, Generating Structured TV-based Priors and Associated Primal-dual Methods, Graph-based Optimization Approaches for Machine Learning, Uncertainty Quantification and Networks, Extrinsic Shape Analysis from Boundary Representations, Efficient Numerical Methods for Gradient Flows and Phase-field Models, Recent Advances in Denoising of Manifold-Valued Images, Optimal Registration of Images, Surfaces and Shapes, and much more. Covers contemporary developments relating to the analysis and learning of images, shapes and forms Presents mathematical models and quick computational techniques relating to the topic Provides broad coverage, with sample chapters presenting content on Alternating Diffusion and Generating Structured TV-based Priors and Associated Primal-dual Methods

Processing, Analyzing and Learning of Images, Shapes, and Forms: Part 2, Volume 20, surveys the contemporary developments relating to the analysis and learning of images, shapes and forms, covering mathematical models and quick computational techniques. Chapter cover Alternating Diffusion: A Geometric Approach for Sensor Fusion, Generating Structured TV-based Priors and Associated Primal-dual Methods, Graph-based Optimization Approaches for Machine Learning, Uncertainty Quantification and Networks, Extrinsic Shape Analysis from Boundary Representations, Efficient Numerical Methods for Gradient Flows and Phase-field Models, Recent Advances in Denoising of Manifold-Valued Images, Optimal Registration of Images, Surfaces and Shapes, and much more. Covers contemporary developments relating to the analysis and learning of images, shapes and forms Presents mathematical models and quick computational techniques relating to the topic Provides broad coverage, with sample chapters presenting content on Alternating Diffusion and Generating Structured TV-based Priors and Associated Primal-dual Methods

A description of perturbation-based methods developed in machine learning to augment novel optimization methods with strong statistical guarantees. In nearly all machine learning, decisions must be made given current knowledge. Surprisingly, making what is believed to be the best decision is not always the best strategy, even when learning in a supervised learning setting. An emerging body of work on learning under different rules applies perturbations to decision and learning procedures. These methods provide simple and highly efficient learning rules with improved theoretical guarantees. This book describes perturbation-based methods developed in machine learning to augment novel optimization methods with strong statistical guarantees, offering readers a state-of-the-art overview. Chapters address recent modeling ideas that have arisen within the perturbations framework, including Perturb & MAP, herding, and the use of neural networks to map generic noise to distribution over highly structured data. They describe new learning procedures for perturbation models, including an improved EM algorithm and a learning algorithm that aims to match moments of model samples to moments of data. They discuss understanding the relation of perturbation models to their traditional counterparts, with one chapter showing that the perturbations viewpoint can lead to new algorithms in the traditional setting. And they consider perturbation-based regularization in neural networks, offering a more complete understanding of dropout and studying perturbations in the context of deep neural networks.

Volume Two of this two-volume sequence presents a comprehensive overview of protein structure prediction methods and includes protein threading, De novo methods, applications to membrane proteins and protein complexes, structure-based drug design, as well as structure prediction as a systems problem. A series of appendices review the biological and chemical basics related to protein structure, computer science for structural informatics, and prerequisite mathematics and statistics.