Optimization Techniques In Statistics

Statistics help guide us to optimal decisions under uncertainty. A large variety of statistical problems are essentially solutions to optimization problems. The mathematical techniques of optimization are fundamentalto statistical theory and practice. In this book, Jagdish Rustagi provides full-spectrum coverage of these methods, ranging from classical optimization and Lagrange multipliers, to numerical techniques using gradients or direct search, to linear, nonlinear, and dynamic programming using the Kuhn-Tucker conditions or the Pontryagin maximal principle. Variational methods and optimization in function spaces are also discussed, as are stochastic optimization in simulation, including annealing methods. The text features numerous applications, including: Finding maximum likelihood estimates, Markov decision processes, Programming methods used to optimize monitoring of patients in hospitals, Derivation of the Neyman-Pearson lemma, The search for optimal designs, Simulation of a steel mill. Suitable as both a reference and a text, this book will be of interest to advanced undergraduate or beginning graduate students in statistics, operations research, management and engineering sciences, and related fields. Most of the material can be covered in one semester by students with a basic background in probability and statistics. Covers optimization from traditional methods to recent developments such as Karmarkars algorithm and simulated annealing Develops a wide range of statistical techniques in the unified context of optimization Discusses applications such as optimizing monitoring of patients and simulating steel mill operations Treats numerical methods and applications Includes exercises and references for each chapter Covers topics such as linear, nonlinear, and dynamic programming, variational methods, and stochastic optimization

Optimization techniques are used to find the values of a set of parameters which maximize or minimize some objective function of interest. Such methods have become of great importance in statistics for estimation, model fitting, etc. This text attempts to give a brief introduction to optimization methods and their use in several important areas of statistics. It does not pretend to provide either a complete treatment of optimization techniques or a comprehensive review of their application in statistics; such a review would, of course, require a volume several orders of magnitude larger than this since almost every issue of every statistics journal contains one or other paper which involves the application of an optimization method. It is hoped that the text will be useful to students on applied statistics courses and to researchers needing to use optimization techniques in a statistical context. Lastly, my thanks are due to Bertha Lakey for typing the manuscript.

This book covers several bases at once. It is useful as a textbook for a second course in experimental optimization techniques for industrial production processes. In addition, it is a superb reference volume for use by professors and graduate students in Industrial Engineering and Statistics departments. It will also be of huge interest to applied statisticians, process engineers, and quality engineers working in the electronics and biotech manufacturing industries. In all, it provides an in-depth presentation of the statistical issues that arise in optimization problems, including confidence regions on the optimal settings of a process, stopping rules in experimental optimization, and more.

A concise text that presents and analyzes the fundamental techniques and methods in optimization that are useful in data science.

A description of perturbation-based methods developed in machine learning to augment novel optimization methods with strong statistical guarantees. In nearly all machine learning, decisions must be made given current knowledge. Surprisingly, making what is believed to be the best decision is not always the best strategy, even when learning in a supervised learning setting. An emerging body of work on learning under different rules applies perturbations to decision and learning procedures. These methods provide simple and highly efficient learning rules with improved theoretical guarantees. This book describes perturbation-based methods developed in machine learning to augment novel optimization methods with strong statistical guarantees, offering readers a state-of-the-art overview. Chapters address recent modeling ideas that have arisen within the perturbations framework, including Perturb & MAP, herding, and the use of neural networks to map generic noise to distribution over highly structured data. They describe new learning procedures for perturbation models, including an improved EM algorithm and a learning algorithm that aims to match moments of model samples to moments of data. They discuss understanding the relation of perturbation models to their traditional counterparts, with one chapter showing that the perturbations viewpoint can lead to new algorithms in the traditional setting. And they consider perturbation-based regularization in neural networks, offering a more complete understanding of dropout and studying perturbations in the context of deep neural networks.

A guide to modern optimization applications and techniques in newly emerging areas spanning optimization, data science, machine intelligence, engineering, and computer sciences Optimization Techniques and Applications with Examples introduces the fundamentals of all the commonly used techniques in optimization that encompass the broadness and diversity of the methods (traditional and new) and algorithms. The author—a noted expert in the field—covers a wide range of topics including mathematical foundations, optimization formulation, optimality conditions, algorithmic complexity, linear programming, convex optimization, and integer programming. In addition, the book discusses artificial neural network, clustering and classifications, constraint-handling, queueing theory, support vector machine and multi-objective optimization, evolutionary computation, nature-inspired algorithms and many other topics. Designed as a practical resource, all topics are explained in detail with step-by-step examples to show how each method works. The book’s exercises test the acquired knowledge that can be potentially applied to real problem solving. By taking an informal approach to the subject, the author helps readers to rapidly acquire the basic knowledge in optimization, operational research, and applied data mining. This important resource: Offers an accessible and state-of-the-art introduction to the main optimization techniques Contains both traditional optimization techniques and the most current algorithms and swarm intelligence-based techniques Presents a balance of theory, algorithms, and implementation Includes more than 100 worked examples with step-by-step explanations Written for upper undergraduates and graduates in a standard course on optimization, operations research and data mining, Optimization Techniques and Applications with Examples is a highly accessible guide to understanding the fundamentals of all the commonly used techniques in optimization.

Optimizing Methods in Statistics.