Maximum Entropy And Bayesian Methods

Cambridge, England, 1988

The Twelfth International Workshop on Maximum Entropy and Bayesian Methods in Sciences and Engineering (MaxEnt 92) was held in Paris, France, at the Centre National de la Recherche Scientifique (CNRS), July 19-24, 1992. It is important to note that, since its creation in 1980 by some of the researchers of the physics department at the Wyoming University in Laramie, this was the second time that it took place in Europe, the first time was in 1988 in Cambridge. The two specificities of MaxEnt workshops are their spontaneous and informal charac ters which give the participants the possibility to discuss easily and to make very fruitful scientific and friendship relations among each others. This year's organizers had fixed two main objectives: i) to have more participants from the European countries, and ii) to give special interest to maximum entropy and Bayesian methods in signal and image processing. We are happy to see that we achieved these objectives: i) we had about 100 participants with more than 50 per cent from the European coun tries, ii) we received many papers in the signal and image processing subjects and we could dedicate a full day of the workshop to the image modelling, restoration and recon struction problems.

This collection of papers by leading researchers in their respective fields contains contributions showing the use of the maximum entropy method in many of the fields in which it finds application. In the physical, mathematical and biological sciences it is often necessary to make inferences based on insufficient data. The problem of choosing one among the many possible conclusions or models which are compatible with the data may be resolved in a variety of ways. A particularly appealing method is to choose the solution which maximizes entropy in the sense that the conclusion or model honours the observed data but implies no further assumptions not warranted by the data. The maximum entropy principle has been growing in importance and acceptance in many fields, perhaps most notably statistical physics, astronomy, geophysics, signal processing, image analysis and physical chemistry. The papers included in this volume touch on most of the current areas of research activity and application, and will be of interest to research workers in all fields in which the maximum entropy method may be applied.

The 10th International Workshop on Maximum Entropy and Bayesian Methods, MaxEnt 90, was held in Laramie, Wyoming from 30 July to 3 August 1990. This volume contains the scientific presentations given at that meeting. This series of workshops originated in Laramie in 1981, where the first three of what were to become annual workshops were held. The fourth meeting was held in Calgary. the fifth in Laramie, the sixth and seventh in Seattle, the eighth in Cambridge, England, and the ninth at Hanover, New Hampshire. It is most appropriate that the tenth workshop, occurring in the centennial year of Wyoming's statehood, was once again held in Laramie. The original purpose of these workshops was twofold. The first was to bring together workers from diverse fields of scientific research who individually had been using either some form of the maximum entropy method for treating ill-posed problems or the more general Bayesian analysis, but who, because of the narrow focus that intra-disciplinary work tends to impose upon most of us, might be unaware of progress being made by others using these same techniques in other areas. The second was to introduce to those who were somewhat aware of maximum entropy and Bayesian analysis and wanted to learn more, the foundations, the gestalt, and the power of these analyses. To further the first of these ends, presenters at these workshops have included workers from area. s as varied as astronomy, economics, environmenta.

In 1978 Edwin T. Jaynes and Myron Tribus initiated a series of workshops to exchange ideas and recent developments in technical aspects and applications of Bayesian probability theory. The first workshop was held at the University of Wyoming in 1981 organized by C.R. Smith and W.T. Grandy. Due to its success, the workshop was held annually during the last 18 years. Over the years, the emphasis of the workshop shifted gradually from fundamental concepts of Bayesian probability theory to increasingly realistic and challenging applications. The 18th international workshop on Maximum Entropy and Bayesian Methods was held in Garching / Munich (Germany) (27-31. July 1998). Opening lectures by G. Larry Bretthorst and by Myron Tribus were dedicated to one of th the pioneers of Bayesian probability theory who died on the 30 of April 1998: Edwin Thompson Jaynes. Jaynes revealed and advocated the correct meaning of 'probability' as the state of knowledge rather than a physical property. This inter pretation allowed him to unravel longstanding mysteries and paradoxes. Bayesian probability theory, "the logic of science" - as E.T. Jaynes called it - provides the framework to make the best possible scientific inference given all available exper imental and theoretical information. We gratefully acknowledge the efforts of Tribus and Bretthorst in commemorating the outstanding contributions of E.T. Jaynes to the development of probability theory.

This volume has its origin in the Fifth, Sixth and Seventh Workshops on and Bayesian Methods in Applied Statistics", held at "Maximum-Entropy the University of Wyoming, August 5-8, 1985, and at Seattle University, August 5-8, 1986, and August 4-7, 1987. It was anticipated that the proceedings of these workshops would be combined, so most of the papers were not collected until after the seventh workshop. Because all of the papers in this volume are on foundations, it is believed that the con tents of this volume will be of lasting interest to the Bayesian community. The workshop was organized to bring together researchers from different fields to critically examine maximum-entropy and Bayesian methods in science and engineering as well as other disciplines. Some of the papers were chosen specifically to kindle interest in new areas that may offer new tools or insight to the reader or to stimulate work on pressing problems that appear to be ideally suited to the maximum-entropy or Bayesian method. A few papers presented at the workshops are not included in these proceedings, but a number of additional papers not presented at the workshop are included. In particular, we are delighted to make available Professor E. T. Jaynes' unpublished Stanford University Microwave Laboratory Report No. 421 "How Does the Brain Do Plausible Reasoning?" (dated August 1957). This is a beautiful, detailed tutorial on the Cox-Polya-Jaynes approach to Bayesian probability theory and the maximum-entropy principle.

Maximum entropy and Bayesian methods have fundamental, central roles in scientific inference, and, with the growing availability of computer power, are being successfully applied in an increasing number of applications in many disciplines. This volume contains selected papers presented at the Thirteenth International Workshop on Maximum Entropy and Bayesian Methods. It includes an extensive tutorial section, and a variety of contributions detailing application in the physical sciences, engineering, law, and economics. Audience: Researchers and other professionals whose work requires the application of practical statistical inference.

This volume records papers given at the fourteenth international maximum entropy conference, held at St John's College Cambridge, England. It seems hard to believe that just thirteen years have passed since the first in the series, held at the University of Wyoming in 1981, and six years have passed since the meeting last took place here in Cambridge. So much has happened. There are two major themes at these meetings, inference and physics. The inference work uses the confluence of Bayesian and maximum entropy ideas to develop and explore a wide range of scientific applications, mostly concerning data analysis in one form or another. The physics work uses maximum entropy ideas to explore the thermodynamic world of macroscopic phenomena. Of the two, physics has the deeper historical roots, and much of the inspiration behind the inference work derives from physics. Yet it is no accident that most of the papers at these meetings are on the inference side. To develop new physics, one must use one's brains alone. To develop inference, computers are used as well, so that the stunning advances in computational power render the field open to rapid advance. Indeed, we have seen a revolution. In the larger world of statistics beyond the maximum entropy movement as such, there is now an explosion of work in Bayesian methods, as the inherent superiority of a defensible and consistent logical structure becomes increasingly apparent in practice.