BIPS:
Bayesian Inference for the Physical Sciences
Rev. Thomas Bayes (1702-1761) and Pierre Simon Laplace (1749-1827)
ANNOUNCEMENT:
Penn State's Center for Astrostatistics and SAMSI will jointly
host a winter school in astrostatistics 18-25 January 2006,
including 3 days devoted to Bayesian methods and 2 days devoted
to nonparametric and machine learning methods. The school also
includes 2 days of tutorials on astronomy for statisticians.
See the
CASt School page for info.
Welcome to BIPS: Bayesian Inference for the Physical Sciences, an annotated online index/clearinghouse for information
on the Bayesian approach to statistical inference of special
relevance to applications in the physical sciences.
This page is constantly evolving, so I
hope you'll visit regularly. To guide you in your
repeat visits, the What's New AT BIPS page
summarizes recent changes in reverse chronological order.
If you come across an online resource that you think should be referenced
here, please don't hesitate to pass it along to the BIPS webslave, via
the links at the bottom of this and other pages.
Bayes's Theorem
Directory
- Probability Theory: The Logic of Science
- This site hosts PDF and PostScript files of physicist Ed Jaynes's
monumental treatise on Bayesian inference, the first volume of which
will be published in 1999/2000 by Cambridge University Press. The same
site hosts an excellent brief
biography of Ed Jaynes written by his last graduate student, Larry
Bretthorst.
- Jaynes: Probability
Theory---The Logic of Science
- Yet another site archiving the chapters of Jaynes's book in
PostScript, made available
here by mathematician Carlos Rodriguez. The version here is
not the latest version.
- Bayesian Spectrum Analysis and Parameter Estimation
- This 1988 book by Larry Bretthorst is a very readable and practical
introduction to Bayesian inference applied to the analysis of time series
data with additive Gaussian noise. Published by Springer-Verlag, it is
now out of print, but by arrangement with the publishers is available
online at the Washington University Bayesian Reprints web site maintained
by Larry.
- Washington University Bayesian Reprints
- This site archives tutorial and application papers by Ed Jaynes,
Larry Bretthorst, Stephen Gull, David Mackay, Devinder Sivia,
Phil Gregory, and Tom
Loredo (physical scientists associated with the MaxEnt conferences).
The tutorial papers include the following (PDF links provided here;
PostScript links also available via the URL above):
- Data Analysis: A Bayesian Tutorial
- This is an undergraduate text on
Bayesian inference written for physical scientists by Devinder Sivia,
and published by Oxford University Press. It is available in paperback.
A bit more info, including some readers' comments, can be found
at Sivia's home page. Originally published in Summer of 1996, this was the first
"modern" Bayesian book written expressly for physical scientists,
and built on the work of the leading physicist exponents of
Bayesian methods (Jaynes, Bretthorst, Gull, and Skilling).
Please eschew the strange use of
the term "evidence" in this otherwise fine book; this usage is unique
to the Cambridge/Oxford MaxEnt practitioners, and the term is used
differently in other Bayesian literature. A revision is in progress
that will update the book with material on modern computational
techniques.
- Bayesian Logical Data Analysis for the Physical Sciences: A Comparative Approach with Mathematica Support
- This is the newest book (May 2005) on Bayesian methods for physical
scientists, written by astronomer Phil Gregory. Aimed at graduate
students, it covers the fundamentals at a level between that
of the Jaynes and Sivia books. It has a much more "applied" focus than
Jaynes's book, and covers several applications at a level well beyond
what is addressed in Sivia's book. There is particularly extensive
coverage of spectral analysis (detecting and measuring periodic
signals), including a self-contained introduction to Fourier
methods. It also includes some treatment of
frequentist methods. It is currently the only book written for physical
scientists that covers modern Bayesian computational methods (i.e.,
MCMC), though only a small part of the "computational landscape" is
explored. It is available in hardcover and in eBook format. A
Mathematica notebook with material supporting
the book is available for download from the BLDA support page.
Gregory has actively taught Bayesian methods to physics and astronomy
students for over a decade; the book certainly benefits from his
extensive classroom experience.
- Bayesian Reasoning in High Energy Physics: Principles and Applications
- Giulio D'Agostini, an outspoken proponent of Bayesian methods in
high energy physics, has extensive course notes on basic Bayesian
statistics archived at the CERN Lecture Program web site (see the 1997/1998 section) as a series of 5 PostScript files with over 150 pages:
This report is also available as
CERN "Yellow" Report CERN-99-03.
- Maximum Entropy Data Consultants: References
- Included on this page is John Skilling's article from the
Journal of Microscopy (1998), "Probabilistic Data Analysis; An Introductory Guide." This gives a brief introduction to the Bayesian
approach, and an introductory overview of various Monte Carlo methods
for doing Bayesian calculations. Please eschew the strange use of
the term "evidence" in this otherwise fine review; this usage is unique
to the Cambridge/Oxford MaxEnt practitioners, and the term is used
differently in other Bayesian literature.
- Kenneth Hanson's Home Page
- Hanson worked at the Los Alamos National Lab using Bayesian methods to deal
with problems in uncertainty quantification for physics models.
This site includes articles documenting the
work of Hanson and his collaborators, as well as tutorial articles by Hanson and by
guest speakers
visiting his Uncertainty Quantification Working Group. These include:
- Tom Loredo's Bayesian Reprints
- Tutorial and research papers by the editor of the BIPS web site.
Three general tutorials and reviews that may be of particular interest to BIPS visitors include:
- From Laplace to Supernova SN 1987A: Bayesian Inference in Astrophysics (1990)
- The Promise of Bayesian Inference for Astrophysics (1992)
- The Return of the Prodigal: Bayesian Inference in Astrophysics (1994)
- Bayesian Inference: A Practical Primer (slides from a talk at MaxEnt 2000)
Visit the link above to find descriptions and links for these papers.
- Bayesian Analysis E-print
Archive
- Hosted by the Los Alamos XXX server that hosts the main reprint
archives in physics and astronomy, this part of the archive sees
little traffic, but the papers that do appear here are usually
of specific relevance to physical scientists.
- ISBA/SBSS Bayesian Abstract Archive
- An archive of Bayesian publications, including technical reports
and theses as well as more publicly available publications, hosted
by ISDS at Duke and sponsored by
ISBA (International Society for Bayesian Analysis) and
SBSS (Section on Bayesian Statistical Sciences of the American
Statistical Association).
- MCMC Preprints
- A well-maintained collection of pointers and bibliographic information
for publications on Markov Chain Monte Carlo (MCMC) methods.
- Perfectly Random Sampling from Markov Chains
- David Wilson's web page providing annotated bibliographic information
for articles describing a new (1996) approach
to using Markov Chains to simulate distributions arising in statistical
mechanics, Bayesian inference, and other fields. This approach, "perfectly
random sampling" or "exact sampling," removes the problem of assessing convergence of Markov chains
(for some problems), and can be used to create a set of iid samples
from the Markov chain's stationary distribution.
A brief and basic tutorial
on exact sampling is available at Radford Neal's
Miscellaneous
Documents page.
- Washington University Bayesian Reprints
- This site archives tutorial and application papers by Ed Jaynes, Larry Bretthorst,
David Mackay, and Tom Loredo (physical scientists associated with
the MaxEnt conferences).
- Bayesian Model Averaging Home Page
- Chris Volinsky's collection of links to articles and software on
Bayesian model averaging, a technique for accounting for uncertainty
in model specification when making inferences.
- Space Telescope Science Institute Search Page
- STSci archives a number of papers in HTML format describing Bayesian
methods for "inverting" complicated imaging data. You can locate these
via the search facility linked here (try searching for "bayesian").
- G. D'Agostini - Probability and Statistics
- D'Agostini, a leading proponent of Bayesian methods in particle
physics, collects here many of his papers on the Bayesian approach.
- Geophysical Inverse Theory
- John Scale's web site with extensive course notes, a review paper,
and other papers on
Bayesian approaches to solving geophysical inverse problems, including
software resources in Xlist-Stat and Mathematica.
- Maximum Entropy Data Consultants: References
- This page from the MEDC web site collects a few of the references
by John Skilling and Sibusiso Sibisi on the "Massive Inference" approach
to Bayesian density estimation (using infinitely divisible process
models, such as gamma, Dirichlet, and compound Poisson processes). Further references on this topic
are available via ftp
from DAMTP at U. Cambridge.
- Non-Subjective Bayesian Statistical Methodology
- This site maintains a PostScript catalog of "noninformative" priors
of various types for various problems (conventional priors, reference
priors, etc.), as well as lists of conferences and researchers
concerned with "default" Bayesian methods.
- Tom Loredo's Bayesian Reprints
- Tutorial and research papers by the editor of the BIPS web site.
Note: (P) indicates a package with multiple functions, documentation, etc.
Other programs are often single-purpose subroutines with little on-line documentation.
- StatLib (CMU)
- StatLib, hosted by Carnegie Mellon University, is a system for
distributing statistics software, much of it Bayesian.
- The AutoClass Project
- Unsupervised Bayesian classification system that seeks a maximum posterior
probability classification for multivariate data, including mixed real-value and discrete data with
missing values. (P)
- BAYESPACK, etc. (Alan Genz's Home Page)
- Genz is a leader in the development of new algorithms for numerical
computation of mulitiple integrals; his recent research focuses on integrals
that arise in Bayesian inference. His homepage provides many of his
papers in PostScript form, and collections of FORTRAN software implementing
and demonstrating his algorithms. His BAYESPACK collection will be of
particular interest to those doing Bayesian calculations of
modest (up to 10 or so) dimensions. SunOS Unix users may prefer the
gzipped tar file version of the package
here at BIPS to Genz's piecemeal distribution (the tar version also
fixes a minor incompatibility with the SunOS f77 compiler).
(P)
- BUGS: Bayesian inference Using Gibbs Sampling (also an ftp directory)
- BUGS is a program for Bayesian inference using the
Gibbs Sampler Markov chain Monte Carlo technique produced by the
Biostatistics Unit of the
Medical Research Council of the United Kingdom.
It is written in Modula 2 and distributed as compiled
code for a variety of platforms. The sites above host the software
and documentation in dowloadable files.
There is also an extensive online HTML manual for BUGS.
(P)
- Radford Neal's software for flexible Bayesian modeling
- "This software supports Bayesian regression and classification
models based on neural networks and Gaussian processes, and Bayesian
mixture models. It also supports a variety of Markov chain sampling
methods, which may be applied to distributions specified by simple
formulas, including simple Bayesian models defined by formulas for the
prior and likelihood. The latest version, of 1999-03-13, allows you to
try lots of Markov chain sampling methods on Bayesian models defined
using a BUGS-like notation." The software consists of a suite of
command-line programs that can be chained together (interacting through
data stored in files) to make inferences. (ANSI C for Unix, with
support for xgraph under X-windows.)
(P)
- Bayesian Model Averaging Home Page
- Chris Volinsky's collection of links to articles and software on
Bayesian model averaging, a technique for accounting for uncertainty
in model specification when making inferences. Includes links to
several S-PLUS packages.
- Gibbs sampler iterations
- Calculates the
number of iterations needed in a Markov chain Monte Carlo run.
- First Bayes
- Teaching package for elementary Bayesian statistics. (P)
- Belief Networks
- Links to software for graphical belief functions, Bayesian networks and influence
diagrams. (P)
- B/D
- Bayes linear methods based on expectation and covariance structures.
- Math 648 -
Bayesian Inference
- Minitab examples.
- MatLab Scripts for Bayesian Blocks
- Jeff Scargle here provides the text of his 1998 Astrophysical Journal article on Bayesian blocks (a Poisson changepoint model
for detecting variability) and MatLab scripts and sample data for
doing Bayes Blocks calculations.
- Bayesian Time Series Analysis Software by Mike West and Colleagues
- This site provides pointers to three collections of software Mike
West (ISDS) and his colleagues have produced, as well as collections
of data sets for software testing and development. Included are
the BATS software (DOS/Win) for the book Applied Bayesian Forecasting
and Time Series Analysis, and Fortran90/S-plus software
for nonstationary time series analysis and analysis with autoregressive
component models. (P)
- Maximum Entropy Data Consultants
- This commercial firm based in the UK was founded by astronomers
John Skilling and Steve Gull, and sells the influential and widely
used MEMSYS code for quantified (i.e., Bayesian) maximum entropy
"deconvolution" of data. They also develop software implementing
"massive inference" techniques for Bayesian density estimation.
- Organizations/Societies/Directories:
- ISBA: International Society for Bayesian Analysis
- The main international organization promoting the development
and application of Bayesian methods, at www.bayesian.org.
There is also still some ISBA info
at the old ISBA site at Albany.
- ASA Section on
Bayesian Statistical Sciences
- Bayesians Worldwide: Bayesian
Statistics Personal Web Pages
- Home Pages of Bayesian Groups/Centers:
- ISDS: Institute for Statistics and Decision Sciences
- Based at Duke University in North Carolina, ISDS hosts a graduate program
in statistics that includes a strong focus on Bayesian inference.
- Bayesian Model-Based Learning Group
- Located at NASA Ames Research Center, this group works on the theory
and associated algorithms for various kinds of general data analysis
techniques using Bayesian inference. The AutoClass home
page is here, as well as a site on Bayesian search methods.
- Cambridge University Inferential Sciences Group (also here)
- This is the home page for the Cambridge Bayesian Inference/MaxEnt group. They are
perhaps most closely associated with Maximum Entropy methods, but
in the 90s their work has taken on a more general Bayesian flavor.
- Harvard/CfA Astronomy & Statistics Working Group
- This group consists of astronomers from the Harvard-Smithsonian Center
for Astrophysics and statisticians from Harvard University's Department of Statistics,
collaborating on astrostatistics problems. Their emphasis is on using
hierarchical Bayesian models and MCMC algorithms to analyze data from
the Chandra X-ray satellite. Some of their methodology
is available in the Chandra Interactive Analysis of Observations (CIAO)
software.
- MPIPP Bayesian Data Analysis Group
- This is the home page for a group of scientists at the Max Planck
Institute for Plasma Physics who develop Bayesian methodology and
software for various general data analysis problems. They hosted
the 1998 Workshop on Maximum Entropy and Bayesian Methods. Their
interests include inverse problems,
wavelets, and signal and background separation.
- Albany MaxEnt Page
- This small site has links with information about the recent annual
workshops on Maximum Entropy and Bayesian methods.
- Non-Subjective Bayesian Statistical Methodology
- This site hosts lists of publications, reports, conferences, and
contacts regarding the use of "default" priors in Bayesian inference.
- Dept. of Statistics at Carnegie Mellon University
- Though not a Bayesian group, with Department members like Jay Kadane,
Rob Kass, and Larry Wasserman, this group produces a lot of Bayesian
work. They are especially interested in interdisciplinary applications,
and run an annual workshop on applied Bayesian statistics. CMU also
hosts the StatLib statistics software archive.
- Statistics at Univeristy of Washington
- A site with notable Bayesian content from Adrian Raftery, Julian Besag,
Peter Hoff, Matthew Stephens and their colleagues on such topics
as spatial statistics, nonparametrics, model averaging, clustering, and
classification.
- Conference Information:
- MaxEnt 2000: 20th International Workshop on Maximum Entropy and Bayesian Methods
(alternate URL)
- This year's workshop will be hosted by Ali Mohammad-Djafari
in France, 8-13 July 2000.
- 6th Valencia International Meeting on Bayesian Statistics
- The major regular gathering of Bayesian statisticians,
held every three to four years in Spain. The 6th meeting will be held
from June 6 to June 10, 1998.
- Fourth Workshop
on Bayesian Statistics in Science and Technology
- This annual workshop, hosted by the Statistics Department at
Carnegie-Mellon University, offers a variety of lecture and poster
presentations, with special in-depth lectures and discussion of a
few chosen case studies of practical applications of Bayesian inference.
The 4th Workshop will be held September 25-27, 1997.
- Bayesian Signal Processing
- This is a 1-week session (20-29 July 1998) offered as part of a
six month (July to December 1998) program on
Nonlinear and Nonstationary Signal Processing hosted by the
Isaac Newton Institute for Mathematical Sciences in Cambridge, England. Topics to be covered include
Bayesian numerical methods, nonlinear/nonstationary time series,
forecasting, changepoint modeling, dynamical systems, and applications
in econometrics and environmental and spatial data analysis.
- The Bayesian Songbook
- Every three to four years there is a large gathering of Bayesian
statisticians in Valencia, Spain; these are the famous and influential
"Valencia" meetings that have produced a series of conference
proceedings simply titled Bayesian Statistics 5, etc..
A tradition of these meetings is a "cabaret" performance by
participants, typically featuring at least one "adaptation" of
a popular song with lyrics altered to refer to some aspect of
Bayesian statistics. This web site collects the lyrics in LaTeX
and PostScript, and even a few audio files (WAV format).
- On Thomas Bayes:
- MacTutor Bio
- Portrait
- Portrait/Outline
- Virtual Library
Entry on Statistics
- StatLib (CMU)
- StatLib, hosted by Carnegie Mellon University, is a system for
distributing statistics software, much of it Bayesian.
- StatCodes
- "StatCodes is a metasite with links to over 200 sites providing free
on-line codes implementing statistical methods that may be useful to
astronomers. Methods include time series analysis, multivariate
analysis, censored and truncated data, nonparametric statistics,
correlation and regression, Bayesian methods (hosted by Tom Loredo,
Cornell University [the site you are at right now!]),
density estimation and smoothing, image analysis,
spatial statistics, visualization tools, interactive Web tools, and
multipurpose statistics packages. In some fields like signal processing
and wavelet analysis, the resources are vast and links are given to
other metasites. Some codes are single subroutines (usually in Fortran
or C), while others are full packages with documentation. The contents
of StatCodes is searchable through the Astronomical Software Directory
Service (http://doright.stsci.edu/asds/docindex.html). StatCodes
welcomes links to new codes, particularly those written by astronomers.
StatCodes is maintained by Penn State astronomer
Eric Feigelson (edf@astro.psu.edu)."
- The DATA Center Home Page
- The Center for Data Analysis Technology and Applications is
an informal forum for exchanging ideas about new data analysis
technology with astrophysical applications and is hosted by
Jeff Scargle at NASA/Ames. The web site hosts descriptions of
talks presented at DATA Center meetings and a bibliography of
publications of members. Scargle and a few other members work
on astrophysical applications of Bayesian inference.
- PDG: Particle Data Group
- The home page for the Particle Data Group that produces the
annual Review of Particle Physics which includes
recommendations of statistical methods for common data analysis tasks in
high energy physics. The little Bayesian content in the Review
remains a subject of continuing controversy. The statistical
content can be found in the
"Mathematical Tools" section of the Review.
- The Data Analysis BriefBook
- An on-line version of a standard particle physics reference on
data analysis methods, prepared and hosted by CERN. Bayesian content
is virtually nonexistant, but it's a useful reference for its description
of current statistical practice in high energy physics.
Thanks to Eric Feigelson for providing a head start in locating online
sources of Bayesian software.
Tom Loredo's Astro Home
Page /