Tom Loredo's Bayesian Reprints

A resource from the BIPS web site.

The papers and reports listed below are those hosted here; many additional publications are listed in my ADS public library and in my recent CV and publications list (PDF). But these are the ones likely to be of most general interest.

Thank you for your interest in my work!

Tutorial and Review Papers

From Laplace to Supernova SN 1987A: Bayesian Inference in Astrophysics (1990)

A review of the fundamentals of Bayesian inference, with a somewhat conceptual or philosophical (rather than practical) emphasis. It's a bit more polemical in places than it would be if I wrote it today, but Bayesian methods have come a long way since 1989! I provide some simple examples with the Gaussian and Poisson distribution, and outline more complicated applications. Published in Maximum-Entropy and Bayesian Methods, Dartmouth, 1989, ed. P. Fougere (Dordrecht, The Netherlands: Kluwer Academic Publishers) pp. 81--142 (1990). Available here as a scanned PDF and in preprint form (PDF and PS) at the Washington University Bayesian Reprints site. Note the errata at the end.

The Promise of Bayesian Inference for Astrophysics (1992)

Discusses the difference between Bayesian and frequentist calculations, emphasizing methodological (rather than conceptual or philosophical) differences between the approaches. I use simple examples throughout, and include an introduction to Bayesian inference with Poisson counting and point processes, including a Bayesian analysis of "on/off" data, and the Bayesian counterpart to the Rayleigh test for measuring periodicity in arrival time series. Published in Statistical Challenges in Modern Astronomy, ed. E.D. Feigelson and G.J. Babu (New York: Springer-Verlag) pp. 275--297 (1992). The published version is a shortened version of a longer manuscript available here as a PDF file. Mike West was one of the discussants of this paper and his valuable comments are also available online as a PostScript file at the ISDS FTP site.

The Return of the Prodigal: Bayesian Inference in Astrophysics (1994)

This 37 page review article, including a simplified analysis of the COBE CBR quadrupole data as a case study (using ``real spherical harmonics,'' spherical harmonics with a Hartley rather than Fourier basis for the azimuthal dependence), was prepared as an invited review for the Bayesian Statistics 5 conference in Valencia in June 1994. It was printed in the preliminary conference proceedings distributed to participants, but was too long for the published proceedings. A revised version was requested by Statistical Science, but at the time I was too occupied by other projects to make the necessary revisions. Available here as a PDF file and as a gzipped PostScript file.

Computational Technology for Bayesian Inference (1999)

A preprint of a small part of my talk on "Bayesian Inference: A Developer's Perspective," given at the 1999 ADASS conference. It provides a nontechnical 10pp survey of some of the current approaches and algorithms for computing the integrals that arise in Bayesian calculations, including asymptotic methods for low-dimensional models (the Laplace approximation), subregion-adaptice quadrature (or cubature) and Monte Carlo integration for modest-dimensional models, and posterior sampling implemented via Markov Chain Monte Carlo (MCMC) for high-dimensional models. Available here as a PDF file and as a gzipped PostScript file.

Unpublished Presentations and Reports

Bayesian Inference With Log-Fourier Arrival Time Models and Event Location Data (1993)

An unpublished technical report from 1993 describing work dating back to 1989 on Bayesian methods for detecting periodic signals in arrival time series (or other 1-D point processes), such as astronomers obtain from observing X-ray and gamma-ray pulsars. This report has a detailed discussion of a Bayesian counterpart to the Rayleigh test, using a log-sinusoid event rate model (i.e., the log of the event rate is a sinusoid, so the event rate density is proportional to a von Mises distribution; some call this an exponentiated sinusoid or exponentiated trigonometric model). It also discusses generalizing the model to include harmonics (a log-Fourier model), and to account for event weighting using an instrument's imaging point spread function. Some of this work was summarized in the unabridged version of Promise, above; it motivated an alternative, piecewise-constant model described in Gregory & Loredo (1992) (this work is described below). The report is available here as a PDF file.

Bayesian Inference: A Practical Primer

Slides (minus two slides illustrating MCMC results from Phillips & Smith 1996 and Sibisi & Skilling 1997) for an hour+ tutorial on Bayesian methods given as the opening talk at the International Workshop on Maximum Entropy and Bayesian Methods, Gif-sur-Yvette, France (2000). Available here as a PDF file and as a gzipped 2-up PostScript file (the latter for more economical printing).

Bayesian Adaptive Exploration

A description of the Bayesian theory of experimental design, with a "toy problem" example worked out showing how to optimally schedule radial velocity observations for estimating the orbital properties of an extrasolar planet. Presented at the Statistical Challenges in Modern Astronomy conference at Penn State in the summer of 2001. Available here as a PDF file.

Bayesian Data Analysis with Applications in Astrophysics: A Survey

Slides from a talk given at the June 2002 CDF/D0 Advanced Analysis Group Workshop at Fermilab. The talk begins with a survey/tutorial on Bayesian inference, and continues with a description of two astrophysical applications: the "on/off" problem (Poisson counting process with uncertain background), and analysis of neutrino data from SN 1987A. (Participants who wanted to see the omitted material will find some of it at the end of the PDF file; see also the Primer and Adaptive Exploration presentations above). Available here as a PDF file and as a gzipped 4-up PostScript file for more economical printing.

Statistical Tools in Python: A NASA AISR Project

Slides from a talk given at the June 2002 CDF/D0 Advanced Analysis Group Workshop at Fermilab. Available here as a PDF file.

The Two-Envelope Paradox

An article written for the "Critical Thinking" column in Cornell's Astronomy Department newsletter, about the famous "two-envelope paradox" or "exchange paradox" (related to the St. Petersburg paradox). The column's editor, Yervant Terzian, posed the paradox in a previous issue and asked readers to resolve it. His version begins:

In a game show there are two closed envelopes containing money. One contains twice as much as the other. You [randomly] choose one envelope and then the host asks you if you wish to change and prefer the other envelope. Should you change? You can take a look and know what your envelope contains.

The article discusses the resolution to the paradox, using some basic algebra and probability theory. It includes a brief review of some of the existing literature on this fascinating puzzle. PDF file

Bayesian Inference in Astronomy & Astrophysics: A Short Course

Five lectures on basic Bayesian inference with applications in astronomy and astrophysics, given by invitation at the Center for Interdisciplinary Plasma Science of the Max Planck Institute for Plasma Physics in Garching, Germany, October 2002. Available here as five PDF files:

Selected Research Papers

Bayesian Adaptive Exploration (TJL, David Chernoff)

We describe a framework for adaptive astronomical exploration based on iterating an Observation--Inference--Design cycle that allows adjustment of hypotheses and observing protocols in response to the results of observation on-the-fly, as data are gathered. The framework uses a unified Bayesian methodology for the inference and design stages: Bayesian inference to quantify what we have learned from the available data; and Bayesian decision theory to identify which new observations would teach us the most. In the design stage, the utility of possible future observations is determined by how much information they are expected to add to current inferences as measured by the (negative) entropies of the probability distributions involved. Such a Bayesian approach to experimental design dates back to the 1970s, but most existing work focuses on linear models. We use a simple nonlinear problem---planning observations to best determine the orbit of an extrasolar planet---to illustrate the approach and demonstrate that it can significantly improve observing efficiency (i.e., reduce uncertainties at a rate faster than the familiar ``root-N'' rule) in some situations. We highlight open issues requiring further research, including dependence on model specification, generalizing the utility of an observation (e.g., to include observing ``costs''), and computational issues. This paper will appear in Statistical Challenges in Modern Astronomy III (in press; expected 2003). Available here as a PDF file.

Type Ia Supernovae, Evolution, and the Cosmological Constant (Persis Drell, TJL, Ira Wasserman)

An analysis of the evidence for a nonzero cosmological constant in the SNe Ia redshift-luminosity data reported by the High-z Supernova Search Team and the Supernova Cosmology Project that gained so much attention in the scientific and popular press in 1998. The original analyses used Bayesian methods to find credible regions for the two density parameters (due to mass and to a possible cosmological constant), but used an incorrect summary of the evidence for a nonzero cosmological constant (a tail probability rather than a Bayes factor or odds ratio). This exaggerated the evidence for a nonzero cosmological constant. More importantly, we study how the strength of the evidence depends on what one assumes about sources of systematic error such as evolution of SNe Ia properties. We find a sensitive dependence; allowing for a small amount of evolution dramatically weakens the evidence, and allowing for a moderate amount leads to no significant preference for models with nonzero cosmological constant. We also use exploratory methods to see if the data themselves indicate evolution may be present. We find suggestive evidence for evolution. Most significantly, we find that the SNe Ia lightcurve analysis methods, used to make them appear more like ``standard candles'' at low redshift, do not decrease the dispersion in the luminosities of distant SNe Ia, indicating that the distant SNe Ia are in some way different from the nearby ones. The text includes an introduction to Bayes factors as Appendix A. Available here as gzipped PostScript files: text and figures. The original data was obtained by two groups:

High Redshift Supernova Search Team (Harvard)
Supernova Cosmology Project (LBL)

Bayesian Harmonic Analysis for Audio Testing and Measurement (TJL; presented at the Audio Engineering Society Convention #111, 2001)

Abstract: Many common audio test and measurement procedures require characterization of the output signal of the device under test in terms of harmonic (sinusoidal) components and residual noise when the device processes sinusoidal imput signals. This work uses the Bayesian approach to statistical inference to address such problems as parameter estimation problems when discrete samples of the output signal are given. In the resulting Bayesian harmonic analysis the power spectrum computed from the discrete-time Furier transform appears as the logarithm of the posterior probability for the frequency of a single sinusoid rather than as an estimate of the signal spectrum; more complicated functions of the transform arise when analyzing signals with multiple sinusoids. Problems such as spectral leakage are addressed by nonlinear processing of the Fourier transform, offering several advantages over methods that use (linear) windowing of data.
Preprint available here as a PDF file.

Bayesian Analysis of the Polarization of Distant Radio Sources: Limits on Cosmological Birefringence (TJL, Eanna Flanagan, Ira Wasserman)

An analysis of the data used in a paper erroneously claiming evidence for a large-scale anisotropy in the universe published in Physical Review Letters in Spring 1997. As even a cursory reading of the original paper made clear, the claimed effect is not present, as is clearly demonstrated by a Bayesian analysis. My personal title for this paper is "The Screwy Statistics of Screwy Light," but I didn't think I'd get away with that in print! To appear in The Physical Review D15, Dec 1997. Available online at the Los Alamos astro-ph archive: http://xxx.lanl.gov/abs/astro-ph /9706258. Some web sites covering this topic include (dead links might be recoverable via the Wayback Machine):

Inferring the Spatial and Energy Distribution of Gamma Ray Burst Sources. II. Isotropic Models (TJL, Ira Wasserman)

This paper and the next are the second and third in a series applying Bayesian methods to the analysis of the distribution of directions and strengths of gamma-ray bursts. The data can be modeled as a multidimensional Poisson point process, but with the "events" comprising the process measured with significant uncertainty. The first paper in the series, deriving the methodology in detail, is available in The Astrophysical Journal Supplement, 96, 261-301 (1995). To appear in The Astrophysical Journal, 1998. Available online at the Los Alamos astro-ph archive: http://xxx.lanl.gov/abs/astro-ph/9701111.

Inferring the Spatial and Energy Distribution of Gamma Ray Burst Sources. III. Anisotropic Models (TJL, Ira Wasserman)

To appear in The Astrophysical Journal, 1998. Available online at the Los Alamos astro-ph archive: http://xxx.lanl.gov/abs/astro-ph/9701112.

A New Method for the Detection of a Periodic Signal of Unknown Shape and Period (Phil Gregory and Tom Loredo)

A Bayesian approach to detection of periodic signals in arrival time series data. This paper documents one of my most rewarding early moments developing Bayesian methods for astronomers. Astronomers had methods for detecting periodic signals that involved folding detected events according to a trial period, and testing the folded distribution for departure from uniformity. Phil Gregory had the clever intuition that entropy might make a good statistic for such a test. He shared this with me just after I'd learned of Bayesian inference with the Dirichlet distribution (from Ed Jaynes's article, ``Monkeys, Kangaroos and N''). So I came up with a Bayesian model designed so that combinatorial entropy would emerge as a kind of sufficient statistic. Basically, if one adopts a uniformly-binned piecewise-constant model for the periodic event rate, many model parameters can be integrated over analytically, allowing easy calculation of marginal distributions for the unknown frequency and signal shape, and straightforward calculation of odds in favor of the presence of a periodic signal; the expressions involve the multiplicity (combinatorial entropy) of the binned event counts. Section 2 includes a brief introduction to Bayesian inference and terminology that may be of more general interest. Section 3 contains a general discussion of the likelihood function for a Poisson point process. The text only is available online at the Washington University Bayesian Reprints site as a gzipped PostScript file or PDF file. For the full paper, including 11 figures, see the published version in The Astrophysical Journal, 398, 146-168 (1992) (ADS link). Although I did nearly all the analytic work and the numerical calculations documented here, Phil took the approach further in several subsequent papers, applying it to several pulsar data sets, and deriving a version of the model for sampled data with additive Gaussian noise.

Tom Loredo's Astro Home Page /