Astronomy 6523: Signal Modeling, Statistical Inference & Data Mining in Astrophysics

Spring 2017     Tuesday-Thursday 1:25-2:40 pm     105 Space Sciences Building
Jim Cordes     520 Space Sciences Building

Syllabus:   pdf
Assessment Quiz:   pdf

Assignment 1: Due Feb 16 pdf
Assignment 2: Due Mar 21 pdf
pulsar_data (compressed ascii data file for Assignment 2)
Assignment 3: Due Apr 25 pdf
Assignment 3a: Group Bayesian project (birthdays) pdf


Lecture 1: Introduction to the Course (link after lecture given) pdf
1. Either one of: Laws of Probability, Bayes' theorem, and the Central Limit Theorem (Babu) or
Chapters 1-2 from Gregory (Role of probability theory in science; Probability theory as extended logic.)
2. Introduction to astroML: Machine Learning in Astrophysics (VanderPlas et al. )
3. Simple example to illustrate frequentist and Bayesian approaches
Jupyter notebook for a simple frequentist/Bayesian example: Frequentist_Bayesian_Example.ipynb
Lecture 2: Basics of Probability and Processes pdf

Lecture 3: Probability examples, transformations, periodogram statistics. pdf

Linear, Shift-invariant Systems and Fourier Transforms pdf
Scanned notes on PDFs, transformations, exponential variables, DFT example pdf
Power spectrum of sinuosoid with plotting, etc.

Lecture 4: Fourier transform utility examples, periodogram properties pdf

Utility of the shift theorem pdf
Mapping Bins to Frequency pdf

Lecture 5: Power spectra estimation issues, Sampling Theorem, Examples pdf
1. Stochastic process checklist pdf
2. Stochastic processes (detail) pdf
3. Useful processes pdf
4. Correlation functions as a diagostic tool pdf
5. Generating correlated random variables pdf
6. Wave Propagation and Diffraction (background + FFT based simulations) pdf
False positive analysis in Fourier-based ppower spectra ipynb file
Examples of correlation functions ipynb file

Lecture 6: Fourier examples: searching in Fourier space; harmonic summing pdf

Lecture 7: Sideband summing for frequency modulated signals: spin + orbit. pdf

Lecture 8: Stochastic processes, Wiener-Khinchin theorem, additional Fourier examples (wave propagation) pdf
Lecture 9: Advanced Spectral Analysis (data gaps, red-noise processes) pdf
1. "Studies in astronomical time series analysis.
    II - Statistical aspects of spectral analysis of unevenly spaced data," Scargle (1982) ADS link
2. A Bayesian Approch to Spectral Estimation (based on Jaynes) pdf
3. Prewhitening and Cholesky decomposition pdf
4. Spectral leakage pdf

Lecture 10: Advanced Spectral Analysis (CLEAN, missing data) pdf

CLEAN vs other approaches pdf

Lecture 11: Advanced Spectral Analysis, Information and Entropy, ME Spectral Estimator pdf

1. Jupyter notebook on autoregressive (AR) processes ipynb file

Lecture 12: ME Spectral Estimator, AR modeling, Examples pdf

Missing data pdf

1. Jupyter notebook spectral leakage with two sinusoids ipynb file

Lecture 13: Quadratic forms, Gaussian process modeling, Principal Component Analysis pdf

Missing data pdf

1. Tutorial on PCA (J. Schlens, UCSD) pdf
2. PCA matrix algebra and examples pdf
1. (Code description) Quick overview of spectral analysis methods (non-FT methods) pdf
2. Maximum entropy spectral estimates using Burg algorithm ipynb file
3. Gaussian process modeling ipynb file
    Data file needed for notebook (regression_data.npz) Python npz file
4. Principal component analysis (PCA) examples ipynb file

Lecture 14: Principal Component Analysis pdf

Lecture 15-16: Matched filtering: theory and applications pdf

1. Matched filtering principles and examples ipynb file
2. Detection example with MF (ROC curves) ipynb file

Lecture 17: Matched filtering and localization applications: pdf

Lecture 18: Advanced localization methods and modeling I: pdf

Lecture 19: Modeling II: pdf

Lecture 20: Bayesian Model Fitting and MCMC (first pass): pdf

Lecture 21: Estimators, optimal weighting, and Weighted least squares: pdf

Lecture 22: Discussion of PS3 and aliasing of red power-law processes: pdf

Lecture 23: The birthday Bayesian problem; construction of likelihood functions: pdf

1. Birthday data (small set) csv file
2. Birthday data (large set) csv file
3. Python code to read data and do (partial) Bayesian analysis python script
Lecture 24: Nonlinear least squares pdf

Lecture 25: Markov processes, MCMC pdf

Lecture 26: MCMCII, Bayesian spectral estimation, Bayesian inference on a chirped sinusoid, Prewhitening and Sinusoid Detection pdf

Miscellaneous notes:
1. Structure functions and Allan variance pdf
2. Power spectrum of the output of an LSI system pdf
3. Time Averages and Ergodicity: Example and Counterexample pdf
4. Bispectrum: a role for third moments pdf

Articles and Links:
1.    Studies in Astronomical Time Series Analysis. I. Modeling Random Processes in the Time Domain (Scargle 1980)
2.    A Guided Tour of the FFT (Bergland 1969, IEEE)
3.    Spectral Analysis of Signals Stoica & Moses (427 pp)
4.    Bayesian Spectrum Analysis and Parameter Estimation Bretthorst (220 pp)
5.   Monte Carlo Methods (Chapter 29 of Information Theory, Inference, and Learning Algorithms, D. MacKay.)
      (Book and chapters available at
6.   Efficient Monte Carlo Methods (Chapter 30 of Information Theory, Inference, and Learning Algorithms, D. MacKay.)
7.   An Introduction to MCMC for Machine Learning (Andrieu et al. 2003, Machine Learning, 50, 5)
8 .   Genetic Algorithms: Principles of Natural Selection Applied to Computation (Stephanie Forrest, Science 1993, 261, 872)

James M. Cordes