Description
For a set of real-valued random variables, we are well familiar with the concept of a probability density function (often short: "pdf" or "density"). The pdf provides a complete characterization of the probabilistic behavior of the system, from which a plethora of further quantities of interest (expectation, variance, modes, ...) could be derived.For practical data sets, the underlying pdf is usually unknown, except for some specialized scenarios involving designed experiments. Hence, there is a strong practical need to be able to estimate probability density functions from real data. The purpose of this project will be to do exactly this; i.e. we will study statistical methods which take a multivariate data set as input, and produce an estimated pdf as output. Relevant techniques include mixture models (a parametric approach, making strong modelling assumptions) and kernel smoothing (a nonparametric approach, making few assumptions apart from smoothness). Some figures exemplifying these two approaches are provided in the right hand panel. We will study the methodology behind such techniques, investigate some of their theory and properties, and apply them on real data sets. This can be considered both as an end in itself (here the sole aim is to compute or visualize the density), or just as an intermediate step to achieve something else, which may include applications such as clustering, classification, or regression.
Prerequisites
- Statistical Concepts II
Resources
- Scott, D. W. (1992) Multivariate density estimation: theory, practice, and visualization. New York; Chichester: Wiley, in library
- Hastie T., Tibhsirani R., and Friedman, J. (2001) The Elements of Statistical Learning. Springer, New York. PDF , Chapter 6, Section 8.5.
- Chacón, J. (2018) Multivariate Kernel Smoothing and its Applications. Boca Raton: CRC Press, Chapman & Hall. Electronic via library
Examples
- Wikipedia: Kernel density estimation
- Brilliant.org: Mixture models
- Fun example: Where's Waldo?
