MATH3431: Machine Learning and Neural Networks
Preface
1
Preliminaries
1.1
Data matrices
1.1.1
Sample mean vector
1.1.2
Sample variance matrix
1.1.3
Correlation matrix
1.1.4
Univariate summaries of variability
1.1.5
Linear combinations
1.1.6
Eigen decomposition
1.2
Distances between individuals
1.2.1
Euclidean distance
1.2.2
Pearson distance
1.2.3
Mahalanobis distance
1.2.4
Manhattan distance
1.2.5
Distances for categorical data
1.3
Linear regression
1.3.1
Using different distance metrics
1.3.2
Partitioning of variance
1.4
Linear discriminant analysis
2
Fundamental concepts
2.1
Types of learning
2.2
Model performance
2.2.1
Classification performance
2.2.2
Regression performance
2.2.3
Overfitting
2.2.4
Model selection criteria
2.3
Cross-validation
3
Supervised Learning — Classification
3.1
Historical notes
3.2
Running examples
3.3
\(k\)
-nearest neighbours
3.3.1
Basic algorithm
3.4
Naive Bayes classifier
3.5
Decision trees
3.6
Perceptron
3.7
Balancing classes
4
Supervised Learning — Regression
4.1
Historical notes
4.2
Running example
4.3
Regularised regression
4.3.1
Lasso regression
4.3.2
Ridge regression
4.3.3
Elastic Net regression
4.3.4
Setting the hyperparameters
4.4
Multivariate Adaptive Regression Splines
4.5
\(k\)
-nearest neighbours
4.6
Decision trees
5
Ensemble methods
5.1
Weak and strong learners
5.2
Stacking
5.2.1
Stacking of classifiers
5.3
Bagging
5.3.1
Bootstrapping
5.3.2
Random forests
5.4
Boosting
5.4.1
AdaBoost
5.4.2
XGBoost
5.5
Error-Correcting Output Codes
5.6
Other ensemble methods
6
Model Interpretation
6.1
Variable importance
6.1.1
Decision tree structure
6.1.2
Permutation-based feature importance
6.2
Main effect visualisations
6.2.1
Main effects
6.2.2
Partial dependence plots
6.2.3
Individual conditional expectation plots
6.2.4
Accumulated local effects
6.2.5
Shapley plots
7
Unsupervised learning
7.1
Traditional statistical approaches
7.1.1
Clustering
7.1.2
Dimension reduction
7.2
Modern unsupervised learning techniques
7.2.1
Clustering with Density-Based Spatial Clustering of Applications with Noise (DBSCAN)
7.2.2
Dimensionality Reduction with t-SNE
7.3
Other ideas in unsupervised learning
References
Published with bookdown. Copyright Durham University
MATH3431: Machine learning and neural networks
References