Making machines learn like kids

Description

Kids are amazing learning machines. Even if they have never seen a cat before, you only need to show them one single image of a cat, and they will be able to recognise cats in a wide variety of other situations. Computers, by comparison, don't do nearly as well. Despite the 'Deep Learning' craze that has taken industry and research by storm, most computer algorithms need to be trained using millions of images of cats before they are able to generalise their knowledge to other situations.

The problem of how to make machines learn from just a single example is known as the 'one-shot learning problem'. This remains a tough research problem, as witnessed by the large body of literature on the topic. There is some recent progress, but there is plenty left to be understood and explored.

The goal of the present project is to introduce you to the basic concepts of machine learning and make you understand the underlying mathematics. You will start by learning the basic concepts in machine learning theory, such as 'classifiers', 'dimension reduction', 'auto-encoders' and 'neural networks', both in terms of the mathematical theory and at the level of simple computer programs which you can implement yourself. You will then delve into some more modern developments in the field, analysing one or more recent research papers. There are connections with all sorts of topics in mathematics (e.g. Bayesian statistics) as well as physics (e.g. disordered systems).

Prerequisites

You must have some programming experience in Python, C/C++ or R, e.g. from having taken Programming I, Mathematical Modelling II or a related module in Physics. It helps if you have taken a statistics module, e.g. Statistical Concepts II or Statistical Methods III but this is not an absolute requirement.

Resources and references

• "Human-level concept learning through probabilistic program induction", by Lake, Salakhutdinov and Tenenbaum. This is a quite readable Science article from 2015 giving an overview of one-shot learning and related topics.
• Deep learning can be learned from a large number of resources; see e.g. the "Deep Learning Book" by Goodfellow, Bengio and Courville.
• There are a few freely available sample programs on the net which implement aspects of one-shot learning and are good ways to get familiar with the logic. An example is "An embarrassingly simple approach to one-shot learning".