Learning about the world

One of the most impactful moments in my research life was a presentation by G. Hinton showing how an algorithm can 'dream'. I am referring here to his wake-sleep algorithm, where there is some sort of a model of the world learned during the wake phase and during the sleep phase, we just let our imagination free, discovering some novel combinations of previously seen patterns. Why was it so interesting? Because, it showed a promise of something that could lead to creative machines and intelligent machines. Here is an example of a digit '3' generated during the sleep phase (Link to the paper) :

Ever since I have been thinking quite a lot about extending the idea from the MNIST world to more complex environments which involve both spatial and temporal patterns. I like to imagine the process of thinking as driving through some internal mental highway which connects concepts.

Predictive world model - Manifolds in our minds

There has been some work on the idea that our brains indeed create internal representations based on local manifolds. Simply put, we can easily traverse the observation space in some latent internal, possibly low dimentional space. Think of faces and facial features as the dimensions. Here's an illustration (adapted from Saul et al 2003, Link to the paper)

This is a nice illustration adapted from Bengio et al (who have inspired me even further to look into this):

I, myself tried the idea of purely spatial low dimentional manifolds learned in a completely unsupervised way on MNIST and CIFAR-10 datasets). Here are some short clips showing what traversing a 3D internal manifold world look like ( Think of the highway animation ).

This is the tool I wrote for visualizing manifolds - GITHUB

The way these are learned is though a simple Denoising Autoencoder which compresses the high dimentioal input into low-dimentional (here 3D) representation and shows where each point lies in the latent low dimentional space.

It is quite interesting to explore such an internal world. My initial observation is that the manifolds resemble some sort of N-dimentional torus, where there is no beginning and no end, with endless 'intersections'

Here's an illustration of 1,2 and 3D torus, think of a n-d torus as a collection of those

Internal world model for solving complex RL tasks

There has been quite a lot of work recently (i.e. World models ) which seems to support the idea that it would be very useful to build some sort of internal model of the world which could be used to accelerate RL training.

I built a Nintendo Learning Environment ( Link ) to perform research on what kind of representations, algorithms to use to drive progress in meta-learning and trasferring skills from one environment to another.

Here are some of my slides showing the idea in action