Member-only story
Understanding the Mathematical Concepts behind Langevin Dynamics
Physics (as always) Enriches
In this post I aim to summarize a pretty “old” paper composed by Max Welling and Yee Whye Teh. It presents the concept of Stochastic Gradient Langevin Dynamics (SGLD). A method that nowadays is used increasingly.
My motivation is to present the mathematical concepts that pushed SGLD forward. For those of you who may find it tedious, you can simply take the methodology and ignore the rigorous parts
Zeitgeist
As I mentioned the paper was written about 10 years ago. During that time The world of ML was both similar and different compared to today:
- Similar — Massive amount of data that the web and modern computation provided boosted significantly the ML and many researchers became enthusiastic with it
- Different — DL was not yet common
ML research topics were therefore slightly different:
Researchers considered neural network a promising tool but still just another tool, such as decision trees or Naïve Bayes. Bayesian methods (it is essential for understanding the authors’ motivation), were mandatory in huge amount of academical training.
