Algorithm for Training Neural Nets with Bayesian Linear Regression

Algorithm for Training Neural Nets with Bayesian Linear Regression

Algorithm for Training Neural Nets with Bayesian Linear Regression – Peter Barnett

“Here I propose a method for training neural networks using Bayesian updating. This method has a few advantages:

• Learns from just a single pass of the data
• Because it isn’t using a gradient-based method we don’t need to specific a loss function
• Also don’t need to specify a learning rate
• Should be applicable to online learning

It has the main disadvantage of probably being strictly worse than normal back propagation for training a neural net…”

Source: peterbarnett.org/2021/03/04/bnet_algorithm/