Mismatches between Traditional Optimization Analyses and Modern Deep Learning

Mismatches between Traditional Optimization Analyses and Modern Deep Learning

Mismatches between Traditional Optimization Analyses and Modern Deep Learning

“Today’s post (based upon our paper with Kaifeng Lyu at NeurIPS20) identifies other surprising incompatibilities between normalized nets and traditional analyses. We hope this will change the way you teach and think about deep learning…”

Source: www.offconvex.org/2020/10/21/intrinsicLR/