Deep Learning is Not So Mysterious or Different

The paper argues that deep learning's unusual generalisation behaviours—like benign overfitting and double descent—are neither unique nor mysterious, and can be explained using established frameworks such as PAC-Bayes. It proposes that soft inductive biases, which favour simpler solutions without limiting model complexity, offer a unified explanation.

Visit Original Article →