Generalization Ideas in Deep-Learning

This repository holds the code and report for the Seminar: Optimization and Generalization in Deep Learning held by Thomas Frerix from the Chair of Computer Vision & Artificial Intelligence at Technische Universität München.

Overview

I will explore different measures for the generalization ability of neural nets.

Literature

• Exploring Generalization in Deep Learning
• Generalization in Deep Learning
• Path-SGD: Path-Normalized Optimization inDeep Neural Networks
• Rademacher and Gaussian Complexities: Risk Bounds and Structural Results
• Understanding Deep Learning requires rethinking Generalization
• Norm-Based Capacity Control in Neural Networks

Important links

• Margin
• Norms
• Spectral Radius
• VC-Dim & Capacity