InTDS ArchivebyEster Hlav·Feb 15, 2023Kaiming He Initialization in Neural Networks — Math ProofDeriving optimal initial variance of weight matrices in neural network layers with ReLU activation functionA response icon4A response icon4
InTDS ArchivebyEster Hlav·Dec 23, 2022Xavier Glorot Initialization in Neural Networks — Math ProofDetailed derivation for finding optimal initial distributions of weight matrices in deep learning layers with tanh activation functionA response icon4A response icon4
InTDS ArchivebyEster Hlav·Sep 2, 20205 Derivatives to Excel in Your Machine Learning InterviewCalculus behind Machine Learning: Review of Derivatives, Gradient, Jacobian, and HessianA response icon1A response icon1
InTDS ArchivebyEster Hlav·Aug 26, 2020Activation Functions in Deep Learning: From Softmax to Sparsemax — Math ProofComplete mathematical derivation of Sparsemax activation function: Softmax alternative for sparse outputsA response icon3A response icon3