<![CDATA[Stories by Stan Kriventsov on Medium]]> https://medium.com/@stankriventsov?source=rss-a3093c146491------2 https://cdn-images-1.medium.com/fit/c/150/150/1*1-4HX-jHHacPs66uTUHX9Q.jpeg Stories by Stan Kriventsov on Medium https://medium.com/@stankriventsov?source=rss-a3093c146491------2 Medium Sat, 14 Nov 2026 18:27:13 GMT <![CDATA[Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity…]]>

Review of paper by William Fedus, Barret Zoph, and Noam Shazeer, Google Brain, 2021.

Continue reading on Deep Learning Reviews »

]]> https://medium.com/deep-learning-reviews/switch-transformers-scaling-to-trillion-parameter-models-with-simple-and-efficient-sparsity-691f6941b27?source=rss-a3093c146491------2 https://medium.com/p/691f6941b27 Thu, 11 Feb 2021 03:33:20 GMT 2021-02-11T03:33:20.691Z <![CDATA[AutoDropout: Learning Dropout Patterns to Regularize Deep Networks (paper review)]]>

Review of paper by Hieu Pham¹ ² and Quoc V. Le¹, ¹Google Research and ²Carnegie Mellon University, 2021.

Continue reading on Deep Learning Reviews »

]]> https://medium.com/deep-learning-reviews/autodropout-learning-dropout-patterns-to-regularize-deep-networks-paper-review-bcad62cda63e?source=rss-a3093c146491------2 https://medium.com/p/bcad62cda63e Thu, 21 Jan 2021 16:27:59 GMT 2021-01-21T16:32:19.852Z <![CDATA[Point Transformer (paper review)]]>

Review of paper by Hengshuang Zhao¹, Li Jiang², Jiaya Jia², et al, ¹University of Oxford, ²The Chinese University of Hong Kong, 2020.

Continue reading on Deep Learning Reviews »

]]> https://medium.com/deep-learning-reviews/point-transformer-paper-review-caa081cb691?source=rss-a3093c146491------2 https://medium.com/p/caa081cb691 Mon, 04 Jan 2021 03:21:13 GMT 2021-01-11T01:22:44.777Z <![CDATA[Every Model Learned by Gradient Descent Is Approximately a Kernel Machine (paper review)]]>

Review of paper by Pedro Domingos, University of Washington, 2020

Continue reading on Deep Learning Reviews »

]]> https://medium.com/deep-learning-reviews/every-model-learned-by-gradient-descent-is-approximately-a-kernel-machine-paper-review-11bc11716485?source=rss-a3093c146491------2 https://medium.com/p/11bc11716485 Tue, 15 Dec 2020 04:19:40 GMT 2020-12-19T17:25:05.767Z <![CDATA[Scaling *down* Deep Learning (paper review)]]>

Review of paper by Sam Greydanus, Oregon State University and the ML Collective, 2020

Continue reading on Deep Learning Reviews »

]]> https://medium.com/deep-learning-reviews/scaling-down-deep-learning-paper-review-e16771202e42?source=rss-a3093c146491------2 https://medium.com/p/e16771202e42 Mon, 07 Dec 2020 16:03:07 GMT 2020-12-07T16:38:40.620Z <![CDATA[Gradient Starvation: A Learning Proclivity in Neural Networks (paper review)]]>

Review of paper by Mohammad Pezeshki¹ ², Sekou-Oumar Kaba¹ ³, Yoshua Bengio¹ ², et al, ¹Mila, ²Université de Montréal, ³McGill University…

Continue reading on Deep Learning Reviews »

]]> https://medium.com/deep-learning-reviews/gradient-starvation-a-learning-proclivity-in-neural-networks-paper-review-23f269d81a07?source=rss-a3093c146491------2 https://medium.com/p/23f269d81a07 Tue, 01 Dec 2020 04:18:06 GMT 2020-12-10T03:27:01.773Z <![CDATA[AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients (paper review)]]>

By Juntang Zhuang¹, Tommy Tang², Yifan Ding³, et al, ¹Yale University, ²University of Illinois at Urbana-Champaign, and ³University of…

Continue reading on Deep Learning Reviews »

]]> https://medium.com/deep-learning-reviews/adabelief-optimizer-adapting-stepsizes-by-the-belief-in-observed-gradients-paper-review-8e1d1a842725?source=rss-a3093c146491------2 https://medium.com/p/8e1d1a842725 Tue, 24 Nov 2020 18:44:47 GMT 2020-11-24T18:44:47.154Z <![CDATA[Attention Augmented Differentiable Forest for Tabular Data (paper review)]]>

Review of paper by Yingshi Chen, Xiamen University, 2020

Continue reading on Deep Learning Reviews »

]]> https://medium.com/deep-learning-reviews/attention-augmented-differentiable-forest-for-tabular-data-paper-review-cdd78c3d4bfe?source=rss-a3093c146491------2 https://medium.com/p/cdd78c3d4bfe Wed, 11 Nov 2020 15:27:54 GMT 2021-01-04T03:29:26.051Z <![CDATA[Deep Learning for Symbolic Mathematics (paper review)]]>

Review of paper by Guillaume Lample and François Charton, Facebook AI Research, 2019.

Continue reading on Deep Learning Reviews »

]]> https://medium.com/deep-learning-reviews/deep-learning-for-symbolic-mathematics-paper-review-7ad72e8c5a02?source=rss-a3093c146491------2 https://medium.com/p/7ad72e8c5a02 Wed, 21 Oct 2020 22:45:02 GMT 2020-10-22T16:08:24.449Z <![CDATA[Compounding the Performance Improvements of Assembled Techniques in a Convolutional Neural Network…]]>

Review of paper by Jungkyu Lee, Taeryun Won, and Kiho Hong, Clova Vision, NAVER Corp, 2019

Continue reading on Deep Learning Reviews »

]]> https://medium.com/deep-learning-reviews/compounding-the-performance-improvements-of-assembled-techniques-in-a-convolutional-neural-network-dcf6e0477a88?source=rss-a3093c146491------2 https://medium.com/p/dcf6e0477a88 Wed, 21 Oct 2020 22:35:33 GMT 2020-10-21T22:35:33.759Z