Medium

Stan Kriventsov

In

Deep Learning Reviews

by

Stan Kriventsov

·

Feb 11, 2021

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity…

Review of paper by William Fedus, Barret Zoph, and Noam Shazeer, Google Brain, 2021.

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity…

In

Deep Learning Reviews

by

Stan Kriventsov

·

Jan 21, 2021

AutoDropout: Learning Dropout Patterns to Regularize Deep Networks (paper review)

Review of paper by Hieu Pham¹ ² and Quoc V. Le¹, ¹Google Research and ²Carnegie Mellon University, 2021.

AutoDropout: Learning Dropout Patterns to Regularize Deep Networks (paper review)

In

Deep Learning Reviews

by

Stan Kriventsov

·

Jan 4, 2021

Point Transformer (paper review)

Review of paper by Hengshuang Zhao¹, Li Jiang², Jiaya Jia², et al, ¹University of Oxford, ²The Chinese University of Hong Kong, 2020.

Point Transformer (paper review)

In

Deep Learning Reviews

by

Stan Kriventsov

·

Dec 15, 2020

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine (paper review)

Review of paper by Pedro Domingos, University of Washington, 2020

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine (paper review)

In

Deep Learning Reviews

by

Stan Kriventsov

·

Dec 7, 2020

Scaling down Deep Learning (paper review)

Review of paper by Sam Greydanus, Oregon State University and the ML Collective, 2020

Scaling *down* Deep Learning (paper review)

In

Deep Learning Reviews

by

Stan Kriventsov

·

Dec 1, 2020

Gradient Starvation: A Learning Proclivity in Neural Networks (paper review)

Review of paper by Mohammad Pezeshki¹ ², Sekou-Oumar Kaba¹ ³, Yoshua Bengio¹ ², et al, ¹Mila, ²Université de Montréal, ³McGill University…

Gradient Starvation: A Learning Proclivity in Neural Networks (paper review)

In

Deep Learning Reviews

by

Stan Kriventsov

·

Nov 24, 2020

AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients (paper review)

By Juntang Zhuang¹, Tommy Tang², Yifan Ding³, et al, ¹Yale University, ²University of Illinois at Urbana-Champaign, and ³University of…

AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients (paper review)

In

Deep Learning Reviews

by

Stan Kriventsov

·

Nov 11, 2020

Attention Augmented Differentiable Forest for Tabular Data (paper review)

Review of paper by Yingshi Chen, Xiamen University, 2020

Attention Augmented Differentiable Forest for Tabular Data (paper review)

In

Deep Learning Reviews

by

Stan Kriventsov

·

Oct 21, 2020

Deep Learning for Symbolic Mathematics (paper review)

Review of paper by Guillaume Lample and François Charton, Facebook AI Research, 2019.

Deep Learning for Symbolic Mathematics (paper review)

In

Deep Learning Reviews

by

Stan Kriventsov

·

Oct 21, 2020

Compounding the Performance Improvements of Assembled Techniques in a Convolutional Neural Network…

Review of paper by Jungkyu Lee, Taeryun Won, and Kiho Hong, Clova Vision, NAVER Corp, 2019

Compounding the Performance Improvements of Assembled Techniques in a Convolutional Neural Network…

Stan Kriventsov

Stan Kriventsov

Software/ML Engineer at Google. Founder of Deep Learning Reviews: https://www.dl.reviews. Former pro chess and poker player.

Following

Help

Status

About

Careers

Press

Blog

Store

Privacy

Rules

Terms

Text to speech