<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mohammed R. Osman on Medium]]></title>
        <description><![CDATA[Stories by Mohammed R. Osman on Medium]]></description>
        <link>https://medium.com/@tiraldj?source=rss-3b89e6336fb5------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*MY32mgx50DLnlj0m.jpg</url>
            <title>Stories by Mohammed R. Osman on Medium</title>
            <link>https://medium.com/@tiraldj?source=rss-3b89e6336fb5------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 15:07:25 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@tiraldj/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Quickie: Avoid the ‘Dummy Variable Trap’]]></title>
            <link>https://blog.roundtableml.com/quickie-avoid-the-dummy-trap-c0664d1524dc?source=rss-3b89e6336fb5------2</link>
            <guid isPermaLink="false">https://medium.com/p/c0664d1524dc</guid>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[one-hot-encoding]]></category>
            <category><![CDATA[dummy-variable]]></category>
            <category><![CDATA[variance-inflation-factor]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Mohammed R. Osman]]></dc:creator>
            <pubDate>Thu, 08 Jun 2023 18:21:22 GMT</pubDate>
            <atom:updated>2023-06-19T15:52:51.323Z</atom:updated>
            <content:encoded><![CDATA[<p>‘Dummy Variable Trap’ is when you use ‘one-hot’ encoding to handle a categorical variable and introduce multicollinearity. Solution: choose one category as the reference and drop it from the DataFrame.</p><h3>Context</h3><p>In many datasets there are features that have categorical variables. To use these features in data science and machine learning, “one-hot” encoding is applied to these features. Meaning they are converted into multiple new columns that are binary, showing the presence or lack of for each category inside the original feature.</p><p>For example: the original feature was something like “Cities” and had values “New York”, “San Francisco”, “Boston”, and “Washington DC”. We could convert each city into a number (“New York” : 1, “Boston” : 2, etc.) but that could make the feature ordinal, meaning New York is less than Boston, etc. which probably doesn’t make sense in most analysis. Instead, we can convert “Cities” into 4 new features, each one signifying the city. So now there’s four columns with binary values (0,1) for “New York”, “San Francisco”, “Boston”, and “Washington DC”. If the row has to do with “New York”, the “New York” column would have the value ‘1’ and the other 3 city columns will have the value ‘0.’ We’d remove the original “Cities” feature out of any further processing.</p><p>The Problem: statistical analysis with this setup will have issues. In the previous example, if one of the city columns has the ‘1’ value, we know that all the other city columns will have ‘0’ values. Also, outside of data points with missing values in the original “Cities” feature, there should not be any rows all zeroes in the 4 city columns. But every (I think!) statistical process will assume that there has to be a case where all of them have a ‘0’ value. These two issues result in a violation of basic statistical assumptions: multicollinearity.</p><h3>What is Multicollinearity &amp; Why’s it a Problem?</h3><p>With the use of ‘one-hot’ encoding, the new features made are linearly correlated with each other to significant level.</p><p>From Wikipedia’s entry on <a href="https://en.wikipedia.org/wiki/Multicollinearity">multicollinearity</a>:</p><blockquote>In <a href="https://en.wikipedia.org/wiki/Statistics">statistics</a>, <strong>multicollinearity</strong> (also <strong>collinearity</strong>) is a phenomenon in which one predictor <a href="https://en.wikipedia.org/wiki/Variable_(mathematics)">variable</a> in a <a href="https://en.wikipedia.org/wiki/Multiple_regression">multiple regression</a> model can be linearly predicted from the others with a substantial degree of accuracy.</blockquote><blockquote>In this situation, the <a href="https://en.wikipedia.org/wiki/Regression_coefficient">coefficient estimates</a> of the multiple regression may change erratically in response to small changes in the model or the data. Multicollinearity does not reduce the predictive power or <a href="https://en.wikipedia.org/wiki/Reliability_(statistics)">reliability</a> of the model as a whole, at least within the sample data set; it only affects calculations regarding <a href="https://en.wikipedia.org/wiki/Dependent_and_independent_variables#Statistics">individual predictors</a>. That is, a multivariable regression model with collinear predictors can indicate how well the entire bundle of predictors predicts the <a href="https://en.wikipedia.org/wiki/Dependent_variable#Statistics">outcome variable</a>, but it may not give valid results about any individual predictor, or about which predictors are redundant with respect to others.</blockquote><blockquote>…</blockquote><blockquote><strong>Multicollinearity</strong> refers to a situation in which <em>more than two</em> explanatory variables in a <a href="https://en.wikipedia.org/wiki/Multiple_regression">multiple regression</a> model are highly linearly related. There is perfect multicollinearity if, for example as in the equation above, the correlation between two independent variables equals 1 or −1. In practice, perfect multicollinearity in a data set is rare. More commonly, the issue of multicollinearity arises when there is an approximate linear relationship among two or more independent variables.</blockquote><blockquote>Mathematically, a set of variables is perfectly multicollinear if there exist one or more exact linear relationships among some of the variables.</blockquote><p>Multicollinearity reduces predictive value of models. In regression models it makes it harder to find statistical significance for independent variables and even small changes within the dataset can change the coefficients. In multi-class classification problems, multicollinearity is a major issue. It ruins the ‘one vs. one’ (OVO) and ‘one vs all’ (OVA) setups necessary for those processes.</p><p>Mainly, it reduces how good the predictions your models make.</p><p>How do we test for multicollinearity? Calculate Variance Inflation Factor (VIF)</p><p>From <a href="https://en.wikipedia.org/wiki/Variance_inflation_factor">Wikipedia</a>:</p><blockquote>In <a href="https://en.wikipedia.org/wiki/Statistics">statistics</a>, the <strong>variance inflation factor</strong> (<strong>VIF</strong>) is the ratio (<a href="https://en.wikipedia.org/wiki/Quotient">quotient</a>) of the variance of estimating some parameter in a model that includes multiple other terms (parameters) by the variance of a model constructed using only one term.<a href="https://en.wikipedia.org/wiki/Variance_inflation_factor#cite_note-1">[1]</a> It quantifies the severity of <a href="https://en.wikipedia.org/wiki/Multicollinearity">multicollinearity</a> in an <a href="https://en.wikipedia.org/wiki/Ordinary_least_squares">ordinary least squares</a> <a href="https://en.wikipedia.org/wiki/Linear_regression">regression</a> analysis. It provides an index that measures how much the <a href="https://en.wikipedia.org/wiki/Variance">variance</a> (the square of the estimate’s <a href="https://en.wikipedia.org/wiki/Standard_deviation">standard deviation</a>) of an estimated regression coefficient is increased because of collinearity. <a href="https://en.wikipedia.org/wiki/Cuthbert_Daniel">Cuthbert Daniel</a> claims to have invented the concept behind the variance inflation factor, but did not come up with the name.<a href="https://en.wikipedia.org/wiki/Variance_inflation_factor#cite_note-2">[2]</a></blockquote><p>The consensus threshold for multicollinearity among variables are VIF values &gt; 10.</p><h3>Solution</h3><p>Drop one of the new columns made by ‘one-hot’ encoding.</p><p>If possible, drop the column that most reflects the ‘base’ or ‘reference’ case for your work. What is the norm in the environment or within the context of the question/problem you are trying to address.</p><p>In the previous city example, I live in New York so would be interested in seeing how the other cities compare to it. So the city feature I’d drop would be “New York.”</p><h3>Illustration</h3><p>This example is using Python. First load Pandas and Statsmodels tools. Statsmodels has the VIF calculator and a tool that creates a constant feature that it requires.</p><pre>import pandas as pd<br><br>#need statsmodels installed: https://www.statsmodels.org/stable/index.html<br><br>from statsmodels.stats.outliers_influence import variance_inflation_factor<br>from statsmodels.tools.tools import add_constant</pre><p>Here’s a toy dataset based off of an example found on stackoverflow.</p><pre>data ={<br>&#39;gender&#39; : [&#39;female&#39;, &#39;male&#39;, &#39;male&#39;, &#39;male&#39;, &#39;male&#39;, &#39;male&#39;,&#39;female&#39;, &#39;male&#39;, &#39;male&#39;, &#39;female&#39;,&#39;female&#39;, &#39;male&#39;, &#39;male&#39;, &#39;male&#39;, &#39;male&#39;, &#39;male&#39;,&#39;female&#39;, &#39;male&#39;, &#39;male&#39;, &#39;female&#39;],<br>&#39;race&#39; : [&#39;white&#39;, &#39;hispanic&#39;, &#39;african&#39;, &#39;asian&#39;, &#39;asian&#39;, &#39;white&#39;, &#39;african&#39;, &#39;hispanic&#39;,&#39;white&#39;, &#39;african&#39;,&#39;white&#39;, &#39;hispanic&#39;, &#39;african&#39;, &#39;asian&#39;, &#39;asian&#39;, &#39;white&#39;, &#39;african&#39;, &#39;hispanic&#39;,&#39;white&#39;, &#39;african&#39;],<br>&#39;age&#39; : [12,15,22,21,29,42,17, 25,14,47,21,36,47,55,43,33,27,73,81,67],<br>&#39;income&#39;: [25,34,42,50,48,39,25,73,86,61,13,16,24,25,27,40,12, 29,16,42]}<br><br>df = pd.DataFrame(data, columns = data.keys())</pre><p>There are two features with categorical values: gender and race. The other two are numerical and there just for illustration’s sake.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/250/1*_7mmbmjKj8RpraojvTAoXw.png" /></figure><p>Here’s how the dummy trap happens: the user goes ahead and uses ‘get_dummies’ Pandas function and leaves it at that.</p><pre>dummy_trap = pd.get_dummies(df, prefix=[&#39;gender&#39;,&#39;race&#39;])</pre><p>An extra step, we made sure to put “gender_” and “race_” in front of the new features to make it easier to identify them by using the ‘prefix’ argument.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/677/1*m-DHBfYRThZmv6nVw_3mnw.png" /><figcaption>Notice that get_dummies() removes the original features</figcaption></figure><p>Let’s do a regular check of the correlation chart to see if there are any obvious problems.</p><pre>dummy_trap.corr()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/1*utAgcuJhhx-eZSlB0qB_FQ.png" /><figcaption>nothing too bad</figcaption></figure><p>Maybe some issues with “race_african” and both sexes? Dataset could be imbalanced with too many of those identifying as African. Nothing problematic enough to stop going. So the unwary researcher keeps going…</p><p>Let’s stop and see what’s actually going on by checking for multicollinearity.</p><p>First generate a constant value in a new DataFrame.</p><pre>X = add_constant(dummy_trap)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/725/1*DUwrDvPACExxMxTQ_YN5dA.png" /><figcaption>‘const’ feature added, it only has one value and is meaningless</figcaption></figure><p>The ‘const’ feature doesn’t really matter because in the actual analysis we will never choose it. But just to prevent any mistakes from people going on autopilot, I made it into a different DataFrame (‘X’) that will not get used again. The value of ‘1’ doesn’t really matter either, only that it’s 1 for all 20 cases.</p><p>Let’s check for multicollinearity.</p><pre>#Check for Multicollinearity using VIF from statsmodels<br><br>vif_data = pd.DataFrame()<br><br>vif_data[&quot;feature&quot;] = dummy_trap.columns<br><br>pd.Series([variance_inflation_factor(X.values, i) <br>               for i in range(X.shape[1])], index=X.columns)</pre><p>Errors immediately pop up. Multicollinearity is so high, we end up dividing by zero.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/607/1*08N8rEKzB8UBfStnShPVcg.png" /><figcaption>Infinity values detected: Completely unusable</figcaption></figure><p>Doing one-hot encoding by itself is not enough. We have to remove columns.</p><p><strong>RECOMMENDED</strong>: Choose which category or categories you want to be the base case. As in, what are we looking at when you have ‘O’ values for all the features of a certain category. This will help make your interpretation of results easier or more logical within the context of the project.</p><pre>avoid_dummy_trap = pd.get_dummies(df, prefix=[&#39;gender&#39;,&#39;race&#39;])<br>avoid_dummy_trap= avoid_dummy_trap.drop(labels=[&#39;gender_male&#39;,&#39;race_white&#39;], axis=1)</pre><p>Here, I choose to make Male and White the base cases by removing ‘gender_male’ and ‘race_white.’ When ‘gender_female’ is zero, that means Male. When ‘race_african’, ‘race_asian’, and ‘race_hispanic’ are zeros, it means White.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/507/1*EBKltyoEffBHADMMKX9K7A.png" /></figure><p><strong>FOR SPEED</strong>: Tell Pandas to drop the first column alphabetically for each category if the dataset is large, there is no context available to make a decision on what should be the reference cases, pipelines are being used or you just want to get things done fast. We do this using the ‘drop_first = True’ argument.</p><pre>fast_avoid_dummy_trap = pd.get_dummies(df, prefix=[&#39;gender&#39;,&#39;race&#39;], <br>              drop_first=True)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*u98M8wq-Rd1O9DWcxhuE1A.png" /><figcaption>‘gender_female’ and ‘race_african’ columns were dropped automatically</figcaption></figure><p>Not necessary but for consistency’s sake in this example here’s the correlation chart.</p><pre>fast_avoid_dummy_trap.corr()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/609/1*-WmufozNoc9PBvCzoyfsqA.png" /><figcaption>Not a single pairing is problematic</figcaption></figure><p>Now check for multicollinearity after dropping columns.</p><pre>vif_data2 = pd.DataFrame()<br>vif_data2[&quot;feature&quot;] = fast_avoid_dummy_trap.columns<br>Z = add_constant(fast_avoid_dummy_trap)<br>pd.Series([variance_inflation_factor(Z.values, i) <br>               for i in range(Z.shape[1])], index=Z.columns)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/232/1*Z1GSX_G4269E9dw-Uypxdw.png" /><figcaption>No more multicollinearity</figcaption></figure><p>Only the meaningless constant had a VIF value above the threshold value of 10. This dataset can be used for statistical analysis and modeling.</p><h3>For more info</h3><p>Learn more about the dummy variable trap: <br><a href="https://www.learndatasci.com/glossary/dummy-variable-trap/">https://www.learndatasci.com/glossary/dummy-variable-trap/</a></p><p>Much deeper discussion on multicollinearity: <a href="https://towardsdatascience.com/everything-you-need-to-know-about-multicollinearity-2f21f082d6dc">Understanding Multicollinearity and How to Detect it in Python</a> by <a href="https://terenceshin.medium.com/">Terence Shin</a> at <a href="https://towardsdatascience.com/">Towards Data Science</a></p><p>Technical info on VIF and how to use it: “<a href="https://online.stat.psu.edu/stat462/node/180/">10.7 — Detecting Multicollinearity Using Variance Inflation Factors</a>” STAT 462: Applied Regression Analysis, PennState Eberly College of Science</p><p>To understand recommended VIF thresholds: <a href="https://quantifyinghealth.com/vif-threshold/#:~:text=Most%20research%20papers%20consider%20a,of%205%20or%20even%202.5.">What is an Acceptable Value for VIF?</a> by George Choueiry.</p><p>More examples using VIF with code over at <a href="https://stackoverflow.com/questions/42658379/variance-inflation-factor-in-python">stackoverflow</a></p><p>For more info on multi-class classification, a great write up “<a href="https://towardsdatascience.com/multi-class-classification-one-vs-all-one-vs-one-94daed32a87b">Multi-class Classification — One-vs-All &amp; One-vs-One</a>” by <a href="https://ameyband.medium.com/?source=post_page-----94daed32a87b--------------------------------">Amey Band</a> at Towards Data Science.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c0664d1524dc" width="1" height="1" alt=""><hr><p><a href="https://blog.roundtableml.com/quickie-avoid-the-dummy-trap-c0664d1524dc">Quickie: Avoid the ‘Dummy Variable Trap’</a> was originally published in <a href="https://blog.roundtableml.com/">RoundtableML</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Study Guide: Deep Learning by Yoshua Bengio, Yann Lecun & Geoffrey Hinton- Nature (2015, 2021)]]></title>
            <link>https://medium.com/@tiraldj/study-guide-deep-learning-by-yoshua-bengio-yann-lecun-geoffrey-hinton-nature-2015-7cbdd5077947?source=rss-3b89e6336fb5------2</link>
            <guid isPermaLink="false">https://medium.com/p/7cbdd5077947</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[study-guide]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Mohammed R. Osman]]></dc:creator>
            <pubDate>Wed, 19 Apr 2023 22:14:50 GMT</pubDate>
            <atom:updated>2023-05-01T13:54:47.527Z</atom:updated>
            <content:encoded><![CDATA[<p>Three Grandfathers of AI review where AI has been and future thoughts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ZwSNNZagDolyN8J0" /><figcaption>Photo by <a href="https://unsplash.com/@miikola?utm_source=medium&amp;utm_medium=referral">Nicolas I.</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>This study guide written for an online machine learning study group “ML Reading Club” from Reddit’s r/machinelearning &amp; r/learnmachinelearning.</p><p>The point of this is not to summarize the paper but explain the important points and provide help for people trying to learn and get deeper on to the topics. There are plenty of summaries available elsewhere, I wanted to help people using a different angle.</p><p>The structure I’m using is an expanded outline. Regular points and explanations in my own words are written regularly. Direct quotes from sources are put into quote blocks directly after links.</p><p>I hope people are better able to learn from these seminal machine learning papers by using these guides. All feedback is appreciated to help make it better.</p><p><a href="https://www.nature.com/articles/nature14539">Deep learning - Nature</a></p><p><a href="https://www.nature.com/"><em>Nature</em></a> volume 521, pages 436–444 (2015)</p><p>This guide is about the classic review paper by Bengio, LeCun, and Hinton about the state of Deep Learning. This paper was published in the journal <em>Nature</em> and is behind a paywall. You can find multiple places to download it anyways with a simple google search. But just to be safe, there will not be any direct quotes from it.</p><p><a href="https://cacm.acm.org/magazines/2021/7/253464-deep-learning-for-ai/fulltext?mobile=false">Deep Learning for AI</a></p><p>By Yoshua Bengio, Yann Lecun, Geoffrey Hinton<br><em>Communications of the ACM</em>, July 2021, Vol. 64 №7, Pages 58–65<br>10.1145/3448250</p><p>In 2021, these authors revisited their review paper and published an update, which is fully open and free to anyone. I will be breaking things down and heavily quoting that paper in the second half of this guide. In my opinion it is a better use of time to focus on their most current thoughts.</p><p>They also did a sit down podcast about their thoughts which is a worthwhile listen:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fplayer.vimeo.com%2Fvideo%2F554817366%3Fh%3D2314613e90%26app_id%3D122963&amp;dntp=1&amp;display_name=Vimeo&amp;url=https%3A%2F%2Fvimeo.com%2F554817366&amp;image=https%3A%2F%2Fi.vimeocdn.com%2Fvideo%2F1161218097-93638bdfed67b88411a51762c6e6bc596459bfa1fa03570322b41637b1de3ffb-d_1280&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=vimeo" width="1920" height="1080" frameborder="0" scrolling="no"><a href="https://medium.com/media/5f26f2ce04d1d3cbaf3f37721432199a/href">https://medium.com/media/5f26f2ce04d1d3cbaf3f37721432199a/href</a></iframe><p>After reviewing the 2021 paper, the guide will end on a little bit of biography on the 3 authors for those interested.</p><h3>Deep Learning (2015)</h3><p>Success of Deep Learning</p><p>Deep Learning requires less domain knowledge, can process more raw data, and improve with more data</p><p>Applied Domains</p><ul><li>Speech Recognition (translation)</li><li>Visual Object Recognition (ImageNet)</li><li>Object Detection (Face, Pedestrian)</li><li>Drug Discovery</li><li>Genomics (predicting effects of mutations)</li><li>Particle Accelerators</li><li>Brain Circuits</li><li>Natural Language (Topic Classification, Sentiment Analysis, Question &amp; Answer, Language Translation)</li></ul><p>“Conventional” Machine Learning Weaknesses</p><ul><li>Processing “natural” or raw data</li><li>Feature selection: time &amp; domain knowledge intensive</li><li>Low-Level Sensing</li><li>Pre-processing</li><li>Feature Extraction</li><li>Inference (Prediction, Recognition)</li></ul><p>Input -&gt; Feature Representation -&gt; Learning Algorithm</p><p>Supervised Learning: Active model making.</p><p>dataset -&gt; labeling -&gt; training (errors, tuning parameters, gradient descent) -&gt; testing</p><p>Gradient Descent</p><ul><li>Imagine prediction errors or ‘error gradient’ as a high dimensional (multiple features) landscape, learning is finding the minimal points or “hill climbing” by adjusting the weights.</li><li>Stochastic gradient descent speeds up the process by randomly picking points in the (error) terrain to do calculations, then averaging the results from multiple runs.</li><li>Great <a href="https://mccormickml.com/2014/03/04/gradient-descent-derivation/">general overview</a> of the math</li></ul><blockquote>Showing input vector, computing outputs and errors, computing average gradient, adjusting weights accordingly. Repeating for many small sets until objective function stop decreasing. Why stochastic: small set gives a noisy estimate of the average gradient over all examples</blockquote><ul><li>Not as good techniques: Linear classifiers or shallow classifiers (must have good features). Input space -&gt; half-spaces; Kernel methods: do not generalize well</li></ul><p>Unsupervised Deep Learning -&gt; Representation Learning</p><ul><li>Raw Data</li><li>Layers of features generated without human involvement</li><li>Automatically discover representations</li><li>Non-linear modules combined to higher/abstract representation</li><li>Learning pushes functions to go from from simple to complex</li></ul><p>Basics of Neural Networks</p><ul><li>Inputs, Outputs (X, y)</li><li>Parameters: weights (w), bias (b)</li><li>Activation Function (phi) (non-linearity)</li></ul><p>Common Activation Functions:</p><ul><li><a href="https://machinelearningmastery.com/rectified-linear-activation-function-for-deep-learning-neural-networks/">Rectified Linear Unit</a> (ReLU) &lt;- Most important</li><li><a href="https://paperswithcode.com/method/elu">Exponential Linear Unit</a> (ELU)</li><li><a href="https://paperswithcode.com/method/leaky-relu">Leaky ReLU</a> (ReLU mod)</li><li><a href="https://paperswithcode.com/method/maxout">Maxout</a> (ReLU/Leaky ReLU mod)</li><li><a href="https://machinelearningmastery.com/a-gentle-introduction-to-sigmoid-function/">Sigmoid</a> &lt;- Classic</li><li><a href="https://paperswithcode.com/method/tanh-activation">tanh</a> (Sigmoid mod)</li><li>Great general <a href="https://towardsdatascience.com/activation-functions-neural-networks-1cbd9f8d91d6">overview</a> on activation functions</li></ul><p>A multi-layer neural network can distort the input space to make the classes of data linearly separable</p><p>Forward &amp; Back Propagation</p><p>Feed-forward neural networks use Forward Propagation: straight shot from inputs to outputs w/o adjustment.</p><ul><li>Strengths: Fast, computationally less demanding, easier math</li><li>Weaknesses: Gets ‘stuck’ in local minima, needs lots of runs</li></ul><p>Back Propagation: After Forward Pass, working backwards and adjusting weights &amp; bias by comparing final output prediction to ground truth. Primary feature of ‘Supervised’ Learning. Goes through all the layers.</p><ul><li>Strengths: Introduces optimization, yields great results</li><li>Weakness: Computationally intensive</li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fjc2IthslyzM%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Djc2IthslyzM&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fjc2IthslyzM%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/318463c6d43447c5c5df08b50ef5e637/href">https://medium.com/media/318463c6d43447c5c5df08b50ef5e637/href</a></iframe><blockquote>Chain Rule: The chain rule of derivatives tells us how two small effects (that of a small change of x on y, and that of y on x) are composed. A small change ∆x in x gets transformed first into a small change ∆y in y by getting multiplied by ∂ y / ∂ x (that is, the definition of partial derivative). Similarly, the change ∆y creates a change ∆z in z. Substituting one equation into the other gives the chain rule of derivatives — how ∆x gets turned into ∆z through multiplication by the product of ∂ y ⁄∂ x and ∂ z⁄∂ x. It also works when x, y and z are vectors (and the derivatives are Jacobian matrices).</blockquote><p>“Practical application” of the chain rule for derivatives.</p><blockquote>Revived around 2006 by unsupervised learning procedures with unlabeled data. In practice, local minima are rarely a problem.</blockquote><p>Fully Connected Neural Net — resource intensive, complex, hard to interpret, but contains all possible connections between adjacent layers.</p><p>Locally Connected Neural Net — ‘Stationarity’ or statistics are similar at different locations. This is how neural networks start looking at data in parts (ex. separate objects in an image file, snippets of audio clip, trends in time series)</p><p>Convolutional Net (CNN, sometimes referred to as ConvNet)— Learning multiple filters. The filtering operation is termed ‘discrete convolution’ , hence the name. Filtering is a bit of a hard concept when thinking of neural networks, try to think of each node or perceptron over multiple layers combining to specialize in a particular idea/symbol/concept. So when the signal from the input goes all the way to the output layer, a certain path takes prominence.</p><p>Convolution: ‘A form or shape that is folded in curved or tortuous windings’ — Webster’s Dictionary</p><p><a href="https://programmathically.com/what-is-pooling-in-a-convolutional-neural-network-cnn-pooling-layers-explained/">Pooling</a>:</p><blockquote>A technique for generalizing features extracted by convolutional filters and helping the network recognize features independent of their location in the image.</blockquote><p><a href="https://livebook.manning.com/concept/deep-learning/feature-map#:~:text=In%20CNNs%2C%20the%20feature%20map,%2C%20edges%2C%20or%20even%20objects.">Feature map</a> (sometimes called ‘Activation Map’)</p><blockquote>In CNNs, the feature map is the output of one filter applied to the previous layer. It is called a feature map because it is a mapping of where a certain kind of feature is found in the image. Convolutional Neural Networks look for “features” such as straight lines, edges, or even objects.</blockquote><p><a href="https://machinelearningmastery.com/pooling-layers-for-convolutional-neural-networks/#:~:text=Pooling%20layers%20provide%20an%20approach,presence%20of%20a%20feature%20respectively.">Pooling Layer</a> (also known as ‘Pooled Feature Maps’)</p><blockquote>Pooling layers provide an approach to down sampling feature maps by summarizing the presence of features in patches of the feature map. Two common pooling methods are average pooling and max pooling that summarize the average presence of a feature and the most activated presence of a feature respectively.</blockquote><p>Classification (sometimes “Recognition”): Predicted labels</p><p>Object Detection = <a href="https://www.einfochips.com/blog/understanding-object-localization-with-deep-learning/">Classification + Localization</a>: Predicted labels and location</p><blockquote>Different Object recognition tasks. Image classification is a task where an image is classified into one or multiple classes based on the task. Image/Object localization is a regression problem where the output is x and y coordinates around the object of interest to draw bounding boxes.</blockquote><p><a href="https://www.v7labs.com/blog/instance-segmentation-guide">Instance Segmentation</a>: Classification + Localization + Boundaries</p><blockquote>a special form of image segmentation that deals with detecting instances of objects and demarcating their boundaries. It finds large-scale applicability in real-world scenarios like self-driving cars, medical imagining, aerial crop monitoring, and more.</blockquote><p><a href="https://www.ibm.com/topics/recurrent-neural-networks">Recurrent Neural Networks</a> (RNN):</p><blockquote>artificial neural network which uses sequential data or time series data. These deep learning algorithms are commonly used for ordinal or temporal problems, such as language translation, natural language processing (nlp), speech recognition, and image captioning</blockquote><p>Data with time &amp; order or sequential data qualities are key to RNNs.</p><p>From a great <a href="https://stanford.edu/~shervine/teaching/cs-230/cheatsheet-recurrent-neural-networks">cheat sheet</a>:</p><blockquote><em>Advantages</em>:</blockquote><blockquote>- Possibility of processing input of any length</blockquote><blockquote>- Model size not increasing with size of input</blockquote><blockquote>- Computation takes into account historical information</blockquote><blockquote>- Weights are shared across time</blockquote><blockquote><em>Disadvantages</em>:</blockquote><blockquote>- Computation being slow</blockquote><blockquote>- Difficulty of accessing information from a long time ago</blockquote><blockquote>- Cannot consider any future input for the current state</blockquote><p>For more details on <a href="https://d2l.ai/chapter_recurrent-neural-networks/rnn.html">math inside RNNs</a>:</p><p>Vectors (Thought, State)</p><ul><li>State or Hidden State contains sequence information up to (but not including) the current time step</li></ul><blockquote>Hidden layers are… layers that are hidden from view on the path from input to output. Hidden states are technically speaking <em>inputs</em> to whatever we do at a given step, and they can only be computed by looking at data at previous time steps.</blockquote><ul><li><a href="https://wiki.pathmind.com/thought-vectors">Thought Vector</a>:</li></ul><blockquote>“Thought vector” is a term popularized by Geoffrey Hinton… which is using <a href="http://www.bloomberg.com/news/articles/2015-10-26/google-turning-its-lucrative-web-search-over-to-ai-machines">vectors based on natural language</a> to improve its search results.</blockquote><blockquote>A thought vector is like a <a href="http://deeplearning4j.org/word2vec.html#embed"><em>word vector</em></a>, which is typically a vector of 300–500 numbers that represent a word. A word vector represents a word’s meaning as it relates to other words (its context) with a single column of numbers.</blockquote><blockquote>That is, the word is embedded in a vector space using a shallow neural network like <a href="http://deeplearning4j.org/word2vec.html">word2vec</a>, which learns to generate the word’s context through repeated guesses.</blockquote><blockquote>A thought vector, therefore, is a vectorized thought, and the vector represents one thought’s relations to others. A thought vector is trained to generate a thought’s context. Just as a words are linked by grammar (a sentence is just a path drawn across words), so thoughts are linked by a chain of reasoning, a logical path of sorts.</blockquote><p><a href="https://d2l.ai/chapter_recurrent-modern/lstm.html">Long Short Term Memory network</a> (LSTM): Improvement on RNN via “memory cell” composed of gates and hidden states. LSTMs are allowed to <em>forget</em>.</p><blockquote>One of the first and most successful techniques for addressing vanishing gradients came in the form of the long short-term memory (LSTM) model due to Hochreiter and Schmidhuber (<a href="https://d2l.ai/chapter_references/zreferences.html#id118">1997</a>). LSTMs resemble standard recurrent neural networks but here each ordinary recurrent node is replaced by a <em>memory cell</em>. Each memory cell contains an <em>internal state</em>, i.e., a node with a self-connected recurrent edge of fixed weight 1, ensuring that the gradient can pass across many time steps without vanishing or exploding.</blockquote><blockquote>The term “long short-term memory” comes from the following intuition. Simple recurrent neural networks have <em>long-term memory</em> in the form of weights. The weights change slowly during training, encoding general knowledge about the data. They also have <em>short-term memory</em> in the form of ephemeral activations, which pass from each node to successive nodes. The LSTM model introduces an intermediate type of storage via the memory cell. A memory cell is a composite unit, built from simpler nodes in a specific connectivity pattern, with the novel inclusion of multiplicative nodes.</blockquote><p>States &amp; Gates:</p><ul><li>Gates: Input, Forget, Output</li><li>States: Gated Hidden State, Memory Cell Internal State</li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FYCzL96nL7j0%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DYCzL96nL7j0&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FYCzL96nL7j0%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/38617d4f71ec3a0e5f57cbbf4f42f4bb/href">https://medium.com/media/38617d4f71ec3a0e5f57cbbf4f42f4bb/href</a></iframe><p>Slightly off topic: if you’re interested in LSTMs check out its next evolution: Bidirectional LSTM (BiLSTM). BiLSTM has an additional LSTM layer that reverses the direction of information flow. So the input flows in both directions, allowing the LSTM to use information from both sides and model words (and grouping of words like phrases) sequential dependencies. Good explainer “<a href="https://medium.com/@raghavaggarwal0089/bi-lstm-bc3d68da8bd0">Bi-LSTM</a>” by <a href="https://medium.com/@raghavaggarwal0089?source=post_page-----bc3d68da8bd0--------------------------------">Raghav Aggarwal</a> on Medium.</p><p>Distributed Representations and Language Processing</p><p>Distributed Representations: Features (or ‘elements’) are not mutually exclusive and the variety of different combinations and configurations could <em>surpass</em> the number of variations found in the training data. This is because learning distributed representations enable generalization to new or novel combinations. Layers of representation in a deep networks creates the potential for exponential performance gains (“exponential in the depth”).</p><blockquote>Memory Networks can answer questions that require complex inference.</blockquote><p>Future of Deep Learning</p><ul><li>Potential of Unsupervised Learning</li><li>CNNs &amp; RNNs + reinforcement learning in computer vision</li><li>NLP</li></ul><h3>Deep Learning for AI (2021)</h3><blockquote>Research on artificial neural networks was motivated by the observation that human intelligence emerges from highly parallel networks of relatively simple, non-linear neurons that learn by adjusting the strengths of their connections.</blockquote><blockquote>This observation leads to a central computational question: How is it possible for networks of this general kind to learn the complicated internal representations that are required for difficult tasks such as recognizing objects or understanding language?</blockquote><p>Neural network research is heavily inspired by biological functions. It’s hilarious that even the legends in the field are in awe of how brains work and even more amazed at how well artificial neural networks perform in the real world.</p><blockquote>Deep learning seeks to answer this question by using many layers of activity vectors as representations and learning the connection strengths that give rise to these vectors by following the stochastic gradient of an objective function that measures how well the network is performing.</blockquote><blockquote>It is very surprising that such a conceptually simple approach has proved to be so effective when applied to large training sets using huge amounts of computation and it appears that a key ingredient is depth: shallow networks simply do not work as well.</blockquote><p>Complexity through multiple layers of nodes leads to learning. Here “layers of activity vectors” aka the hidden layers in neural networks create emergent “representations.” Representations here is a catchall word for abstract ideas, symbols, similarity/dissimilarity, etc. As a neural network processes inputs through all the layers, the layers start to ‘filter’ them into silos, ending in the output layer. The more layers or deeper a neural network is, the more likely it will accurately understand what the input data has. This is how Deep Learning works.</p><blockquote>These challenges include learning with little or no external supervision, coping with test examples that come from a different distribution than the training examples, and using the deep learning approach for tasks that humans solve by using a deliberate sequence of steps which we attend to consciously — tasks that Kahneman calls <em>system 2</em> tasks as opposed to <em>system 1</em> tasks like object recognition or immediate natural language understanding, which generally feel effortless.</blockquote><p>The authors here are starting to bring up the differences between Supervised vs Unsupervised machine learning. Unsupervised machine learning is able to handle ‘raw’ or ‘natural’ data much better than Supervised. Supervised requires much more active participation by the users as well.</p><blockquote>Put simply, the logic-inspired paradigm views sequential reasoning as the essence of intelligence and aims to implement reasoning in computers using hand-designed rules of inference that operate on hand-designed symbolic expressions that formalize knowledge.</blockquote><blockquote>The brain-inspired paradigm views learning representations from data as the essence of intelligence and aims to implement learning by hand-designing or evolving rules for modifying the connection strengths in simulated networks of artificial neurons.</blockquote><blockquote>In the logic-inspired paradigm, a symbol has no meaningful internal structure: Its meaning resides in its relationships to other symbols which can be represented by a set of symbolic expressions or by a relational graph.</blockquote><p>Inspired by our own biology, this is how researchers first started out designing and using machine learning. It’s much more logical (and intuitive) to our human minds to handcraft AIs. But it didn’t perform nearly as well as hoped. Research in neural networks stagnated a bit because it was stuck in the Supervised paradigm. Both leaps in compute power and theory were needed.</p><blockquote>In the brain-inspired paradigm the external symbols that are used for communication are converted into internal vectors of neural activity and these vectors have a rich similarity structure. Activity vectors can be used to model the structure inherent in a set of symbol strings by learning appropriate activity vectors for each symbol and learning non-linear transformations that allow the activity vectors that correspond to missing elements of a symbol string to be filled in.</blockquote><blockquote>The main advantage of using vectors of neural activity to represent concepts and weight matrices to capture relationships between concepts is that this leads to automatic generalization. If Tuesday and Thursday are represented by very similar vectors, they will have very similar causal effects on other vectors of neural activity. This facilitates analogical reasoning and suggests that immediate, intuitive analogical reasoning is our primary mode of reasoning, with logical sequential reasoning being a much later development…</blockquote><p>The hardest part of this passage is “activity vectors.” Let’s go back to the physics definition of vectors: a value that describes magnitude and direction. In the machine learning context, feature vectors represent an object’s numeric or symbolic characteristics (basically features). I interpret activity vectors, as written by the authors here, as a grouping of feature vectors during learning. The AI is learning concepts, symbols and groupings of them, which allows it to identify &amp; include similar objects in the absence of explicit labeling in the training data or missing information.</p><blockquote>Deep learning re-energized neural network research in the early 2000s by introducing a few elements which made it easy to train deeper networks. The emergence of GPUs and the availability of large datasets were key enablers of deep learning and they were greatly enhanced by the development of open source, flexible software platforms with automatic differentiation such as Theano Torch, Caffe, TensorFlow, and PyTorch. This made it easy to train complicated deep nets and to reuse the latest models and their building blocks.</blockquote><p>More compute, larger datasets, and advancements in software platforms were a big help.</p><blockquote>But the composition of more layers is what allowed more complex non-linearities and achieved surprisingly good results in perception tasks</blockquote><p>More hidden layers gives more opportunities for the neural network to form ‘’complex non-linearities’ and correctly filter the input.</p><blockquote>It is important to realize that it is not simply a question of having more parameters, since deep networks often generalize better than shallow networks with the same number of parameters. The practice confirms this. The most popular class of convolutional net architecture for computer vision is the ResNet family<a href="https://cacm.acm.org/magazines/2021/7/253464-deep-learning-for-ai/fulltext?mobile=false#R43">43</a> of which the most common representative, ResNet-50 has 50 layers.</blockquote><p>Very interesting take, parameters (think number of nodes or perceptrons in a layer) is less important than the number of layers used. Depth is the key quality.</p><blockquote>Other ingredients not mentioned in this article but which turned out to be very useful include image deformations, drop-out, and batch normalization.</blockquote><p>Image Deformation: from <a href="https://en.wikipedia.org/wiki/Free-form_deformation">Wikipedia</a>, “In computer graphics, free-form deformation (FFD) is a geometric technique used to model simple deformations of rigid objects. It is based on the idea of enclosing an object within a cube or another hull object, and transforming the object within the hull as the hull is deformed.”</p><p><a href="https://towardsdatascience.com/regularization-dropout-in-deep-learning-5198c2bf6107">Drop-out Regularization</a>: A method used to prevent over-fitting by the neural network. Randomly selected neurons are ignored during training, temporally on the forward pass and weight changes are skipped during backward propagation. This prevents overspecialization and forces the neural network to be more robust by having multiple pathways for recognition of the same object. Hopefully this increases the generalizability of the network.</p><p><a href="https://arxiv.org/abs/1502.03167">Batch Normalization</a>: Standardization or Normalization of inputs between layers to increase training speeds. The compute time required to pre-process the signals is dwarfed by the speed gains when the number of layers is high.</p><blockquote>We believe that deep networks excel because they exploit a particular form of compositionality in which features in one layer are combined in many different ways to create more abstract features in the next layer.</blockquote><blockquote>For tasks like perception, this kind of compositionality works very well and there is strong evidence that it is used by biological perceptual systems.</blockquote><p>From <a href="https://ai.stanford.edu/~jingweij/cicv/">Compositionality in Computer Vision</a>:</p><blockquote>People understand the world as a sum of its parts. Events are composed of other actions, objects can be broken down into pieces, and this sentence is composed of a series of words. When presented with new concepts, people can decompose the novelty into familiar parts. Our knowledge representation is naturally compositional. Unfortunately, many of the underlying architectures that catalyze vision tasks generate representations that are not compositional…compositionality in computer vision — — the notion that the representation of the whole should be composed of the representation of its parts. As humans, our perception is intertwined greatly by reasoning through composition: we understand a scene by components, a 3D shape by parts, an activity by events, etc.</blockquote><p>What the authors in the paper are saying is that computer vision by deep networks are able to breakdown inputs (pictures, scenes, etc) into smaller components to understand both what it contains inside and the visual data as a whole. Again, the multitude of layers in the network allow it to form symbols and representations that it can filter for.</p><blockquote><strong>Unsupervised pre-training.</strong> When the number of labeled training examples is small compared with the complexity of the neural network required to perform the task, it makes sense to start by using some other source of information to create layers of feature detectors and then to fine-tune these feature detectors using the limited supply of labels.</blockquote><p>This is a really interesting comment snuck in before talking about transfer learning. Essentially, the authors are thinking that Transfer Learning is the key to deal with situations where labels are too few or labeling is too onerous (cost, time-wise). IMO this recommendation is not conventional thinking, but I could be wrong.</p><blockquote>In transfer learning, the source of information is another supervised learning task that has plentiful labels. But it is also possible to create layers of feature detectors without using any labels at all by stacking auto-encoders.</blockquote><blockquote>First, we learn a layer of feature detectors whose activities allow us to reconstruct the input.</blockquote><blockquote>Then we learn a second layer of feature detectors whose activities allow us to reconstruct the activities of the first layer of feature detectors.</blockquote><blockquote>After learning several hidden layers in this way, we then try to predict the label from the activities in the last hidden layer [breaking this sentence into two -MO, pt. 1]</blockquote><blockquote>and we backpropagate the errors through all of the layers in order to fine-tune the feature detectors that were initially discovered without using the precious information in the labels. [pt. 2]</blockquote><p>So what’s going on here?:</p><blockquote>The pre-training may well extract all sorts of structure that is irrelevant to the final classification but, <strong>in the regime where computation is cheap and labeled data is expensive</strong>, this is fine so long as the pre-training transforms the input into a representation that makes classification easier. [bold used for emphasis -MO]</blockquote><blockquote>In addition to improving generalization, unsupervised pre-training initializes the weights in such a way that it is easy to fine-tune a deep neural network with back propagation.</blockquote><p>So transfer learning is dependent on re-optimizing the neural network originally trained on something else for a new use case, hoping the symbols in the original have some relevance to the new training data. Otherwise more compute time is necessary to do more iterations of optimization using back propagation.</p><blockquote>The effect of pre-training on <em>optimization</em> was historically important for overcoming the accepted wisdom that deep nets were hard to train, but it is much less relevant now that people use rectified linear units and residual connections.</blockquote><p>Rectified linear units (ReLU) was already covered earlier. <a href="https://paperswithcode.com/method/residual-connection">Residual connections</a> are defined as:</p><blockquote>Residual Connections are a type of skip-connection that learn residual functions with reference to the layer inputs, instead of learning unreferenced functions.</blockquote><blockquote>The intuition is that it is easier to optimize the residual mapping than to optimize the original, unreferenced mapping. To the extreme, if an identity mapping were optimal, it would be easier to push the residual to zero than to fit an identity mapping by a stack of nonlinear layers.</blockquote><p>This figure from the residual connections paper helps in understanding it better:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6HDuqhUzP92iXhHoS0Wl3w.png" /><figcaption>Clipped from Resnet documentation</figcaption></figure><p>So going top to bottom, it looks like a regular neural network with connections from input through 2 hidden layers. Except now we got this ‘x identity’ that carries information that skips internal transformation by the layers (and thus not altered), this is the residual connection.</p><blockquote>However, the effect of pre-training on <em>generalization</em> has proved to be very important. It makes it possible to train very large models by leveraging large quantities of unlabeled data, for example, in natural language processing, for which huge corpora are available.</blockquote><blockquote>The general principle of pre-training and fine-tuning has turned out to be an important tool in the deep learning toolbox, for example, when it comes to transfer learning or even as an ingredient of modern meta-learning.</blockquote><p>This is confusing to me because the original model used for transfer learning was supposed to have been made using data sets with labeling but here the authors are saying the original model used for transfer learning can be unsupervised too. So…maybe trial and error is necessary to see if it works? Hope the back propagation works its magic?</p><blockquote><strong>Unsupervised and self-supervised learning.</strong> Supervised learning, while successful in a wide variety of tasks, typically requires a large amount of human-labeled data. Similarly, when reinforcement learning is based only on rewards, it requires a very large number of interactions. These learning methods tend to produce task-specific, specialized systems that are often brittle outside of the narrow domain they have been trained on.</blockquote><p>Very critical considerations. So how do the authors advise overcoming them?</p><blockquote>Reducing the number of human-labeled samples or interactions with the world that are required to learn a task and increasing the out-of-domain robustness is of crucial importance for applications such as low-resource language translation, medical image analysis, autonomous driving, and content filtering.</blockquote><blockquote>Humans and animals seem to be able to learn massive amounts of background knowledge about the world, largely by observation, in a task-independent manner. This knowledge underpins common sense and allows humans to learn complex tasks, such as driving, with just a few hours of practice. A key question for the future of AI is how do humans learn so much from observation alone?</blockquote><blockquote>In supervised learning, a label for one of <em>N</em> categories conveys, on average, at most <em>log</em>2(<em>N</em>) bits of information about the world. In model-free reinforcement learning, a reward similarly conveys only a few bits of information. In contrast, audio, images and video are high-bandwidth modalities that implicitly convey large amounts of information about the structure of the world. This motivates a form of prediction or reconstruction called self-supervised learning which is training to “fill in the blanks” by predicting masked or corrupted portions of the data. Self-supervised learning has been very successful for training transformers to extract vectors that capture the context-dependent meaning of a word or word fragment and these vectors work very well for downstream tasks.</blockquote><blockquote>For text, the transformer is trained to predict missing words from a discrete set of possibilities. But in high-dimensional continuous domains such as video, the set of plausible continuations of a particular video segment is large and complex and representing the distribution of plausible continuations properly is essentially an unsolved problem.</blockquote><p>This is the first time in either paper transformers were brought up. In the seminal paper <a href="https://arxiv.org/abs/1706.03762">“Attention is All You Need”</a>, transformer models are:</p><blockquote>The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/350/1*NpUwn4uIX6nGhGhrn2duNA.png" /><figcaption>from “Attention is All You Need” by Vaswani et al.</figcaption></figure><p>From <a href="https://blogs.nvidia.com/blog/2022/03/25/what-is-a-transformer-model/">Nvidia’s official blog</a>:</p><blockquote>A transformer model is a neural network that learns context and thus meaning by tracking relationships in sequential data like the words in this sentence.</blockquote><blockquote>Transformer models apply an evolving set of mathematical techniques, called attention or self-attention, to detect subtle ways even distant data elements in a series influence and depend on each other.</blockquote><p>Truely paradigm shifting, next level work. To fully describe transformers would require a whole post on its own. But these two links are a good start. More info available in ‘Dive into Deep Learning’ Chapter 11 (Section 7) <a href="https://d2l.ai/chapter_attention-mechanisms-and-transformers/transformer.html">Transformer Architecture</a></p><blockquote><strong>The mysterious success of rectified linear units.</strong> The early successes of deep networks involved unsupervised pre-training of layers of units that used the logistic sigmoid nonlinearity or the closely related hyperbolic tangent. Rectified linear units had long been hypothesized in neuroscience and already used in some variants of RBMs and convolutional neural networks. It was an unexpected and pleasant surprise to discover that rectifying non-linearities (now called ReLUs, with many modern variants) made it easy to train deep networks by backprop and stochastic gradient descent, without the need for layerwise pre-training.</blockquote><p>That list of activation functions is more important than you think!</p><blockquote><strong>Soft attention and the transformer architecture.</strong> A significant development in deep learning, especially when it comes to sequential processing, is the use of multiplicative interactions, particularly in the form of soft attention.</blockquote><blockquote>This is a transformative addition to the neural net toolbox, in that it <strong>[Transformers] changes neural nets from purely vector transformation machines into architectures which can dynamically choose which inputs they operate on, and can store information in differentiable associative memories. </strong>[bold for emphasis -MO]</blockquote><p>Now the neural network not just has memory like the RNNs, but can be selective about the what inputs go through the layers and how.</p><blockquote>A key property of such architectures is that they can effectively operate on different kinds of data structures including sets and graphs.</blockquote><p>So greater flexibility in data types, hallmark of Unsupervised, but with additional flexibility in terms of order.</p><blockquote>Soft attention can be used by modules in a layer to dynamically select which vectors from the previous layer they will combine to compute their outputs. This can serve to make the output independent of the order in which the inputs are presented (treating them as a set) or to use relationships between different inputs (treating them as a graph).</blockquote><p>Soft and Hard attention are concepts in computer vision that are on the harder end and explaining them would be too involved for this post. I’ll post some links to the topic area. From the Institute of Physics “<a href="https://iopscience.iop.org/article/10.1088/1742-6596/1693/1/012173/pdf">An Overview of the Attention Mechanisms in Computer Vision</a>”, the Jonathan Hui blog post “<a href="https://jhui.github.io/2017/03/15/Soft-and-hard-attention/">Soft &amp; hard attention</a>,” and “<a href="https://medium.com/heuritech/attention-mechanism-5aba9a2d4727">Attention mechanism</a>” by <a href="https://medium.com/@heuritech?source=post_page-----5aba9a2d4727--------------------------------">Heuritech</a> on Medium. Very, very simply put: Attention is having the neural network look at specific areas of the image to classify certain things that the AI expects to find there, as opposed to looking at the image as a whole. Soft attention is deterministic and Hard attention is stochastic (so an estimate) in terms of gradient descent calculations.</p><blockquote>Another way to think about attention mechanisms is that they make it possible to dynamically route information through appropriately selected modules and combine these modules in potentially novel ways for <strong>improved out-of-distribution generalization</strong>. [bold for emphasis -MO]</blockquote><p>That last part of the excerpt is key: Transformers can be robust enough handle situations the training set didn’t have. The incredible performance of transformers cannot be understated.</p><blockquote>Transformers have produced dramatic performance improvements that have revolutionized natural language processing...</blockquote><blockquote>Perhaps more surprisingly, transformers have been used successfully to solve integral and differential equations symbolically. A very promising recent trend uses transformers on top of convolutional nets for object detection and localization in images with state-of-the-art performance. The transformer performs post-processing and object-based reasoning in a differentiable manner, enabling the system to be trained end-to-end.</blockquote><blockquote><strong>Contrastive learning.</strong> One way to approach this problem is through latent variable models that assign an energy (that is, a badness) to examples of a video and a possible continuation.</blockquote><p>From V7 Lab’s blog post “<a href="https://www.v7labs.com/blog/contrastive-learning-guide#:~:text=Contrastive%20Learning%20is%20a%20technique,a%20data%20class%20from%20another.">The Beginner’s Guide to Contrastive Learning</a>”:</p><blockquote>Contrastive Learning is a Machine Learning paradigm where unlabeled data points are juxtaposed against each other to teach a model which points are similar and which are different. That is, as the name suggests, samples are contrasted against each other, and those belonging to the same distribution are pushed towards each other in the embedding space. In contrast, those belonging to different distributions are pulled against each other…</blockquote><blockquote>Contrastive Learning mimics the way humans learn. For example, we might not know what otters are or what grizzly bears are, but seeing the images… we can at least infer which pictures show the same animals. The basic contrastive learning framework consists of selecting a data sample, called “anchor,” a data point belonging to the same distribution as the anchor, called the “positive” sample, and another data point belonging to a different distribution called the “negative” sample. The SSL model tries to minimize the distance between the anchor and positive samples, i.e., the samples belonging to the same distribution, in the latent space, and at the same time maximize the distance between the anchor and the negative samples.</blockquote><p>So some labeling is required, hence Contrastive learning is sometimes regarded as ‘Semi-supervised’ machine learning. If you see ‘Self-supervised’, that means that the AI is actively labeling the unstructured and or unlabeled raw input data as it flows through the model. The AI needs to have some base truth from which to see the similarities or dissimilarities to of other pictures. Back to the review paper…</p><blockquote>The key difficulty with contrastive learning is to pick good “negative” samples: suitable points <em>Y</em> whose energy will be pushed up. When the set of possible negative examples is not too large, we can just consider them all. This is what a softmax does, so in this case contrastive learning reduces to standard supervised or self- supervised learning over a finite discrete set of symbols. But in a real-valued high-dimensional space, there are far too many ways a vector Ŷ could be different from <em>Y</em> and to improve the model we need to focus on those <em>Y</em>s that should have high energy but currently have low energy. Early methods to pick negative samples were based on Monte-Carlo methods, such as contrastive divergence for restricted Boltzmann machines and noise-contrastive estimation.</blockquote><p>Check out Analytics Step’s post “<a href="https://www.analyticssteps.com/blogs/what-restricted-boltzmann-machine-gibbs-sampling-and-contrastive-divergence">What is a Restricted Boltzmann Machine? Gibbs Sampling and Contrastive Divergence</a>” and KDNuggets’ <a href="https://www.kdnuggets.com/2019/07/introduction-noise-contrastive-estimation.html">“A Gentle Introduction to Noise Contrastive Estimation”</a> for more info on these two examples.</p><p>Ok now we take the principles of transformers and contrastive learning and start cooking with GANs:</p><blockquote>Generative Adversarial Networks (GANs) train a generative neural net to produce contrastive samples by applying a neural network to latent samples from a known distribution (for example, a Gaussian). The generator trains itself to produce outputs Ŷ to which the model gives low energy <em>E</em>(Ŷ). The generator can do so using backpropagation to get the gradient of <em>E</em>(Ŷ) with respect to Ŷ. <strong>The generator and the model are trained simultaneously, with the model attempting to give low energy to training samples, and high energy to generated contrastive samples</strong>. [bold for emphasis -MO]</blockquote><p>Basically the GAN and Model are to separate things that work together, one judges the output of the other with the target images. Original paper on <a href="https://arxiv.org/abs/1406.2661">Generative Adversarial Networks</a> by Goodfellow et al. From Google Developer blog “<a href="https://developers.google.com/machine-learning/gan/gan_structure">Overview of GAN Structure</a>”:</p><blockquote>A generative adversarial network (GAN) has two parts:</blockquote><blockquote>The <strong>generator</strong> learns to generate plausible data. The generated instances become negative training examples for the discriminator.</blockquote><blockquote>The <strong>discriminator</strong> learns to distinguish the generator’s fake data from real data. The discriminator penalizes the generator for producing implausible results.</blockquote><blockquote>When training begins, the generator produces obviously fake data, and the discriminator quickly learns to tell that it’s fake:</blockquote><p>The point of a GAN is that eventually the model learns how to generate images that can fool the Discriminator.</p><p>A fantastic blog post on GANs is “<a href="https://www.freecodecamp.org/news/an-intuitive-introduction-to-generative-adversarial-networks-gans-7a2264a81394">An intuitive introduction to Generative Adversarial Networks (GANs</a>)” by Thalles Silva on freecodecamp.com. The following schematic by Silva is great:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/675/1*ZsAS0-h8_bVOimMFCcl14g.png" /></figure><p>Along with IBM’s explainer video:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FTpMIssRdhco%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DTpMIssRdhco&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FTpMIssRdhco%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/e71cb2466f9c8ba736a821c07c087926/href">https://medium.com/media/e71cb2466f9c8ba736a821c07c087926/href</a></iframe><blockquote>GANs are somewhat tricky to optimize, but adversarial training ideas have proved extremely fertile, producing impressive results in image synthesis, and opening up many new applications in content creation and domain adaptation as well as domain or style transfer.</blockquote><p>Style transfer generates an new image by combining data or content from one image with the style of another. The difficulty lies in training a computer vision model to understand (what humans) consider a style. Having enough images of the specific style in the training set is crucial. Very thorough blog post on the topic by Fritz AI “<a href="https://www.fritz.ai/style-transfer/">Style Transfer Guide</a>.” How to use <a href="https://www.tensorflow.org/tutorials/generative/style_transfer">TensorFlow</a> to ‘neural’ style transfer and paper with code.</p><blockquote><strong>Making representations agree using contrastive learning.</strong> Contrastive learning provides a way to discover good feature vectors without having to reconstruct or generate pixels. The idea is to learn a feed-forward neural network that produces very similar output vectors when given two different crops of the same image or two different views of the same object but dissimilar output vectors for crops from different images or views of different objects.</blockquote><p>Two things are going on here. First, the cropping (not mentioned here but also turning the image by an angle to change its orientation). This does two things, it trains the model to recognize vectors (or features) in the training set images without too much focus on location within the image. Two, it multiplies the number of images in the training data without the risk of overfitting. Warning: this only applies for this and several other computer vision techniques, absolutely not for machine learning or data science in general.</p><p>It’s interesting that the GAN methodology has one feed forward or classical neural network set up. So no cycling or feedback mechanisms involved. I think this is because the separate Discriminator model or module is tasked with giving the feedback within the GAN setup.</p><blockquote>The squared distance between the two output vectors can be treated as an energy, which is pushed down for compatible pairs and pushed up for incompatible pairs.</blockquote><p>Ok this is a strange concept, mainly because we’re using the word ‘energy’ here. This is very much not the typical way ‘energy’ is used, here is one of the authors, Yann LeCun, doing a conference session on them.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FBqgnnrojVBI%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DBqgnnrojVBI&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FBqgnnrojVBI%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/15c70e836f51bf5d6987500b69b0d78a/href">https://medium.com/media/15c70e836f51bf5d6987500b69b0d78a/href</a></iframe><p>From LeCun on the <a href="https://atcold.github.io/pytorch-Deep-Learning/en/week07/07-1/">NYU Center for Data Science’s github</a>:</p><blockquote>Energy-based models observe a set of variables <em>x</em> and output a set of variables <em>y</em>. There are 2 major problems with feed-forward nets:</blockquote><blockquote>What if the inference procedure is a more complex calculation than stacked layers of weighted sums?</blockquote><blockquote>What if there are multiple possible outputs for a single input? Example: Predicting future frames of video. Essentially in a classification net, we train this net to emit a score for each class. However, this is not possible to do in a continuous high dimensional domain like images. (We cannot have softmax over images!). Even if the output is discrete, it could have a large sample space. For example, the text is compositional leading to a huge number of possible combinations. Energy-based models provide a better framework to model these modalities.</blockquote><blockquote><strong>EBM approach</strong></blockquote><blockquote>Instead of trying to classify <em>x</em>’s to <em>y</em>’s, we would like to predict if a certain pair of (<em>x</em>, <em>y</em>) fit together or not. Or in other words, find a <em>y</em> compatible with <em>x</em>. We can also pose the problem as finding a <em>y</em> for which some <em>F</em>(<em>x</em>,<em>y</em>) is low.</blockquote><blockquote><strong>Definition</strong></blockquote><blockquote>We define an energy function <em>F</em>:X×Y→R where <em>F</em>(<em>x</em>,<em>y</em>) describes the level of dependency between (<em>x</em>,<em>y</em>) pairs. (<strong>Note</strong> that this energy is used in inference, <strong>not</strong> in learning.) The inference is given by the following equation:</blockquote><blockquote>ŷ = argminy​{<em>F</em>(<em>x</em>,<em>y</em>)}</blockquote><blockquote><strong>Solution: gradient-based inference</strong></blockquote><blockquote>We would like the energy function to be smooth and differentiable so that we can use it to perform the gradient-based method for inference. In order to perform inference, we search this function using gradient descent to find compatible <em>y</em>’s. There are many alternate methods to gradient methods to obtain the minimum.</blockquote><p>This is hard to understand but LeCun later explains why an ‘energy-based’ model can be better than the standard probabilistic way of doing things:</p><blockquote><strong><em>Question: Can you elaborate on the advantage that energy-based models give? In probability-based models, you can also have latent variables, which can be marginalized over.</em></strong></blockquote><blockquote>The difference is that in probabilistic models, you basically don’t have the choice of the objective function you’re going to minimize, and you have to stay true to the probabilistic framework in the sense that every object you manipulate has to be a normalized distribution (which you may approximate using variational methods, etc.). Here, we’re saying that ultimately what you want to do with these models is make decisions. If you build a system that drives a car, and the system tells you “I need to turn left with probability 0.8 or turn right with probability 0.2”, you’re going to turn left. The fact that the probabilities are 0.2 and 0.8 doesn’t matter — what you want is to make the best decision, because you’re forced to make a decision. So probabilities are useless if you want to make decisions. If you want to combine the output of an automated system with another one (for example, a human, or some other system), and these systems haven’t been trained together, but rather they have been trained separately, then what you want are calibrated scores so that you can combine the scores of the two systems so that you can make a good decision. There is only one way to calibrate scores, and that is to turn them into probabilities. All other ways are either inferior or equivalent. But if you’re going to train a system end-to-end to make decisions, then whatever scoring function you use is fine, as long as it gives the best score to the best decision. Energy-based models give you way more choices in how you handle the model, way more choices of how you train it, and what objective function you use. If you insist your model be probabilistic, you have to use maximum likelihood — you basically have to train your model in such a way that the probability it gives to the data you observed is maximum. The problem is that this can only be proven to work in the case where your model is “correct” — and your model is never “correct”. There’s a quote from a famous statistician [Goerge Box] that says <strong><em>“All models are wrong, but some are useful.”</em></strong> So probabilistic models, particularly those in high-dimensional spaces, and in combinatorial spaces such as text, are all approximate models. They’re all wrong in a way, and if you try to normalize them, you make them more wrong. So you’re better off not normalizing them.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*VRa6DTARwvlof0WW.png" /><figcaption><strong>Fig. 2</strong>: Visualization of the energy function that captures dependency between x and y (LeCun, github)</figcaption></figure><blockquote>This is an energy function that’s meant to capture the dependency between <em>x</em> and <em>y</em>. It’s like a mountain range if you will. The valleys are where the black dots are (these are data points), and there are mountains all around. Now, if you train a probabilistic model with this, imagine that the points are actually on an infinitely thin manifold. So the data distribution for the black dots is actually just a line, and there are three of them. They don’t actually have any width. So if you train a probabilistic model on this, your density model should tell you when you are on this manifold. On this manifold, the density is infinite, and just <em>ε</em> outside of it should be zero. That would be the correct model of this distribution. Not only should the density be infinite, but the integral over [x and y] should be 1. This is very difficult to implement on the computer! Not only that, but it’s also basically impossible. Let’s say you want to compute this function through some sort of neural net — your neural net will have to have infinite weights, and they would need to be calibrated in such a way that the integral of the output of that system over the entire domain is 1. That’s basically impossible. The accurate, correct probabilistic model for this particular data example is impossible. This is what maximum likelihood will want you to produce, and there’s no computer in the world that can compute this. So in fact, it’s not even interesting. Imagine that you had the perfect density model for this example, which is a thin plate in that (x, y) space — you couldn’t do inference! If I give you a value of x, and ask you “what’s the best value of y?” You wouldn’t be able to find it because all values of y except a set of zero-probability have a probability of zero, and there are just a few values that are possible. For these values of x for example:</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*uAu0ueXRc5MmjB1j.png" /><figcaption><strong>Fig. 3</strong>: Example for multiple prediction of EBM as an implicit function (LeCun, github)</figcaption></figure><blockquote>There are 3 values of y that are possible, and they are infinitely narrow. So you wouldn’t be able to find them. There’s no inference algorithm that will allow you to find them. The only way you can find them is if you make your contrast function smooth and differentiable, and then you can start from any point and by gradient descent you can find a good value for y for any value of x. But this is not going to be a good probabilistic model of the distribution if the distribution is of the type I mentioned. So here is a case where insisting to have a good probabilistic model is actually bad. Maximum likelihood sucks [in this case]!</blockquote><blockquote>So if you are a true Bayesian, you say “oh but you can correct this by having a strong prior where the prior says your density function has to be smooth”. You could think of this as a prior. But, everything you do in Bayesian terms — take the logarithm thereof, forget about normalization — you get energy-based models. Energy-based models that have a regulariser, which is additive to your energy function, are completely equivalent to Bayesian models where the likelihood is exponential of the energy, and now you get exp⁡(energy)exp⁡(regulariser)exp(energy)exp(regulariser), and so it’s equal to exp⁡(energy+regulariser)exp(energy+regulariser). And if you remove the exponential you have an energy-based model with an additive regulariser.</blockquote><blockquote><strong>So there is a correspondence between probabilistic and Bayesian methods there, but insisting that you do maximum likelihood is sometimes bad for you, particularly in high-dimensional spaces or combinatorial spaces where your probabilistic model is very wrong. It’s not very wrong in discrete distributions (it’s okay) but in continuous cases, it can be really wrong. And all the models are wrong. </strong>[bold for emphasis- MO]</blockquote><p>Ok back to the review paper:</p><blockquote>A series of recent papers that use convolutional nets for extracting representations that agree have produced promising results in visual feature learning. The positive pairs are composed of different versions of the same image that are distorted through cropping, scaling, rotation, color shift, blurring, and so on.</blockquote><p>All these preprocessing of images in the training set help make the end model more robust (especially to bad data or low quality inputs) and multiplies the size of the training set. Color shift and blurring are examples of more extreme distortions, not always recommended.</p><blockquote>The negative pairs are similarly distorted versions of different images which may be cleverly picked from the dataset through a process called hard negative mining or may simply be all of the distorted versions of other images in a minibatch.</blockquote><p>Hard negative mining is including in the training dataset explicit examples of things you explicitly <strong>don’t want the classifier to identify</strong> amongst the universe of things you’re designing the AI to catch. Common procedure is to generate random bounding boxes in the training images, remove the ones that overlap with actual boxes you want the model to recognize and label the remaining ones in the negative. So now the training data has <em>positive</em> labeled examples of things you want the AI to classify and bound (segment analysis) and <em>negative</em> examples of things you want it to ignore. This helps prevent false positives and improves overall accuracy.</p><p>For further reading, two papers using hard negative mining are: “<a href="https://arxiv.org/abs/2010.04592">Contrastive Learning with Hard Negative Samples</a>” by Joshua Robinson et al., and “<a href="https://ieeexplore.ieee.org/document/9923534/authors#authors">Improved Single Shot Detector with Enhanced Hard Negative Mining Approach</a>” by <a href="https://ieeexplore.ieee.org/author/37087885120">Niranjan Ravi</a> and <a href="https://ieeexplore.ieee.org/author/38182856300">Mohamed El-Sharkawy</a>. Hard negative mining can be used in NLP too, check out<a href="https://medium.com/@dunnzhang0/hard-negative-mining-in-nature-language-processing-how-to-select-negative-examples-in-66f59da316a4"> “Hard Negative Mining in Nature Language Processing (How to Select Negative Examples in Classification and Rank Task)</a>” by <a href="https://medium.com/@dunnzhang0">infgrad</a> on Medium.</p><blockquote>The hidden activity vector of one of the higher-level layers of the network is subsequently used as input to a linear classifier trained in a supervised manner. This Siamese net approach has yielded excellent results on standard image recognition benchmarks. Very recently, two Siamese net approaches have managed to eschew the need for contrastive samples. The first one, dubbed SwAV, quantizes the output of one network to train the other network, the second one, dubbed BYOL, smoothes the weight trajectory of one of the two networks, which is apparently enough to prevent a collapse.</blockquote><p>“Siamese net approach” is to input two different (but usually similar) pictures in to two instances of the same model (both instances of the model have the same exact design and weights &amp; biases) then analyzing the similarity/dissimilarity of the outputs. The point of this procedure is increase the accuracy of classifying objects in the same class and to increase the distance or perceived differences with other classes. Great blog post on it: “<a href="https://towardsdatascience.com/siamese-networks-introduction-and-implementation-2140e3443dee">Siamese Networks Introduction and Implementation</a>” by <a href="https://adityadutt.medium.com/?source=user_profile-------------------------------------">Aditya Dutt</a> on Medium. A good technical paper on the topic for those interested in the approach: “<a href="https://www.cs.cmu.edu/~rsalakhu/papers/oneshot1.pdf">Siamese Neural Networks for One-shot Image Recognition</a>” by Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov from University of Toronto.</p><p>The paper that the review authors referenced to about SwAV is “<a href="https://arxiv.org/abs/2011.10566">Exploring Simple Siamese Representation Learning</a>” by <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Chen%2C+X">Xinlei Chen</a> and <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=He%2C+K">Kaiming He</a> from Facebook AI Research. They put out a video to explain their work:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fk-PcMBYQsOY%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dk-PcMBYQsOY&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fk-PcMBYQsOY%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/5e760b9d33387447a5c57240facb0c21/href">https://medium.com/media/5e760b9d33387447a5c57240facb0c21/href</a></iframe><blockquote><strong>Variational auto-encoders.</strong> A popular recent self-supervised learning method is the Variational Auto-Encoder (VAE). This consists of an encoder network that maps the image into a latent code space and a decoder network that generates an image from a latent code. The VAE limits the information capacity of the latent code by adding Gaussian noise to the output of the encoder before it is passed to the decoder. This is akin to packing small noisy spheres into a larger sphere of minimum radius. The information capacity is limited by how many noisy spheres fit inside the containing sphere. The noisy spheres repel each other because a good reconstruction error requires a small overlap between codes that correspond to different samples. Mathematically, the system minimizes a free energy obtained through marginalization of the latent code over the noise distribution. However, minimizing this free energy with respect to the parameters is intractable, and one has to rely on variational approximation methods from statistical physics that minimize an upper bound of the free energy.</blockquote><p>At the moment I do not feel comfortable in putting all this into my own words so will link to people who do. The original paper cited by the authors on this topic is “<a href="https://arxiv.org/abs/1312.6114">Auto-Encoding Variational Bayes</a>” by <a href="https://arxiv.org/search/stat?searchtype=author&amp;query=Kingma%2C+D+P">Diederik P Kingma</a> and <a href="https://arxiv.org/search/stat?searchtype=author&amp;query=Welling%2C+M">Max Welling</a>. In Towards Data Science, <a href="https://medium.com/@joseph.rocca?source=post_page-----f70510919f73--------------------------------">Joseph Rocca</a> and <a href="https://medium.com/@baptiste.rocca">Baptiste Rocca</a> wrote a very popular Medium blog post on the topic: “<a href="https://towardsdatascience.com/understanding-variational-autoencoders-vaes-f70510919f73">Understanding Variational Autoencoders (VAEs)</a>.” Jaan Altosaar also wrote a great piece on VAE for both deep learning and graph models in “<a href="https://jaan.io/what-is-variational-autoencoder-vae-tutorial/">Tutorial — What is a variational autoencoder?</a>” Nice video on VAE:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F9zKuYvjFFS8%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D9zKuYvjFFS8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F9zKuYvjFFS8%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/96f7a858ebed359bfc6c58c7a7591d88/href">https://medium.com/media/96f7a858ebed359bfc6c58c7a7591d88/href</a></iframe><p>The rest of the 2021 review paper had to do with the three authors’ views on the future of Machine Learning and the challenges it must tackle. There isn’t too much to comment on as the writing is clear. Not much technical material for me to explain either. It’s very much the thoughts of these pioneers for the upcoming future, so I will separate their last portion of the paper into relevant sections and keep commentary to a minimum.</p><blockquote>The Future of Deep Learning</blockquote><blockquote>The performance of deep learning systems can often be dramatically improved by simply scaling them up. With a lot more data and a lot more computation, they generally work a lot better. The language model GPT-3 with 175 billion parameters (which is still tiny compared with the number of synapses in the human brain) generates noticeably better text than GPT-2 with only 1.5 billion parameters. The chatbots Meena and BlenderBot also keep improving as they get bigger. Enormous effort is now going into scaling up and it will improve existing systems a lot, <strong>but there are fundamental deficiencies of current deep learning that cannot be overcome by scaling alone,</strong> as discussed here. [bold for emphasis — MO]</blockquote><p>3 of the biggest challenges:</p><blockquote>Comparing human learning abilities with current AI suggests several directions for improvement:</blockquote><blockquote>Supervised learning requires too much labeled data and model-free reinforcement learning requires far too many trials. Humans seem to be able to generalize well with far less experience.</blockquote><blockquote>Current systems are not as robust to changes in distribution as humans, who can quickly adapt to such changes with very few examples.</blockquote><blockquote>Current deep learning is most successful at perception tasks and generally what are called system 1 tasks. Using deep learning for system 2 tasks that require a deliberate sequence of steps is an exciting area that is still in its infancy.</blockquote><blockquote><strong>What needs to be improved.</strong> From the early days, theoreticians of machine learning have focused on the iid assumption, which states that the test cases are expected to come from the same distribution as the training examples. Unfortunately, this is not a realistic assumption in the real world: just consider the non-stationarities due to actions of various agents changing the world, or the gradually expanding mental horizon of a learning agent which always has more to learn and discover. As a practical consequence, the performance of today’s best AI systems tends to take a hit when they go from the lab to the field.</blockquote><blockquote>Our desire to achieve greater robustness when confronted with changes in distribution (called out-of-distribution generalization) is a special case of the more general objective of reducing sample complexity (the number of examples needed to generalize well) when faced with a new task — as in transfer learning and lifelong learning— or simply with a change in distribution or in the relationship between states of the world and rewards.</blockquote><blockquote>Current supervised learning systems require many more examples than humans (when having to learn a new task) and the situation is even worse for model-free reinforcement learning since each rewarded trial provides less information about the task than each labeled example. It has already been noted that humans can generalize in a way that is different and more powerful than ordinary iid generalization: we can correctly interpret novel combinations of existing concepts, even if those combinations are extremely unlikely under our training distribution, so long as they respect high-level syntactic and semantic patterns we have already learned. Recent studies help us clarify how different neural net architectures fare in terms of this systematic generalization ability. How can we design future machine learning systems with these abilities to generalize better or adapt faster out-of-distribution?</blockquote><blockquote><strong>From homogeneous layers to groups of neurons that represent entities.</strong> Evidence from neuroscience suggests that groups of nearby neurons (forming what is called a hyper-column) are tightly connected and might represent a kind of higher-level vector-valued unit able to send not just a scalar quantity but rather a set of coordinated values. This idea is at the heart of the capsules architectures, and it is also inherent in the use of soft-attention mechanisms, where each element in the set is associated with a vector, from which one can read a key vector and a value vector (and sometimes also a query vector).</blockquote><blockquote>One way to think about these vector-level units is as representing the detection of an object along with its attributes (like pose information, in capsules). Recent papers in computer vision are exploring extensions of convolutional neural networks in which the top level of the hierarchy represents a set of candidate objects detected in the input image, and operations on these candidates is performed with transformer-like architectures.</blockquote><blockquote>Neural networks that assign intrinsic frames of reference to objects and their parts and recognize objects by using the geometric relationships between parts should be far less vulnerable to directed adversarial attacks, which rely on the large difference between the information used by people and that used by neural nets to recognize objects.</blockquote><blockquote><strong>Multiple time scales of adaption.</strong> Most neural nets only have two timescales: the weights adapt slowly over many examples and the activities adapt rapidly changing with each new input. Adding an overlay of rapidly adapting and rapidly, decaying “fast weights” introduces interesting new computational abilities.</blockquote><blockquote>In particular, it creates a high-capacity, short-term memory, which allows a neural net to perform true recursion in which the same neurons can be reused in a recursive call because their activity vector in the higher-level call can be reconstructed later using the information in the fast weights. Multiple time scales of adaption also arise in learning to learn, or meta-learning.</blockquote><blockquote><strong>Higher-level cognition.</strong> When thinking about a new challenge, such as driving in a city with unusual traffic rules, or even imagining driving a vehicle on the moon, we can take advantage of pieces of knowledge and generic skills we have already mastered and recombine them dynamically in new ways. This form of systematic generalization allows humans to generalize fairly well in contexts that are very unlikely under their training distribution. We can then further improve with practice, fine-tuning and compiling these new skills so they do not need conscious attention anymore. How could we endow neural networks with the ability to adapt quickly to new settings by mostly reusing already known pieces of knowledge, thus avoiding interference with known skills? Initial steps in that direction include Transformers and Recurrent Independent Mechanisms.</blockquote><blockquote>It seems that our implicit (system 1) processing abilities allow us to guess potentially good or dangerous futures, when planning or reasoning. This raises the question of how system 1 networks could guide search and planning at the higher (system 2) level, maybe in the spirit of the value functions which guide Monte-Carlo tree search for AlphaGo.</blockquote><blockquote>Machine learning research relies on inductive biases or priors in order to encourage learning in directions which are compatible with some assumptions about the world. The nature of system 2 processing and cognitive neuroscience theories for them suggests several such inductive biases and architectures, which may be exploited to design novel deep learning systems. How do we design deep learning architectures and training frameworks which incorporate such inductive biases?</blockquote><blockquote>The ability of young children to perform causal discovery suggests this may be a basic property of the human brain, and recent work suggests that optimizing out-of-distribution generalization under interventional changes can be used to train neural networks to discover causal dependencies or causal variables. How should we structure and train neural nets so they can capture these underlying causal properties of the world?</blockquote><blockquote>How are the directions suggested by these open questions related to the symbolic AI research program from the 20th century? Clearly, this symbolic AI program aimed at achieving system 2 abilities, such as reasoning, being able to factorize knowledge into pieces which can easily recombined in a sequence of computational steps, and being able to manipulate abstract variables, types, and instances. We would like to design neural networks which can do all these things while working with real-valued vectors so as to preserve the strengths of deep learning which include efficient large-scale learning using differentiable computation and gradient-based adaptation, grounding of high-level concepts in low-level perception and action, handling uncertain data, and using distributed representations.</blockquote><p>Thats it, that’s the end of the two review papers by three of the greatest pioneers in Machine Learning. For those who want to know more about the authors themselves, the final section here will describe their careers quickly and link to interviews/presentations they’ve done.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FUTfQwTuri8Y%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DUTfQwTuri8Y&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FUTfQwTuri8Y%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/15f2beac6c5ee8737e1a2a70edce2301/href">https://medium.com/media/15f2beac6c5ee8737e1a2a70edce2301/href</a></iframe><p><a href="https://twitter.com/geoffreyhinton">Geoffrey Hinton</a></p><p>Geoffrey Hinton is an Emeritus Distinguished Professor at the University of Toronto and was a Google Brain researcher</p><p>Best known for <a href="https://www.analyticsvidhya.com/blog/2018/04/introductory-guide-understand-how-anns-conceptualize-new-ideas/?utm_source=blog&amp;utm_medium=heroes-of-machine-learning-experts-researchers">artificial neural networks (ANNs)</a>. Often called the “Godfather of Deep Learning.” His research on the back propagation algorithm significantly improved of deep learning model performance. Other research works are Boltzmann machines and <a href="https://www.analyticsvidhya.com/blog/2018/04/essentials-of-deep-learning-getting-to-know-capsulenets/?utm_source=blog&amp;utm_medium=heroes-of-machine-learning-experts-researchers">Capsule neural networks</a>.</p><p>edit: most recently Hinton quit Google and wrote about how concerned he is about the direction LLM Ai is headed:</p><p><a href="https://www.nytimes.com/2023/05/01/technology/ai-google-chatbot-engineer-quits-hinton.html">&#39;The Godfather of A.I.&#39; Leaves Google and Warns of Danger Ahead (Published 2023)</a></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FpnTLZQhFpaE%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DpnTLZQhFpaE&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FpnTLZQhFpaE%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/ea635956e62f8c779a4b23b699525195/href">https://medium.com/media/ea635956e62f8c779a4b23b699525195/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FuawLjkSI7Mo%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DuawLjkSI7Mo&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FuawLjkSI7Mo%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/5d6cff31f1c8e385f6f15903118423bc/href">https://medium.com/media/5d6cff31f1c8e385f6f15903118423bc/href</a></iframe><p><a href="https://www.linkedin.com/in/yoshuabengio/?originalSubdomain=in">Yoshua Bengio</a></p><p>Yoshua Bengio is a professor at the Department of Computer Science and Operations Research at the Université de Montréal. He is also the co-founder of Element AI</p><p>Yoshua is known for his work on artificial neural networks and deep learning in the 1980s and 1990s. He co-founded the ICLR conference with Yann LeCun. He is one of the most-cited computer scientists in the areas of Deep Learning, RNN, probabilistic learning, and NLP.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F0tEhw5t6rhc%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D0tEhw5t6rhc&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F0tEhw5t6rhc%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/1032caa6510f97283bd77cde524d079b/href">https://medium.com/media/1032caa6510f97283bd77cde524d079b/href</a></iframe><p><a href="https://twitter.com/ylecun">Yann LeCun</a></p><p>Currently the Chief AI Scientist and VP at Facebook. Founding father of <a href="https://www.analyticsvidhya.com/blog/2017/06/architecture-of-convolutional-neural-networks-simplified-demystified/https://www.analyticsvidhya.com/blog/2018/12/guide-convolutional-neural-network-cnn/?utm_source=blog&amp;utm_medium=heroes-of-machine-learning-experts-researchers">convolutional nets</a>. Made convolutional neural networks work with backpropagation. Titan.</p><p>Thanks for reading! Hope this helped.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7cbdd5077947" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Killers Outta the Gate ‘22: Reviewing Offensive Linemen’s First Year]]></title>
            <link>https://medium.com/@tiraldj/killers-outta-the-gate-22-reviewing-offensive-linemans-first-year-9fcfbe9578?source=rss-3b89e6336fb5------2</link>
            <guid isPermaLink="false">https://medium.com/p/9fcfbe9578</guid>
            <category><![CDATA[american-football]]></category>
            <category><![CDATA[offensive-line]]></category>
            <category><![CDATA[nfl-draft]]></category>
            <category><![CDATA[nfl]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Mohammed R. Osman]]></dc:creator>
            <pubDate>Tue, 11 Apr 2023 20:38:37 GMT</pubDate>
            <atom:updated>2023-04-12T21:13:45.930Z</atom:updated>
            <content:encoded><![CDATA[<p>Simplified look into Offensive Linemen picked in 2022 NFL Draft to see who popped positively &amp; negatively in their first year in the league.</p><p>TLDR: The best tackles to show up on the field right away: Ikem Ekwonu; Charles Cross, and Tyler Smith; Guards: Zion Johnson; Center: Tyler Linderbaum. Amazing finds by NFL teams later on in the draft at Tackle: Braxton Jones, Abraham Lucas, and Jamaree Salyer; Guard: Dylan Parham, Cordell Volson, and Spencer Burford; Center: Luke Fortner.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*cvs4TyH5_NQ4DKrM" /><figcaption>Photo by <a href="https://unsplash.com/@hi_i_am_steph?utm_source=medium&amp;utm_medium=referral">Stephan Coudassot</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Table of Contents</h3><p>(skip to the Offensive Tackles, Guards, &amp; Centers for the good stuff!)</p><p><a href="#d839">Evaluating the First Year of Offensive Linemen</a></p><p><a href="#0bb2">Methodology</a></p><p><a href="#e18d">Definitions and Formulas</a></p><p><a href="#5182">Team Context</a></p><p><a href="#8312">Offensive Tackles</a></p><p><a href="#7933">Offensive Guards</a></p><p><a href="#f561">Centers</a></p><p><a href="#a3d9">Concluding Remarks</a></p><h3>Evaluating the First Year of Offensive Linemen</h3><p>Let’s get this out of the way: Evaluating NFL players is hard, evaluating NFL offensive linemen is extremely hard. The line produces very few positive statistics to show a player doing something well. The majority of offensive linemen stats produced are negative. Most of which are focused on the passing game (pressures, hurries, touches, sacks, pass knockdowns, etc.). Making the situation even worse is that the NFL does not formally assign negative statistics (other than penalties) to individual offensive linemen.</p><p>The logic behind this is that assigning blame on individual offensive linemen is too subjective. It’s very clear when a Defensive End beats a Offensive Tackle and gets the sack, but most cases it isn’t. Defensive linemen and Linebackers do multiple techniques to confuse the offensive line (like stunts, switches, stacking, etc.) Defensive coaches call plays to put numbers in the box in their favor, bring down secondary to blitz, ignore one side of the offensive line to focus on the other (presumably weaker) side, and so many other techniques.</p><p>It cannot be discounted how much impact the players behind the offensive line impact the statistics. Quarterbacks that scramble or get out of the pocket often move towards defensive players or allow them to get off blocks easily to make plays. Conversely, quarterbacks that have fast reactions and throw speeds can throw the ball out before the defense can bring them down. Ultra-talented running backs can make an offensive line look like they’re playing much better than they really are through sheer talent or by making the defense play more reactive and less aggressive. The opposite holds true if the running backs do not scare the defense, defensive coordinators will ignore the run game and focus completely on getting to the Quarterback. All of these factor into how the defense approaches the offensive line, and how the line performs blocking.</p><p>Finally, college has been alleged to under-train offensive linemen for the NFL due to the prevalence of Air Raid, RPO, and systems where the QB is in shotgun almost 100% of the time in college. Offensive linemen drafted into the NFL are expected to take several years of development, both physical and mental, to take on starting roles. The offensive line has to work as a total unit to deal with the defense to confront all these challenges, and therefore the NFL officially puts out negative statistics at the team level. Independent sports journalists do grade and assign these stats to players but do not publish them openly.</p><p>Despite these conditions, I want to look at the players that are contributing to their teams right away. The best way of doing this, with limited dependence on subjective player grades and rankings, is to look at snaps played. The more snaps an offensive lineman plays, the greater his contributions to the team are. Even if they’re playing badly, the coaching staff concluded that having the player on the field was gave the team it’s best chance to win. If a rookie is playing large number of snaps straight out of the gate, it says volumes about their potential to have great careers in the NFL. To see how they played, I used two player evaluation sites, and only their open data, to supplement the snap counts. This may be argued as a gross over simplification of judging players but due to all the difficulties and constraints outlined, this was the best solution I came up with.</p><h3>Methodology</h3><p>To do this project I pulled the vast majority of my data from <a href="https://www.pro-football-reference.com/">Pro Football Reference</a> website. Along with the Pro Football Reference grades for every player, I used the overall seasonal grades from <a href="https://www.pff.com/">PFF</a> (or Pro Football Focus). Most of the data for offensive linemen required putting several data tables together and not much else.</p><p>While Pro Football Reference has a very open data policy, PFF is much more restrictive with theirs, the vast majority of their data is behind a private paywall and distributing it without permission is frowned upon. Even getting their publicly available season grades data requires going through hoops and personal data input. I made the decision to use PFF grades for offensive linemen because there are so few places that grade them individually, instead of a team by team basis). PFF, generally speaking, is the leading purveyor of player grades and info on the NFL. Out of caution, I decided to ‘mask’ the two variables used from them, the grade and NFL positional rank. From PFF, individual player’s sacks allowed was also used and left unmasked. This way we can still get the insights from their grades but without violating anyone’s copyright.</p><p>For offensive line grades at the team level, I used PFF and Sharp Football Analysis.</p><p>I’m more than obsessed with quantifying athleticism when trying to predict hitting on NFL draft picks, to do that finalized values from the <a href="https://ras.football/">Relative Athletic Score</a> database were used. RAS scores are an attempt to condense several athletic test scores and physical measurements, then compare them to NFL players historically.</p><p>In most of the data tables presented, the order of the players is by Offense Snap Rate or the percentage of offensive snaps the lineman played.</p><h3>Definitions and Formulas</h3><p><strong>Round</strong>: The round of the NFL Draft the player was picked in. From 1 to 7, the earlier the round, the greater the expectations. Finding players that can contribute in the lower rounds is a big credit to the NFL team’s scouting team and General Manager. Teams that successfully draft in the first round have the ability to extend the player on a 5th year extension that is very team friendly, further increasing the pressure to hit in this round. Rounds 6 and 7 are considered picking players you didn’t want to compete for in the Undrafted Free Agent (UDFA) market and have zero expectations to make the active roster by regular season cut downs.</p><p><strong>Pick</strong>: Exact order when the player is picked. Used as a measure of ‘cost.’ Low numbered draft picks are more intensive than high numbered draft picks in terms of resources given to each NFL team every year.</p><p><strong>Player</strong>: Player name. Sometimes there were names that were repeated that required removing by hand.</p><p><strong>Games</strong>: The number of regular season NFL games played out of 17. Playoff games are not covered. Baring unforeseen injuries, the more games played the more the player contributed to the team.</p><p><strong>Games Started</strong>: Number of games where the offensive lineman had a starting role. Offensive linemen are expected to start and finish complete games, so this is a very important indicator of player contribution. I was not able to find Games Started within the official NFL datasets available and even the primary Pro-Football Reference datasets were oddly missing this category. Late in writing the post I randomly found that their dataset on penalties did have this statistic but it had missing data values for many players (indicated by ‘9999’ dummy variable).</p><p><strong>Total Snaps</strong>: The number of snaps played, on Offense, Defense, and Special Teams. The project focused on snaps on Offense and Special Teams. The basic assumption is that the more a player played, the greater value he brought to the team.</p><p><strong>Offense Snaps</strong>: Number of plays played on Offense, the primary focus for Offensive Lineman.</p><p><strong>Offense Snap Rate</strong>: Offense Snaps / Total Offensive Plays over the Season. Gives percentage of plays where the player played. As a general rule of thumb, 80% or over shows the player was a starter on the team. ~100% means the rookie player was a Day 1 starter throughout the entire season, the most optimal condition. Players with Offensive Snap Rates from 30–60% are considered to be substantial contributors, either playing heavy reps or played starting roles for the team throughout the season.</p><p><strong>wAV</strong>: From Pro Football Reference, “Weighted Career Approximate Value” is a value from 0–10 that gives a numerical value for how much the site felt the player contributed to the team over the season. This reflects all three phases of the game (offense, defense, and special teams). The higher the value, the better the player did for his team. Players with low scores most likely played few to zero snaps for their teams. Those that scored highly both played a ton of snaps and played well.</p><p><strong>PFF Positional Rank Quotient</strong>: Fancy name for masking PFF data I made for this project. Simply speaking, if a player played what PFF considered ‘significant’ number of snaps for the team, he is ranked amongst those in his positions. I took this rank number, divided by the total number of players officially in the position, then subtracted this number from 1. This should result in % of the NFL this player is better than at their position, according to PFF. If the player didn’t play enough plays to be ranked by PFF, a standard missing value dummy variable ‘-9998&#39; is used.</p><p><strong>PFF Position Grades Normalized</strong>: Is what it says on the tin to mask the PFF grades. I took all the PFF overall season grades for the rookies and normalized them using the min-max approach to re-scale the grades to [0,1] by first subtracting the minimum from the grade, then dividing by the range. So take these values as only relational to other rookies at their position, not a generalize-able grade to the NFL overall. I was more interested in how players performed in relation to each other in this draft than a objective performance grade.</p><p>Let’s use PFF Positional Rank Quotient to compare the players to the rest of the league and PFF Position Grades Normalized to judge distances between players. The first should give us a real sense of where the player is performance-wise straight out of the gate. The second will be used to ascertain how much value the rookie brought vs. their draft pick position. The expectation is that If the the normalized grade between two players is small, but the difference in pick is large, than the player picked later gave more value to their team because they ‘cost’ less.</p><p>‘<strong>Pen</strong>’: Short for Penalty, the number of penalties a player committed over the season. Offensive Line penalties have a horrible tendency to shut down scoring drives or put the offense in very sub-optimal situation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/552/1*dDlvI4b-a3-D8iPU5cZ_6g.png" /><figcaption>Histogram of penalties done by Offensive Linemen in the NFL in 2022</figcaption></figure><p>The histogram of penalties assigned to players throughout the 2022 season showed that most offensive linemen (with high snap counts) committed somewhere between 6–8 penalties over the season. Players who had over 8 will be assumed to have a particular issue with penalties.</p><p><strong>Holding</strong>: Holding penalties committed over the season. In my opinion, this penalty should be looked at on its own because there are many scenarios (like preventing your QB from getting killed) where holding is the smart play.</p><p><strong>False Start</strong>: Penalty where the Offensive Lineman move before the ball is snapped to the QB by the Center. One of the worst unforced errors in football.</p><p>‘<strong>Decl/Offs</strong>’: The penalty on the offensive lineman was recorded in the logbook but was declined by the opposing team’s coach. Worth noting to see just how many penalties were committed that actually hurt the team.</p><p><strong>Sacks</strong>: Number of times the QB was tackled behind the line of scrimmage, with the player being assigned the blame by PFF. This number should not be considered ground truth, but instead regarded as judgement call made by the scouts at PFF. The only sack numbers I would completely trust are the team wide sack figures published by the NFL. Assigning sacks to offensive linemen is a far trickier to do than most stats. Quarterbacks could cause sacks to happen by holding the ball too long, scrambling towards defensive linemen or at angles the offensive linemen couldn’t maintain blocks. Or they could’ve just not thrown the ball away when all the recievers were covered (usually called ‘coverage sacks’). Running backs or Full backs could’ve also missed their blocks behind the offensive linemen, in which case both are said to be at fault. Within this analysis I thought it was important to atleast acknowledge this statistic. Pressures, hurries, and QB hits allowed are all stats that are behind paywalls at PFF so were left unused.</p><p><strong>Sacks per Snap</strong>: Problematic variable I generated by dividing PFF’s Sack numbers by Pro-Football Reference’s snaps on offense. I thought it was important to try and contextualize the sacks number to convey how often the player is blamed for the QB getting tackled. The reason this is problematic is that I couldn’t find a breakdown of offensive snaps by run and pass plays. Only the combination of both was was freely available online. This issue lowers just how much insight we can get from this variable since a run heavy offenses will lead to lower values and vice versa. But I decided to include it as a broad attempt to account for how much the offensive linemen played in games.</p><p><strong>RAS</strong> / <strong>AllTime RAS</strong>: Short for Relative Athletic Score. A score out from 1–10, with 10 being the absolute rarest and &lt; 4 usually meaning incomplete or half hearted testing. Some top end eligible players decide that testing would risk decreasing their draft status and don’t do enough testing for a RAS score to be made. These players were given a dummy variable. RAS score calculation appears to me to put more weight to physical attributes (height, wingspan, hand size etc.). Due to college Pro Days that happen after the combine occurring all over the calendar, the finalized RAS scores happen close to the Draft. The database is broken down to RAS score, which is a score made with the latest testing, and ‘All Time RAS’ score which was the best score previous to the latest testing. In my experience there is often very little difference between RAS and All Time RAS except in cases where the player was injured during the NFL Combine.</p><h3>Team Context</h3><p>Before moving on to individual players. let’s look at how offensive lines did at the team level using three publicly available sources. Grades from PFF as published by <a href="https://seahawkswire.usatoday.com/2023/01/12/seahawks-offensive-line-rankings-week-18-pff/">USA Today</a>. Sharp Football Analysis which had <a href="https://www.sharpfootballanalysis.com/analysis/2022-nfl-offensive-line-rankings-by-positional-unit/">grading</a> using a ranked ballot voting system for their scouts and short breakdowns of their reasoning. The last service used was Football Outsiders (FO) which is more like PFF in that they take a very mathematical approach and they posted <a href="https://www.footballoutsiders.com/stats/nfl/basic-offensive-line">“2022 Basic Offensive Line Stats.”</a> FO provided ranking for the offensive lines overall, and more specific ranking for Power Run (North South run plays), Open Field (sideline to sideline run plays), and Pass only ranking. To summarize all this information, I averaged the overall rankings.</p><p>Honestly these team wide ranking/grading is more important than individual grades but that’s no fun…and my conclusion based off of these three sports stats &amp; evaluation services?<strong> Even grading offensive lines is too damn subjective.</strong> Both PFF and Sharp made these grades prior to the playoffs, and Football Outsiders published this after the season was over, but as far as I know didn’t include playoff statistics. Yet the average ranking showed the three having both Super Bowl teams, the Philadelphia Eagles and Kansas City Chiefs as the top two offensive lines. Despite this making intuitive sense, it’s still amazing to see happen.</p><p>But after these two teams, the dissimilarity between the three lists is alarming. Teams that made the playoffs did tend to be up top (the top 5th?) which made sense. But teams were all over the place. For example, check out the Atlanta Falcons who had a nightmare season get ranked as #5 offensive line in the league by PFF and Football Outsiders but 31 by Sharp. Or the Cleveland Browns (ranked #1 by Sharp, #6 by PFF, and #14 by FO) and Baltimore Ravens (ranked #2 by PFF, #6 by Sharp, and #21 by FO) both having a horrible season and no one can agree on how their offensive line played. There wasn’t consensus even in the bottom of the table. Profootball Focus, Sharp, and Football Outsiders couldn’t even agree on which offensive lines performed badly. Of the very bottom of the table, Houston Texans, New York Jets, and Seattle Seahawks, you can argue that the three rankings weren’t in agreement.</p><p>When scanning the Football Outsider rankings based on the run and pass games separately, the overall rankings make even less sense. The Seahawks had the best open field running, and all three looked middling to OK in passing, power running, and open field. The top ranked Philadelphia Eagles and Kansas City Chiefs both were ranked middling in the open field but the Eagles were good with their power running and the Chiefs did great with pass blocking. Tampa Bay being #1 at pass blocking makes sense because they have Tom Brady there to bail out the line. But <em>the Detroit Lions and Jacksonville Jaguars </em>as #2 and #3 in pass blocking?! Detroit was 5th best oline overall, wow. I could go on and on about how crazy these rankings seem to me but all it proves is that evaluating offensive line play is extremely difficult, subjective, and hard to use statistics on. Evaluating individual players must be even more difficult.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/886/1*0ex-EbEPNT33S6LACrClcg.png" /><figcaption>*PFF = Pro Football Focus, Sharp = Sharp Analytics, FO = Football Outsiders. These are the OL rankings by team</figcaption></figure><p>Since the NFL is a passing league and protecting the QB is utmost importance, it’s worth keeping in mind how often each team’s offensive line allowed sacks to happen. “Sack yards” is the distance lost through sacks, just as a rule of thumb if this number is low most likely the QB was taken down in the pocket or very close to the line. If this number is high, most likely the team is running an offensive system with a scrambling QB. Chiefs, no surprise, were close to the top of the list. But the Philadelphia Eagles and Minnesota Vikings being at the bottom was head scratching. Broncos, Colts and Bears all had nightmare seasons and it’s no surprise they let their Quarterback get sacked the most. The LA Rams being the third worst when it comes to sacks, and bottom third in the overall rankings showed just how badly their team dropped off from Super Bowl strength.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/702/1*19-aCn_MZnkzqK9gpqc6Sg.png" /><figcaption>I still can’t get over how well the Lions and Jaguars keep popping up.</figcaption></figure><h3>Offensive Tackles</h3><p>The 2022 NFL Draft was <strong>AMAZING</strong> for teams that took Offensive Tackles.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/739/1*IIBy6rbFdf03GUt2tNljSw.png" /><figcaption>2022 produced 7 starters at OT, with another 3 players playing more than half the snaps</figcaption></figure><p>Of the 14 Offensive Tackles picked in the draft, four were starters for their team on Day 1. Two more players were one game away from starting every game. Three more players were significant contributors throughout the season. That’s 9 out of 14 (64%) offensive tackles picked showing up for their teams on their first season. Another 4 players could be reasonably expected to come back from injury and develop into starting caliber players. This level of success at the position is definitely not the norm.</p><p>Let’s look at Offensive Tackles in detail, starting with the four players that started every game of the season. Regardless of how they were graded, to do so in their first season is a testament to their future potential in the league. All four held the blind side, the premium position of the line.</p><p><a href="https://www.panthers.com/team/players-roster/ickey-ekwonu/">Ikem Ekwonu (OT): Bio, News, Stats &amp; more</a></p><p>Ikem Ekwonu played every single snap on offense for the Carolina Panthers. The highest Offensive Tackle picked in the 2022 Draft, Ekwenu took over at the left tackle position. Despite rookie year struggles, you can’t ask more from him. <a href="https://youtu.be/_vFp0aWiWFo">Highlights</a></p><p><a href="https://www.chicagobears.com/team/players-roster/braxton-jones/">Braxton Jones (OT): Bio, News, Stats &amp; more</a></p><p>Braxton Jones, in my opinion, is in the running for the greatest steal in the 2022 Draft. The Chicago Bears used a 5th round pick (168) on Jones and got themselves a Day 1 starter who played every single snap on offense. That’s absolutely incredible. Braxton Jones played his first season on the left side and has the play flexibility to swing over to the right as well. Possibly Jones was able to start because the offensive line talent on the team was wanting, as shown by the overall grades for Chicago being in the bottom third of the league. But for any team to get a starter in the 3rd day of the Draft round is a tremendous coup. <a href="https://youtu.be/Pv29inMmNek">Highlights</a>.</p><p><a href="https://www.seahawks.com/team/players-roster/charles-cross/">Charles Cross (OT): Bio, News, Stats &amp; more</a></p><p>Charles Cross almost played every snap on offense for the Seahawks at left tackle. Picked 2 players after Ikem Ekwenu in the 1st round, the expectations were very high for Cross but the season was a rough one for the offensive line and the player. Both were graded very poorly. Despite this, there is an <a href="https://www.si.com/nfl/seahawks/news/seattle-seahawks-charles-cross-rookie-offense-pete-carroll-geno-smith-kenneth-walker-iii">argument</a> that underneath the surface, Cross actually played well but was put under difficult circumstances. <a href="https://www.youtube.com/watch?v=QAm_AMRT994">Highlights</a>.</p><p><a href="https://www.dallascowboys.com/team/players-roster/tyler-smith/">Tyler Smith (OT): Bio, News, Stats &amp; more</a></p><p>Tyler Smith was picked near the bottom of the 1st round and was an immediate starter on the Dallas Cowboys for the full season. He played almost every snap and was well regarded in his first year. Prior to the season, many draft analysts thought Smith was drafted too early as he had a <a href="https://www.nfl.com/prospects/tyler-smith/3200534d-4976-5890-507b-4d662371e33a">consensus</a> 2nd–3rd round grade out of college. After his rookie year the Cowboys look like they hit on a winner instead. <a href="https://www.youtube.com/watch?v=lVTmOT2cIMM">Highlights</a>.</p><p>Let’s dig into the player evaluations and see how the Offensive Tackles of the 2022 Draft class played.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/716/1*-IuX6Pjo6XzUs787x4f9vw.png" /><figcaption>4 picks were better than half the starting OTs in the NFL. Almost half the players picked played well</figcaption></figure><p>On the positive side Pro Football Reference and Pro Football Focus highlighted the play of several offensive tackles. Of the four already covered, Braxton Jones was held at a very high standard with the best PFF grade amongst tackles in the draft. Jones may have played <em>better than 77% of the starting offensive tackles in the league</em>. Again, he’s a strong contender for steal of the draft. Tyler Smith played nearly as well as Braxton Jones and is on the top end of all tackles in the league, too. Ikem Ekwenu did not play as well as either of them and was in the bottom half of the offensive tackles. Not too far off from average, which for a rookie is more than good enough. Charles Cross is an interesting case where the two evaluation services vastly disagreed on his evaluation. Pro Football Reference gave Cross a grade of 9 out 10 possible points for contribution to the team while PFF graded him slightly more than half of Braxton Jones and in the bottom 33% of the league. From perusing articles on the Seattle Seahawks season, my takeaway has been that Cross had a challenging rookie season on a bad offensive line. He did hold on to his starting job and showed promise as the season went along.</p><p>Let’s take a look at two unheralded rookies picked on Draft days 2 and 3 that performed much better than expected, Abraham Lucas and Jamaree Salyer. Both have performed so well in their rookie years that they’re considered better than half the offensive tackles that played in 2022.</p><p><a href="https://www.seahawks.com/team/players-roster/abraham-lucas/">Abraham Lucas (OT): Bio, News, Stats &amp; more</a></p><p>Abraham Lucas was the second offensive tackle Seattle Seahawks picked in the 2022 Draft to remake their offensive line. The overall line played really badly yet Lucas stood out for a 3rd rounder who started 16 games out of 17. He came out of college known for his pass blocking but got put on the right side of the line. Lucas did well here in pass blocking and wasn’t bad on run plays either. Pro Football Reference was extremely positive about his performance, giving him a score of 9 out of 10 points, tying teammate Charles Cross for the highest grades in the tackle group. Despite a bad year overall, the Seattle Seahawks drafted two of the best tackles in 2022. <a href="https://www.youtube.com/watch?v=99WZ3rJnqWo">Highlights</a>.</p><p><a href="https://www.chargers.com/team/players-roster/jamaree-salyer/">Jamaree Salyer (G): Bio, News, Stats &amp; more</a></p><p>Jamaree Salyer is a bonafide draft day steal found in the 6th round. With pick 195, the LA Chargers got themselves a offensive tackle that played all 17 games, starting 14 out of them. Salyer didn&#39;t just hang on to the job because the Chargers couldn’t find someone better, either. He played OK on both run and pass blocking. The Chargers offensive line overall played average this season. Pro Football Focus even graded him slightly better than Abraham Lucas above. Pro Football Reference gave him a lesser score of 7 out of 10 but still a good one. It’s hard to get past that the Chargers found a player in the 6th round that gave 14 quality starts his first year. <a href="https://www.youtube.com/watch?v=jymLJuSaT_8">Highlights</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/669/1*S1Cj4OAwjWlcFbEmlzvumg.png" /><figcaption>Please note: both Sacks and Penalty numbers aren’t official</figcaption></figure><p>Tyler Smith, Ikem Ekwonu and Charles Cross definitely need to reduce their penalties. Most of Cross’s penalties were False Starts, which can be addressed easily. All those penalties is holding Tyler Smith back from being recognized as a top tier young tackle in the NFL. Jamaree Salyer had a particularly low number of penalties relative to the snaps he played. Abraham Lucas, who’s strength is in pass blocking, gave up the most sacks? Surprising. I thought Bernhard Raimann was responsible for more sacks than PFF recorded. In general the rookies that played didn’t have a particular problem with getting their QBs killed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/578/1*499_ems-PDyD4qTAqInUQQ.png" /><figcaption>I am surprised how many top end OTs were also contributing to Special Teams</figcaption></figure><p>Dallas Cowboy Tyler Smith contributing the most to special teams, while holding down a starting left tackle position speaks volumes of his ability and toughness. Charles Cross, Jamaree Salyer and Ikem Ekwonu were no slouches either.</p><p><a href="https://www.raiders.com/team/players-roster/thayer-munford-jr/">Thayer Munford Jr. (OT): Bio, News, Stats &amp; more</a></p><p>Wanted to highlight Daniel Faalele and Thayler Munford contributing to special teams in their rookie seasons. Munford is the quintessential story of a player with zero expectations who fought into the active lineup by playing on special teams. He showed enough to the Raiders that in an pinch they relied on him to start 4 games. Did Munford play well? No, not really. But what do you want out of a rookie picked 238 in the 7th round? Munford only allowed one sack for the Chargers offensive line in his limited time and that’s a win for the team. Definitely a underdog story to watch.</p><p><a href="https://www.baltimoreravens.com/team/players-roster/daniel-faalele/">Daniel Faalele (OT): Bio, News, Stats &amp; more</a></p><p>Daniel Faalele is a giant of a player with crazy athletic traits but is a project when it comes to developing into a right tackle. That he is playing so much on special teams shows the Baltimore Ravens think he’s on the right path. Faalele gave the Ravens one emergency start and was active in 16 games.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/420/1*moZNQcmOmAMAHjrw6hFCYw.png" /><figcaption>Athletic freaks at OT didn’t dominate their first year</figcaption></figure><p>Abraham Lucas, Bernhard Raimann, Zach Tom, and Trevor Penning were extreme outliers on the RAS scores and were mostly drafted based on these elite athletic traits. Abraham Lucas combined his athleticism with good fundamentals to have a great rookie season, he was not a project. The rest of this group are definitely development players who shouldn’t be forced to play...</p><p><a href="https://www.colts.com/team/players-roster/bernhard-raimann/">Bernhard Raimann (OT): Bio, News, Stats &amp; more</a></p><p><strong><em>EDIT: A Colts fan hip checked parts of my write up, please go to ‘Extra Notes’ a the bottom of this post for his corrections.</em></strong></p><p>Berhnard Raimann may have had the hardest rookie season of all the offensive linemen picked in 2022. He was drafted completely based on his physical traits because he has minimal experience. Raimann started his football very late and was still learning the basics when he left college. The Indianapolis Colts had a plan to develop this raw player slowly… then injuries hit. Berhnard Raimann got forced into starting lineup at left tackle in week 5. The results were atrocious. After two games, he was sat down again till week 9 against the New England Patriots which could be argued as his worst game of the season. But Raimann was kept on in the starting lineup for the rest of the season and slowly got better. Here is where things get strange, Pro Football Focus is delusionally in love with the guy. Whereas Profootball Reference gave him a grade of 5 out of 10, which all things considered is pretty generous…Pro Football Focus gave Bernhard Raimann near the highest grade among tackles in this draft class. And ranked him better than 69% of the active offensive tackles in the NFL. I don’t understand what’s going on. The Indianapolis Colts got to be looking forward to Raimann making a serious second year jump.</p><p>Two players whose seasons got cut short due to injuries: Max Mitchell and Trevor Penning.</p><p><a href="https://www.newyorkjets.com/team/players-roster/max-mitchell/">Max Mitchell (OT): Bio, News, Stats &amp; more</a></p><p>Life threatening blot clots derailed Max Mitchell’s rookie season for the New York Jets. Team doctors were able to detect the nightmare issue in week 13 and shut down his season. Prior to that Mitchell’s season was uneven. He started out the season as the starting right tackle for 4 games then went down with a knee injury in that 4th game. He was on the active roster in week 12 and started week 13. The Jets offensive line situation was a major issue the entire 2022 season. How did Max Mitchell look in the 5 games he started? Hard to say. The grades are low but he was on a bad offense (that the star rookie RB Breece Hall bailed out), a QB that threw interceptions when trying to throw out of bounds (Zach Wilson, baby!) and offensive line that kept getting injured. Let’s just hope Mitchell’s blood clot issues are behind him and he can continue to play in the NFL.</p><p><a href="https://www.neworleanssaints.com/team/players-roster/trevor-penning/">Trevor Penning (OT): Bio, News, Stats &amp; more</a></p><p>Speaking of nightmare seasons, Trevor Penning just couldn’t get past foot injuries. He started off the season with a serious case of turf toe on his right foot and then started in the last game of the season. Where he suffered a serious Lisfranc injury on the same foot, requiring surgery. So all we have is one game of game tape, much more was expected out of the 19th overall pick in the 1st round. Despite high grades on Pro Football Focus, we can’t really say much about Trevor Penning until we see enough reps on the field.</p><p>Finally, offensive tackles that played their first year but there are reasons to worry: Nicholas Petit-Frere and Evan Neal.</p><p><a href="https://www.tennesseetitans.com/team/players-roster/nicholas-petit-frere/">Nicholas Petit-Frere (OT): Bio, News, Stats &amp; more</a></p><p>Nicholas Petit-Frere started at right tackle in every game in his rookie season except the final game due to an ankle injury. While drafted in the 3rd round as a developmental offensive tackle, he made the starting lineup and kept it…despite poor play over the season. Just how bad he did is up for debate. PFF had him the 2nd lowest graded tackle from this draft that played significant number of snaps and ranked in the bottom 10% of the entire league. Pro Football Reference was much kinder and gave him a grade of 6 out of 10 points. It’s hard to tell if the blame lays on Petit-Frere or on the Tennessee offensive line as a whole. The team struggled on offense and was one of the bottom five in 2022 in offensive line play. For a third round pick to immediately start for his team is an accomplishment, let’s see if he makes a second year jump in 2023.</p><p><a href="https://www.giants.com/team/players-roster/evan-neal/">Evan Neal (OT): Bio, News, Stats &amp; more</a></p><p>Fairly or unfairly, Evan Neal is getting labeled as a potential bust. Picked 7th overall in the 2022 NFL Draft, the expectations the New York Giants had of him were sky high. At the time, Neal’s draft stock was also high and the pick at that spot was applauded by most sports journalists, case in point from <a href="https://www.si.com/nfl/2022/04/29/2022-nfl-draft-first-round-grades">Kevin Hanson of Sports Illustrated</a>:</p><blockquote>“Neal offers the Giants versatility, as he started at left guard, right tackle and left tackle over the past three seasons, respectively. The former five-star recruit has an impressive combination of size, length and power, but he’s a smooth mover for a prospect his size. Neal is a freakish athlete who topped Bruce Feldman’s annual freaks list for <em>The Athletic</em>. Left tackle Andrew Thomas improved in 2021 from his rookie season, but the Giants’ offensive line needed an upgrade at right tackle, and Neal provides that on day one. Neal was my top-ranked offensive linemen and fourth-ranked prospect. <strong>GRADE: A+</strong>’</blockquote><p>In Evan Neal’s rookie season, what we saw instead was a player who was not position versatile and struggled with the move to right tackle. The athleticism did not translate, which is really strange for the position. With his physical attributes, Evan Neal should be a monster on the right side. And maybe he is but the statistics aren’t conveying it? Pro Football Reference did give the New York Giants the #2 ranking for Power Run blocking. The offensive line was very bad blocking for their quarterback Daniel Jones but actively helping Saquon Barkley with his bounce back year at running back. PFF gave him the worst grade of any tackle that got significant play time in this draft. Even worse, they ranked him 2nd worst in the league. Pro Football Reference gave him a 6 out 10 points, which would be OK but not for a first rounder. It didn’t help that he lost 4 games in the middle of the season to injury.</p><p>Based off of videotape review, it looks to me that Evan Neal was much more of a developmental prospect than a finished product as many had assumed. Meaning, through coaching and natural development he could reach the potential the NY Giants saw in him. Probably in another year or two. Maybe. This offseason he has taken <a href="https://www.si.com/nfl/giants/big-blue-plus/giants-rt-evan-neal-looks-to-take-big-step-in-second-season">personal coaching</a> to find the stance &amp; technique that best fits him at right tackle. Neal has been talking about a complete rework of his technique. Not sure how long that could take but it is way to early to slap the bust label on any player after just one season.</p><h3>Offensive Guards</h3><p><strong>Most of the offensive guard rookies were pushed into starting and did not play great.</strong> While disappointing, it’s not unexpected. A few played average but most performed well below NFL standards. Teams who picked guard in the 2022 draft will be hoping for solid 2nd year jumps.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/727/1*pJJxDiomgGc_S8a8LTcCkw.png" /><figcaption>6 Day 1 starters at OG were picked in 2022, with 2 more major contributors. Lots of diamonds found</figcaption></figure><p>When it came to drafting offensive guards, teams would’ve been better off picking from the middle of the 2022 Draft, than anyone high. Of course it’s way too early to judge draft picks after one year, but it seems like teams found winners on the 2nd day of the Draft. Most of the players picked in rounds 1st–2nd struggled. Four players were season long starters for their NFL teams and three more played 15 or more games. Eight out of sixteen players in total guards drafted showed enough promise to turn into full time starters their rookie seasons.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/698/1*NiNkqdFfE-lDJAztH704fg.png" /><figcaption>Only 2 players were graded as ‘average’ by PFF but Pro-football Reference thought half were good picks</figcaption></figure><p>The offensive guard ratings by both Pro Football Focus and Pro Football Reference are not glowing for this class. PF ranked most of the players very low compared to other NFL guards. Pro Football Reference was much higher on these players and gave good grades (7+ out of 10) to 6 out of 8 players that started multiple games. Whether many of these players got the start because of their draft position or if the offensive lines they were playing in just didn’t have anyone who could replace them, I would have to go on a team by team basis. In my opinion, most of these players should not have been starting and instead should’ve been developing behind the scenes.</p><p><a href="https://www.chargers.com/team/players-roster/zion-johnson/">Zion Johnson (G): Bio, News, Stats &amp; more</a></p><p>Let’s start the evaluations with the two of the four Day 1 starters who performed well for their teams: Zion Johnson and Dylan Parham.</p><p>Los Angeles Charger Zion Johnson and Las Vegas Raider Dylan Parham had the first and second highest PFF grades for drafted offensive guards with meaningful snaps last season. Both would rank at the middle of all the offensive guards playing in the league. Pro Football Reference liked Johnson more (8 points out of 10) than Parham (7 out of 10), both are good grades for the season. Johnson was the second offensive guard picked (17th in round 1 of the draft) Chargers fans can make a case that he played up to that pick. It’s an valid argument because Zion Johnson arguably played the best out of all these guards, without giving out many sacks (5) or taking excessive penalties (7). Johnson has stabilized the right guard position for the Chargers. <a href="https://www.youtube.com/watch?v=jNhyzKyKK6Q">Highlights</a>.</p><p>But his competition for the best season is against Dylan Parham and Cordell Volson, who were picked in 3rd and 4th rounds.</p><p><a href="https://www.raiders.com/team/players-roster/dylan-parham/">Dylan Parham (G): Bio, News, Stats &amp; more</a></p><p>In the 3rd round with the 90th pick, the Las Vegas Raiders found a diamond in Dylan Parham. He played the entire season at left guard, with few penalties (4) and 6 sacks. In the offseason, Parham’s play allowed the Raiders to cut former first round pick Alex Leatherwood. Was he dominant during the season? No. But he was more than good enough and on a offensive line that collectively played really well (average rank of 6 between the three grading services). What held Parham back were issues in pass blocking throughout the season. The Raiders have themselves a solid starter if he keeps improving.</p><p>Two starters who gave their teams almost every rep but could definitely improve their play: Ed Ingram and Cole Strange.</p><p><a href="https://www.vikings.com/team/players-roster/ed-ingram/">Ed Ingram (G): Bio, News, Stats &amp; more</a></p><p>Ed Ingram didn’t light up the NFL, but as a second rounder (pick 59) Engram played every single offensive snap for the Vikings. The season was not as good as Zion Johnson or Dylan Parham, as graded by Pro Football Focus. Pro Football Reference did give him a solid 7 out of 10 grade. On run plays he was OK (the Vikings were especially bad on run blocks his first year). It’s the pass protection where Ingram’s struggles got highlighted. Ingram’s critics got quite loud during the rookie’s first season with the playoff bound Vikings. From “<a href="https://www.si.com/nfl/vikings/news/espn-lists-rg-ed-ingram-vikings-biggest-flaw-playoffs-approach">ESPN Lists RG Ed Ingram as Vikings’ Biggest Flaw as Playoffs Approach</a>” on SI.com:</p><blockquote>Vikings rookie right guard Ed Ingram isn’t merely struggling in pass protection, he’s on pace to give up a historic number of pressures this season.</blockquote><blockquote>Ingram has allowed 44 pressures this season, according to PFF, which leads all players through 12 weeks. No one else has more than 35. The most pressures ever allowed by a guard in the PFF era (since 2006) is 65, achieved by the Raiders’ Alex Leatherwood last season and the Giants’ David Diehl in 2011. Both players spent time at tackle during those seasons but played the majority of their snaps at guard.</blockquote><blockquote>Ingram, with 44 pressures in 11 games, is on pace to allow 68 this season (granted, with a 17-game schedule). The second-round pick out of LSU has done that while playing exclusively at guard. He ranks 57th out of 60 qualified guards in PFF’s pass blocking grade and has been charged with giving up eight sacks, three more than the next-closest player at his position.</blockquote><p>By the end of the season, Pro Football Focus ranked Ingram in the bottom quarter of all offensive guards. He well below average at guard, for an offensive line that was also below average (average ranking of 21 out of 32). The Vikings absolutely need him to improve on pass blocking specifically. For a second round pick, Ed Ingram did alright. But if he keeps causing problems for his QB, patience will run out fast.</p><p><a href="https://www.patriots.com/team/players-roster/cole-strange/">Cole Strange (G): Bio, News, Stats &amp; more</a></p><p>I’m a Patriots fan, so I’m probably biased. Cole Strange started every game and almost played every offensive snap for the team at left guard. But the whole season he was up and down, depending on who was alongside him on the average (15.7 average rank out of 32) New England Patriots offensive line. If it was starters David Andrew at center and Trent Brown at left tackle? He played well. If it were backups at either, Strange’s performance fell down. He got benched a few times for <em>Isiah Wynn </em>who sucks, one of the worst uses of the 5th rookie extension ever. Pro Football Focus ranked Strange in the bottom fifth among NFL offensive guards. Strange especially got abused on bull rushes so he needs to increase playing weight and/or play strength. Cole Strange also needs to improve hand usage &amp; technique and awareness versus stunts, as shown by Taylor Kyles here:</p><h3>Taylor Kyles on Twitter: &quot;Strange&#39;s other weak spot has been mental lapses vs stunts, sometimes looking caught off guard or unsure of his assignment.Understandable why Strange said his mistakes have been fundamental. Not like he&#39;s getting bull rushed on solo rep. Technique + recognition need to improve pic.twitter.com/GT8bI4wPEC / Twitter&quot;</h3><p>Strange&#39;s other weak spot has been mental lapses vs stunts, sometimes looking caught off guard or unsure of his assignment.Understandable why Strange said his mistakes have been fundamental. Not like he&#39;s getting bull rushed on solo rep. Technique + recognition need to improve pic.twitter.com/GT8bI4wPEC</p><p>Now, Cole Strange isn’t a bust or anywhere near that label. But the Patriots picked a developmental player from the FCS at the bottom of the 1st round and he had the 4th best season among the rookie guards. I wish he played better but it has to be repeated, the guy was a small college developmental player who got picked because of crazy good athletic testing. Cole Strange needs time and coaching to become a solid starter.</p><p>Two 4th round picks that gave their teams starter play for almost the entire season: Cordell Volson and Spencer Burford. Both of these players started 16 games for their teams and played well above expectations. Did they play great? Nope. But the Bengals and 49ers got themselves starters in the 4th round who didn’t let their QBs get killed so no worries!</p><p><a href="https://www.bengals.com/team/players-roster/cordell-volson/">Cordell Volson (OT): Bio, News, Stats &amp; more</a></p><p>Pro Football Reference absolutely loved Cordell Volson, giving him 9 out of 10 points for his contributions to the team. Pro Football Focus graded him below Cole Strange to be the 5th best offensive guard to be drafted, better than 16% of the active offensive guards in the NFL. It’s hard to tell who’s closer to the truth here. Considering Cordell Volson was #2 in performance based pay adjustment, Pro Football Reference might have a case here.</p><h3>Tom Pelissero on Twitter: &quot;Former #Eagles safety Marcus Epps, who left in free agency for the #Raiders, led the NFL with a $880,384 performance-based pay distribution for 2022. pic.twitter.com/S9OiYxULBi / Twitter&quot;</h3><p>Former #Eagles safety Marcus Epps, who left in free agency for the #Raiders, led the NFL with a $880,384 performance-based pay distribution for 2022. pic.twitter.com/S9OiYxULBi</p><p>Cordell Volson came out of North Dakota State seen as a very raw developmental player. Very limited in pass protection, and generally inexperienced but has athletic traits for the NFL. Not someone who should be cracking the active roster anytime soon. Volson kept improving throughout the season, outlined by Mike Santagata of Sports Illustrated, and got good enough to shed the developmental lable:</p><p><a href="https://www.si.com/nfl/bengals/allbengals-insiders-plus/bengals-film-breakdown-analyzing-cordell-volson-improvement-throughout-rookie-season">Bengals Film Breakdown: Analyzing Cordell Volson&#39;s Improvement Throughout Rookie Season</a></p><p>Volson’s technique got better and the game slowed down for him, so stunts and other defensive line tricks effected him less and less. While pass protection because a strength for him, it’s also a place the Bengals need him to keep improving. He needs to increase his play strength and/or playing weight, improve his balance (Volson got criticized for being stiff and would stand still during blocks). Would he start on a good offensive line? Probably not at the start of the season, definitely by the end. Volson hardly ever got hit with penalties (4) and gave up 5 sacks over the season. Bengals on average got ranked 19.7, dealt with instability in the starting lineup and ended up needing Cordell Volson to be good for their playoff rush. And he played more than OK. If Volson keeps improving at the rate he’s going, who knows how good he can be?</p><p><a href="https://www.49ers.com/team/players-roster/spencer-burford/">Spencer Burford (OT): Bio, News, Stats &amp; more</a></p><p>Spencer Burford had a weird season. He was a developmental player who ended up starting every game but would regularly get sub’d out by the previous starting right guard (67.7% Offensive Snap Rate). Prior to the draft he was <a href="https://www.nfl.com/prospects/spencer-burford/32004255-5205-4996-13f9-5da6af74dbe3">seen</a> as a player with the athletic traits for the NFL but not the experience or skills. Pick 134 seemed reasonable for Burford, but he showed the San Francisco 49ers that they got something in him. At training camp, Burford got the chance to work the starting right guard position when they tried moving the starter there (Daniel Brunskill) to center. Then Brunskill got injured, making Burford the starter for the rest of the season.</p><p>By December 1 of the 2022 season, he had the best statline of all rookie offensive linemen. From “<a href="https://www.ninersnation.com/2022/12/1/23488406/49ers-spencer-burford-sack">Spencer Burford is the only rookie offensive linemen in the NFL who hasn’t allowed a sack</a>” by Jordan Elliott:</p><blockquote>Of the rookies who have played at least 50 percent of snaps on the offensive line this season, Burford is the only one who has yet to allow a sack this season. Taking it a step further, Burford’s 11 pressures allowed is the fewest among rookie offensive linemen, with the next closest being Ravens center Tyler Linderbaum who has allowed 17…Burford’s consistency in pass protection has been on display over the course of this season as well, as he has only allowed more than one pressure two times in 11 games while never allowing more than three in a given game.</blockquote><p>How did Spencer Burford do his rookie year? Pro Football Reference gave him a solid 7 out of 10 grade. Pro Football Focus graded him harshly, with the 6th highest grade on a rookie guard, and ranked Burford in the bottom 13% of the league. My take is that he was the starting right guard on a top 10 offensive line in the NFL and his team was one game away from the Super Bowl. That means Spencer Burford was good. He didn’t give up many sacks (5) and penalties were not a problem (4). He led all rookie guards in special team snaps. Spencer Burford does need to get stronger at all aspects of the position. Especially as a run blocker. In his rookie year, Burford got overwhelmed by very talented defensive lines and in the playoffs it became a problem. He really needs time and development to solidify his starting job at right guard.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/652/1*ZOibyYu46V5lc294j7gJiw.png" /></figure><p>Ed Ingram had an eye popping problem with giving up sacks at 11. This is a major concern. Zion Johnson should focus on reducing the penalties but in general there wasn’t a egregious issue with fouling. Until you get to Kenyan Green, who had 7 penalties with another 5(!) that were declined, for a grand total of 12, which is well outside the curve. The Houston Texans have to figure out how to fix Green’s problems. Holding was the issue for most of these players, a harder to problem to address than False Starts. Dylan Parham, Cordell Volson and Ed Ingram were particularly good at avoiding penalties. Luke Wattenberg probably get more playing reps if his pass protection was better.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/567/1*njuRt80l2FUV8CEM74r2Bw.png" /><figcaption>Late round picks contributed on Special Teams as expected</figcaption></figure><p>Spencer Burford, Cordell Volson, Ed Ingram, Cole Strange, and Dylan Parham were particularly active contributors to special teams. Lewis Kidd is able to make the NFL active roster as a UDFA through special teams play. Looking at the guards who also did special teams, a surprisingly high amount of starters are included and too few of the lower drafted players participated.</p><p><a href="https://www.neworleanssaints.com/team/players-roster/lewis-kidd/">Lewis Kidd (G): Bio, News, Stats &amp; more</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/407/1*EofQ2BbIyZ5c4WNaArLVHg.png" /><figcaption>Half of the 6 best OGs picked had &gt;8.5 RAS scores…But so did many who didn’t play much</figcaption></figure><p>It seems like RAS was mixed in predicting who played well. One one hand some players with high RAS scores excelled on the field: Zion Johnson, Dylan Parham, and Spencer Burford. But quite a few players with high RAS scores didn’t get many reps (Nick Zakelj, Chris Paul, Luke Wattenberg). Cole Strange, who got the highest RAS score among guards started every game but didn’t play well. Kenyon Greene, who didn’t play well, had the lowest RAS score among drafted rookies. Cordell Volson’s RAS score was slightly above Green’s, and he did play well. So it’s all a mixed bag.</p><p>Who I’m worried about: Luke Goedeke and Kenyon Green</p><p><a href="https://www.buccaneers.com/team/players-roster/luke-goedeke/">Luke Goedeke (OT): Bio, News, Stats &amp; more</a></p><p>Luke Goedeke gave the Tampa Bay Buccaneeners half a season of starts, and not much good play. Pro Football Reference gave him a 3 out of 10 grade, and Pro Football Focus graded him poorly, ranking him in the bottom 10% of the guards in the league. The Buccaneers converted him from right tackle to left guard, and Goedeke struggled with both the position and side change. He was also jumping from a small school (MAC conference) to the NFL. For a 2nd round draft pick (pick #57) known for his versatility, this pushed him too hard. Goedeke started for the Buccaneers the first six games of the season at left guard, then went down with a foot injury in game 7. From then on Nick Leverett took over as starter. Luke Goedeke doesn’t start again until week 18 where he replaced injured Tristan Wirfs at right tackle. In the playoff game the next week, Goedeke was moved back to left guard to replace an injured Leverrett. Profootball Focus thought his best game of the season was at right tackle. Tampa Bay is <a href="https://www.tampabay.com/sports/bucs/2022/10/19/luke-goedeke-bucs-rookie-offensive-line-todd-bowles/">desperate</a> to make something of their 2nd round pick and test Luke Goedeke at left guard, right guard and right tackle at this year’s training camp.</p><p><a href="https://www.houstontexans.com/team/players-roster/kenyon-green/">Kenyon Green (OT): Bio, News, Stats &amp; more</a></p><p>The Houston Texans have a problem, they picked Kenyon Green in the first round, with pick #15. He was the first offensive guard picked. This was the kind of player the Texans thought they got:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FyYNsqddRh3c%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DyYNsqddRh3c&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FyYNsqddRh3c%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/6af0a83dc114a04e572f1538a80a398b/href">https://medium.com/media/6af0a83dc114a04e572f1538a80a398b/href</a></iframe><p>He played like the worst guard picked this draft. Pro Football Focus ranked him as the worst player among all offensive guards in the NFL that played at least 50% of all snaps on offense and a horribly low overall grade for the season. Pro Football Reference gave him a 5 out of 10 score, which is not a good grade for someone who started 14 games. Kenyon Green didn’t get the starting position out of training camp due to concussion issues. He had a serious problem with committing penalties during the season. Against the Dallas Cowboys in week 14, Green fell down with a ankle or knee (?) injury that took till week 17 to start again. After the season, he had arthroscopic knee surgery. The 2022 Houston Texans had the worst offensive line in the NFL (ranked #13 when it comes to sacks tho! Somehow). Kenyon Green’s highlight of the year happened when a Titans player ragdolled him into the running back.</p><p><a href="https://www.tennesseetitans.com/video/can-t-miss-play-jeffery-simmons-throws-kenyon-green-into-dameon-pierce-for-insan">Can&#39;t-Miss Play: Jeffery Simmons Throws Kenyon Green into Dameon Pierce for Insane TFL | Game Highlights</a></p><p>Yeah, it hasn’t been great. What can Kenyon Green improve on? Everything. Forget the bust label, that’ll take another season or two. But at this point he needs to earn his roster spot and not be handed a starting job. Green needs to be treated like a pure development player and not rushed back into the starting lineup.</p><h3>Centers</h3><p>Only 3 centers got drafted in the 2022 NFL Draft, two became instant starters. Tyler Linderbaum is already considered one of the best centers in the league.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/717/1*GtCqKjhzFFFUjmq_00--4A.png" /><figcaption>2 out of 3 Centers picked were day 1 starters that played the entire season</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/687/1*Fq7gi6vuROKrBy-q-w8UcA.png" /><figcaption>Linderbaum was ranked better than 80% of NFL starting Centers. Fortner was ranked better than only 2 others</figcaption></figure><p>Let’s start out with the player who played most snaps at center: Luke Fortner.</p><p><a href="https://www.jaguars.com/team/players-roster/luke-fortner/">Luke Fortner (C): Bio, News, Stats &amp; more</a></p><p>Luke Fortner, picked #65 in the 3rd round, came to the Jacksonville Jaguars and immediately became their starting center. Fortner played pretty consistantly at a position of need for the team. There’s even an argument that he was the Jaguar’s best draft pick in 2022. It was clear throughout the season that Luke Fortner went through tough patches as a rookie, but the team felt good about him. Pro Football Focus did not see him in the same light, giving him a poor grade for the year and ranked him in the bottom 10% of active centers in the NFL. Pro Football Reference definitely loved his play, giving Fortner 8 points out of 10 for the year. The NFL also gave him the third largest salary adjustment due to great performance. Luke Fortner played very clean, with only 4 penalties and 1 sack his rookie season. He also contributed heavily to special teams.As the leader of the offensive line, the Jacksonville Jaguars had the offensive line ranked on average 21. So there is definitely room for Luke Fortner to grow. Overall the Jaguars must be very happy with this pick.</p><p><a href="https://www.baltimoreravens.com/team/players-roster/tyler-linderbaum/">Tyler Linderbaum (C): Bio, News, Stats &amp; more</a></p><p>Tyler Linderbaum was expected to be a great center prospect, that could slide into the NFL immediately, and that’s exactly what happened. There’s really not much more that can be said about him. Linderbaum got picked around the end of the first round, pick #25, by the Baltimore Ravens and played every snap on offense. Pro Football Focus ranked him in the top 80% of active centers in the NFL and Pro Football Reference gave him a solid 7 out of 10 points for the season. He didn’t have problems with penalties (6) or sacks (3). Maybe one knock on the player is that he played zero snaps on special teams, but that could have been a coaches decision. Athletically, Tyler Linderbaum is very gifted (RAS score of 8.77). With him at center, the Baltimore Ravens offensive line played at a NFL top 10 level. Tyler Linderbaum more than justified where his first round draft position. <a href="https://www.youtube.com/watch?v=HnFy97bl5Ek">Highlights</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/647/1*p-GIPbU0s8CEIBZ2LlKePA.png" /><figcaption>Neither starter had issues with penalties or sacks. Fortner was especially clean.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/555/1*qW2c57m4o4_5D4rt1NcPpw.png" /><figcaption>Fortner, along with starting, put in the work on special teams too</figcaption></figure><p><a href="https://www.philadelphiaeagles.com/team/players-roster/cam-jurgens/">Cam Jurgens (C): Bio, News, Stats &amp; more</a></p><p>Cam Jurgens made all his contributions to the Philadelphia Eagles through special teams. Despite getting picked in the 2nd round, pick #51, this isn’t considered a disappointment. The Eagles have a great center in veteran Jason Kelce who considered retirement last year but decided to play for one more year. The Eagles went all the way to the Super Bowl but lost. Kelce decided to play one more year to see if the team can go back. So Cam Jurgens has been the insurance policy for the Philadelphia offensive line in case Kelce didn’t return or got hurt. Basically factors outside of Cam Jurgens’ control. Could Philadelphia have gotten Jurgens or another center later? Definitely. But they didn’t know Kelce’s career plans at the time and Cam Jurgens had serious athletic traits (RAS score of 9.94!) to develop around. Either way the Philadelphia Eagles have their center of the future on the roster.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/370/1*d_rIJS3SsKpSEt3LKYn5bw.png" /><figcaption>Very small dataset but RAS scores showed athleticism at Center may be underappreciated</figcaption></figure><h3>Concluding Remarks</h3><p>It’s clearly nonsensical to hand out grades and judgements on offensive linemen just based off of their first year. Hopefully the majority of players drafted will develop into full time players after 3–4 seasons. But when they are getting starting jobs and playing a ton of reps on the field, its plenty obvious that the teams who drafted them got winners.</p><p>I was very surprised at how many success stories happened at the tackle spot. Guards felt more natural or typical a draft experience, where the majority of players will need a few seasons to catch up to the NFL game. Centers was strange, only 3 got picked and two of them started immediately. The third one didn’t start because the player he was supposed to replace changed their mind about retiring. But the main conclusion is, if your team needed a tackle for their offensive line, this draft had them.</p><h3>Extra Notes</h3><p>Next up: Wide Receivers!</p><p>Great reader responses:</p><blockquote>Volson definitely played well, but he’s older and was already cerebral …Volson scored really high on a specific test that covers a range of metrics that are supposed to be indicative of success, a lot of which mental. <a href="https://bleacherreport.com/articles/1549204-nfl-combine-new-player-assessment-test-examines-prospects-psychological-makeup">The new test for player’s psychological makeup</a>. <a href="https://www.cincyjungle.com/platform/amp/2022/5/11/23062488/why-the-bengals-drafted-cordell-volson">Why the Bengals drafted Cordell Volson</a>— Jay</blockquote><p>Great point about why Cordell Volson dropped to the 4th round in the first place but compensated by being smart.</p><blockquote>I don’t think you’re understanding wAV/AV correctly. First, wAV is a career stat that combines players’ individual season AVs together with a weighted sum. Luckily, the highest value is given a 100% weight and these players are all rookies, so wAV = AV in this case. Second, you refer to AV as a “grade” fairly often, and reference a maximum of 10 AV. AV is not a grade assigned to players by scouts at PFR (PFR doesn’t employ scouts), it’s a calculated metric, and for OL, it’s entirely based on the player’s team’s offensive points per drive relative to league average and whether the player made the Pro Bowl or an AP All-Pro team. That’s it, nothing more goes into the calculation for OL AV. There’s no maximum. Maybe no rookie OLs had higher than 10 AV, but Josh Allen and Jalen Hurts led the league with 20 AV this year. The single-season record for AV is 26 by LT in 2006. There’s also not a minimum; players have ended seasons with negative AV before. — Yedic</blockquote><p>So this is a much accurate explanation and description of Pro Football Reference’s method of evaluating players. In this case offensive linemen get points for pass and run plays which is then fed to a formula. I actually found these values in a different sue SIS Datahub well after I was done data gathering. In the future I’ll use the components for analysis</p><p>So my write up on Bernhard Raimann used wrong details and Colts fan Matthew corrected me hard on Reddit. Here’s his post:</p><blockquote>So onto the Colts and Raimann.</blockquote><blockquote>I’ll start with the objectively wrong, he didn’t start week 5 because of injury, we had rotated him in on snaps prior to this and then the performance of Matt Pryor was so bad it forced Raimann to basically be chucked in sink or swim for two games.</blockquote><blockquote><a href="https://coltswire.usatoday.com/2022/10/13/indianaoolis-colts-bernhard-raimann-starting-left-tackle/">https://coltswire.usatoday.com/2022/10/13/indianaoolis-colts-bernhard-raimann-starting-left-tackle/</a></blockquote><blockquote>He was then benched for two games and then brought back for the remainder of the season. Now onto the more subjective things.</blockquote><blockquote>Raimann was a ‘bright spot’ for the Colts last year with his play down the stretch. As you mentioned he had limited experience, having grown up abroad and coming to the game late and playing tight end initially only having one season at tackle. But the traits were there. And he developed and improved drastically as the season went on. Now I personally like the PFF ability to grade lineman, they feel strongly about their ability to grade them and I back them on it. He also had the second toughest schedule per PFF pass rusher grade in that second half stretch. Matt Judon, Chandler Jones, Brandon Graham, Alex Highsmith, DeMarcus Lawrence, Micah Parsons, Danielle Hunter and Khalil Mack, among others, in the second half of the season</blockquote><blockquote>That’s not to mention the disfunction with QB (3 different QB starts two of which were completely immobile) a change of head coach and play caller for the worse and an underperformed group of veterans around him (even BigQ was ‘worse’ then his usual self this year). Context is important for all roles in this league. And then finally the stats. I won’t use PFF as you’ve mentioned you don’t see eye to eye with them. But just pressures and snaps given up. 27 pressures and 7 sacks on 709 snaps played.</blockquote><blockquote>Not bad for a developing rookie and player at the position overall.</blockquote><p>So not only was Matthew more accurate about Raimann’s playing time/snaps from the start of the season (and how he became a starter) but also how the Colts viewed him as a player. This outlining of his season would also explain why PFF gave Raimann such (surprisingly!) good grades.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9fcfbe9578" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[cuDF for Data Scientists: Functions for Data Wrangling]]></title>
            <link>https://medium.com/@tiraldj/cudf-for-data-scientists-part-1-2-functions-for-data-wrangling-12a8f889b33e?source=rss-3b89e6336fb5------2</link>
            <guid isPermaLink="false">https://medium.com/p/12a8f889b33e</guid>
            <category><![CDATA[data-wrangling]]></category>
            <category><![CDATA[rapids-ai]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[pandas]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Mohammed R. Osman]]></dc:creator>
            <pubDate>Thu, 05 Jan 2023 23:36:10 GMT</pubDate>
            <atom:updated>2023-02-21T18:06:05.766Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ux0WVAUYn5ndiNvO" /><figcaption>Photo by <a href="https://unsplash.com/@nicolasarnold?utm_source=medium&amp;utm_medium=referral">Nicolas Arnold</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Reference guide made to help people transition from Pandas to NVIDIA’s RAPIDS.AI cuDF package.</p><h3>Purpose</h3><p>Think of a one stop shop to find data wrangling help to use for cuDF.</p><p>I hope to make it a living document that people can have bookmarked and use to quickly &amp; easily find help using both RAPIDS.AI cuDF <em>and</em> standard Pandas. The focus will be on loading, organizing and manipulating data on the GPU. The list of functions are based on personal experience as the most common or important pandas data wrangling functions. Functions that analyze, predict, or used in machine learning are outside of the scope. This post should have everything you need to take raw data into the GPU as a DataFrame, clean &amp; organize it, and output it in other forms.</p><p>While there is a serious effort to make the transition from pandas to cuDF as pain-free as possible, there can be important differences. This often goes beyond the scope of one post, so links to specific articles and I’ll try to point out things to look out for. For comparisons’ sake links to both cuDF and pandas docs will be provided and examples (and some personal commentary) included.</p><p>As RAPIDS evolves, expect this post to go through multiple revisions as mistakes are fixed, new content added from suggestions, and new cuDF capabilities are integrated.</p><h3>Table of Contents</h3><p><a href="#69d7">Setup</a></p><p>cuDF Functions</p><ul><li><a href="#d8b1"><strong>cuDF DataFrame</strong></a></li><li><a href="#f2f3"><strong>agg</strong></a></li><li><a href="#e7ee">Using functions across the dataframe without loops (<strong>apply</strong>, <strong>applymap</strong>, <strong>map, apply_rows</strong>, <strong>groupby().transform()</strong>, <strong>items</strong>, <strong>scatter_by_map</strong>)</a></li><li><a href="#5ebe">Assign binning values (<strong>cut, qcut</strong>)</a></li><li><a href="#4858">columns (<strong>columns, lis</strong>t)</a></li><li><a href="#6f65">Combining dataframes (<strong>concat, join, merge</strong>)</a></li><li><a href="#70f0"><strong>crosstab</strong></a></li><li><a href="#2435"><strong>drop</strong></a></li><li><a href="#4a1c">Dealing with duplicate data (<strong>duplicated, drop_duplicates</strong>)</a></li><li><a href="#18f6"><strong>explode</strong></a></li><li><a href="#aa6b">Finding values (<strong>eval</strong>, <strong>filter</strong>, <strong>query, groupby.get_group</strong>)</a></li><li><a href="#6f06">Frequency (<strong>count, unique, nunique, value_counts, groupby.ngroup / ngroups</strong>):</a></li><li>Coding data from categorical to numerical: <a href="#b4ca"><strong>(“category” data type, factorize, get_dummies</strong></a><strong>)</strong></li><li><a href="#5276"><strong>groupby</strong></a></li><li><a href="#0ca2">Quick dataframe information (<strong>describe, head &amp; tail, info, len, shape</strong>)</a></li><li><a href="#dd40">Directly access data (<strong>at</strong>, <strong>get_loc</strong>, <strong>insert, iloc, isin, loc, xs</strong>)</a></li><li><a href="#a4ac">Pivot tables <strong>(melt, pivot, pivot_table, stack, unstack, wide_to_long):</strong></a></li><li><a href="#b9aa"><strong>rank</strong></a></li><li><a href="#770e"><strong>rename</strong></a></li><li><a href="#eb66"><strong>resample</strong></a></li><li><a href="#803a">Exchanging values (<strong>mask</strong>, <strong>replace</strong>, <strong>where</strong>)</a></li><li><a href="#2ddb">Indexing (<strong>index, MultiIndex, </strong>names<strong>, reset_index, reindex, reindex_like, set_index, sort_index, truncate</strong>)</a></li><li><a href="#3a4f"><strong>sample</strong></a></li><li><a href="#bc70">sorting values (<strong>sort_values, nlargest, nsmallest, groupby.nth</strong>)</a></li><li><a href="#457e">value formats (<strong>astype, dtypes, select_dtypes, to_datetime</strong>)</a></li></ul><p>Extras:</p><ul><li>Description of Dataset Used for Illustrative Purposes</li><li>Generating the NFL Databowl 2022 DataFrame in cuDF</li><li>Simplified NFL Database Using Pandas</li></ul><h3><strong>Setup</strong></h3><p>Function descriptions will use the following format:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/846/1*OjnxGGsOdulLEhRGJJqg9Q.png" /><figcaption>Notice that Pandas insert() has more options available and cuDF has a ‘nan_as_null’ option</figcaption></figure><p>Arrow 1: Instead of going alphabetical, the functions are arranged in groupings or families of functions.</p><p>Arrow 2: The actual function will be defined, usually word for word as written by <a href="https://pandas.pydata.org/docs/index.html">Pydata</a> or <a href="https://docs.rapids.ai/api/cudf/stable/">RAPIDS</a> teams. Personal notes are written here afterwards.</p><p>Arrow 3: The format on how the function is used, with arguments is written out. Usually the cuDF and pandas formats are exactly the same but not always. This is a good way of being alerted to that. Both cuDF and pandas official docs links are provided so you can dig into the functions and see examples.</p><p>Arrow 4: Example provided with code. Provided on a case by case basis</p><p>Arrow 5: Comment provided to explain what is going on and what insight was gained in the example.</p><h3>cuDF Functions</h3><p><strong>cuDF DataFrame</strong>:</p><p>The fundamental basis of managing data. Two-dimensional, size-mutable, potentially heterogeneous tabular data. Data structure also contains labeled axes (rows and columns). Arithmetic operations align on both row and column labels.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe">cuDF</a>: (<em>data=None</em>, <em>index=None</em>, <em>columns=None</em>, <em>dtype=None</em>, <em>nan_as_null=True</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html?highlight=dataframe#pandas.DataFrame">pandas</a>: <em>(data=None, index=None, columns=None, dtype=None, copy=None)</em></p><p><strong>agg</strong>:</p><p>Aggregate using one or more operations over the specified axis. When DataFrame.agg is called with single operation/function, a Series is returned. With several operations/functions, a DataFrame is returned</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.agg#cudf.DataFrame.agg">cudf</a>: (<em>aggs</em>, <em>axis=None</em>); note: axis not yet supported<br><a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.agg.html">pandas</a>: (<em>func=None</em>, <em>axis=0</em>, <em>*args</em>, <em>**kwargs</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/671/1*OzIps5jZBerIjIfcagfd7w.png" /><figcaption>After creating the DataFrame, agg() was used to condense it down.</figcaption></figure><p>agg() is a very flexible tool, this is just a small look into what it can do.</p><p>Using functions across the dataframe without loops (<strong>apply</strong>, <strong>applymap</strong>, <strong>map, apply_rows</strong>, <strong>groupby().transform(), items</strong>, <strong>scatter_by_map</strong>). Unfortunately, as of this writing I myself am not an expert in these functions so will not include examples.</p><p><strong>apply</strong> (apply, groupby.apply):</p><p>Is highly bespoke in Rapids.AI packages, please look at the doc linked before using. Apply works in parallel, always use this instead of for loops where possible.</p><blockquote>Apply a function along an axis of the DataFrame. apply relies on Numba to JIT compile func. Thus the allowed operations within func are limited to those supported by the CUDA Python Numba target. For more information, see the cuDF guide to user defined functions.</blockquote><blockquote>Support for use of string data within UDFs is provided through the strings_udf RAPIDS library. Supported operations on strings include the subset of functions and string methods that expect an input string but do not return a string. Refer to caveats in the UDF guide referenced above.</blockquote><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.apply#cudf.DataFrame.apply">cuDF</a>: (func, axis=1, raw=False, result_type=None, args=(), **kwargs)<br> <a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.apply.html">pandas</a>: (func, axis=0, raw=False, result_type=None, args=(), **kwargs)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/476/1*FrkVAzMpt4zfcxixVCl0GA.png" /><figcaption>using the user defined function over the DataFrame row by row</figcaption></figure><p><strong>apply_chunks</strong>:</p><p>Transform user-specified chunks using the user-provided function. (Untested)</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.apply_chunks">cuDF</a>: <em>(func, incols, outcols, kwargs=None, pessimistic_nulls=True, chunks=None, blkct=None, tpb=None)</em></p><p><strong>applymap</strong>:</p><p>Apply a function to a Dataframe elementwise. This method applies a function that accepts and returns a scalar to every element of a DataFrame. (Couldn’t get to work)</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.applymap#cudf.DataFrame.applymap">cuDF</a>: (<em>func: Callable[[Any], Any], na_action: Optional[str] = None, **kwargs)<br></em><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.applymap.html?highlight=applymap#pandas.DataFrame.applymap">pandas</a>: (<em>func</em>, <em>na_action=None</em>, <em>**kwargs</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/546/1*jlJqdVvstn9SxpjvbsL95g.png" /><figcaption>Example using applymap() in Pandas</figcaption></figure><p><strong>apply_rows</strong>:</p><p>Apply a row-wise user defined function. Not available in pandas</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.apply_rows">cuDF</a>: <em>(func, incols, outcols, kwargs, pessimistic_nulls=True, cache_key=None)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/493/1*TMdcnPZ87QFENVKAxThNOw.png" /><figcaption>Example from RAPIDS doc</figcaption></figure><p><strong>groupby().transform()</strong></p><p>Passes a function to the whole series and returns a series. Apply an aggregation, then broadcast the result to the group size. Shares builtin functions with agg(). Pandas version uses cython engine as default but can be switched to numba.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.core.groupby.groupby.groupby.transform#cudf.core.groupby.groupby.GroupBy.transform">cudf</a>: <em>groupby(categorical)[column function applied].transform(‘function’)</em><br><a href="https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.DataFrameGroupBy.transform.html?highlight=groupby%20transform#pandas.core.groupby.DataFrameGroupBy.transform">pandas</a>: (<em>func</em>, <em>*args</em>, <em>engine=None</em>, <em>engine_kwargs=None</em>, <em>**kwargs</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/694/1*v6GulqALiVMzweYtvL4JFA.png" /><figcaption>Using groupby().transform() on the DataFrame puts information per category within the rows</figcaption></figure><p><strong>items</strong>:</p><p>Iterate over column names and series pairs. Used to take chunks of the data, count the elements, and then reassemble the counts. <a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.items#cudf.DataFrame.items">cuDF</a>, <a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.items.html?highlight=items#pandas.DataFrame.items">pandas</a>. Better write up in Pydata.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/891/1*aq-eDXHbonA_-c1t4R7d7g.png" /><figcaption>items() example in Pydata.org done in cuDF</figcaption></figure><p><strong>map</strong> (Series):</p><p>Map values of Series according to input correspondence. Used for substituting each value in a Series with another value, that may be derived from a function, a dict or a Series. Values that are not found in the dict are converted to NaN. Default values in dicts are currently not supported. Think of map() as taking a dataframe, telling it the function you want to apply, and the inputs it needs. cuDF’s map() functionality may seem more limited than the Pandas version. This has to do with how cuDF store data and interacts with ‘object’ datatype.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.series.map">cuDF</a>: (<em>arg</em>, <em>na_action=None</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.Series.map.html?highlight=map#pandas.Series.map">pandas</a>: <em>(arg, na_action=None)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/863/1*p7rKhzBkK1iJPlw6esYB3Q.png" /><figcaption>Utilizing a dictionary to switch values in the DataFrame with map()</figcaption></figure><p>Be careful about what kinds of data you use on maps():</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/486/1*Kvr8LfcqpdeIlWqj_34vdg.png" /><figcaption>Using cupy.arange() to generate an array of numbers instead of inputting a list caused a complicated situation</figcaption></figure><p><strong>scatter_by_map</strong>:</p><p>Scatter to a list of dataframes. Uses map_index to determine the destination of each row of the original DataFrame. Not available in pandas. (Not tested)</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.scatter_by_map#cudf.DataFrame.scatter_by_map">cuDF</a>: <em>(map_index, map_size=None, keep_index=True, **kwargs)</em></p><p>assign binning values (<strong>cut, qcut</strong>):</p><p><strong>cut</strong>:</p><p>Bin values into discrete intervals. Can be used to convert continuous values into categorical. Is being debugged in cuDF as of writing.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.cut.html#cudf.cut">cuDF</a>: (<em>x</em>, <em>bins</em>, <em>right: </em><a href="https://docs.python.org/3/library/functions.html#bool"><em>bool</em></a><em> = True</em>, <em>labels=None</em>, <em>retbins: </em><a href="https://docs.python.org/3/library/functions.html#bool"><em>bool</em></a><em> = False</em>, <em>precision: </em><a href="https://docs.python.org/3/library/functions.html#int"><em>int</em></a><em> = 3</em>, <em>include_lowest: </em><a href="https://docs.python.org/3/library/functions.html#bool"><em>bool</em></a><em> = False</em>, <em>duplicates: </em><a href="https://docs.python.org/3/library/stdtypes.html#str"><em>str</em></a><em> = ‘raise’</em>, <em>ordered: </em><a href="https://docs.python.org/3/library/functions.html#bool"><em>bool</em></a><em> = True</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.cut.html?highlight=cut#pandas.cut">pandas</a>: (<em>x</em>, <em>bins</em>, <em>right=True</em>, <em>labels=None</em>, <em>retbins=False</em>, <em>precision=3</em>, <em>include_lowest=False</em>, <em>duplicates=’raise’</em>, <em>ordered=True</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/921/1*4QD8GpjnhGNFcv5paEoqng.png" /><figcaption>cut() created 4 categorical values or bins from a range of numbers</figcaption></figure><p><strong>qcut</strong>:</p><p>Quantile-based discretization function. Cut with percentages of the dataframe. Use qcut() to create a numerical categorical variable that separate the DataFrame to analyze. Not available in cuDF as of writing</p><p><a href="https://pandas.pydata.org/docs/reference/api/pandas.qcut.html#pandas.qcut">pandas</a>: <em>(x, q, labels=None, retbins=False, precision=3, duplicates=’raise’)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F1KpRBILQbkD6JKpPpq8Vw.png" /><figcaption>Categorizing the DataFrame into 25% sections</figcaption></figure><p>columns (<strong>columns, lis</strong>t):</p><p>Give the columns in the dataframe. Columns truncates, use list when you need the full rundown.</p><p><strong>columns</strong>:</p><p>Returns a tuple of columns. <a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.columns#cudf.DataFrame.columns">cuDF</a>, <a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.columns.html">pandas</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/641/1*vXmdFlIzgQGkLJpjkQjpng.png" /><figcaption>Using dataframe options you can specify when output truncates. In general use list when you want everything.</figcaption></figure><p><strong>list</strong>:</p><p>List method from Python applied to a Data Frame or Series</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/238/1*UwW4_BpkNWk1ZBytIIW0Tg.png" /><figcaption>List shows everything as a vertical output.</figcaption></figure><p>Combining dataframes (<strong>concat, join, merge</strong>).</p><p>Honestly, I use the <a href="https://pandas.pydata.org/docs/user_guide/merging.html">pydata.org guide</a> all the time.</p><p><strong>concat</strong>:</p><p>Concatenate DataFrames, Series, or Indices row-wise</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.concat.html#cudf-concat">cuDF</a>: (<em>objs</em>, <em>axis=0</em>, <em>join=’outer’</em>, <em>ignore_index=False</em>, <em>sort=None</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.concat.html?highlight=concat#pandas.concat">pandas</a>: (<em>objs</em>, <em>*</em>, <em>axis=0</em>, <em>join=’outer’</em>, <em>ignore_index=False</em>, <em>keys=None</em>, <em>levels=None</em>, <em>names=None</em>, <em>verify_integrity=False</em>, <em>sort=False</em>, <em>copy=True</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FJ-P-OzVfqJ1Ec6GmsHDeg.png" /><figcaption>Putting two DataFrames together vertically using concat()</figcaption></figure><p><strong>join</strong>:</p><p>Join columns with other DataFrame on index or on a key column. Usually used when putting together using an index. Join is often useful when the data files is unlabeled and don’t have any common columns but are using the same index.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.join#cudf.DataFrame.join">cuDF</a>: (<em>other</em>, <em>on=None</em>, <em>how=’left’</em>, <em>lsuffix=’’</em>, <em>rsuffix=’’</em>, <em>sort=False</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.join.html#pandas.DataFrame.join">pandas</a>: (<em>other</em>, <em>on=None</em>, <em>how=’left’</em>, <em>lsuffix=’’</em>, <em>rsuffix=’’</em>, <em>sort=False</em>, <em>validate=None</em>)</p><p><strong>merge</strong>:</p><p>Merge (GPU) DataFrame objects by performing a database-style join operation by columns or indexes. You are able to join using more than one column.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.merge">cuDF</a>: (<em>right</em>, <em>on=None</em>, <em>left_on=None</em>, <em>right_on=None</em>, <em>left_index=False</em>, <em>right_index=False</em>, <em>how=’inner’</em>, <em>sort=False</em>, <em>lsuffix=None</em>, <em>rsuffix=None</em>, <em>indicator=False</em>, <em>suffixes=(‘_x’, ‘_y’)</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.merge.html#pandas.DataFrame.merge">pandas</a>: (<em>right</em>, <em>how=’inner’</em>, <em>on=None</em>, <em>left_on=None</em>, <em>right_on=None</em>, <em>left_index=False</em>, <em>right_index=False</em>, <em>sort=False</em>, <em>suffixes=(‘_x’, ‘_y’)</em>, <em>copy=True</em>, <em>indicator=False</em>, <em>validate=None</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/730/1*MS2q5U9jK_wRaOcMSe1afQ.png" /><figcaption>Using the merge() function to combine two DataFrames with 2 identifying columns in common</figcaption></figure><p><strong>crosstab</strong>:</p><p>Compute cross tabulation of two (or more) factors. Computes a frequency table of the factors. Unless an array of values and an aggregation function are specified.</p><p>cuDF: No documentation available and some technical issues occur but it works for numerical data, not string/object. Use pandas crosstab instead.<br><a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.crosstab.html?highlight=crosstab#pandas.crosstab">pandas</a>: (<em>index</em>, <em>columns</em>, <em>values=None</em>, <em>rownames=None</em>, <em>colnames=None</em>, <em>aggfunc=None</em>, <em>margins=False</em>, <em>margins_name=’All’</em>, <em>dropna=True</em>, <em>normalize=False</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/560/1*I7RYcmaf01Sal4SHs9MvVg.png" /><figcaption>In the crosstab table, ‘a’ here is the y axis and the ‘b’ &amp; ‘c’ are the x axis.</figcaption></figure><p><strong>drop:</strong></p><p>Drop specified labels from rows or columns.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.drop#cudf.DataFrame.drop">cuDF</a>: (<em>labels=None</em>, <em>axis=0</em>, <em>index=None</em>, <em>columns=None</em>, <em>level=None</em>, <em>inplace=False</em>, <em>errors=’raise’</em>)<br><a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.drop.html?highlight=drop#pandas.DataFrame.drop">pandas</a>: (<em>labels=None</em>, <em>*</em>, <em>axis=0</em>, <em>index=None</em>, <em>columns=None</em>, <em>level=None</em>, <em>inplace=False</em>, <em>errors=’raise’</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*whomTM0Yr0LSXVvfrPbDsw.png" /><figcaption>Could’ve also used ‘inplace=True’ to save the dataframe with dropped columns</figcaption></figure><p>Dealing with duplicate data (<strong>duplicated, drop_duplicates</strong>)</p><p><strong>duplicated</strong>:</p><p>Return boolean Series denoting duplicate rows. Not available as of writing in cuDF.</p><p>cuDF: Unavailable<br><a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.duplicated.html">pandas</a>: (<em>subset=None</em>, <em>keep=’first’</em>)</p><p><strong>drop_duplicates</strong>:</p><p>Return DataFrame with duplicate rows removed, can optionally consider specified subset of columns.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.drop_duplicates">cuDF</a>: (<em>subset=None</em>, <em>keep=’first’</em>, <em>inplace=False</em>, <em>ignore_index=False</em>)<br><a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.drop_duplicates.html?highlight=drop_duplicates#pandas.DataFrame.drop_duplicates">pandas</a>: <em>(subset=None, *, keep=’first’, inplace=False, ignore_index=False)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/871/1*uwiGzRjmayNP6RCCSex_5Q.png" /><figcaption>Only one play from each game is retained. Shows that there was 122 games played in 8 weeks.</figcaption></figure><p><strong>explode</strong>:</p><p>Transform each element of a list-like to a row, replicating index values. Basically if a value is a series or list, break down the list into separate values.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.explode#cudf.DataFrame.explode">cuDF</a>: (<em>column</em>, <em>ignore_index=False</em>)<br><a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.explode.html?highlight=explode#pandas.DataFrame.explode">pandas</a>: (<em>column</em>, <em>ignore_index=False</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/830/1*v8TiXR6ubFj4pkE-Ai8r6g.png" /><figcaption>Three categories in DataFrame converted into two using explode(). Note the index remaining the same.</figcaption></figure><p>Finding values (<strong>eval</strong>, <strong>filter</strong>, <strong>query, groupby.get_group</strong>)</p><p><strong>eval</strong>:</p><p>Evaluate a string describing operations on DataFrame columns. eval and query are closely connected in functionality by sharing kwargs.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.eval#cudf.DataFrame.eval">cuDF</a>: (<em>expr: </em><a href="https://docs.python.org/3/library/stdtypes.html#str"><em>str</em></a>, <em>inplace: </em><a href="https://docs.python.org/3/library/functions.html#bool"><em>bool</em></a><em> = False</em>, <em>**kwargs</em>)<br><a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.eval.html?highlight=eval#pandas.DataFrame.eval">pandas</a>: (<em>expr</em>, <em>*</em>, <em>inplace=False</em>, <em>**kwargs</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/886/1*Gq_cFNEOH8VwxUi3By7XdA.png" /><figcaption>Created a DataFrame from a subset of columns, used eval() to create a new column &amp; custom data then dropped missing values. Based on what we’ve found bad things happen to QBs 6% of plays. But that’s including all the run plays. I found the number to be really low…</figcaption></figure><p><strong>filter</strong>:</p><p>Subset the dataframe rows or columns according to the specified index labels. Does not filter a dataframe on its contents, applied to the labels of the index. Currently unavailable in cuDF.</p><p><a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.filter.html?highlight=filter#pandas.DataFrame.filter">pandas</a>: (<em>items=None</em>, <em>like=None</em>, <em>regex=None</em>, <em>axis=None</em>)</p><p><strong>query</strong>:</p><p>Query with a boolean expression using Numba to compile a GPU kernel. As of this writing, cuDF’s query is more limited than the pandas version by not dealing with categorical or string/object variables. You will get a error saying: “<em>query only supports numeric, datetime, timedelta, or bool dtypes.”</em> To deal with this, convert the categorical variable into ordinal values or use one-hot encoding. Query and eval are closely connected in functionality by sharing kwargs. Please be aware that in cuDF, query doesn’t have the ‘inplace’ option. to make changes to the dataframe, you have to specify by writing ‘df = df.query(expr)’.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.query">cuDF</a>: (<em>expr</em>, <em>local_dict=None</em>)<br><a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.query.html?highlight=query#pandas.DataFrame.query">pandas</a>: <em>(expr, *, inplace=False, **kwargs)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nhgkQVvUbttlyUSE0FD2Mg.png" /><figcaption>Simple query() filtering all the acceleration values over 4</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/315/1*7rAdf6l2aNECFxytLwXpng.png" /><figcaption>query() using index values</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/398/1*pN_l8ZgfCZydABBYoUU7BQ.png" /><figcaption>Info function of query() can be used to see value counts leftover</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/985/1*IbDrBjkWKI32kEX-pr91uw.png" /><figcaption>query() can use multiple criteria at the same time</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/985/1*3eWx0ZZ_R6Xa-Rrqsh7vDQ.png" /><figcaption>query() can even use criteria using values outside of the DataFrame using the ‘@’</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*wXTQ1g47PgU71hSqnPhBoA.png" /><figcaption>cuDF’s query() doesn’t use objects or string variables, here each team is given a number identifier</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/624/1*jAcSSdyGi6hAoeDyu1XQfw.png" /><figcaption>Using the dictionary, string team values were replaced with numerical values, and query() can be used</figcaption></figure><p><strong>groupby.get_group: (<em>name</em>, <em>obj=None</em>)</strong></p><p>Construct DataFrame from group with provided name. Not available in cuDF as of writing. I’m including it here just to show an alternative to query available to Pandas users.</p><p><a href="https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.GroupBy.get_group.html?highlight=get_group#pandas.core.groupby.GroupBy.get_group">pandas</a>: <em>(name, obj=None)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/596/1*VEga6KyJzjZdH8h2y9Snqw.png" /><figcaption>DataFrame filtered down to Pass Blocking plays where Offensive Linemen are in standard “Pass Pro” or “PP”</figcaption></figure><p>Frequency (<strong>count, unique, nunique, value_counts, groupby.ngroups</strong>):</p><p><strong>count</strong>:</p><p>Count non-NA cells for each column or row.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.count#cudf.DataFrame.count">cuDF</a>: (<em>axis=0</em>, <em>level=None</em>, <em>numeric_only=False</em>, <em>**kwargs</em>)<br><a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.count.html?highlight=count#pandas.DataFrame.count">pandas</a>:<em>(axis=0, level=None, numeric_only=False)</em></p><p><strong>unique</strong>:</p><p>Return unique values</p><p><strong>nunique</strong>:</p><p>Count number of distinct elements in specified axis. Return Series with number of distinct elements. Can ignore NaN values.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.nunique#cudf.DataFrame.nunique">cuDF</a>: (<em>axis=0</em>, <em>dropna=True</em>)<br><a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.nunique.html?highlight=nunique#pandas.DataFrame.nunique">pandas</a>: <em>(axis=0, dropna=True)</em></p><p><strong>value_counts</strong>:</p><p>Return a Series containing counts of unique rows in the DataFrame</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.value_counts#cudf.DataFrame.value_counts">cuDF</a>: (<em>subset=None</em>, <em>normalize=False</em>, <em>sort=True</em>, <em>ascending=False</em>, <em>dropna=True</em>)<br><a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.value_counts.html?highlight=value_counts#pandas.DataFrame.value_counts">pandas</a>: <em>(subset=None, normalize=False, sort=True, ascending=False, dropna=True)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/368/1*NoHPs8Xh6HiA4a--kouGhA.png" /><figcaption>The difference between unique() and nunique() was that one Null value</figcaption></figure><p><strong>groupby.ngroup:</strong></p><p>Number each group from 0 to the number of groups — 1. Usually arranged alphabetically. This is the enumerative complement of cumcount. Note that the numbers given to the groups match the order in which the groups would be seen when iterating over the groupby object, not the order they are first observed. This is a fast way of coding categorical values into numerical ones. This function is analogous to factorize() applied to groupby()</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.core.groupby.groupby.groupby.ngroup">cuDF</a>: (<em>ascending=True</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.GroupBy.ngroup.html?highlight=ngroup">pandas</a>: (<em>ascending=True</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/518/1*-3B8-spQue8Lj6BcCn2UBA.png" /><figcaption>In cuDF, query only supports numeric, datetime, timedelta, or bool dtypes. Use ngroup() to code string or object datatypes into numerical.</figcaption></figure><p><strong>groupby.ngroups:</strong></p><p>Pandas only as of this writing. Use this function as a way of finding out how many unique values a column holds based on other categories. It’s a faster, simpler alternative to using query() + unique(). It can also be used exactly like groupby.ngroup().</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/527/1*kLmYdqQ0nF3pcGpsELIzjg.png" /><figcaption>Let’s look at QB drop backs. There were 8 different types and ngroup was used to code them.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/517/1*y19nSnIZGuiZTI-qArHjbQ.png" /><figcaption>Numerically coded, now analysis can be done using QB drop backs, most of which were ‘Traditional’ (6)</figcaption></figure><p>Coding data from categorical to numerical: (<strong>factorize, get_dummies)</strong>:</p><p><strong>“category” data type:</strong></p><p>cuDF and Pandas got can compute DataFrame columns with datatype assigned ‘object’ as coded numbers while still showing the original input. To do this, change the data type from “object” to “category.” Advantages of doing this that it is easy, fast, and allows the data to be included in statistical processes. The downside is that the user has to figure out the numbering system (usually sorted alphabetically) later.</p><p><strong>factorize</strong>:</p><p>Encode the object as an enumerated type or categorical variable. It creates two outputs, a list of encoded values and a StringIndex or code book that shows what the numbered codes represent. This method is useful for obtaining a numeric representation of an array when all that matters is identifying distinct values. factorize() is available as both a top-level function. cuDF has ‘na_sentinel’ and Pandas has ‘use_na_sentinel’, they are equivalents. If True the sentinel -1 will be used for NaN values. If False, NaN values will be encoded as non-negative integers and will not drop the NaN from the uniques of the values. This is a good function to use for quickly coding categorical values into ordinal values. An alternative to factorize() is use groupby.ngroup().</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.series.factorize#cudf.Series.factorize">cudf</a>: <em>(na_sentinel=-1)</em><br><a href="https://pandas.pydata.org/docs/reference/api/pandas.factorize.html?highlight=factorize#pandas.factorize">pandas</a>: (<em>values</em>, <em>sort=False</em>, <em>na_sentinel=[OLD Avoid]</em>, <em>use_na_sentinel=[default=True]</em>, <em>size_hint=None</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/670/1*jHSjSnw53UIE7pmlvyaJCw.png" /><figcaption>factorize() creates two outputs: an array and a StringIndex. Save them as two separate lists then add a column or replace the original column with the array. Save the StringIndex for interpreting the data afterwards</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/674/1*yUXQLUJuPKBiOnb4i30k_g.png" /><figcaption>Where players lined up in the play, code book has unique values column ranked alphabetically -1.</figcaption></figure><p><strong>get_dummies</strong>:</p><p>Returns a dataframe whose columns are the one hot encodings of all columns in the original dataframe or dataframe column</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.get_dummies.html">cudf</a>: (<em>df</em>, <em>prefix=None</em>, <em>prefix_sep=’_’</em>, <em>dummy_na=False</em>, <em>columns=None</em>, <em>cats=None</em>, <em>sparse=False</em>, <em>drop_first=False</em>, <em>dtype=’uint8&#39;</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.get_dummies.html?highlight=get_dummies#pandas.get_dummies">pandas</a>: (<em>data</em>, <em>prefix=None</em>, <em>prefix_sep=’_’</em>, <em>dummy_na=False</em>, <em>columns=None</em>, <em>sparse=False</em>, <em>drop_first=False</em>, <em>dtype=None</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/466/1*z9EOS3NMkwvWjNeHsjdK6g.png" /><figcaption>get_dummies() is crucial in dealing with categorical variables</figcaption></figure><p><strong>groupby</strong>:</p><p>Group using a mapper or by a Series of columns. A groupby operation involves some combination of splitting the object, applying a function, and combining the results. This can be used to group large amounts of data and compute operations on these groups. Can be <em>very different</em> to use compared to pandas so I’m linking the RAPIDS doc on the <a href="https://docs.rapids.ai/api/cudf/stable/api_docs/groupby.html">groupby</a> functions a second time. Unlike Pandas, cuDF uses sort=False by default to achieve better performance, which does not guarantee any particular group order in the result.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.groupby#cudf.DataFrame.groupby">cuDF</a>: (<em>by=None</em>, <em>axis=0</em>, <em>level=None</em>, <em>as_index=True</em>, <em>sort=False</em>, <em>group_keys=False</em>, <em>squeeze=False</em>, <em>observed=False</em>, <em>dropna=True</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.groupby.html?highlight=groupby#pandas.DataFrame.groupby">pandas</a>: <em>(by=None, axis=0, level=None, as_index=True, sort=True, group_keys=_NoDefault.no_default, observed=False, dropna=True)</em></p><p>Examples of groupby() done after reading <a href="https://sonery.medium.com/">Soner Yıldırım</a>’s excellent <a href="https://towardsdatascience.com/all-about-pandas-groupby-explained-with-25-examples-494e04a8ef56">post</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/926/1*EiTenTqByYLAcczrYzASiQ.png" /><figcaption>Grouping the DataFrame by where players lined up, making a table of the mean stats for bad plays, and then sorting that table by the worst play: allowing the QB to be sacked</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/921/1*fIbIs_QesJNbW4fUhnIGDg.png" /><figcaption>Same as the example above but using agg()</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/674/1*XE3BOItYifpBBlce3YRhvw.png" /><figcaption>Nonsensical example of using groupby() with agg(). cuDF doesn’t sort without being told to.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/702/1*Foh536IdicjkheNq9t09nA.png" /><figcaption>Doing a groupby of (1st) Player’s official position, then breaking further down to the type of block used on the play. The results given are the min, mean, and max, in that order. Screencap of a portion of the readout.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/525/1*sokVwqQkl6q4UGoPOfRlXA.png" /><figcaption>groupby() using the ‘dropna=True’ to see if the means change from sorting out missing values</figcaption></figure><p><em>The following groupby() examples are Pandas-only:</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/542/1*1goV0YQU4UUYiQHBbBZqUg.png" /><figcaption>DataFrame using groupby() on two categories and agg() for custom columns</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/521/1*Bf685d1aZm43t79JGk6iXQ.png" /><figcaption>grioupby() used with as_index() set to False. DataFrame now has it’s own indexing, which now intersperses categories with missing values. In this case it shows players on the defensive side</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/541/1*knSwlNAZeT-FUAfoo4K7SQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/542/1*JnhvwL2HlTwuo-H2HbMgTA.png" /><figcaption>Grouping by position, then block type, the table is sorted by the 10 highest sack rates using nlargest(). I’m guesing a QB gave the ball to another player to throw then tried to block a rusher. It didn’t work. The same thing done to the right, using sort_values() and head()</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/741/1*M1K1c0AiCCAljddnFyhMdA.png" /><figcaption>groupby() with unique() yields a table with all the unique values in each grouping. Saving this table as a DataFrame can be very useful for organizing &amp; coding the data and using explode() later on.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/469/1*bqREv0fghDw-u_4sCv8fGg.png" /><figcaption>groupby() with the nunique() to find the number of unique values after dividing into groups</figcaption></figure><p>Quick dataframe information (<strong>describe, head &amp; tail, info, len, shape</strong>)</p><p><strong>describe</strong>:</p><p>Get descriptive statistics. Will truncate with many columns, use specific functions if you need data on all the columns.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.describe#cudf.DataFrame.describe">cuDF</a>: (<em>percentiles=None</em>, <em>include=None</em>, <em>exclude=None</em>, <em>datetime_is_numeric=False</em>)<br><a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.describe.html?highlight=describe#pandas.DataFrame.describe">pandas</a>: (<em>percentiles=None</em>, <em>include=None</em>, <em>exclude=None</em>, <em>datetime_is_numeric=False</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/259/1*Jo1S0VfXrrP5eqFB4jlXHg.png" /></figure><p><strong>head &amp; tail</strong>:</p><p>get the first (or last, default = 5) rows of the dataframe.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dc99ZfPh-IaXzA7h8ijMKw.png" /></figure><p><strong>info</strong>:</p><p>This method prints information about a DataFrame including the index dtype and column dtypes, non-null values and memory usage.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.info#cudf.DataFrame.info">cuDF</a>: (<em>verbose=None</em>, <em>buf=None</em>, <em>max_cols=None</em>, <em>memory_usage=None</em>, <em>null_counts=None</em>)<br><a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.info.html?highlight=info#pandas.DataFrame.info">pandas</a>: (<em>verbose=None</em>, <em>buf=None</em>, <em>max_cols=None</em>, <em>memory_usage=None</em>, <em>show_counts=None</em>, <em>null_counts=None</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/478/1*O_J-SrQPulo3zbj5uVF5yw.png" /><figcaption>Knowing how much VRAM a DataFrame is taking is crucial</figcaption></figure><p><strong>len</strong>:</p><p>Count how many values in a set/series/dict, applied to a dataframe gives rows. Used outside of the dataframe (example: len(df.columns))</p><p><strong>shape</strong>:</p><p>Returns a tuple representing the dimensionality of the DataFrame.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/136/1*LA_O1JiIeFhxNkc-YISUEg.png" /><figcaption>shape() is used often just to check how the large the DataFrame is and if join/merge/concat() worked correctly</figcaption></figure><p>Directly access data (<strong>at</strong>, <strong>get_loc</strong>, <strong>insert, iloc, isin, loc, xs</strong>)</p><p><strong>insert</strong>:</p><p>Add a column to DataFrame at the index specified by loc.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.insert#cudf.DataFrame.insert">cuDF</a>: (<em>loc</em>, <em>name</em>, <em>value</em>, <em>nan_as_null=None</em>)<br><a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.insert.html?highlight=insert#pandas.DataFrame.insert">pandas</a>: <em>(loc, column, value, allow_duplicates={bool, optional, default lib.no_default</em><strong><em>}</em></strong><em>)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/932/1*tEwpBXlkD4WCEwIrOag9oA.png" /><figcaption>New column with values from a UDF shows more than half the time DEs outplayed Tackles</figcaption></figure><p><strong>get_loc</strong>:</p><p>Get integer location, slice or boolean mask for requested label. For indexes, find what index value a column or row is. Necessary when a DataFrame is converted into a numpy array or other formats where column/row identifiers are stripped out.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.index.get_loc#cudf.Index.get_loc">cudf</a>: (key, method=None, tolerance=None)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.Index.get_loc.html">pandas</a>: (key, method=None, tolerance=None)</p><p><strong>iloc</strong>:</p><p>Select values by position, using the index or another integer based identifier. May also be used with a boolean array. Use like (x,y) coordinate data [row, column]. <a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.iloc#cudf.DataFrame.iloc">cuDF</a>; <a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.iloc.html?highlight=iloc#pandas.DataFrame.iloc">pandas</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/612/1*-lRsYfgXcequtnjFdNM0sA.png" /><figcaption>iloc() is not as often used as loc() but can be important when locating outliers</figcaption></figure><p><strong>isin</strong>:</p><p>Whether each element in the DataFrame is contained in values. <a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.isin#cudf.DataFrame.isin">cuDF</a>; <a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.isin.html?highlight=isin#pandas.DataFrame.isin">pandas</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/391/1*sJjiXBcrM67-NNU4QZ1hnw.png" /><figcaption>isin() can be used further for replacing or changing values</figcaption></figure><p><strong>loc</strong> <strong>/ at</strong>:</p><p>Select rows and columns by label or boolean mask. <a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.loc#cudf.DataFrame.loc">cuDF</a>; <a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.loc.html?highlight=loc#pandas.DataFrame.loc">pandas</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/484/1*r7ADawChusuS-FTh30Wiwg.png" /><figcaption>Used describe() to see how many values were left after using loc()</figcaption></figure><p><strong>xs</strong>:</p><p>Return cross-section from the Series/DataFrame, uses key argument to select data at a particular level of a MultiIndex.</p><p>cuDF: [Unavailable as of writing]<br><a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.xs.html?highlight=xs#pandas.DataFrame.xs">pandas</a>: (key, axis=0, level=None, drop_level=True)</p><p>Missing Values (<strong>dropna, fillna, isna, isnull, isnull().sum, nans_to_nulls</strong>, <strong>notna, notnull</strong>)</p><p><strong>dropna</strong>:</p><p>Drop rows (or columns) containing nulls from a Column.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.dropna#cudf.DataFrame.dropna">cuDF</a>: (<em>axis=0</em>, <em>how=’any’</em>, <em>thresh=None</em>, <em>subset=None</em>, <em>inplace=False</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.dropna.html?highlight=dropna#pandas.DataFrame.dropna">pandas</a>: <em>(*, axis=0, how= </em>{‘any’, ‘all’}<em>, thresh= {int, optional}, subset=None, inplace=False)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/390/1*YzeRKWpSz8oBG2XCjX-AjQ.png" /><figcaption>nflId had 48,614 missing values which were removed</figcaption></figure><p><strong>fillna:</strong></p><p>Fill null values with value or specified method. Same as using <strong>backfill</strong> when using<strong> bfill </strong>method.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.fillna#cudf.DataFrame.fillna">cuDF</a>: (<em>value=None</em>, <em>method=None</em>, <em>axis=None</em>, <em>inplace=False</em>, <em>limit=None</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.fillna.html?highlight=fillna#pandas.DataFrame.fillna">pandas</a>: <em>(value=None, *, method=None, axis=None, inplace=False, limit=None, downcast=None)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/388/1*okoSx6PKbb5CETGmV6l8NQ.png" /><figcaption>fillna() is good to use when using numerical coding, ‘9999999’ here is used as missing value</figcaption></figure><p><strong>interpolate</strong>:</p><p>Interpolate data values between some points. Currently, only ‘linear` is supported, which ignores the index and treats each value as equally spaced.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.interpolate#cudf.DataFrame.interpolate">cuDF</a>: (<em>method=’linear’</em>, <em>axis=0</em>, <em>limit=None</em>, <em>inplace=False</em>, <em>limit_direction=None</em>, <em>limit_area=None</em>, <em>downcast=None</em>, <em>**kwargs</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.interpolate.html">pandas</a>: <em>(method=’linear’, *, axis=0, limit=None, inplace=False, limit_direction=None, limit_area=None, downcast=None, **kwargs)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/448/1*SgQflcI6fF2QPFwKn1tW4Q.png" /><figcaption>interpolate() is a good way of generating new values for indexes and identifiers.</figcaption></figure><p><strong>isna / isnull:</strong></p><p>Identify missing values. <a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.isna#cudf.DataFrame.isna">cuDF</a>, <a href="https://pandas.pydata.org/docs/reference/api/pandas.isna.html?highlight=isna#pandas.isna">pandas</a>; isnull: <a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.isnull#cudf.DataFrame.isnull">cuDF</a>; <a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.isnull.html?highlight=isnull#pandas.DataFrame.isnull">pandas</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/368/1*wORXcC3_djYlmc8hPESCDg.png" /><figcaption>isna().sum() is a good way of finding how much of a problem missing values are.</figcaption></figure><p><strong>notna / notnull:</strong></p><p>Identify non-missing values. <a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.notna#cudf.DataFrame.notna">cuDF</a>, <a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.notna.html?highlight=notna#pandas.DataFrame.notna">pandas</a>; notnull: <a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.notnull#cudf.DataFrame.notnull">cuDF</a>; <a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.notnull.html?highlight=notnull#pandas.DataFrame.notnull">pandas</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/364/1*uPatkgoMlB7fL3ONu-88nw.png" /><figcaption>notna() tells you how many datapoints aren’t missing values</figcaption></figure><p>Pivot tables <strong>(melt, pivot, pivot_table, stack, unstack, wide_to_long):</strong></p><p><strong>melt</strong>:</p><p>Unpivots a DataFrame from wide format to long format, optionally leaving identifier variables set. Also wide_to_long() can be used in pandas.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.melt.html">cuDF</a>: (<em>frame</em>, <em>id_vars=None</em>, <em>value_vars=None</em>, <em>var_name=None</em>, <em>value_name=’value’</em>, <em>col_level=None</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.melt.html?highlight=melt#pandas.DataFrame.melt">pandas</a>: <em>(id_vars=None, value_vars=None, var_name=None, value_name=’value’, col_level=None, ignore_index=True)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/414/1*NuwDH82rUo3NuaJH6fsHKA.png" /><figcaption>Created a DataFrame then used melt() to isolate ‘tax’ values</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/472/1*jcfA7-78gKLkBhbd1pbQZw.png" /><figcaption>When you add to ‘value_vars’, melt() lengthens the DataFrame</figcaption></figure><p><strong>pivot</strong>:</p><p>Return reshaped DataFrame organized by the given index and column values</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.pivot.html">cuDF</a>: (<em>data</em>, <em>index=None</em>, <em>columns=None</em>, <em>values=None</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.pivot.html?highlight=pivot#pandas.DataFrame.pivot">pandas</a>:<em>(*, index=None, columns=None, values=None)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/612/1*NwZVwA8ahbRYMYvdhKrSXA.png" /><figcaption>Large DataFrame created for illustration</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/372/1*8dmlTVEK_e9IrImGe9wCAw.png" /><figcaption>Pivot table created using an ‘index’ and ‘columns’. The third column of values makes up the values on the chart to create a 3 dimensional table.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/513/1*PkiH0QTIaQvowQky-CJIpQ.png" /><figcaption>If you put the ‘columns’ input under ‘[ ]’, then the pivot table expands and becomes 2 dimensional</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/995/1*W9__jPLY4vINAGewfGahCg.png" /><figcaption>Multiple values under ‘columns’ greatly expands the 2 dimensional pivot table</figcaption></figure><p><strong>pivot_table</strong>:</p><p>Create a spreadsheet-style pivot table as a DataFrame. The levels in the pivot table will be stored in MultiIndex objects (hierarchical indexes) on the index and columns. More visually appealing than pivot().</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.pivot_table.html">cuDF</a>: (<em>data</em>, <em>values=None</em>, <em>index=None</em>, <em>columns=None</em>, <em>aggfunc=’mean’</em>, <em>fill_value=None</em>, <em>margins=False</em>, <em>dropna=None</em>, <em>margins_name=’All’</em>, <em>observed=False</em>, <em>sort=True</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.pivot_table.html#pandas.DataFrame.pivot_table">pandas</a>: (<em>values=None</em>, <em>index=None</em>, <em>columns=None</em>, <em>aggfunc=’mean’</em>, <em>fill_value=None</em>, <em>margins=False</em>, <em>dropna=True</em>, <em>margins_name=’All’</em>, <em>observed=False</em>, <em>sort=True</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/907/1*o8MJzQGmG_LiI_uDxNFTCA.png" /><figcaption>DataFrame created and then turned into a pivot table</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/886/1*VWj7X6I_mLOuZysVjrOvBQ.png" /><figcaption>Same as above but ‘fill_values’ was used to replace null values with 0.</figcaption></figure><p><strong>stack</strong>:</p><p>Stack the prescribed level(s) from columns to index. Return a reshaped Series. Basically take away a column while still retaining the data.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.stack#cudf.DataFrame.stack">cudf</a>: (<em>level=-1</em>, <em>dropna=True</em>)<br><a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.stack.html">pandas</a>: (<em>level=- 1</em>, <em>dropna=True</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/168/1*NyeIE_f7BKXPlCxBYn6iiA.png" /><figcaption>stack() could be useful in organizing wide DataFrames</figcaption></figure><p><strong>unstack</strong>:</p><p>Pivot one or more levels of the (necessarily hierarchical) index labels.Pivots the specified levels of the index labels of df to the innermost levels of the columns labels of the result. unstack() doesn’t change the pivot table back to a regular DataFrame, it changes the orientation of the data.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.unstack.html#cudf-unstack">cuDF</a>: (<em>df</em>, <em>level</em>, <em>fill_value=None</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.unstack.html#pandas.DataFrame.unstack">pandas</a>: (<em>level=- 1</em>, <em>fill_value=None</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/290/1*s2FGAzUmj9FQolhIzi72dQ.png" /><figcaption>Change the ‘level’ argument to change the orientation of the data presented even further.</figcaption></figure><p><strong>wide_to_long</strong>:</p><p>Wide panel to long format. Pandas only melt() alternative.<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.wide_to_long.html#pandas.wide_to_long">pandas</a>: (<em>df</em>, <em>stubnames</em>, <em>i</em>, <em>j</em>, <em>sep=’’</em>, <em>suffix=’\\d+’</em>)</p><p><strong>rank</strong>:</p><p>Compute numerical data ranks (1 through n) along axis. By default, equal values are assigned the average of the ranks of those values.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.rank#cudf.DataFrame.rank">cuDF</a>: (<em>axis=0</em>, <em>method=’average’</em>, <em>numeric_only=None</em>, <em>na_option=’keep’</em>, <em>ascending=True</em>, <em>pct=False</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.rank.html?highlight=rank#pandas.DataFrame.rank">pandas</a>: <em>(axis=0, method=’average’, numeric_only=_NoDefault.no_default, na_option=’keep’, ascending=True, pct=False)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/377/1*P6q2DA9IoOEgHVtyiVoYCQ.png" /><figcaption>rank() used to find the best plays in terms of yardage.</figcaption></figure><p><strong>rename</strong>:</p><p>Alter column and index labels. Function / dict values must be unique (1-to-1). Labels not contained in a dict / Series will be left as-is.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.rename#cudf.DataFrame.rename">cuDF</a>: (<em>mapper=None</em>, <em>index=None</em>, <em>columns=None</em>, <em>axis=0</em>, <em>copy=True</em>, <em>inplace=False</em>, <em>level=None</em>, <em>errors=’ignore’</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.rename.html?highlight=rename#pandas.DataFrame.rename">pandas</a>: <em>(mapper=None, *, index=None, columns=None, axis=None, copy=None, inplace=False, level=None, errors=’ignore’)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/766/1*0iLcdjYanxHGmopGVGRXIw.png" /></figure><p><strong>resample </strong>(asfreq):</p><p>Convert time series to specified frequency. Very necessary when working with time series data. Pandas’ asfreq() function is not available in cuDF, use resample() instead. As of this writing I had some problems using the apply function with resample, am looking into it.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.resample">cuDF</a>: (<em>rule</em>, <em>axis=0</em>, <em>closed=None</em>, <em>label=None</em>, <em>convention=’start’</em>, <em>kind=None</em>, <em>loffset=None</em>, <em>base=None</em>, <em>on=None</em>, <em>level=None</em>, <em>origin=’start_day’</em>, <em>offset=None</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.resample.html?highlight=resample#pandas.DataFrame.resample">pandas</a>: (<em>rule</em>, <em>axis=0</em>, <em>closed=None</em>, <em>label=None</em>, <em>convention=’start’</em>, <em>kind=None</em>, <em>loffset=None</em>, <em>base=None</em>, <em>on=None</em>, <em>level=None</em>, <em>origin=’start_day’</em>, <em>offset=None</em>, <em>group_keys=_NoDefault.no_default</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/552/1*P_V9kE-0k_qsxZdIOu6Idg.png" /><figcaption>Created a DataFrame containing time series minute bins. Then resampled the frequency into 15 min bins</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/244/1*tGsmy9J3AMa52M6P1PoXuA.png" /><figcaption>If you resample upwards, cuDF will create the time data values but leave the values black. User should fill those empty values using fillna() or interpolate()</figcaption></figure><p>Exchanging values (<strong>mask</strong>, <strong>replace</strong>, <strong>where</strong>)</p><p><strong>mask</strong>:</p><p>Replace values where the condition is True. When you want to use change values using boolean methods, can be chained using ‘and.’</p><p><a href="https://docs.rapids.ai/api/cudf/legacy/api_docs/api/cudf.series.mask">cuDF</a>: <em>(cond, other=None, inplace=False)</em><br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.mask.html">pandas</a>:<em> (cond, other=nan, *, inplace=False, axis=None, level=None)</em></p><p><strong>replace</strong>:</p><p>Replace values given in ‘to_replace’ with alternate value</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.replace#cudf.DataFrame.replace">cuDF</a>: (<em>to_replace=None</em>, <em>value=None</em>, <em>inplace=False</em>, <em>limit=None</em>, <em>regex=False</em>, <em>method=None</em>)<br><a href="https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.replace.html">pandas</a>: <em>(to_replace=None, value=_NoDefault.no_default, *, inplace=False, limit=None, regex=False, method={‘pad’, ‘ffill’, ‘bfill’})</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/651/1*j4Vwp-N_4OA7IWUsQ9Ki5A.png" /><figcaption>I’m a Patriots fan.</figcaption></figure><p><strong>where</strong>:</p><p>Where condition is True, keep the original value. Where False, replace with corresponding value from other.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.where#cudf.DataFrame.where">cuDF</a>: (<em>cond</em>, <em>other=None</em>, <em>inplace=False</em>)<br><a href="https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.where.html">pandas</a>: <em>(cond, other={scalar, Series/DataFrame, or callable}, *, inplace=False, axis=None, level=None, errors=’raise’, try_cast=_NoDefault.no_default)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/662/1*lE7_yS9IMA7N7hJ4vkdUow.png" /><figcaption>where() is good way of exchanging values</figcaption></figure><p>Indexing (<strong>index, MultiIndex, names, reset_index, reindex, reindex_like, set_index, sort_index, truncate</strong>).</p><p>Not too many examples will be provided for this section because index functions are pretty niche, until you need them. Fantastic write up from PyData <a href="https://pandas.pydata.org/docs/user_guide/indexing.html">here</a>. As stupid as it sounds, you’ll find yourself messing with indexes much more than you’d ever expect.</p><p><strong>index</strong>:</p><p>The basic object storing row labels for all cuDF objects. The options for manipulating dataframe index using cuDF is extensive, especially when combining dataframes or data cleaning. Its worth a look at the official documents.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.index#cudf-index">cuDF</a>: (<em>data=None</em>, <em>dtype=None</em>, <em>copy=False</em>, <em>name=None</em>, <em>tupleize_cols=True</em>, <em>nan_as_null=True</em>, <em>**kwargs</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.index.html?highlight=index#pandas.DataFrame.index">pandas</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/752/1*RWtmnYWDkPk7UUkqnUFnsA.png" /><figcaption>Not the most useful output but could help when hunting down missing values</figcaption></figure><p><strong>MultiIndex</strong>:</p><p>A multi-level or hierarchical index. Provides N-Dimensional indexing into Series and DataFrame objects. Good post on advanced uses for MultiIndex / advanced indexing <a href="https://pandas.pydata.org/docs/user_guide/advanced.html">here</a>.</p><p><a href="https://docs.rapids.ai/api/cudf/nightly/api_docs/api/cudf.multiindex"><em>cuDF</em></a><em>: </em>(<em>levels=None</em>, <em>codes=None</em>, <em>sortorder=None</em>, <em>names=None</em>, <em>dtype=None</em>, <em>copy=False</em>, <em>name=None</em>, <em>**kwargs</em>)<em><br></em><a href="https://pandas.pydata.org/docs/reference/api/pandas.MultiIndex.html#pandas.MultiIndex"><em>pandas</em></a><em>: (levels=None, codes=None, sortorder=None, names=None, dtype=None, copy=False, name=None, verify_integrity=True)</em></p><p><strong>index.names</strong></p><p>Rename the index in a DataFrame using index.names function. I find naming the index and using reset_index() as good practice during data filtering, merging/joining, and modification. Remember to drop the old index if it’s not relevant to the new DataFrame.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/839/1*lgm1fSUuqp9WZgUWjcffxg.jpeg" /><figcaption>Let’s look at the 2022 Rookie Running Backs. After uploading a database on all the Rookies, and specifying relevant columns, I sorted for Rushing Yards</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/722/1*44LMMu_200I1bptb7_9zlQ.jpeg" /><figcaption>The DataFrame’s index reset to the new sort. The first index value is moved from ‘0’ to ‘1’</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/818/1*-4vH9xTiXfKwmIN24wojwQ.jpeg" /><figcaption>Changed the name of the index to ‘Rushing Rank’ and called for the top 10 Rookie rushers in the 2022 season.</figcaption></figure><p><strong>reset_index</strong>:</p><p>Reset the index of the DataFrame, or a level of it.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.reset_index#cudf.DataFrame.reset_index">cuDF</a>: (<em>level=None</em>, <em>drop=False</em>, <em>inplace=False</em>, <em>col_level=0</em>, <em>col_fill=’’</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.reset_index.html?highlight=reset_index#pandas.DataFrame.reset_index">pandas</a>: <em>(level=None, *, drop=False, inplace=False, col_level=0, col_fill=’’, allow_duplicates= {bool, optional, default lib.no_default}, names=None)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/735/1*-Sh4Nh-cphkKUC1Qlw24EQ.png" /><figcaption>Taking a random sample of 1000 data points from the entire DataFrame.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/773/1*TISxIVsqpDNwgPQ42fITuA.png" /><figcaption>reset_index() is used here to clean the sample DataFrame and used to create randomized DataFrames</figcaption></figure><p><strong>reindex:</strong></p><p>Conform DataFrame to new index. Places NA/NaN in locations having no value in the previous index. A new object is produced unless the new index is equivalent to the current one and copy=False. I mostly use this to reorder DataFrame columns &amp; rows alphabetically.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.reindex#cudf.DataFrame.reindex">cuDF</a>: (<em>labels=None</em>, <em>index=None</em>, <em>columns=None</em>, <em>axis=None</em>, <em>method=None</em>, <em>copy=True</em>, <em>level=None</em>, <em>fill_value=&lt;NA&gt;</em>, <em>limit=None</em>, <em>tolerance=None</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.reindex.html">pandas</a>: (<em>labels=None</em>, <em>index=None</em>, <em>columns=None</em>, <em>axis=None</em>, <em>method=None</em>, <em>copy=None</em>, <em>level=None</em>, <em>fill_value=nan</em>, <em>limit=None</em>, <em>tolerance=None</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/603/1*iPp5UakTtoc9ubSd3KZ5KA.png" /><figcaption>reindex() used to organize DataFrame columns alphabetically</figcaption></figure><p><strong>reindex_like</strong>:</p><p>Conform DataFrame to new index. Places NA/NaN in locations having no value in the previous index. A new object is produced unless the new index is equivalent to the current one and copy=False.</p><p>cuDF: [unavailable as of writing]<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.reindex_like.html?highlight=reindex_like#pandas.DataFrame.reindex_like">pandas</a>: <em>(other, method=None, copy=True, limit=None, tolerance=None)</em></p><p><strong>set_index</strong>:</p><p>Return a new DataFrame with a new index</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.set_index#cudf.DataFrame.set_index">cuDF</a>: (<em>keys</em>, <em>drop=True</em>, <em>append=False</em>, <em>inplace=False</em>, <em>verify_integrity=False</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.sort_index.html#pandas.DataFrame.sort_index">pandas</a>: (<em>keys, *, drop=True, append=False, inplace=False, verify_integrity=False)</em></p><p><strong>sort_index</strong>:</p><p>Sort object by labels (along an axis).</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.sort_index#cudf.DataFrame.sort_index">cuDF</a>: (<em>axis=0</em>, <em>level=None</em>, <em>ascending=True</em>, <em>inplace=False</em>, <em>kind=None</em>, <em>na_position=’last’</em>, <em>sort_remaining=True</em>, <em>ignore_index=False</em>, <em>key=None</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.sort_index.html#pandas.DataFrame.sort_index">pandas</a>: (<em>*</em>, <em>axis=0</em>, <em>level=None</em>, <em>ascending=True</em>, <em>inplace=False</em>, <em>kind=’quicksort’</em>, <em>na_position=’last’</em>, <em>sort_remaining=True</em>, <em>ignore_index=False</em>, <em>key=None</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/665/1*PBM0K-Sl29b767oeIipCJw.png" /><figcaption>sort_index() used here to take away some of the randomness of the selection</figcaption></figure><p><strong>sample</strong>:</p><p>Return a random sample of items from an axis of object. random_state parameter can be specified to get same samples. If reproducible results are required, a random number generator may be provided via the random_state parameter.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.sample#cudf.DataFrame.sample">cuDF</a>: (<em>n=None</em>, <em>frac=None</em>, <em>replace=False</em>, <em>weights=None</em>, <em>random_state=None</em>, <em>axis=None</em>, <em>ignore_index=False</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.sample.html?highlight=sample#pandas.DataFrame.sample">pandas</a>: <em>(n=None, frac=None, replace=False, weights=None, random_state=None, axis=None, ignore_index=False)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/618/1*dpICrxuJJFQCXu2XFX91FA.png" /><figcaption>Two ways of taking samples, by explicitly choosing how many or a fraction of the entire DataFrame</figcaption></figure><p>sorting values (<strong>sort_values, nlargest &amp; nsmallest, groupby.nth, truncate</strong>):</p><p><strong>sort_values:</strong></p><p>Sort by the values along either axis</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.sort_values#cudf.DataFrame.sort_values">cuDF</a>: (<em>by</em>, <em>axis=0</em>, <em>ascending=True</em>, <em>inplace=False</em>, <em>kind=’quicksort’</em>, <em>na_position=’last’</em>, <em>ignore_index=False</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.sort_values.html?highlight=sort_values#pandas.DataFrame.sort_values">pandas</a>: <em>(by, *, axis=0, ascending=True, inplace=False, kind=’quicksort’, na_position=’last’, ignore_index=False, key=None)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/568/1*i3_iYIGHXPUwI64tECHJvQ.png" /></figure><p><strong>nlargest &amp; nsmallest:</strong></p><p>Return the first n rows with the largest values in columns, in descending (&amp; ascending for nsmallest) order. The columns that are not specified are returned as well, but not used for ordering. Including these two functions because it saves time from writing out: “ sort_values(columns, ascending=False).head(n) “ and is computationally faster.</p><p><a href="https://docs.rapids.ai/api/cudf/nightly/api_docs/api/cudf.dataframe.nlargest">cuDF</a>: (<em>n</em>, <em>columns</em>, <em>keep=’first’</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.nlargest.html">pandas</a>: (<em>n</em>, <em>columns</em>, <em>keep=’first’</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/302/1*1AV7KaEQK9uwwFQxN6ICqQ.png" /><figcaption>How the hell did a team get to the point where they needed 39 yards for 1st down? [Sack]</figcaption></figure><p><strong>groupby.nth:</strong></p><p>For use either after creating a DataFrame using groupby() or attached to the end of the line utilizing groupby(). Take the nth row from each group if n is an int, otherwise a subset of rows. Can be either a call or an index. dropna is not available with index notation. Index notation accepts a comma separated list of integers and slices.</p><p>Pandas only: if dropna, will take the nth non-null row, dropna is either ‘all’ or ‘any’; this is equivalent to calling dropna(how=dropna) before the groupby. Even though it is not specified in the docs, you can use the following args: slice, dropna (<em>=’any’</em>), as_index (=True or False). This is equivalent to calling dropna(how=dropna) before the groupby.</p><p>cuDF: (n)<br>pandas: (n, dropna=None)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/456/1*EYamCzeuV-8NoZsJQ9_WEw.png" /><figcaption>Looking at the 100th row of each position group</figcaption></figure><p>value formats (<strong>astype, dtypes, select_dtypes, to_datetime</strong>)</p><p><strong>astype</strong>:</p><p>Cast the object to the given dtype.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.astype#cudf.DataFrame.astype">cuDF</a>: (<em>dtype</em>, <em>copy=False</em>, <em>errors=’raise’</em>, <em>**kwargs</em>)<br><a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.astype.html">pandas</a>: (<em>dtype</em>, <em>copy=True</em>, <em>errors=’raise’</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/471/1*vT_Aq23xuH0yLn5w4xn8Og.png" /><figcaption>Changing the datatype of a column. Key in reducing the DataFrame size by downgrading datatypes</figcaption></figure><p><strong>dtypes</strong>:</p><p>find the data types in a dataframe</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/392/1*0gZlJKVix3fZKtOqL-jXvg.png" /></figure><p><strong>select_dtypes</strong>:</p><p>dtypes gives the format/type of data. select_dtypes returns a subset of the DataFrame’s columns based on the column dtypes.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.dataframe.select_dtypes#cudf.DataFrame.select_dtypes">cuDF</a>: <em>(include=None, exclude=None)</em><br><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.select_dtypes.html?highlight=select_dtypes">pandas</a>: <em>(include=None, exclude=None)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/524/1*4gJKWLQBTAlZo2S_Q8dGDw.png" /><figcaption>Looking at the ‘float64’ columns. Can be converted to integers to gain efficiency</figcaption></figure><p>For <a href="https://jakevdp.github.io/PythonDataScienceHandbook/02.01-understanding-data-types.html">reference</a> on how data types work in python:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/744/1*Ao0rMmguuQ8uJ33RRMGlyA.png" /><figcaption>from the Python Data Science Handbook by Jake VanderPlas</figcaption></figure><p><strong>to_datetime </strong>(astype(‘datetime[]’):</p><p>Convert argument to datetime. cuDF likes it’s dates in the “Year-Month-Day” format. “Month/Day/Year” has to get converted. Converting dates (in multiple columns) into datetime can also be done by using the astype() function. As of writing there are some bugs with the ‘dayfirst’ arg in cuDF so avoid changing it to ‘True.’ Some have found using the astype() function faster than to_date() but I haven’t tested it out myself.</p><p><a href="https://docs.rapids.ai/api/cudf/stable/api_docs/api/cudf.to_datetime.html">cuDF</a>: (<em>arg</em>, <em>errors=’raise’</em>, <em>dayfirst=False</em>, <em>yearfirst=False</em>, <em>utc=None</em>, <em>format=None</em>, <em>exact=True</em>, <em>unit=’ns’</em>, <em>infer_datetime_format=False</em>, <em>origin=’unix’</em>, <em>cache=True</em>)<br><a href="https://pandas.pydata.org/docs/reference/api/pandas.to_datetime.html">pandas</a>: (<em>arg</em>, <em>errors=’raise’</em>, <em>dayfirst=False</em>, <em>yearfirst=False</em>, <em>utc=None</em>, <em>format=None</em>, <em>exact=True</em>, <em>unit=None</em>, <em>infer_datetime_format=False</em>, <em>origin=’unix’</em>, <em>cache=True</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/449/1*1CUgglkSmCtaSUD0IiODMA.png" /><figcaption>Using to_date()</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/412/1*UdQib2ECvhhsrnJq3uAIHQ.png" /><figcaption>Using astype()</figcaption></figure><h3>Description of Dataset Used for Illustrative Purposes</h3><p>The dataset used for all the examples were provided by the NFL on Kaggle for their 2023 Data Bowl competition. I love the NFL so thought it’d be fun to use that dataset for this post.</p><p><a href="https://www.kaggle.com/competitions/nfl-big-data-bowl-2023/data">NFL Big Data Bowl 2023</a></p><p>The data provided was a combination of general NFL player &amp; game descriptions (players.csv, games.csv), the first 8 weeks of last season (week1.csv, etc.), NFL NextGen data (plays.csv), and ProFootball Focus scouting reports on each game (pffScoutingData.csv).</p><p>The competition was focused on offensive and defensive line play, so had lots of data on players along the line of scrimmage and what they did each play.</p><h3>Generating NFL Databowl 2023 DataFrame in cuDF</h3><p>The Kaggle data files can be downloaded and used using the Kaggle API, directly downloaded as zip files (using wget and zipfile, etc.) into pandas, or just downloading to your directory. I did the last just to keep things simple and allow me to rerun the jupyter notebook over and over again.</p><p>For this exercise all we need is to load the cuDF module. I included the others in the code just to show other things Rapids users can use.</p><pre>#import pandas as pd<br>#import numpy as np<br><br>import cudf<br><br>#import cupy as cp<br>#import cuml<br>#import cugraph as cg<br>#import cuspatial as cs<br>#import cuxfilter as cf<br>#import cusignal as csig<br>#import cucim as cc<br><br># data viz<br>#import matplotlib.pyplot as plt<br>#from matplotlib import rcParams<br>#import seaborn as sns</pre><p>Load all the files provided by the NFL Databowl competition organizers provided. Using cuDF, all these files are loaded directly into the GPU’s VRAM. As long as all processes are kept inside the GPU using Rapids, the computer’s CPU, RAM, and disk storage will be minimally involved.</p><pre># Load given datasets<br><br>games = cudf.read_csv(&#39;games.csv&#39;)<br>pffScoutingData = cudf.read_csv(&#39;pffScoutingData.csv&#39;)<br>players = cudf.read_csv(&#39;players.csv&#39;)<br>plays = cudf.read_csv(&#39;plays.csv&#39;)<br>week1 = cudf.read_csv(&#39;week1.csv&#39;)<br>week2 = cudf.read_csv(&#39;week2.csv&#39;)<br>week3 = cudf.read_csv(&#39;week3.csv&#39;)<br>week4 = cudf.read_csv(&#39;week4.csv&#39;)<br>week5 = cudf.read_csv(&#39;week5.csv&#39;)<br>week6 = cudf.read_csv(&#39;week6.csv&#39;)<br>week7 = cudf.read_csv(&#39;week7.csv&#39;)<br>week8 = cudf.read_csv(&#39;week8.csv&#39;)</pre><p>The first 8 weeks NFL NextGen stats of the 2021 season have to be put together. The concat() function will do that work.</p><p>Notice that as soon as I’m done with a dataframe, it is deleted immediately to conserve VRAM. This is not necessary if your dataset is small relative to your total VRAM capacity but becomes crucial as you get near the limit.</p><pre># Combine the 8 weeks of data<br><br>weeks = [week1, week2, week3, week4, week5, week6, week7, week8]<br>weeks = cudf.concat(weeks)<br>del[week1, week2, week3, week4, week5, week6, week7, week8]</pre><p>Edit: the following alternative was suggested in feedback that is faster and more flexible:</p><pre>#Load weekly data and concat in one line<br><br>weeks = cudf.concat([cudf.read_csv(&quot;week&quot; + str(x)) for x in range(1,9)])</pre><p>Using merge() function, combine the weekly data frame with the PFF scouting numbers.</p><pre># Combine weekly data with PFF Scouting data<br><br>df = weeks.merge(pffScoutingData, on = [&#39;gameId&#39;, &#39;playId&#39;, &#39;nflId&#39;])<br>del(weeks, pffScoutingData)</pre><p>Add game information to the data frame.</p><pre># Add in the basic Games data in front<br><br>df = games.merge(df, on =&#39;gameId&#39;)<br>del(games)</pre><p>Attach more detailed information on what happened in each play</p><pre># Add the details of the plays at the end of the dataframe<br><br>df = df.merge(plays, on = [&#39;gameId&#39;, &#39;playId&#39;])<br>del(plays)</pre><p>Our primary data frame now has all the game data for the first half of the 2021 NFL season. Now attach information about the players.</p><pre># Add in Player data, putting it in front of the dataframe<br><br>df = players.merge(df, on = &#39;nflId&#39;)<br>del(players)</pre><p>The base database is now complete. We have all the information the contest providers have given us in an organized, singular data frame. There are many categorical and ordinal values throughout. To do statistical analysis on these columns, we use one-hot encoding to split everything into 0 (False) and 1 (True). This will vastly increase the number of columns the data frame will contain.</p><pre># One Hot Encoding of variables with multiple categories using Pandas<br><br>officialPosition = cudf.get_dummies(df.officialPosition)<br>down = cudf.get_dummies(df.down)<br>passResult = cudf.get_dummies(df.passResult)<br>offenseFormation = cudf.get_dummies(df.offenseFormation)<br>personnelO = cudf.get_dummies(df.personnelO)<br>personnelD = cudf.get_dummies(df.personnelD)<br>dropBackType = cudf.get_dummies(df.dropBackType)<br>pff_passCoverage = cudf.get_dummies(df.pff_passCoverage)<br>pff_passCoverageType = cudf.get_dummies(df.pff_passCoverageType)<br>pff_role = cudf.get_dummies(df.pff_role)<br>pff_blockType = cudf.get_dummies(df.pff_blockType)<br>foulName1 = cudf.get_dummies(df.foulName1)<br>pff_positionLinedUp = cudf.get_dummies(df.pff_positionLinedUp)</pre><p>Here we meet our first difference between pandas and cuDF but it’s a very minor one. In pandas when you combine dataframes that happen to have the same column names but are treated as unique, they are treated as repeats. Pandas simply changes the name of the columns (like adding a ‘-x’ suffix).</p><p><em>Rapids AI’s cuDF does not do this</em>.</p><p>It will stop the kernel and give you a warning. If you see columns with the same name in data frames you are combining, give each repeat column a unique name.</p><pre># Renaming columns to avoid issues with repeats<br><br>down.columns = [&#39;SP/NaN&#39;, &#39;FirstD&#39;, &#39;SecondD&#39;, &#39;ThirdD&#39;, &#39;FourthD&#39;]<br>officialPosition.columns = [&#39;Center&#39;, &#39;CornerBack&#39;, &#39;DBack&#39;, &#39;DEnd&#39;, &#39;DTackle&#39;, &#39;FullBack&#39;, &#39;FreeSafety&#39;, &#39;Guard&#39;, &#39;InsideLB&#39;, &#39;LineBacker&#39;, &#39;MiddleLB&#39;, &#39;NoseTackle&#39;, &#39;OutsideLB&#39;, &#39;QuarterBack&#39;, &#39;RunningBack&#39;, &#39;StrongSafety&#39;, &#39;Tackle&#39;, &#39;TightEnd&#39;, &#39;WideReciever&#39;]<br>passResult.columns = [&#39;Complete&#39;,&#39;Incomplete&#39;, &#39;Interception&#39;, &#39;Scramble&#39;, &#39;Sack&#39;]</pre><p>Now it adding the one-hot encoded information to the primary data frame should go problem free.</p><pre># Join One Hot Encoded info into the main dataframe<br><br>df = cudf.concat([df, passResult, down, officialPosition, pff_positionLinedUp, pff_role, pff_blockType, dropBackType, offenseFormation, personnelO, personnelD, pff_passCoverage, pff_passCoverageType, foulName1], axis=1)<br>del([down, officialPosition, pff_positionLinedUp, pff_role, pff_blockType, passResult, dropBackType, offenseFormation, personnelO, personnelD, pff_passCoverage, pff_passCoverageType, foulName1])</pre><p>And that’s it. we have our complete database for analysis.</p><p>There’s about 8 million rows, 290 columns, and takes up around 8 GBs of VRAM. Pretty damn big. Using RAPIDS, it gets done in less than 5 seconds and almost all of it is taken up by moving the initial csv files from the SSD hard drive to the GPU.</p><p>From here on in the database will be used to illustrate how to use the most common pandas functions used to prepare (data munging) data sets for data science and machine learning.</p><p>In the vast majority of cases, cuDF and pandas are like for like. There will be explicit notes in where there are differences</p><h3>Thanks for reading.</h3><h3>EXTRA: Simplified NFL Database Using Pandas</h3><p>Since I plan on posting this on the Kaggle discussion boards and most people will want to use regular old pandas, here’s how to create the same data set, minus a few columns and the information in the weekly csvs. If your PC can handle all of it (32+ GB RAM), take out the # marks in the code.</p><pre>cd ~/Dropbox/NFL_Databowl_2023<br><br># Load up modules<br><br># data manipulation<br>import pandas as pd<br><br><br># Load up the data files from NFL NextGenStats<br><br>games = pd.read_csv(&#39;games.csv&#39;)<br>pffScoutingData = pd.read_csv(&#39;pffScoutingData.csv&#39;)<br>players = pd.read_csv(&#39;players.csv&#39;)<br>plays = pd.read_csv(&#39;plays.csv&#39;)<br>#week1 = pd.read_csv(&#39;week1.csv&#39;)<br>#week2 = pd.read_csv(&#39;week2.csv&#39;)<br>#week3 = pd.read_csv(&#39;week3.csv&#39;)<br>#week4 = pd.read_csv(&#39;week4.csv&#39;)<br>#week5 = pd.read_csv(&#39;week5.csv&#39;)<br>#week6 = pd.read_csv(&#39;week6.csv&#39;)<br>#week7 = pd.read_csv(&#39;week7.csv&#39;)<br>#week8 = pd.read_csv(&#39;week8.csv&#39;)</pre><pre># Drop Unnecessary data from games, players and plays dataframes<br>games = games.drop([&#39;season&#39;, &#39;gameDate&#39;, &#39;gameTimeEastern&#39;, &#39;homeTeamAbbr&#39;, &#39;visitorTeamAbbr&#39;], axis=1)<br>players = players.drop([&#39;height&#39;, &#39;weight&#39;, &#39;birthDate&#39;, &#39;collegeName&#39;], axis=1)<br>plays = plays.drop([&#39;playDescription&#39;, &#39;quarter&#39;, &#39;possessionTeam&#39;, &#39;defensiveTeam&#39;, &#39;yardlineSide&#39;, &#39;gameClock&#39;, &#39;preSnapHomeScore&#39;, &#39;preSnapVisitorScore&#39;, &#39;absoluteYardlineNumber&#39;], axis=1)</pre><pre># Merge games, pffScoutingData, and players into a single dataframe &#39;df&#39;<br>df = games.merge(pffScoutingData, on = &#39;gameId&#39;)<br>df = players.merge(df, on = &#39;nflId&#39;)<br>df = df.merge(plays, on = [&#39;gameId&#39;,&#39;playId&#39;])<br><br>del[plays, players, games, pffScoutingData]<br><br># combine the 8 playing weeks of data into &#39;weeks&#39;<br># weeks = pd.concat([week1,week2,week3,week4,week5,week6,week7,week8])<br><br># merge it with the dataframe<br># df = pd.merge(df, weeks, on=&#39;gameId&#39;, how=&#39;outer&#39;)<br><br># delete unused dataframes<br>#del[weeks, week1,week2,week3,week4,week5,week6,week7,week8]</pre><pre># Let&#39;s encode Position and Down<br><br>officialPosition = pd.get_dummies(df.officialPosition)<br>down = pd.get_dummies(df.down)<br><br># One Hot Encoding of variables with multiple categories using Pandas<br><br>passResult = pd.get_dummies(df.passResult)<br>offenseFormation = pd.get_dummies(df.offenseFormation)<br>personnelO = pd.get_dummies(df.personnelO)<br>personnelD = pd.get_dummies(df.personnelD)<br>dropBackType = pd.get_dummies(df.dropBackType)<br>pff_passCoverage = pd.get_dummies(df.pff_passCoverage)<br>pff_passCoverageType = pd.get_dummies(df.pff_passCoverageType)<br>pff_role = pd.get_dummies(df.pff_role)<br>pff_blockType = pd.get_dummies(df.pff_blockType)<br>foulName1 = pd.get_dummies(df.foulName1)<br>pff_positionLinedUp = pd.get_dummies(df.pff_positionLinedUp)<br><br><br><br># Join all the codes to main dataframe<br><br>df = pd.concat([df, down, officialPosition, pff_role, pff_blockType, passResult, dropBackType, offenseFormation, personnelO, personnelD, pff_passCoverage, pff_passCoverageType, foulName1, pff_positionLinedUp], axis=1)</pre><pre># Save Work<br><br>df.to_csv(&#39;NFL_2023_Databowl_Data_with_codes.csv&#39;)</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=12a8f889b33e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Elite Athletes of the 2022 NFL Draft Class: A Data Exploration]]></title>
            <link>https://medium.com/@tiraldj/the-elite-athletes-of-the-2022-nfl-draft-class-a-data-exploration-f526ed533fa7?source=rss-3b89e6336fb5------2</link>
            <guid isPermaLink="false">https://medium.com/p/f526ed533fa7</guid>
            <category><![CDATA[nfl-draft]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[athleticism]]></category>
            <category><![CDATA[nfl]]></category>
            <category><![CDATA[2022]]></category>
            <dc:creator><![CDATA[Mohammed R. Osman]]></dc:creator>
            <pubDate>Wed, 27 Apr 2022 23:55:15 GMT</pubDate>
            <atom:updated>2022-04-28T16:37:58.713Z</atom:updated>
            <content:encoded><![CDATA[<p>I’m a big fan of the NFL and love the draft. These next few pieces will be for fans of the draft that can’t get enough. I wanted to look at the draft from a different angle than others and help inform people about the players their favorite teams can take. But not as a paid scout, draft analyst, or pundit. Let’s look at as many of the players that made themselves available, got tested, and evaluated by national analysts from a data point of view. Specifically data that measures athleticism.</p><p>Who are the athletic freaks of the NFL Draft 2022? Which prospects at the top of draft charts are also mega athletes? Who are players we don’t know much about or have gotten little attention from the national media? What does athleticism look like at the positional level?</p><p>The rest of this first post is broken up into 3 parts: data preparation, understanding the final database for 2022, and “The Elites” where we go through the best of the best athletes this year’s draft has to offer. If you don’t want the data talk (it’s simple don’t worry!), skip to the end for the good stuff.</p><h3>Quick Summary</h3><ol><li>NFL Draft Evaluations are not tied to pure athleticism</li><li>NFL Draft Analysts strongly agree with each others’ player evaluations</li><li>It’s harder to rank players by position groups than all at once</li><li>2022 has a large number of prospects with very good athletic qualities</li><li>NFL Draft Analysts do not like this year’s draft class, especially at the top</li><li>2022 top end talent is deep at DE, OT, and CB</li><li>2022 top end talent is wanting at QB, IOL, and DT</li></ol><h3><strong><em>Data Preparation</em></strong></h3><p>For this analysis I wanted to use publicly available information that has info athleticism measures and how draft analysts viewed players. The first part I will take advantage of the work of <a href="https://twitter.com/MathBomb">Kent Lee Platte</a> and use his Relative Athletic Scores (RAS) database, along with the raw test results from the 2022 NFL Draft Combine.</p><p><a href="https://ras.football/">RASRelative Athletic Scores grade a player&#39;s measurements on a 0 to 10 scale compared to their peer group.</a></p><p>The RAS is a ‘grey box’ metric that combines popular physical tests that college football players do and standard body measurements as an index to measure athleticism. This raw score is then compared to prior college football players that have made themselves available to the NFL Draft to yield a final RAS score. It is not public exactly how the score is calculated but we do know what inputs go into it. Going position by position, the RAS score goes from 1 (lowest %) to 10 (highest, rarest) level of athleticism seen. Each RAS score is only comparative to players from the past in the same position. While this doesn’t allow for a straight listing of the best athletes in the draft, it does prevent problems like wide receivers from being compared to offensive guards. The RAS is a solid tool to quickly take a look at draftees and find the bigger, faster, stronger athletes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/703/1*XJ-i_Z94uuvjnpKBYUCW1A.jpeg" /><figcaption>RAS card for Jordan Davis, one of the best athletes at Defensive Tackle ever seen</figcaption></figure><p>For the raw test scores, the 2022 NFL Draft Combine results will get used. These tests are 40 yard dash, Broad Jump, Vertical Jump, 3 Cone Drill, 20 &amp; 60 yard shuttle, and the Bench Press. The 40 yard dash (and it’s breakdown into ‘splits’) is the standard method to measure speed. There is nothing special about 40 yards other than tradition and 4.4 seconds is considered to be the gold standard. The 20 &amp; 60 yard shuttles also measure speed but are not nearly as seen as important in talent evaluations. Both Vertical and Broad Jumps are meant to measure lower body explosiveness. The Vertical Jump is also scrutinized by the NFL to estimate how high defensive players can swat the ball away. The 3 Cone Drill is a measure of agility and how fast players can change directions. This test is heavily used in scouting for almost every position but the offensive line and quarterbacks. The Bench Press is testing for upper body strength by having prospects bench press 225 pounds as many times as they can. Outside of offensive and defensive linemen, the Bench Press is not heavily weighted in NFL draft evaluations.</p><p><a href="https://www.nfl.com/combine/tracker/live-results/40-yards-dash/all-positions/all-colleges/">2022 NFL Combine Results - NFL.com</a></p><p>It should be noted that for both RAS and NFL Draft Combine data, missing values is a major issue. Due to choice, postponement to College Pro Days, injury or lack of invites, many prospects do not have testing results for all the metrics. Their profiles are incomplete, creating missing values in the database. This is a regular occurrence and was expected, even prospects at the top of the draft. What <em>was </em>unexpected was the problems in using the NFL’s home page. Their database is missing whole sections (the 60 Yard Shuttle) or oddly specific missing data (3 Cone Drill for Running Backs, actually very important for that position).</p><p>To scan the conventional wisdom on how good these college players are as actual football players, where athleticism is one of many components of their evaluation, I used three player rankings and scores by draft analysts and Amazon. In choosing these rankings, priority was given to those that had grades on the most players possible, popularity amongst NFL fans and was free to use.</p><p><a href="https://twitter.com/TonyPauline">Tony Pauline</a> at The Pro-Football Network (PFN) covered the most players, as far as I could find. There was a consensus PFN ranking available but it was limited to 300 players. Tony Pauline’s draft board (“PFN-TP”) had over 800.</p><p><a href="https://www.profootballnetwork.com/tony-paulines-big-board-top-players-2022-nfl-draft/">Tony Pauline&#39;s Big Board of top players for the 2022 NFL Draft</a></p><p><a href="https://twitter.com/LanceZierlein">Lance Zierlein</a> had the second most players covered and is a popular long time scout used by the NFL’s homepage for the Draft. The NFL has been collaborating with Amazon’s Machine Learning / Big Data arm to produced “NFL Next Gen Stats” and for the draft they have their own draft grades available. I used both Zierlein’s and Next Gen Stats grades.</p><p><a href="https://www.nfl.com/prospects/participants/all/all-positions/all-colleges/">NFL.com | Official Site of the National Football League</a></p><p>Bleacher Report is a very popular sports news and rumor site. They’ve branched out into NFL Draft scout evaluations.</p><p><a href="https://bleacherreport.com/articles/2955006-2022-nfl-draft-big-board-br-nfl-scouting-depts-rankings-as-draft-nears">2022 NFL Draft Big Board: B/R NFL Scouting Dept.&#39;s Rankings as Draft Nears</a></p><p>Profootball Focus (PFF) is one of the most popular NFL and Draft sports websites and their (post-Combine) ‘Big Board’ of top 344 players was used.</p><p><a href="https://www.pff.com/draft/big-board?season=2022">NFL Draft Big Board | PFF</a></p><p>There are other draft websites, paid &amp; free, and rankings available but limiting to four of the most popular sources felt adequate. Fair warning to anyone looking to make their own database: spelling mistakes on names are all over the place! A massive time sink to fix.</p><p>Using Python, I created two new rankings from each source: overall grade rank and rank by position. If the draft database had actual scores (PFN, NFL), then the entire player universe was ranked by score. For all data sources, players were separated by position and ranked by either score or player rank (PFF). Both rankings were done for each of the Combine tests as well. The reasoning behind creating these new data points is so that we can more easily understand and contextualize how each source sees draft prospects. I am also very interested in seeing where analysts perceive talent and athletic drop offs between players at the same position.</p><p>Final database can be downloaded <a href="https://www.mediafire.com/file/rldcetnqzybrl2o/NFL2022_Draft_Chart_MOsman.csv/file">here</a>.</p><p>Feel free to poke around and use during the Draft.</p><h3><strong><em>Understanding the Final Database</em></strong></h3><p>The complete databases has 2300 players, with 43 headings. Of these players, approximately 350 have data for most or all the measurements. Beyond that, lots of missing data abounds where the players never had any draftable grades or were ranked in the top 350 players available. Considering that the 2022 draft has 262 total draft picks over 7 rounds, the database can be assumed to be comprehensive.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/432/1*LeA-1_AbAGyvecU1hYqmBQ.png" /><figcaption>Final Measurements covered in the database</figcaption></figure><p>Note: “RAS” is the Relative Athletic Score; “PFN-TP” is from Tony Pauline’s draft board for PFN; “NFL” is Lance Zierlein’s grading and “NFL NG” is the grades from NFL Next Gen Stats; “PFF” is from Pro-Football Focus; “BR” is from Bleacher Report; the rest are from the Combine by event.</p><p>First thing I’ve noticed is that there were incomplete RAS scores for 32 prospects. Even though this was expected, some of them who are highly regarded and not having their RAS hurts the full evaluation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/330/1*k9z01C9qMO3wdy6qJywSlg.png" /><figcaption>Players with incomplete testing data and no RAS score. In order of expected overall player rank.</figcaption></figure><p>Major players who were not healthy by the Combine or their Pro-Days: Jameson Williams, Drake London, Andrew Booth, John Metchie, and Carson Strong. Williams and Metchie are recovering from ACL tears. London is recovering from a significant ankle injury. Andrew Booth had sports hernia surgery after the college season and Carson Strong had cadaver knee cartilage inserted into one of his knees. All of these players are expected to be picked in the first two days of the Draft. The others on the list, like Drake London, either had chosen not to do enough of the tests to generate a RAS score or had undisclosed ailments. David Ojabo of Michigan, a top end player will be out the next season with a torn Achilles, is not on this list. Ojabo has an RAS score because he completed the tests at the Combine prior to getting injured at his college Pro-Day. Ojabo may still be a 2nd day pick but no one is certain. A scary situation happened with Damone Clark, an athletic middle line backer prospect, who found out he needed spinal fusion surgery for a severely herniated disc at the Combine. Another player of note is Derek Stingley who is still coming off a significant injury and was able to post a RAS score close to 9. Once completely healthy, Stingley should be considered one of the most athletic corner backs to ever come out of the draft.</p><p>Strange and unexpected finding: if we assume the top athletes in the draft have a RAS score 8 or higher, 296 players make the cut. Depending on the draft analyst, <em>only 140–150 of their top 300 are on that list</em>. This is a big indication that <strong>athleticism is not the end all be all </strong>according to the draft analysts. Nearly half of the draft prospects in top 300 charts are on there based on skill sets over raw physical ability.</p><p>To do a sanity check, I looked at the correlation chart of the RAS score with everything else. The lack of correlation strength of RAS with Draft grades clearly indicates that the NFL Draft is not about raw athleticism. Despite all the talk about combine results, height, weight, arm lengths etc., evaluations really come down to game tape.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/289/1*VM7q5D8c3S2suaG0wx5qNw.png" /><figcaption>Aidan Hutchinson. Kayvon Thibodeaux. Travon Walker. George Karlaftix. Jermaine Johnson.The only thing RAS scores are moderately tied to is the 3 Cone Drill.</figcaption></figure><p>If not the RAS metric, maybe the tests at the NFL Combine matter? 40 Yard Dash?3 Cone Drill? Vertical or Broad Jumps? <strong>None of the Combine tests correlated to player evaluations</strong>. I need to check the correlations for speed specific positions like wide receivers, corner backs, and running backs in a future post. There <em>has </em>to be some connections there.</p><p>Another way of looking at these results could be: NFL teams with strong coaching staffs and heavy investment into training &amp; development can snatch up incredible athletes as without using Draft picks and turn them into high end players. There should be so many pure athletes left over after the 262 NFL Draft picks, the potential for developing incredible players is there.</p><p><em>So why don’t we see it happen more often?</em></p><p>Since all the testing data and the RAS score isn’t tied to player evaluations, could this be due to noise? If the draft analysts don’t agree with each other, indicating that their grading is closer to random than not, then we may have been trying to tie these measurements to noisy variables?</p><p>Nope. The draft boards correlate with each other from 70–90% with each other. Meaning analysts are mostly in agreement with on grading players</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/682/1*owmOl-dLB_HDqnUsJvbTzg.png" /><figcaption>NFL Draft Analysts tend to agree with one another when making draft boards</figcaption></figure><p>When you compare ranking by position, the correlations between the draft boards is unexpectedly lower. So when draft analysts look at players one position at a time, they <em>should </em>come to similar conclusions on who is better than the other. Intuition says that once analysts start having to compare and rank players of different positions, that variance increases significantly. Each board values different positions with different weights and personal bias becomes more pronounced.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/893/1*dI8tx45HKxHmvPMxslcCvw.png" /><figcaption>Analysts still align when it comes to ranking players by position, but much less strongly than overall</figcaption></figure><p>But that’s not the case. Analysts seem to align with each other on where players are ranked overall, regardless of position, than when you drill down position to position. I find this odd. Possible explanations could be that everyone reads and listens to each other and unintentionally/intentionally form a consensus. But when they look at players one position at a time, they are more varied and stick to their own opinions. Again, I am very surprised by these results. It should be much easier for people to look at the players in the same position and be similar in their rankings than ranking all the prospects in a draft all at once. Maybe this is all an artificial artifact within the math of the scores and rankings. Might be worth exploring more later…</p><h3><strong><em>Where is the Talent? Looking at Position Groups</em></strong></h3><p>Let’s get closer to the heart of understanding the NFL 2022 Draft, what does the data tell us about athleticism and perceived talent at each position group?</p><p>Here I will use histograms of scores/grades at each position to mine for insights. But first a quick view of the entire draft through the RAS and the NFL analyst grades.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/601/1*mVSK2CJxPgGcviLQ8vI70w.png" /><figcaption>Histograms using the entire draft dataset</figcaption></figure><p>First thing that pops out is that almost 300 players have RAS scores of 8 and above. And blow that there are another 600+ players which the RAS metric says are above the average athleticism seen every year. This is a draft class with lots of prospects worth tracking according to the RAS database. <strong>Draft analysts see the 2022 NFL Draft very negatively</strong>. There has been persistent talk that this year’s draft is not as talented as last years but is very deep. Some have said NFL teams have placed less than 16 players with a first round grade. Quite a few analysts have said that the top players this year wouldn’t beat out the best 12 players from 2021. Looking at these histograms, the draft grades line up with this pessimism. The most negative draft evaluators are PFN’s Tony Pauline and NFL’s Lance Zierlein. Zierlein especially does not see this draft class as a good one overall and both of their overall grades show a lack of overall top end talent. The NFL/Amazon’s Next Gen Stats and Bleacher Report scores say that there are few high end players but graded many players with above average to very good, thus a deep class. These two evaluations and the RAS scores indicates that 2022 NFL Draft might not produce too many superstars but there will be plenty of teams that will get skilled athletes throughout it’s 7 rounds.</p><p>Now let’s go through each position in the 2022 class.</p><p><strong>Defensive Ends</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/592/1*xDn7ypuiGQJ-l5musWt5rw.png" /><figcaption>Defensive Ends Histogram</figcaption></figure><p>There are an incredible number of Defensive End draft prospects with fantastic athleticism! The distribution for RAS scores is highly unusual and unexpected. More than 35 Defensive End prospects have a RAS score of 8 or above. This optimism is not supported by the analysts. According to the histograms of the draft evaluators, most felt that there were only 2–3 high end Defensive End prospects. The majority of the other prospects are of middling talent. PFF ranked 11 Defensive Ends in its top 50…and only two more in th next 50 ranks.</p><p>If your team doesn’t take one of the top 3 DE’s, they are better off looking for one in the middle rounds.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1004/1*DbwdpAdzhB5sboepA6ybcg.png" /></figure><p><em>Amazing Prospects: </em><strong>Aidan Hutchinson</strong> or <strong>Kayvon Thibodeaux</strong> is the best defensive end in the 2022 NFL Draft. For most of this off season Hutchinson was deemed the best but the day before the draft they are close enough to be a toss up. <strong>Travon Walker</strong> has incredible athletic testing but is raw and needs a year or two of coaching. He could end up being better than both of them in the long run. <strong>George Karlaftis </strong>is a traditional all-around prospect that’s good at setting the edge and will not get out of the 1st round. <strong>Jermaine Johnson</strong> is a borderline 1st round talent that relies on speed and pass rush moves/hand fighting to get to the QB.</p><p><em>Sleepers: </em><strong>David Ojabo</strong> prior to the tearing his Achilles at the end of Pro-Day was a late 1st round pick. Now teams are probably looking to take the chance on him late in the 2nd or in the 3rd round.<strong> Boye Mafe</strong>, <strong>Nik Bonitto</strong>, and <strong>Arnold Ebitketie </strong>are active high motor defensive ends that will not succeed in the NFL without getting coached up. They don’t have much in terms of skill set but both are expected to be picked in the 2nd round. Both do have the athleticism required to start. <strong>Sam Williams</strong> can be a great pass rusher but is a serious liability versus the run. Williams should be a 3rd rounder but could sneak into the 2nd round if a team falls in love with his pass rush. Similarly <strong>Isaiah Thomas</strong> is a tall defensive end that can rush the quarter back. But he’s also a complete liability in the run game and bad against tough tackles in general. He should be available late in the draft. <strong>DeAngelo Malone</strong> is a good player who might have to transition to outside line backer due to size concerns. Malone earned several awards in the lower FBS conference and now must show he can hold up at the NFL level. He probably goes in the 3rd. <strong>Myjai Sanders</strong> could also transition to outside linebacker because he has the speed but not ability to get off blocks to play defensive end full time. Sander’s pass rush skills are good enough to be worth a late round pick. <strong>Jeffrey Gunter</strong> is very versatile, able to play both 3–4 and 4–3, and drop into coverage. Gunter should be available in the 4th round. <strong>Cameron Thomas </strong>is a solid defensive end prospect who can handle the run and pass equally well. Thomas might not have the speed or agility teams are looking for at the edge but he could grow into a quality starter there. This potential is enough to get drafted in the 4th or 5th round. <strong>Josh Paschal </strong>and <strong>Amare Barno</strong> have good testing numbers but not the best game tape. They should be taken late 5th or early 6th rounds. <strong>Drake Jackson </strong>is the same but needs even more time for development. Jackson could be found in the final round of the draft.</p><p><em>Who Dat?!: </em>Dominique Robinson, Cullen Wick, Scott Patchan, Kingsley Jonathon, Mike Tverdov, Manny Walker, Luiji Vilain, Charles Wiley, Marcus Moore, Kaden Roy, Michael Clemons, Brandon Barlow, Sam Okuayinonu, Ty Shelby</p><p><strong>Defensive Tackles</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/605/1*b4IHSXdt3_nsgS43Hw0nEQ.png" /><figcaption>Defensive Tackle Histogram</figcaption></figure><p>According to the RAS there are at least 30 Defensive Tackles with the height, weight, strength attributes teams look for. Unfortunately, only one prospect is seen as elite. The draft class for Defensive Tackles is very wanting, with 6 players getting a good grade by all the evaluators.</p><p>This is the year to skip taking a Defensive Tackle in the early rounds and seek develop prospects with athletic traits found in the late or as un-drafted free agents.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1S6N8sKe7V-1dtlijhHjMA.png" /></figure><p><em>Amazing Prospects: </em><strong>Jordan Davis </strong>or ‘The Hulk’ has the best testing scores for defensive tackles in RAS history. Davis is a bona fide Top 10 draft talent and great against stopping the run. For a guy his size, Davis has the endurance to play a high number of snaps too. <strong>Devonte Wyatt </strong>is an excellent pass rushing defensive tackle who will be available at the end of the first round or even into the start of the 3rd round because of off the field issues. <strong>Travis Jones</strong> is the second best run stopping defensive tackle in the draft. Jones is someone a team that missed on Davis can take in the first half of the 2nd round.</p><p><em>Sleepers: </em><strong>Logan Hall </strong>is good against the run, needs to work on his mechanics a to get even better. <strong>Thomas Booker</strong> is a late round possibility. Very talented and smart but needs to gain weight and muscle.</p><p><em>Who Dat?!: </em>Curtis Brooks, Ben Stille. Matt Henningsen, Eric Johnson, Jordan Jackson, Mika Tafua, Cody Roscoe, Elijah Garcia, Kurt Hinish, Alvin Peah, Weston Kramer, Keyshon Camp, Manny Jones, Zachary Carter, Miles Fox</p><p><strong>Strong Safeties</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/592/1*_oFXGwIxXLdvKIhoZIkUJw.png" /><figcaption>Strong Safety Histogram</figcaption></figure><p>The distribution of RAS scores shows that there are a large number of high end athletes at the Strong Safety position this year. Except for a single standout, most are not perceived as very high value. There are maybe 3 good Strong Safeties this year.</p><p>If your team is looking to draft Strong Safety, there are few worth getting and better off to grab one in the second half of the second round or early 3rd round. After that, the drop off in talent is evident but maybe some developmental gems can be found.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/1*2efhGTj6xoGn9Cv_iyPBqA.png" /></figure><p><em>Good Prospects:</em> <strong>Jaquan Brisker</strong> is a tough, instinctive safety with the speed to man cover tight ends and do zone. <strong>Jalen Pitre</strong> has the speed and agility to be a hybrid safety / slot corner.</p><p><em>Sleepers:</em><strong><em> </em>JT Woods </strong>brings serious long speed to the position but needs more development in the other aspects of the job. <strong>Sterling Weatherford </strong>will contribute in the league on special teams on day 1 and has the potential to develop into a tough hitting starting strong safety.</p><p><em>Who Dat?!: </em>Marcel Debo, Scott Nelson, Dane Belton, Rodney Thomas, Jack Koemer, McCrary-Ball, D’Anthony Bell, Chris Moore, Brandon Easterling, Tyrone Hill, Juanyeh Thomas, Genmark-Heath, Bubba Bolden, Van Zandt.</p><p><strong>Free Safeties</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/596/1*7XQK7VTRE3o3LxvoLTHDAw.png" /><figcaption>Free Safety Histogram</figcaption></figure><p>There are some truly elite level athletes at Free Safety. At least 15 of them have the speed and agility to excel in the NFL. Talent-wise, only 1 or 2 Free Safeties have been evaluated as elite. Most of the prospects have been graded as middling talents. PFF thinks that out of the best 100 draft prospects, just 5 are Free Safeties.</p><p>Unless a team can grab a top Free Safety in the 1st or 2nd round, it may be worth waiting into the 5th or later rounds to pick up developmental prospects.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vaZYxaCOjLqLoB5roerFng.png" /><figcaption>Free Safety Histogram</figcaption></figure><p><em>Amazing Prospects: </em><strong>Kyle Hamilton</strong> and <strong>Daxton Hill </strong>are both are hybrids at the position. Kyle Hamilton has both the speed &amp; skill to be free safety and the strength &amp; toughness to play strong safety. Daxton Hill has everything you want at free safety and the agility required to cover and shut down slot receivers. Hamilton has been described as a ‘unicorn’ by many draft analysts and Hill brings rare (~4.3 40 yard dash) speed for the free safety position. Kyle Hamilton is expected to be an early 1st round pick but could possibly drop due to a worse than expected 40 yard dash time at the NFL Combine. Daxton Hill can sneak into the bottom of round 1.</p><p><em>Sleepers: </em><strong>Lewis Cine</strong> is a big hitter that plays with skill. He has to continue developing his pass coverage. <strong>Nick Cross</strong>’s tackling is closer to strong safety in his play style and needs to get more agile and improve pass coverage. But Cross can match up with Tight Ends and become a NFL starter soon. <strong>Markquese Bell </strong>is great at run support but needs to develop pass coverage skills before coaches will trust him. Bell should be a go at special teams and be available in the 7th</p><p><em>Who Dat?!: </em>Joey Blount, Isaac Taylor-Stuart, Tycen Anderson, Mike Griffen, Drew Hartlaub, Nazzeh Johnson, Isaiah Pola-Mao, Abrae Booker-Currin, Reed Blankenship</p><p><strong>Inside Linebackers</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/592/1*N4TcdHOhmd_xPxwHkq6J-A.png" /><figcaption>Inside Linebackers Histogram</figcaption></figure><p>Half of all the Inside Linebacker prospects in the 2022 draft have athletic measurables much higher than the norm coming out of college. There are at least 11 who are at the very high end of the RAS score range. In terms of talent at Inside Linebacker, there are few elites, maybe 2. Most are lowly rated. PFF has 9 Inside Linebackers in the top 150.</p><p>This is not the best draft for Inside Linebackers. If your team can’t get one early, they may be better off not drafting one at all. Grab the athletes very late or as undrafted free agents.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0ijjOIKYBgsRsf0dn9driA.png" /></figure><p><em>Very Good Prospects: </em><strong>Chad Muma</strong> is a tackling machine and is a great middle line backer prospect. <strong>Quay Walker</strong> statistically led the Georgia line backer corps full of NFL level talent and has risen in team draft boards. Walker could go as early as the bottom half of the 1st round. .</p><p><em>Sleepers: </em><strong>Brian Asamoah</strong> is a weak side linebacker with good coverage skills and instincts vs the run. Speed is a key strength but Asamoah has to improve his tackling. An NFL team will take a gamble taking <strong>Damone Clark,</strong> hoping he can bounce back from spinal fusion surgery. <strong>Mike Rose </strong>is a late round prospect who could carve out in the NFL. <strong>Darrian Beavers </strong>is a big, tough throwback middle linebacker but comes with speed concerns. <strong>Terrel Bernard</strong> has had a bad injury history but comes with the speed to do man coverage at the NFL level. If he can get healthy and add play strength to his game, Bernard could make for a quality nickle line backer. <strong>D’Marco Jackson</strong> has some potential on the weak side plus can play special teams right away.</p><p><em>Who Dat?!: </em>Chance Campbell, Jake Hummel, Micah McFadden, Jack Cochrane,Silas Kelly, Joe Beckett</p><p><strong>Outside Linebackers</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/601/1*6P_28mf14eNqeWpJ37jcCw.png" /><figcaption>Outside Linebackers Histogram</figcaption></figure><p>Outside Linebackers make up the 2nd highest number of elite athletes in the RAS database of all defensive position groups. The histogram distribution of this position group doesn’t make sense. It’s <em>more</em> likely you’ll find a prospect with incredible physical traits at any point in the draft than not. It’s very surprising to see, and hints that teams could be aggressive in drafting Outside Linebacker prospects throughout the draft. Of all the prospects available, possibly four could be elite and PFF has 7 in the top 100. Most available are above average or good.</p><p>Almost every Outside Linebacker drafted in the first 262 picks will have great athletic traits this year. Most of them are set to be average to good NFL linebackers. This is not the year to find star Outside Linebackers but ones that will fill out rosters and contribute.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vPkx1ZMyxxoCyc1q8jOA7w.png" /></figure><p><em>Amazing Prospects: </em><strong>Devin Lloyd</strong> is the consensus best outside linebacker in the 2022 draft. Widely considered a low risk pick, Lloyd is a can do everything linebacker. <strong>Leo Chenal </strong>is a unique player who’s great stopping the run and blitzing but wasn’t asked to do much else in college. Most teams him classified as a middle line backer. Maybe he could learn how to get good at pass coverage? Chenal definitely has the physical tools to do so. <strong>Channing Tindall </strong>is <em>the fastest</em> line backer in this draft. <strong>Christian Harris</strong> is very athletic and came out of a great program. But Harris has been accused of not being tough or making enough plays.</p><p><em>Sleepers: </em><strong>Troy Anderson</strong>, with testing this off the charts historically, the right coaching could create a star. <strong>Brandon Smith</strong> is more of an athlete than player, needs lots of development. <strong>Malcolm Rodriguez</strong> fits the definition of underdog sleeper. Highly skilled, fast and capable but has height and size issues. If he can overcome them in the NFL, Rodriquez could be just as effective as any other linebacker in this draft class.</p><p><em>Who Dat?!: </em>Devin Harper, Cameron Goode, Adam Anderson, Jeremiah Moon, Baylon Spector, Tariq Carpenter, Vi Jones, Kyron Johnson (has some sleeper draft hype, probably goes undrafted), Grant Dixon</p><p><strong>Corner Backs</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/595/1*BPL7FNTR1iEaMCo3NaMGrw.png" /><figcaption>Corner Backs Histogram</figcaption></figure><p>There are many Corner Backs with high degree of athleticism in the 2022 NFL Draft according to the RAS scores. So many great athletes are available that at least 50 are at the high end of the RAS range. That being said, there seems to be drop off in talent after the first 2–3 prospects. The NFL draft evaluators graded most corner backs with middling grades or less. This is a deep group of Corner Backs coming out of college.</p><p>Incredible athletes in the entire corner back class, teams will find good prospects in all 7 rounds.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/988/1*I9o5naCEW3BEv9Fx6nQpdA.png" /></figure><p><em>Amazing Prospects: </em><strong>Derek Stingley</strong> has the potential to be an All Pro superstar. <strong>Trent McDuffie </strong>is shorter than people would like but makes up for it with great talent and jumping ability. Both of them will go very high in the 1st round as elite talents. <strong>Kair Elam</strong> may be the best press man corner in the draft (he barely did zone tho) and will be available late 1st round, early 2nd. <strong>Kyler Gordon </strong>is a incredible athlete who can play any where but is on the shorter side. Gordon will be picked in the 2nd round based on his explosiveness and speed. <strong>Martin Emerson</strong> is a good draft prospect that has the height and speed to play press corner as well as contribute to special teams. Emerson should go late 2nd round or 3rd round</p><p><em>Sleepers: </em><strong>Tariq Woolen</strong> and <strong>Zyon McCollum</strong> are two of the biggest risers on NFL draft boards after the Combine. Their combination of tested speed and height is tantalizing. McCollum has a RAS score of 10, meaning he is historically athletic for a corner back, but doesn’t have the best game tape. Woolen on the other hand tested great and his performance on the field showed. Both are from the FBS so there is some hesitancy in taking them too high. Woolen’s testing may have boosted him up to the 2nd round, and McCollum could go late 3rd or early 4th. <strong>Alontae Taylor</strong> has the height and speed, along with the skills to play press man on the edge. Taylor shouldn’t leave the 3rd round with potential like that. <strong>Damarri Mathis</strong> is a quality all around draft prospect but NFL evaluators have him graded all over the place. Its hard to predict where he’s slotted but wouldn’t surprise if Mathis goes in the 3rd or 4th round. <strong>Tariq Castro-Fields </strong>and <strong>Cam Taylor-Britt</strong> have good measurements and testing data but bad game tape. Both should go should go in the middle rounds based on potential. <strong>Akayleb Evans</strong> could get drafted in the final round based on speed alone because his game tape was not good.</p><p><em>Who Dat?!: </em>Triston McCullom, Tony Adams, Josh Thompson, Dallis Flowers, Joshua Williams, Sam Webb, Jaylen Watson, Jaylon Jones, Darrell Baker, Christian Holmes, Chase Lucas, Greg Junior, Tre Avery, Elijah Hamilton, Joshua Flowers, Zyon Gilbert, Allan George, Kader Kohou, Christian Matthew, Dominique Long, Devin Hafford, Ja’Sir Taylor, Josh Blackwell, Raleigh Texada, DaRon Bland, Christian Benford, Don Gardner</p><p><strong>Wide Receivers</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/601/1*WGABl7lBvvWIq7FwNF3isQ.png" /><figcaption>Wide Recievers Histogram</figcaption></figure><p>Wide receivers has distribution of RAS scores where most of the prospects are near the average level of athleticism found every year at the position. But this still means that many great athletes are available for teams to pick. Over 30 wide receivers with 8 or higher RAS scores are there, 20 of which may not even get drafted. Overall, most of the prospects are graded below average by NFL analysts tho. Of the 5 top rated wide receivers, 2 are graded as elite potential. This is a drop off from last year. In this draft, teams will have to take chances on developmental prospects, many who are tall and fast. There are less than average number of short and fast receivers with draftable grades available this year. It should be noted that of the wide receivers that did get high grades, many did not get RAS scores due to injuries.</p><p>There are not many can’t miss wide receivers for teams to choose from this year. There is a deep group of developmental prospects to choose from tho. This is a draft where NFL teams will have to take on higher risks than normal on prospects throughout the draft.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/984/1*5k7p2gmz28Xqh3m4DyDU0A.png" /></figure><p><em>Amazing Prospects: </em><strong>Chris Olave</strong> is smooth and fast, would make a great wide receiver 2. <strong>Christian Watson</strong> and <strong>Alec Pierce</strong> are high risk, high reward players that have rare height and speed combination. The only reason why <strong>George Pickens</strong> is not a 1st round talent is his bad injury history and slight frame. <strong>Jalen Tolbert</strong> can make for a very fast and tall ‘big’ slot receiver found in the 2nd round.</p><p><em>Sleepers: </em><strong>Khalil Shakir</strong> doesn’t have the height, weight, or arm length that the NFL desires in slot receivers but he can <em>play</em>. Shakir may not have the speed either but the game tape shows he’s got talent. <strong>Kevin Austin</strong> hasn’t played too many games in college or posted big statistics due to injuries. But there is raw talent there for a team to develop. Austin on game tape is aggressive getting the ball. <strong>Bo Melton</strong> plays fast and showed up in the Senior Bowl practices. Not the tallest but he can take the top off the defense. Also very fast but short by NFL standards, <strong>Calvin Austin</strong> also adds agility to the mix as slot receiver. He had great statistics in college and could transition that success into the NFL. <strong>Kyle Philips</strong> is another slot receiver who is not fast but a good route runner and has great hands. Philips could be found in rounds 5 or later and be very productive. <strong>Tanner Conner</strong> is a 7th round flyer who has good height-speed combination. <strong>Tyquan Thornton</strong> tested the fastest 40 yard dash at the NFL combine and should be available in the 7th round. <strong>Jalen Nailor</strong> came out of high school as the fastest wide receiver recruit but basically lost his college career to injuries. Some team will take the chance that Nailor gets healthy and regain his speed.</p><p><em>Who Dat?!: </em>Drake Young, Brendon Schooler, Isaiah Weston, Andre Miller, Tyshaun James, Ra’Shaun Henry, KeShunn Abram, Jalen Virgil, Jaquarii Roberson, Jalen Martin, Stone Smartt, Derek Wright, Lance McCutcheon, Irvin Charles, Cyrus Holder, Kaden Davis, Raleigh Webb, Dee Anderson, Sy Barnett, Samson Nacua, Thomas Reid, Jared Wyatt</p><p><strong>Tight Ends</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/609/1*L1YV3aK9NWJcONZhCXN8BA.png" /></figure><p>According to the RAS scores, there are more tight ends with very good athletic qualities but most are not rated very well by the draft analysts. The talent break down looks stratified, with more players in the back end than as elites. Based on my readings before this analysis, consensus opinion is that there aren’t any sure fire, top end tight ends this draft. But the RAS scores show that there is plenty of them that have strength and speed to make it in the NFL. PFF ranked only 2 in the top 50 but 5 more from 50–100 and near the 200 mark.</p><p>Teams looking for tight ends might be better off looking for developmental prospects in the 3rd round and lower. If they need speed, it can be found throughout. No one knows who’s <em>actually</em> good. Or if they’re all good?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/992/1*TbJB9KeibSJeAR705uQlBA.png" /></figure><p><em>OK Prospects: </em><strong>Trey McBride </strong>consensus best tight end but isn’t really good enough to stand out as much as hoped. Should be available in the lower half of the 2nd round.</p><p><em>Sleepers: </em><strong>Jelani Woods</strong> tested better than he actually plays. He’s very large for a tight end and can catch. But actual play speed isn’t what his 40 yard dash times hint at. <strong>Daniel Belinger</strong> very athletic prospect. <strong>Charlie Kolar</strong> is tall and fast, some have nicknamed him ‘Diet Cooper Kupp.’ Don’t ask him to block. <strong>Greg Dulcich</strong> is a very good blocker who could add to his receiving game. Has a good amount of recent draft hype. <strong>Grant Calcaterra</strong> is a talented short area / end zone receiver who is low on draft boards because of a serious history of concussions.</p><p><em>Who Dat?!: </em>Jamal Pettigrew, Andrew Ogletree, Lucas Krull, Rodney Williams, Ben Beise, Nick Muse, Peyton Henderson, Mohn Mitchell, Josh Babicz, Austin Allen</p><p><strong>Running Backs</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/599/1*fi27R4tVPR7iX1N5zdnSYw.png" /></figure><p>Most running backs that declared for the draft are average or above average RAS score relative to previous draft classes. This is not the most athletic group to come out. All the draft analysts graded most of the prospects available with middling grades. Maybe there are 2 high end running backs this year?</p><p>Running back needy teams can take the best late 2nd round or early 3rd. After that there is a cluster of prospects that aren’t well differentiated physically or skills-wise.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/986/1*Y5duehLTwfGytMClpHtYfw.png" /></figure><p><em>Amazing Prospects: </em><strong>Breece Hall</strong> is far and away the best athlete at running back. NFL Next Gen Stats gradded him 99(?!) <strong>Kenneth Walker </strong>may be the better running back. To me it feels like a toss up. Walker is the better all around, do everything running back. Breece is the home run hitter, fast &amp; agile back that goes to the outside.</p><p><em>Sleepers: </em>Like most of the running back prospects this year, there isn’t much differentiation between players. <strong>Zamir White, Pierre Strong and James Cook </strong>helped themselves get noticed with their excellent testing.</p><p><em>Who Dat?!: </em>Rashaad White, Master Teague, Zander Horvath, Bryant Koback, Maurice Burkley, Tyler Goodson, Isiah Pacheco, Kevin Brown, Keontay Ingram, Aaron Shampkin, Kingston Davis</p><p><strong>Offensive Tackles</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/601/1*dlzNv1TMp9wRByR7ztukNg.png" /></figure><p>This offensive tackle class is <em>amazing</em> when it comes to physical qualities. The number of prospects with very good to elite RAS scores is much higher than the norm. Teams will see average athletes at tackle only deep into the later rounds. In terms of talent 3 tackles have been evaluated as elite. Behind them is an unusually high number if well regarded prospects.</p><p>This offensive tackle draft class has both elite talent and depth. It’s a great year to draft one or more.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/989/1*_Zod_p0Vq926C3WjCcW5Cg.png" /></figure><p><em>Amazing Prospects: </em><strong>Ikem Ekwonu</strong> may be a top 5 pick in 2022 and is a very physical player. <strong>Trevor Penning</strong> and <strong>Bernhard Raimann</strong> both have first round grades on most draft charts and are also very gifted physically. Penning is extremely strong and Raimann (despite being 25 and new to football) showed great movement skills for a large player. <strong>Tyler Smith</strong> is considered by many to be the best run blocker in the draft. <strong>Zach Tom</strong> has been going up draft charts as a player who can play guard and tackle on both sides.</p><p><em>Sleepers: </em><strong>Matt Weletzko</strong> tested great but may be available in the 5th round. Despite testing very well, <strong>Kellen Diesch</strong> (a good run blocker) needs to further develop play strength in order to make it in the NFL. He is considered too tall and top heavy and opposing players beat him using the bullrush constantly. <strong>Abraham Lucas </strong>is a true sleeper who could go as high as the 2nd round and has position versatility. <strong>Braxton Jones </strong>is a solid tackle prospect who did well in the Senior Bowl. <strong>Sean Rhyan</strong> is getting more attention as a guard than tackle but could be drafted in the middle rounds.</p><p><em>Who Dat?!: </em>Tristen Taylor, Ryan Van Demark, Logan Bruss, Jake Dixon, Austin Deculus, Lewis Kidd, Clayton Bradley</p><p><strong>Offensive Guards</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/592/1*l135RNs8ZHBs1ZbdV2hdjg.png" /></figure><p>The RAS database has a very uniform distribution of scores for offensive guard prospects. Most of them have been graded as average to slightly below. Maybe there is 1–2 elite guards in this draft, with a drop off afterwards. The middle rounds may be where most of the guards are picked because good players may not be available late.</p><p>If your team needs a guard but misses out on the few elite players, round 3–4 could be where value meets supply. After that good, strong offensive guards will be hard to find.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hMUVGctkCKRQqtEaClXdGw.png" /></figure><p><em>Amazing Prospects: </em><strong>Zion Johnson</strong> is definitely the best guard prospect this year. <strong>Cole Strange</strong> is a fast riser and both them have positional versatility. Strange’s testing must have put him on the map for many teams as a 3rd round prospect.</p><p><em>Sleepers:</em> <strong>Dylan Parham</strong> is getting looked at for center, <strong>Jason Poe</strong> came on late in the draft evaluation season and is compared to pro-bowler Shaq Mason. <strong>Chris Paul</strong> and <strong>Joshua Ezeudu</strong> have not been graded well but their testing numbers made teams give them a second look. Ezeudu is probably a priority undrafted free agent.</p><p><em>Who Dat?!: </em>Sebastian Guitierrez, Nick Zalelj, Blaise Andries (killed it in the bench), Wesley French, Jack Snyder, Derek Schweiger</p><p><strong>Offensive Centers</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/1*7c83-UtPA9HIwAg2XWos0w.png" /></figure><p>While there are 10 offensive center prospects with high RAS scores, draft analysts are very down on the position in 2022 draft. Only 1 center is graded high. After a large step down, two more prospects are available the good evaluations. Expect many guards to be drafted with conversion into centers in mind.</p><p>Most centers drafted this year will be taken late as developmental players. There is no shortage of centers with the physical tools to make it in the NFL and it would not be a surprise if most are signed as undrafted free agents.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3re0KRaf7ZTg5sCzbvtqDw.png" /></figure><p><em>Amazing Prospects: </em><strong>Tyler Linderbaum</strong> is the consensus best center in the draft, and the only one worth taking high. But the RAS score really highlights the potential <strong>Cameron Jurgens</strong> has. He could be a steal in the middle rounds.</p><p><em>Sleepers: </em><strong>Luke Wattenberg, Doug Kramer,</strong> and <strong>Alec Lindstrom</strong> might sneak into NFL teams as late draftees or undrafted free agents</p><p><em>Who Dat?!: </em>Dawson Deaton had an massive RAS score and I never heard of him before. Michael Niese</p><p><strong>Quarter Backs</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/1*CCfJ0jFm9FhHTG88FIOgqQ.png" /></figure><p>There are only 3 high RAS (mobile) QBs with draftable grades this year. Many of the best regarded QB prospects did not do all the testing so the database is not representative. Beyond showcasing how good Pickett and Ridder are, RAS is of little value this year.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/974/1*Y5LbZwBEHB4QN0aOt2ypjg.png" /></figure><p><em>Amazing Prospects: </em><strong>Desmond Ridder</strong> tested very well (as expected) and should get some attention for teams that want a running QB. It is really odd to see K<strong>enny Pickett </strong>getting a great RAS score when he was heavily criticized for having small hands at the combine. Maybe his legs can make up for it?</p><p><em>Sleepers: </em><strong>EJ Perry </strong>would make a good 6–7th round flyer</p><p><em>Who Dat?!: </em>Never heard of Peters or Mancuso</p><p>Ok phew that was a lot. Hope you gained something from all that.</p><p>As a follow up, I hope to see how the RAS scores and NFL analyst predictions fared in the actual draft starting tomorrow.</p><p>Best of luck to your teams and thanks for reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f526ed533fa7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Run NVIDIA’s RAPIDS on a eGPU: How to Do Advanced ML/DS on a Laptop]]></title>
            <link>https://medium.com/@tiraldj/run-nvidias-rapids-on-a-egpu-how-to-do-advanced-ml-ds-on-a-laptop-400d43f0b588?source=rss-3b89e6336fb5------2</link>
            <guid isPermaLink="false">https://medium.com/p/400d43f0b588</guid>
            <category><![CDATA[rapids-ai]]></category>
            <category><![CDATA[nvdia]]></category>
            <category><![CDATA[egpu]]></category>
            <dc:creator><![CDATA[Mohammed R. Osman]]></dc:creator>
            <pubDate>Mon, 11 Apr 2022 16:07:03 GMT</pubDate>
            <atom:updated>2022-04-11T16:07:03.676Z</atom:updated>
            <content:encoded><![CDATA[<p>Access cutting edge computing with a GPU on a budget</p><p>TL/DR: Go to Section 2 on for the How-To steps.</p><h3><strong>What are we trying to do and why?</strong></h3><p>One of the main barriers to entry to learning advanced machine learning and data science is the cost and availability of equipment necessary. Especially when you do not want to use cloud compute resources or are more comfortable with local resources, cost can be prohibitive. Many only have a laptop to use and buying/building a full blown workstation is out of budget.</p><p>Already have a relatively new laptop? You can use an eGPU and a NVIDIA GPU to get around these cost issues to do RAPIDS. This package is one of the most advanced ML &amp; data science suites available and leverages graphics cards to significantly accelerate processes. For data science / ML practitioners that have the resources to build a workstation, you can buy a eGPU instead and reallocate the rest of the remaining funds to purchase a better GPU.</p><p>In my own work, I do onsite demonstrations and being mobile is key. Sometimes there’s poor or zero no wifi / ethernet. There are projects I’m working on where I’m legally prevented from using the cloud and have to maintain strict data security. So I need to be able to have high compute power, but be able to move around easily, and be able to do setup quickly and easily. The eGPU idea came out of these needs. Also I dumped all my resources into a great NVIDIA GPU card for research when one became available and didn’t have much money left for much else, haha.</p><h3>1. What do we need?</h3><p>Starting from the basic, the 3 main things you need are 1) a laptop or computer with USB-C/Thunderbolt connection, 2) an eGPU to contain GPU, and 3) a NVIDIA GPU that meets RAPIDS compute requirements.</p><p>If your computer/laptop is not currently running or dual booting Linux, you will need a free USB stick that can install a Linux distribution.</p><p>In terms PC hardware requirements, the main thing we need from your computer/laptop is a free USB-C/Thunderbolt connection. There are three different types of USB-C connections, the one we need is the latest one, USB-C 3.1, with the ‘<a href="https://www.intel.com/content/www/us/en/products/docs/io/thunderbolt/thunderbolt-technology-general.html">Thunderbolt</a>’ branding. While not as fast as a PCIe 3 connection to the motherboard, it has the speeds necessary for eGPU use. A quick check for USB-C/Thunderbolt is looking for a <a href="https://d33v4339jhl8k0.cloudfront.net/docs/assets/5a610cf50428635d7f43a263/images/5fc81bfbeb7cc612aa354528/file-vzjGAY5TCg.png">visual lightning bolt symbol</a> next to the connection. If your USB-C connection is completely unmarked but you were able to connect a monitor to it, look up the motherboard specs of your machine to see if it has Thunderbolt. If the connection can not drive a monitor, the machine is not viable for this project.</p><p>The more system RAM you have, the better. This way RAPIDS can offload data to system RAM when CPU functions are called without too heavy a hit on performance. Try to have 2x the amount of RAM as your GPU has VRAM. If you do not (like in the setup shown), you have to pay particular attention to do as much work on the GPU as possible. Which is good policy in general.</p><p>RAPIDS runs natively on Linux. Windows 11 runs the suite but for now, we will focus on the standard installation. I will use 64-bit Ubuntu but you can also use the 64-bit CentOS Linux distro.</p><p>GPU requirements are that is and NVIDIA card and has enough VRAM for your purposes. In my experience a good rule of thumb is to have 3–4x as much VRAM compared to size of your typical work datasets. The more VRAM on the card, the less work for the user and the system overall.</p><p>Note: not every NVIDIA GPU is compatible with RAPIDS. Please check NVIDIA’s product page to make sure your GPU has “compute capability” of 6 or above:</p><p><a href="https://developer.nvidia.com/cuda-gpus#compute">CUDA GPUs - Compute Capability</a></p><p>Complete Pre-Reqs for RAPIDS summarized from <a href="https://rapids.ai/start.html">RAPIDS’ “Getting Started”</a> below but main thing is to focus on the Linux OS and GPU compatibility. Everything else we will handle in the how-to:</p><p><strong>GPU:</strong> NVIDIA Pascal™ or better with <a href="https://developer.nvidia.com/cuda-gpus"><strong>compute capability</strong></a> 6.0+</p><p><strong>OS:</strong> Ubuntu 18.04/20.04 or CentOS 7/8 with gcc/++ 9.0+</p><ul><li>See <a href="https://docs.rapids.ai/notices/rdn0008">RDN 8</a> for recent changes to gcc/++ 9.0 requirements</li><li>RHEL 7/8 support is provided through CentOS 7/8 builds/installs</li></ul><p><strong>Docker:</strong> Docker CE v19.03+ and <a href="https://github.com/NVIDIA/nvidia-docker#quickstart"><strong>nvidia-container-toolkit</strong></a></p><ul><li><a href="https://rapids.ai/start.html#-docker-container">Legacy Support</a> — Docker CE v17–18 and <a href="https://github.com/NVIDIA/nvidia-docker/wiki/Installation-(version-2.0)">nvidia-docker2</a></li></ul><p><strong>CUDA &amp; NVIDIA Drivers:</strong> One of the following supported versions:</p><ul><li><a href="https://developer.nvidia.com/cuda-11.0-update1-download-archive">11.0</a> &amp; v450.51+</li><li><a href="https://developer.nvidia.com/cuda-11.2.0-download-archive">11.2</a> &amp; v460.32+</li></ul><p>External GPU (“eGPU”) is how we house the GPU and connect it to the computer/laptop using the USB-C/Thunderbolt connection. Main considerations are the size of eGPU and the power supply it contains. Make sure the eGPU housing’s size can fit the GPU of your choice and meets its electricity demands. Try to get an eGPU with more than the max electricity draw of your GPU. Spring extra for extras like USB connections and the ability to power the laptop. From personal experience, make sure that the eGPU wasn’t optimized for AMD cards and can handle NVIDIA cards. NVIDIA cards that can handle RAPIDS are usually built for high performance, so do not take eGPU compatibility as a given.</p><h3><strong>1.1 What I’m working with:</strong></h3><p>For this how-to, I used my 4 year old HP Spectre 360x. I brought it for writing , casual tablet &amp; media use,and honestly nothing else. It has 16 GB of RAM, a 512 GB SSD and a 2 core Intel CPU. This whole project turns this thin, convertible ultrabook into a machine learning beast.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/573/1*a3PnzI57sRMaeoOAg32ygg.png" /><figcaption>Hp Spectre x360, my “workhorse” laptop</figcaption></figure><p>Next thing is the eGPU. External GPUs come in many sizes and forms, you can even build one yourself What I’m working with:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/765/1*BVGhFHHmpInlEA-c5rfIqQ.png" /><figcaption>Razer’s eGPU enclosures</figcaption></figure><p>I went ahead and brought the Razer Core X Chroma refurbished off of eBay. Thought it was worth spending more for all the extras. Felt laptop power delivery, 700 Watt power supply, several USB connections and Ethernet cable was definitely worth it. It’s also got some dumb lights.</p><p>A spare USB stick was converted to an Ubuntu USB installer. Here are links to make one for <a href="https://ubuntu.com/tutorials/create-a-usb-stick-on-windows#1-overview">Ubuntu</a> and <a href="https://wiki.centos.org/HowTos/InstallFromUSBkey">CentOS</a>.</p><h3><strong>2. Step by Step Procedure</strong></h3><h3><strong>2.1 Preparing the PC/Laptop for Dual Booting Linux &amp; Windows</strong></h3><p>Go to this great <a href="https://itsfoss.com/install-ubuntu-1404-dual-boot-mode-windows-8-81-uefi/">how-to</a> first to familiarize yourself with the process.</p><p>I recommend creating a partition on your PC’s hard drive for Linux to live in. From Windows, look up ‘Create and format hard disk partitions’ or hit the Disk Management tool. Choose a disk drive that has ample space for Linux and your machine learning / data science needs and use the “shrink volume” function. I split my hard drive in half and set aside ~250 GB. Reboot your computer and on startup boot from the USB stick with the Linux distro.</p><p>If you need help on how to do this try going <a href="https://www.instructables.com/Install-any-linux-from-a-usb-the-easy-way/">here</a>. I used a stick with Ubuntu (20.04.2.0) and ran the demo on the laptop. Go ahead and try out the Linux distro. So far nothing will change for your computer. Check to see if everything is working, like mouse, screen(s), etc just to make sure there’s a high level of hardware compatibility. If you have audio jack issues, that unfortunately is common.</p><p>Once you’re ready to go, hit the install icon and start the process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/342/1*o3taSZ8FnLNrbZ4VnfixcQ.png" /><figcaption>Install Ubuntu from the USB</figcaption></figure><p>From there do the normal installation</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*yG5qzFrRQbzDmAO6nWFMGA.png" /><figcaption>Go with the ‘Normal installation’ or use ‘Minimal’ if hard drive space is an issue</figcaption></figure><p>The Linux installer will realize that you have a Windows setup and ask if you want to do a duel boot or erase everything and install Linux only. Go with the duel boot option and install the OS into the free partition created by the ‘shrink volume’ from before.</p><p>Ok, now you got a duel booting machine! Let’s poke around</p><h3><strong>2.2 Understanding Your Setup Before Attaching the eGPU</strong></h3><p>Let’s first see what the machine looks like before adding the eGPU. Go to ‘Terminal’ and use the following command:</p><blockquote><em>lspci | grep VGA</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MxoUu4Da5PB2mv4GEX9cfw.png" /><figcaption>Detecting the internal (CPU) GPU</figcaption></figure><p>Here we can see a single GPU, HD Graphics 620 that is on the laptop’s CPU. Let’s put together the eGPU.</p><h3><strong>2.3 Install the NVIDIA GPU into the eGPU</strong></h3><p>Follow the instructions included with your eGPU.</p><p>It show go something like this: 1) take the enclosure apart, 2) put in the the graphics card on the PCI connector, 3) attach the appropriate power chords from the power supply to the graphics card, 4) close up the enclosure, 5) attach USB-C/Thunderbolt cord from the eGPU to the laptop/pc, 6) attach cord to monitor (<strong>DON’T, </strong>will explain why later), and 7) attach power cord from outlet to the eGPU enclosure.</p><p>Here are some pictures of me in the process:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5mdaE5P57PsEAJGN5gOHGQ.jpeg" /><figcaption>Here’s the inside of the Razer with the Titan RTX installed. Try to keep the GPU fans unimpeded and use plastic ties to organize the power wires</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gNLL5SRU6jtPvj4k9uCiUg.jpeg" /><figcaption>With the enclosure fully closed and locked in.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bz3sfupAysXUOg_Wy6QsMw.jpeg" /><figcaption>Technical Problems! Secure Boot enabled caused this to happen</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X4lGtKYqDa9Dwu4muOV7iA.jpeg" /><figcaption>Neon lights mean the eGPU is working, at least hardware-wise</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lirCR2nNnW3XY6Iq4CXPvw.jpeg" /><figcaption>Ok! the work bench up and running. eGPU to the right side because that’s where the USB-C connections are.</figcaption></figure><h3><strong>2.4 Making the proper connections between the eGPU and the Linux OS</strong></h3><p>Windows will most likely self install the drivers and ask for some permissions once you attach the eGPU to the computer and power it up. Linux will have some extra steps, for security reasons.</p><p>First it’ll let you know that something has to be “authorized.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/516/1*UwJki5z1XHGLC7ai6H4PDQ.png" /><figcaption>Prompt you should see when you first connect the eGPU using a USB-C/Thunderbolt</figcaption></figure><p>Go to Settings and search for “Thunderbolt” , that should bring you to the following screen:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/991/1*6Pi_jp_CTdOpncBZCUghlw.png" /><figcaption>Thunderbolt Settings will ask you to give access to the ‘2’ eGPUs</figcaption></figure><p>Click on both and “authorize and connect” them. Now your eGPU has direct connections to the CPU through PCIe slots.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/987/1*NBIuaePQFm11LGagzmIUkQ.png" /></figure><p>Close Settings and let’s check if our computer now sees the 2nd GPU using the terminal:</p><blockquote><em>lspci | grep VGA</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ucCSH4mWsv6EN3Qqey1RAg.png" /><figcaption>Check to see if Ubuntu is detecting the new hardware</figcaption></figure><p>The NVIDIA graphics card has definitely been found and connected. Most likely it is using the “Nouveau” generic graphics card drivers. Let’s get started on that.</p><h3><strong>2.5 Setting the right graphics drivers for RAPIDS</strong></h3><p>First let’s update the operating system fully, using the following command. You can update using the “Software &amp; Updates” utility in the GUI as well.</p><blockquote><em>sudo apt upgrade &amp;&amp; sudo apt update</em></blockquote><p>This may require your system to reboot. Go ahead and log back on to Ubuntu.</p><p>With the base operating system up to date, we will install the NVIDIA drivers. This is best done using the GUI interface and “Software &amp; Updates” utility.</p><p>When it opens, it default opens to “Updates,” as shown below. Click on the “Additional Drivers” tab.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1005/1*Ch8ssEXNZTRkwj70jKZw6Q.png" /><figcaption>‘Software &amp; Updates’ will open, go to ‘Additional Drivers’</figcaption></figure><p>The OS will scan for hardware and generate a list of drivers available for your NVIDIA GPU (below). As expected the system is currently using the default Nouveau display drivers, which are perfectly fine. But RAPIDS needs to use the 460 (proprietary) drivers. Notice this is <strong>NOT</strong> the latest NVIDIA drivers nor the server drivers. For reliable results do not use those and do not upgrade to the latest (470 as of writing) until its been tested by others to be compatible.</p><p>Click on the “Using NVIDIA driver metapackage from ‘nvidia-driver-460 (proprietary)” and then apply changes. Reboot afterwards if the OS asks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/999/1*0Iv7CP9S7AoAVj7OCTyTMg.png" /><figcaption>At first Ubuntu will default to open source ‘Nouveau’ graphics drivers. Switch to ‘nvidia-driver-460’ metapackage and restart</figcaption></figure><h3><strong>2.6 Final Hardware Check</strong></h3><p>Log back on to Ubuntu check if everything is in working order. With NVIDIA graphics cards, the most commonly used diagnostic to see if the card is interfacing with the operating system correctly is the following command:</p><blockquote><em>nvidia-smi</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/827/1*0CqFj3XfwvCZq0hEq6a4Zg.png" /><figcaption>Use terminal command ‘nvidia-smi’ to check the status of the GPU. Check if CUDA Version is 11.2</figcaption></figure><p>If the above screen pulls up on the terminal, everything is working. The computer is using the eGPU and the graphics card inside it!</p><p>If this install of linux is not fresh and you had a previous installation of CUDA installed, the CUDA version the system may be using could be different from 11.2. RAPIDS is compatible with CUDA versions 11.0 and 11.2 , you will have to take steps to remove CUDA altogether and install one of those.</p><h3><strong>2.7 Install Anaconda Python</strong></h3><p>RAPIDS is a combination of Python and C++ programing, with our eGPU working within Linux, we can now get the package going. First thing to do is install Python in the system. I recommend installing Anaconda or Miniconda as it is very robust and comes with Jupyter Notebook. Go to Anaconda’s <a href="http://www.anaconda.com/">homepage</a> and grab the latest install package appropriate for your machine. In most cases it will be the <a href="https://www.anaconda.com/products/individual">Individual Edition, Python 3.8, 64-bit for x86 systems</a> (scroll all the way down).</p><p>If hard drive space is limited, use Miniconda instead. The installer can be found <a href="https://docs.conda.io/en/latest/miniconda.html">here</a>. Make sure to choose the appropriate Linux package and Python version 3.8.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M7D3_Qy22Obi3SOSNuVSPg.png" /><figcaption>Download Anaconda Python, for most people it will be the x86 installer</figcaption></figure><p>Go to where the installer file was downloaded to and use the bash function to install the Anaconda file (usually Downloads):</p><blockquote><em>bash [Anaconda3/Miniconda3]-[VERSION/STABLE DATE]-Linux-x86_64.sh</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7iLjxVzze3CXNKbvOq6xrw.png" /><figcaption>Using Terminal, go to downloads and use ‘bash’ command to install Anaconda</figcaption></figure><p>After installing Anaconda or Miniconda, do a quick update:</p><blockquote><em>conda update conda &amp;&amp; conda update anaconda</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/707/1*0Lzcby3mms_UxcomVjT8EA.png" /><figcaption>Update Anaconda Python</figcaption></figure><h3><strong>2.8 Install NVIDIA RAPIDS</strong></h3><p>Now we are ready to install the RAPIDS package. Go to the “<a href="https://rapids.ai/start.html">Getting Started” page</a> of the RAPIDS homepage and use the RAPIDS release selector applet to get the right code for your machine.</p><p>For most people the selector should look like this and yield the code to download it. Below I updated the code to add in Jupyter Lab and also install everything automatically:</p><blockquote><em>conda create -n rapids-21.10 -y -c rapidsai -c nvidia -c conda-forge \</em><br><em> rapids-blazing=21.10 python=3.8 cudatoolkit=11.2 jupyterlab</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q9RhjT9HkXYYJD-DqSlNvw.png" /><figcaption>These are the typical settings for install, make sure to change if you decided to use different OS or packages</figcaption></figure><p>Apply that code using the terminal and RAPIDS will install.</p><h3><strong>2.9 RAPIDS Check</strong></h3><p>Afterwards, go ahead and check to see if everything is working by first activating the RAPIDS environment under Python, then running Juptyer Notebook:</p><blockquote><em>conda activate rapids-21.10</em></blockquote><blockquote><em>jupyter notebook</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/502/1*a5vhkqBZDvip-4dBdr2spw.png" /><figcaption>Note: If you have installed a newer or older version of RAPIDS, replace “21.10” with that version number</figcaption></figure><p>With the Jupyter Notebook open, load a dataframe using “cudf” and a csv data file of your choosing. Then use the nvidia-smi function to check that the dataframe is being hosted into the VRAM of your GPU. You can use the following example code. Here is documentation for <a href="https://docs.rapids.ai/api/cudf/stable/">cuDF</a>, if you have questions on how to import datasets online or offline:</p><blockquote><em>import cudf</em></blockquote><blockquote><em>gdf = cudf.read_csv(“</em><a href="https://raw.githubusercontent.com/rapidsai/cugraph/branch-21.10/datasets/karate.csv"><em>https://raw.githubusercontent.com/rapidsai/cugraph/branch-21.10/datasets.karate.csv</em></a><em>”, delimiter=’ ‘, dtype = [‘int32’, ‘int32’, ‘float32’], header=None)</em></blockquote><blockquote><em>!nvidia-smi</em></blockquote><p>Here is the example code</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g-tD0w4E9UjMIDxYYcj3pg.png" /><figcaption>Running a quick test to verify that RAPIDS and NVIDIA GPU are working. Under ‘Processes’ you can see RAPIDS cuDF loading a dataset into the GPU’s VRAM</figcaption></figure><p>Hopefully you see a process running on the GPU, indicating that RAPIDS is running on your system without any problems.</p><p>You are now capable of doing next level data science and machine learning using NVIDIA’s RAPIDS!</p><h3>2.10 Troubleshooting</h3><p>If there is a compatibility issue with your eGPU, the NVIDIA GPU, and the laptop/PC, try using this <a href="http://www.pocketnix.org/posts/eGPUs%20under%20Linux%3A%20an%20advanced%20guide">guide</a>. I found using the open source “<a href="https://github.com/hertg/egpu-switcher">eGPU Switcher</a>” to be helpful when the problem stems from the Linux OS. An <a href="https://developer.nvidia.com/blog/accelerating-machine-learning-on-a-linux-laptop-with-an-external-gpu/">NVIDIA Developer blog</a> on using eGPUs was also helpful when the issue had to do with video drivers. Also from personal experience make sure to keep Secure Boot off, here’s <a href="https://wiki.ubuntu.com/UEFI/SecureBoot/DKMS">how</a>.</p><h3><strong>3. Notes on using and choosing a eGPU</strong></h3><p>One thing that I highly recommend users to do is NOT try to push on the GPU eGPU’s USB-C/Thunderbolt IO due to lower bandwidth than what we’re used to with PCI-E connection on the motherboard. RAPIDS excels because it does all your ETL on the GPU and you’ll need to leverage this as much as you can. Use the connections on your PC as much as possible and avoid using the eGPU’s connections. This even includes using the GPU’s video connection(s) too.</p><ul><li>The more you use the videocard directly, the more processes and VRAM is made unavailable for RAPIDS to use. Even with the 24 GB VRAM of my Titan RTX card, the VRAM can be used up very quickly.</li><li>Regardless of how much VRAM your NVIDIA GPU has, if the size of the data you are working with is too large, RAPIDS will start using system RAM. This causes a significant hit on performance. If the Python processes you are running run out available on board RAM and VRAM, Linux will start using the swap file on the hard drive. This is catastrophic in terms of time, speed, and accuracy.</li><li>Always try to minimize data transfers between the computer and GPU as much as possible. As you get serious with using GPUs to do machine learning and data science, managing memory will be a part of project design. You can use this humble eGPU setup to start learning how.</li><li>The more non-GPU data traffic the USB-C/Thunderbolt connection between the laptop/workstation has, the slower everything will be. So don’t use the eGPU’s USB connections for external hard drives, keyboard, mouse or other peripherals. But this doesn’t mean you have to abandon the use of all of the eGPU’s extra connections and features, you can alternate how the connection is being used. A practical example of this that I do regularly is having portable hard drives and ethernet internet transfer data to and from the laptop’s SSD when the GPU is idle</li></ul><p>Understand the enclosure you are housing the GPU. My experience with the Razer Core X Chroma really showed me the importance of having a tough, beefy enclosure. I’ve been able to take it everywhere, cars, trains, etc. and it was able to take a pounding while protecting the GPU. There were lighter, smaller options with less metal and at cheaper price points. But having a knock around capable housing really let me be mobile without worry. I was able to carry mine around on a small rolling suitcase without problems or attracting attention. It’s significantly better than carrying around a mid tower PC &amp; projector.</p><ul><li>Maybe I could’ve gotten away with a custom small factor machine. For this project a off the rack eGPU was chosen so more people can replicate it.</li><li>If you don’t plan on taking your eGPU out and about, then toughness is not worth paying extra.</li><li>If you plan on using a NVIDIA GPU that doesn’t have a large electricity draw, you can also use a eGPU with a less powerful power power supply. Always make sure your eGPU’s power supply can push out more than the GPU’s max power draw. This will physically protect all the components and prevent throttling of the GPU.</li><li>Enclosures with extra IO available are not necessary or recommended for data science / ML work but can definitely be a nice to have for other activities.</li></ul><h3><strong>4. Conclusions</strong></h3><p>After two months of use I can honestly say that my thin ultrabook built casual has turned into a ML &amp; data science beast. It’s now capable of doing so much, it’s mind blowing. It is viable to first generate a prototype with this setup then use cloud computing to completion using dask and multi-GPU packages. My original plan was to build a custom workstation for work this fall but with this eGPU setup, those plans are delayed by at least a year or two.</p><p>I honestly feel like this experiment with an alternative hardware setup has been a success. Much better than my expectations. By using an eGPU instead of building a new workstation, the money savings were redirected to upgrading the GPU used. Which then allowed me to use RAPIDS at a much higher level and do more advanced work, faster. The eGPU also allowed me to be mobile and do presentations in secure environments or ones without internet. I plan to keep using this setup for prototyping and presentation work. In 2–3 years I will build a full workstation and use this as a full time mobile setup, backup work setup.</p><p>Using the eGPU for RAPIDS is viable and very cost effective.</p><h3>Future Projects</h3><p>RAPIDS recently became compatible with Windows 11, if there is interest I can do a write on how to set that up.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=400d43f0b588" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is Happening Out there? A Deep Dive into Coronavirus (COVID-19) World Data. (Part 1)]]></title>
            <link>https://medium.com/@tiraldj/what-is-happening-out-there-a-deep-dive-into-coronavirus-covid-19-world-data-part-1-74a7b7aaa70d?source=rss-3b89e6336fb5------2</link>
            <guid isPermaLink="false">https://medium.com/p/74a7b7aaa70d</guid>
            <category><![CDATA[covid19]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[coronavirus]]></category>
            <dc:creator><![CDATA[Mohammed R. Osman]]></dc:creator>
            <pubDate>Sun, 22 Mar 2020 02:26:50 GMT</pubDate>
            <atom:updated>2020-04-13T22:05:30.643Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Hwe2CNQkO7IFNk8H" /><figcaption>Photo by <a href="https://unsplash.com/@markusspiske?utm_source=medium&amp;utm_medium=referral">Markus Spiske</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>[Updated: April 6, 2020 ] <em>Part 1: Establishing context.</em> There’s so much we don’t know. We do not know how many cases of new Coronavirus infections are happening every day, how many total infection cases have occurred or even how many deaths are due to the virus. Despite all this uncertainty, nations, states/provinces and cities have to had to make hard decisions.</p><p><strong>Purpose</strong>:</p><p>A series of posts attempting to understand the ‘sparse’ data available at the world, country and regional level using simple statistical techniques. Attempts will be made to generate alternate figures and projections of the future.</p><p>By writing, explaining and showing the data in a readable manner, my hope is to help others grapple with the flood of information on COVID-19.</p><p>This first post is to explain what we know about the Coronavirus descriptively and set the stage for number crunching later. There will be lots of links.</p><p><strong>Disclaimer</strong>:</p><p>I am not a Epidemiologist or in the medical field. My background is in Climate Change and modeling as a PhD in Department of Technology &amp; Society at School of Engineering in Stony Brook University, NY. I did do a Human Biology undergrad at College of Human Ecology at Cornell University. The thoughts and opinions posted here do not necessarily reflect those of the these institutions and are completely my own. I declare no competing interests. All posts will be for public consumption and free. My parents will never forgive me for not being a real doctor but I hope you can when mistakes are made here.</p><p><strong>Corrections</strong></p><p>[EDIT 3, 4/6/2020, Credit to<em> Justin</em>] I made the mistake of thinking that Coronavirus, SARS-CoV-19, and COVID-19 were all referring to the same thing. Coronavirus was the early general name for the virus from the <em>base family of viruses</em> it belonged to. SARS-CoV-19 is the name of the <em>specific virus</em>. COVID-19 is the <em>disease</em> caused by the virus. So you can get infected by the virus but not develop the disease. My brain fell into a haze from reading all this material and the three terms mistakenly became one.</p><p><strong>What is the Coronavirus? What does COVID-19 do to the body?</strong></p><p>As illustrated in Nature’s<a href="https://www.nature.com/articles/d41586-020-00758-2"> “The coronavirus pandemic in five powerful charts,”</a> the Coronavirus is a serious global pandemic. <a href="https://www.cdc.gov/coronavirus/2019-ncov/downloads/2019-ncov-factsheet.pdf">Here’s the US Center of Disease Control’s Fact Sheet</a> for basic information and symptoms. USA Today has great illustrations on the virus <a href="https://www.usatoday.com/in-depth/news/2020/03/13/what-coronavirus-does-body-covid-19-infection-process-symptoms/5009057002/">here</a>. This is a piece from the National Institutes of Health, “<a href="https://www.ncbi.nlm.nih.gov/books/NBK554776/">Features, Evaluation and Treatment Coronavirus (COVID-19)</a>” for a more thorough description. National Public Radio described COVID-19 as deadlier than Influenza and addressed criticisms of how seriously we should take it in “<a href="https://www.npr.org/sections/goatsandsoda/2020/03/20/815408287/how-the-novel-coronavirus-and-the-flu-are-alike-and-different">How The Novel Coronavirus And The Flu Are Alike … And Different</a>.” I also recommend this Science Insider video <a href="https://www.youtube.com/watch?v=OOJqHPfG7pA">“What Coronavirus Symptoms Look Like, Day By Day.”</a></p><p>It is outside of our scope to go into depth on the biological aspects of COVID-19 but for our purposes keep these in mind (it’s a lot to read, feel free to skip to the main points in <strong>bold</strong>):</p><ol><li>80% of the Infected are ‘mild to medium’ in severity and 20% require hospitalization. This <strong>80/20 ratio </strong>can be used to estimate how many infected are not in hospitals. The Chinese Center of Disease Control in “<a href="http://weekly.chinacdc.cn/en/article/id/e53946e2-c6c4-41e9-9a9b-fea8db1a8f51">Vital Surveillances: The Epidemiological Characteristics of an Outbreak of 2019 Novel Coronavirus Diseases (COVID-19) — China, 2020</a>” goes on to break down the 20% into <strong>15% of infected are “severe”</strong> and the remaining <strong>5% as “critically ill.” </strong>These two numbers can be used to backwards estimate the overall infected population based on the number of deaths</li><li>Based on the March 10, 2020 publication of <a href="https://annals.org/aim/fullarticle/2762808/incubation-period-coronavirus-disease-2019-covid-19-from-publicly-reported">“The Incubation Period of Coronavirus Disease 2019 (COVID-19) From Publicly Reported Confirmed Cases: Estimation and Application”</a> the calculations here will assume the virus takes <strong>5 days to incubate inside an infected human before they can be diagnosed as infected.</strong> COVID-19 can take 2–14+ days to incubate after successful unprotected contact with the virus but this assumption will simplify how we look at things. After these 5 days, the infected can go on and infect others.</li><li><strong>The Coronavirus is very stealthy</strong>. It can infect others often before making it’s host victim sick. This is termed as asymptomatic, being sick without showing any signs of it. From <a href="https://www.eurosurveillance.org/content/10.2807/1560-7917.ES.2020.25.10.2000180">“Estimating the asymptomatic proportion of coronavirus disease 2019 (COVID-19) cases on board the Diamond Princess cruise ship, Yokohama, Japan, 2020”</a>, 17.9% of the infected were asymptomatic for their entire experience. According to <a href="https://www.medrxiv.org/content/10.1101/2020.03.05.20031815v1">“Estimating the generation interval for COVID-19 based on symptom onset data”</a>, 48–62% of patients were infectious before showing any symptoms. Meaning that after the first 5 days of incubation<strong>, more than half of victims do not show any signs of infection, almost 1 out of 5 never do. But all are infectious.</strong></li><li>Once symptoms start to show, the Chinese Center of Disease Control and Wuhan University researchers describe <strong>infected as requiring hospitalization at day 7. In day 8, 20% of patients (the “severe” and “critically ill” categories) will experience Acute Respiratory Distress Syndrome</strong> as described in The Lancet, <a href="https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(20)30211-7/fulltext">“Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study.”</a> Acute Respiratory Distress Syndrome(ARDS) is difficulty breathing due to fluid build up in the lungs, preventing oxygen from being supplied to vital organs, and can be fatal. <a href="https://health.clevelandclinic.org/heres-the-damage-coronavirus-covid-19-can-do-to-your-lungs/">“Here’s the Damage Coronavirus (COVID-19) Can Do to Your Lungs”</a> from the Cleveland Clinic is an easy to understand explainer of COVID-19 and ARDS. Information on diagnosis and treatment of this condition can be found from the Mayo Clinic <a href="https://www.mayoclinic.org/diseases-conditions/ards/symptoms-causes/syc-20355576">here</a>. The Coronavirus at this stage can be detected in CT scans of the chest. It should be noted that standard pneumonia and COVID-19 can be differentiated in the CT scans of the chest. For the severely or critically ill that survive, recovering lung function from ARDS <a href="https://www.thetimes.co.uk/article/even-coronavirus-survivors-can-be-left-with-lung-damage-that-takes-15-years-to-heal-msh8zmwtb">may take years</a>. For the 20% that needed to be hospitalized, <strong>approximately Day 17 after symptoms first showed, most patients can leave. </strong>Additional studies on how COVID-19 progresses <a href="https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(20)30183-5/fulltext">here</a> and <a href="http://dx.doi.org/10.1001/jama.2020.1585">here</a>, both using Chinese data.</li><li><strong>The COVID-19 is fatal 14 to 56 days after the first onset of symptoms </strong>according to the World Health Organization’s<strong> </strong><a href="https://www.who.int/docs/default-source/coronaviruse/who-china-joint-mission-on-covid-19-final-report.pdf">“Report of the WHO-China Joint Mission on Coronavirus Disease 2019 (COVID-19).”</a> In every publication and data set I have personally seen, <strong>no deaths have occurred in the COVID-19 infected exhibiting “mild” to “severe” symptoms, or 95% of all infected. Fatalities came from patients in “critically ill” category, 5% of the infected. </strong>PLEASE NOTE: this does not mean it can’t happen to the other 95%, where the infection symptoms progress in severity unstopped. Just that it hasn’t emerged from the current data we have.</li><li><strong>It is extremely hard to estimate how fatal COVID-19 is.</strong> To understand it, a set of estimates will be used. In Epidemiology how fatal a disease can be described by Mortality rate and Case Fatality Rate (CFR). Mortality is the number of deaths from a disease divided by entire population studied. CFR divides the deaths caused by the disease divided by the number of cases of said disease. CFR will be used to describe how deadly the Coronavirus is in this post. At this early stage, there is no one agreed upon CFR rating and the rating itself will change over time as diagnosis and treatment improves. The Centre for Evidence-Based Medicine at Oxford University <a href="https://www.cebm.net/global-covid-19-case-fatality-rates/">posted</a> (updated to March 17, 2020) a CFR of <strong>0.14%</strong> for the general population. They have a great breakdown of case observations by country and explain the variations in the CFR estimations. An earlier assessment (February, 2020) using the Chinese population only, <a href="https://jamanetwork.com/journals/jama/fullarticle/2762130">“Characteristics of and Important Lessons From the Coronavirus Disease 2019 (COVID-19) Outbreak in China”</a> estimated the CFR to be <strong>2.3% </strong>(1023 of 44 672 confirmed cases). Currently the <a href="https://systems.jhu.edu/research/public-health/ncov/">Johns Hopkins COVID-19 database</a> has CFR at<strong> 3.4%</strong> in aggregate for the entire world. The overall trend in all cases is that the mortality metrics are extremely high at the beginning of the pandemic, and at the center of infection locations, then drift lower over time.</li><li><strong>Fatalities are highly correlated to age and comorbidities</strong>. Most deaths happening to those above 60 years old. Without access to COVID-19 data broken down by age or patient health data at the moment, analysis will not be done on this aspect of COVID-19 fatalities. Hopefully the appropriate data at the country &amp; world levels are made available in the not too far future. That being said, the topic should be broached. The Centre of Evidenced-Based Evidence at Oxford University in their CFR post gave estimations by age and comorbities <a href="https://www.cebm.net/global-covid-19-case-fatality-rates/">here</a>. The group does place fatality rates to ages below 60 but the estimates for those above 70 years are alarming. <strong>From 70–79, CFR is estimated to be 8% </strong>(7.2%-8.9% with confidence level). <strong>Above age 90, CFR is estimated to be 14.8%</strong> (13%-16.7%). <strong>For patients with other negative health conditions, nothing is worse than cardiovascular disease (10.5% CFR), followed by diabetes (7.3% CFR). </strong>Surprisingly, these two conditions were worse to have than existing respiratory illnesses (6.3%), hypertension (6.0%) or cancer (5.6%). These estimates align with some of the summary data available at <a href="https://www.worldometers.info/coronavirus/coronavirus-age-sex-demographics/">World of Meter</a>. There is no consensus yet on which conditions interact with COVID-19 to create the worst outcomes. <a href="https://www.ijidonline.com/article/S1201-9712(20)30136-3/fulltext">“Prevalence of comorbidities in the novel Wuhan coronavirus (COVID-19) infection: a systematic review and meta-analysis”</a> in the International Journal of Infectious Diseases, did a meta-analysis of multiple studies and found the most prevalent comorbidities to be hypertension and diabetes followed by cardiovascular disease and respiratory illness. An <a href="https://www.thelancet.com/journals/lanres/article/PIIS2213-2600(20)30116-8/fulltext">post </a>in Lancet tries to explain this finding by asking if those with hypertension and diabetes are more likely to get infected. There is more work in this area to be done.</li><li>Despite much lower fatality statistics and expectations, <a href="https://www.cdc.gov/coronavirus/2019-ncov/faq.html?CDC_AA_refVal=https%3A%2F%2Fwww.cdc.gov%2Fcoronavirus%2F2019-ncov%2Fprepare%2Fchildren-faq.html#anchor_1584387482747"><strong>the Coronavirus is just as likely to infect young people as adults</strong></a>. The US Center of Disease Control <a href="https://www.bloomberg.com/news/articles/2020-03-19/coronavirus-in-young-people-is-it-dangerous-data-show-it-can-be">reported </a><a href="https://www.sciencenews.org/article/coronavirus-covid19-young-adults-can-face-severe-cases">that </a>28.2% of the first 2,500 COVID-19 cases occurred in Americans aged 20 to 44. From February 12 to March 16, 2020, 508 out of reported 2,449 COVID-19 cases had to be hospitalized. Patients aged in that age group made up 12% of these cases. <strong>The infected among the 60 years old and under are most likely to be among the 80% that show no to mild symptoms…and spread the virus.</strong></li><li>One last note, researchers and government reports from Italy (<a href="https://www.epicentro.iss.it/coronavirus/bollettino/Infografica_17marzo%20ITA.pdf">in Italian</a>) indicate that COVID-19 fatalities differs by sex. This issue has been reported upon by the <a href="https://www.latimes.com/science/story/2020-03-21/why-is-the-coronavirus-more-deadly-for-men-than-for-women">L.A. Times</a> and<a href="https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(20)30526-2/fulltext"> the Lancet</a>. <strong>Men are much more likely to die from the Coronavirus than females.</strong> From February 21 to March 12, 2020 in Italy, males represented 58% of the infected but 72% of the fatalities. This observation was much worse than those found in China through <a href="https://onlinelibrary.wiley.com/doi/pdf/10.1002/jmv.25757">meta-analysis</a> (60%) or official Chinese CDC <a href="http://weekly.chinacdc.cn/en/article/id/e53946e2-c6c4-41e9-9a9b-fea8db1a8f51">reporting</a> (male fatality having 65% more likelihood than females). Like the data on age and comorbidities, most of the public data does not include information on sex. [Edit March 28, 2020] We do not know why Males are having these outcomes and there are many confounding factors, including cultural ones, that are at play.</li></ol><p>The following are additions and edits/clarifications written in after the post was originally published.</p><ol><li>[Edit 1, 3/28/2020] In March 27, 2020 the <em>American Journal of Respiratory and Critical Care Medicine </em>pushed the paper<em> “</em><a href="https://www.atsjournals.org/doi/pdf/10.1164/rccm.202003-0524LE">Time Kinetics of Viral Clearance and Resolution of Symptoms in Novel Coronavirus Infection</a>,” where a small sample of recovered COVID-19 Chinese patients were tracked and<strong> 50% were still infectious despite no longer needing care or exhibiting any symptoms</strong>.<strong> Patients remained contagious after the end of their symptoms ranged from one to eight days.</strong> The researchers also posited that victims who had severe or critically ill diagnosis of COVID-19 may be infectious longer than that. These results are supported by an an earlier retrospective study of Chinese patients in <a href="https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(20)30566-3/fulltext#back-bib33">The Lancet</a> that found median duration of surviving patients being infectious was 20 days<strong> </strong>[unclear if this number is from the first sign of symptoms or after the 5 days of incubation], with a maximum of 37 days or 5 weeks observed. Another <a href="https://www.medrxiv.org/content/10.1101/2020.03.05.20030502v1">small sample study</a> from Germany, found COVID-19 patients with mild symptoms tested positive for the virus for weeks afterwards. Both of these papers are very early and used very small numbers of patients but I thought it is incredibly important because they point to Coronavirus victims capable of spreading the virus despite feeling they are cured. And that time frame can be a week or more. The researchers in the first paper advised<strong> those who have recovered from even mild cases of COVID-19 should quarantine themselves for an additional 2 weeks.</strong></li><li>[Edit 1, 3/28/2020]<strong> “Cytokine Storm” Syndrome, the over-reaction by the body’s immune system to the virus, may be the cause of many Coronavirus fatalities.</strong> PLEASE NOTE that this is an early founding with much more research to be done. The University of Alabama in Birmingham wrote a easy to read explainer on possible link between this condition and the Coronavirus <a href="https://www.uab.edu/reporter/know-more/publications/item/8909-here-s-a-playbook-for-stopping-deadly-cytokine-storm-syndrome">here</a>, and the full interview with a specialist <a href="https://www.uab.edu/news/research/item/11176-covid-19-do-not-forget-the-host-in-treating-this-disease">here</a>. Direct information on Cytokine Storm Syndrome (CSS) or hemophagocytic syndromes can be found <a href="https://www.sinobiological.com/resource/cytokines/cytokine-storm">here</a>. A very thorough description &amp; discussion on the specific version of CSS suspected in COVID-19 cases, Macrophage Activation Syndrome (MAS), can be found <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3368378/">here</a>. The primary symptoms of CSS are high fever, swelling and redness, extreme fatigue and nausea. It can result in<strong> “overwhelming systemic inflammation, hemodynamic instability, multiple organ dysfunction, and potentially death… If a cytokine storm occurs in the lungs fluids and immune cells such as macrophages may accumulate and eventually block off the airways, potentially resulting in death.” </strong>(SinoBiological.com). CSS, not the inherent qualities of COVID-19, may be the reason some infected can progress from mild to severe or even critical condition rapidly. There are several tests doctors can run on patients to confirm or detect CSS but the high number of different ways CSS can manifest complicates diagnosis and treatment. The medical staff could have to go through a battery of tests before identifying the exact syndrome the patient is suffering from. [Edit 1, March 29, 2020] One early treatment of CSS specifically from COVID-19 has been <a href="https://www.sciencedirect.com/science/article/pii/S209580992030062X?via%3Dihub">proposed</a>, using an artificial liver interface to filter patients’ blood of the problematic immune cells. Investigation into CSS and COVID-19 are ongoing in its early phases but one aspect of this syndrome may be related to...</li><li>[Edit 1, 3/28/2020] <strong>The Coronavirus can damage the heart. </strong>Previously it was established that patients with cardiac disease and hypertension as comorbidities had worse outcomes than those without. An overly simplistic definition of of hemodynamic instability is acute heart &amp; circulatory problems, based off of <a href="http://eknygos.lsmuni.lt/springer/167/9-17.pdf">“Defining Hemodynamic Instability.”</a> From the paper, “physical signs of acute circulatory failure constitute primary references for shock, including hypotension, abnormal heart rates, cold extremities…” Released March 27, 2020 in JAMA Cardiology,<a href="https://jamanetwork.com/journals/jamacardiology/fullarticle/2763846?resultClick=24"> “Potential Effects of Coronaviruses on the Cardiovascular System: A Review”</a> concludes <strong>“Coronavirus disease 2019 is associated with a high inflammatory burden that can induce vascular inflammation, myocarditis, and cardiac arrhythmias.”</strong> In the same journal two days before publication of this review paper, <a href="https://jamanetwork.com/journals/jamacardiology/fullarticle/2763524?resultClick=24">“Association of Cardiac Injury With Mortality in Hospitalized Patients With COVID-19 in Wuhan, China” </a>supported the claim that individuals with heart conditions are more likely to be infected with COVID-19 and suffer more severe outcomes than the rest of the population. The Coronavirus puts this vulnerable demographic in even worse cardiovascular state after recovery, if they survive.</li><li>[Edit 2, 3/30/2020] The Italian health administration released a report (<a href="https://www.epicentro.iss.it/coronavirus/bollettino/Report-COVID-2019_17_marzo-v2.pdf">original in Italian</a>, English explainer on <a href="https://www.bloomberg.com/news/articles/2020-03-18/99-of-those-who-died-from-virus-had-other-illness-italy-says">Bloomberg World</a>) confirming the impact of age, comorbidities and sex on the health outcomes. Sampling 375 or 18% of the Italian COVID-19 fatalities case data, the median age of patients who succumbed was 80.5, the second age group with the most deaths was 70–79. Victims aged 60–69 and 90+ were nearly equal and far less than the 70–89 age grouping. The Italian analysis sheds important light on comorbidities, <strong>only 1% of fatalities had no previous negative health conditions</strong>.<strong> All deaths of those under 40 years old were males with ‘serious’ health problems</strong> as defined by the Italian health administration. As of writing of the original post there were infographics showing how the chances of death climbs with the number of existing comorbidities, citing WHO data but I could not find the original numbers in the database. The sample the Italian report is based on does show this relationship clearly: <strong>48.5% of deaths came from infected with 3 or more conditions, 25.6% had a two conditions and 25.1% had a single comorbidity.</strong> Of the 375 fatalities , 75%+ had high blood pressure, approximately one third of the cases had diabetes or heart disease (not exclusive).</li><li>[Edit 3, 4/6/2020] According to a small sample study based in China, the percentage of Coronavirus infected that are <strong>asymptomatic may be 80%</strong>, much higher than earlier expected according to the Chinese CDC. The post is available in <a href="https://www.bmj.com/content/369/bmj.m1375">The BMJ</a>.</li><li>[Edit 3, 4/6/2020] Ventilators are a scarce commodity worldwide in this pandemic. Predicting fatalities based on age, comorbidities, and scans of their chest has been accurate. <strong>But guessing who will require ventilators has been a difficult challenge</strong> for hospitals because our current assumptions for age, sex, and health does not correlate nearly as well. There are ongoing statistical and lab research work being done in this area. Adding to the evidence of the ‘Cytokine Storm’ theory, a product of white blood cell activation, <a href="https://www.medrxiv.org/content/10.1101/2020.04.01.20047381v1">Interleukin-6 (IL-6)</a>, was found to be a promising predictor of who would require artificial respiratory intervention in a small 30 patient study. If this finding is true and testing for elevated levels of IL-6 or other products of white blood cells can be done early, fast and cheaply, medical staff can anticipate who needs ventilators use and save more lives. Vox published great post on ‘Cytokine Storm’ and treatments, <a href="https://www.vox.com/2020/3/12/21176783/coronavirus-covid-19-deaths-china-treatment-cytokine-storm-syndrome">“How doctors can potentially significantly reduce the number of deaths from Covid-19</a>,” by Randy Q. Cron, MD, PhD, who literally wrote the <a href="https://www.springer.com/gp/book/9783030220938">book</a> on it, and W. Winn Chatham, MD. In the post, they recommended tests to detect this syndrome and directly recommended ‘drugs targeting interleukin-1 (IL-1), IL-6, IL-18, and <a href="https://www.ncbi.nlm.nih.gov/pubmed/31732958">interferon-gamma</a>.’ Testing for these things in addition to IL-6 may help doctors predict who will need ventilators even more.</li><li>[Edit 3, 4/6/2020] <strong>There are at least 8 strains of the Coronavirus</strong> according to the real-time sequencing aggregation site <a href="https://nextstrain.org/ncov">Nextstrain.org</a>. The differences in how they differ from one another (lethality, transmission, etc) are not completely understood. Some differences have been found, such as the existence of <a href="https://academic.oup.com/nsr/advance-article/doi/10.1093/nsr/nwaa036/5775463">two strains in China (L &amp; S)</a>. The L strain is evolutionary newer and more lethal than the older, less fatal S strain. China currently has fewer new cases of L strain infections than S. Another is the Iranian strain which specifically can cause <a href="https://www.medrxiv.org/content/10.1101/2020.03.23.20041889v1">loss of smell</a>. [Added 4/8/2020] On a positive note, researchers at UC San Diego Health <a href="https://onlinelibrary.wiley.com/doi/abs/10.1002/alr.22579">report </a>that the loss of smell and taste is statistically connected to patients with higher recovery rates. [Edit 5 4/13/2020] A survey using a COVID19 phone app in UK also indicated <a href="https://www.medrxiv.org/content/10.1101/2020.04.05.20048421v1">loss of smell and taste</a> as early predictors of infection. I will add to the descriptions of these strains as they are better defined.</li><li>[Edit 4, 4/7/2020] All over the world many healthcare workers are becoming infected and dying at rates higher than the general population. There is no collected info on medical staff infections and fatalities or formally published research on the issue as of this writing. Medical researchers have speculated two theories on why deaths are high for frontline workers against the Corona virus. The first theory is that viral load can overwhelm workers’ immune systems. Even if they have antibodies capable of protecting the body the virus, their immune system can not withstand the constant exposure to SARS-CoV-19 at work. The second theory is there is a sub-population of people who are uniquely vulnerable to the Coronavirus at a genetic level, some of which are working in healthcare. Both of these theories are highly speculative at this point and require further investigation.</li></ol><p>10) If you have 300 rolls of toilet paper and 40 bottles of Purell, you can’t get infected. That’s just basic science. (left here just in case I missed something. Praise Nurgle, this already went on way too long.)</p><p><strong><em>Too Long/Didn’t Read</em></strong>: The Coronavirus is a serious pandemic. 80% of the Infected are ‘mild to medium’ in severity and 20% require hospitalization. The virus takes 5 days to incubate inside an infected human before they can be diagnosed as infected. The virus is very stealthy, more than half of victims do not show any signs of infection, almost 1 out of 5 never do. But all are infectious. The infected require hospitalization at day 7 since the start of symptoms. In day 8, 20% of patients (the “severe” and “critically ill” categories) will experience Acute Respiratory Distress Syndrome. Most patients can leave approximately Day 17 after symptoms first showed. The Coronavirus is fatal 14 to 56 days after the first onset of symptoms. It is extremely hard to estimate how fatal the Coronavirus is. Fatalities are highly correlated to age and comorbidities, particularly cardiovascular disease, hypertension and diabetes. The Coronavirus is fatal 14 to 56 days after the first onset of symptoms. Even though younger patients are most likely to recover from the virus, they are just as likely to be infected and infect others. The infected among the 60 years old and under are most likely to be among the 80% that show no to mild symptoms and spread the virus. Men are much more likely to die from the Coronavirus than females. Even after patients recover from the virus and all symptoms are gone, they can still be infectious to others and test positive for days or weeks. “Cytokine Storm” Syndrome, the over-reaction by the body’s immune system to the virus, may be the cause of many Coronavirus fatalities.<strong> </strong>Those with heart conditions are more likely to be infected and the Coronavirus can damage the heart itself. Having three or more comorbities greatly increases the chance of fatality from COVID-19. Predicting who need ventilators has been much more difficult than predicting fatalities but looking at immune response looks promising. There are multiple strains of the SARS-CoV-19 virus throughout the world.</p><p>So. Now we have an idea of what we’re dealing with. Thank you for reading this and I hope you read the next installment. Stay safe.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=74a7b7aaa70d" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>