<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Chris Rawles on Medium]]></title>
        <description><![CDATA[Stories by Chris Rawles on Medium]]></description>
        <link>https://medium.com/@thecrawles?source=rss-7b27ba77e1ee------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*RHoydUAmX5OUG56u.jpeg</url>
            <title>Stories by Chris Rawles on Medium</title>
            <link>https://medium.com/@thecrawles?source=rss-7b27ba77e1ee------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 08:21:10 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@thecrawles/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How to deploy interpretable models on Google Cloud Platform]]></title>
            <link>https://medium.com/data-science/how-to-deploy-interpretable-models-on-google-cloud-platform-8da93f2e131d?source=rss-7b27ba77e1ee------2</link>
            <guid isPermaLink="false">https://medium.com/p/8da93f2e131d</guid>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[explainable-ai]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Chris Rawles]]></dc:creator>
            <pubDate>Mon, 24 Feb 2020 15:46:06 GMT</pubDate>
            <atom:updated>2020-02-25T13:28:47.379Z</atom:updated>
            <content:encoded><![CDATA[<h4>Get local and global explanations for production TensorFlow models</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*WbiMH9n_Mhqxpxe7.jpg" /><figcaption>Photo sourced from <a href="https://pixabay.com/photos/splashing-splash-aqua-water-165192/">Pixabay</a>.</figcaption></figure><p><em>Posted by Chris Rawles, Michael Munn, and Michael Abel.</em></p><p>Modern machine learning and AI have demonstrated impressive results to solve very complex problems. However, more complex problems often mean more complex data, which inevitably leads to more complex models. Truly understanding why a model makes a certain prediction can be as complicated as the original problem itself!</p><p>This can be problematic as many of these ML systems have come to affect use cases ranging from medical interventions, autonomous transportation, criminal justice, risk management and many other areas of society. In many cases, the usefulness and fairness of these AI systems is limited by our ability to understand, explain, and control them. As such, considerable effort and research has gone into unlocking the black box of powerful yet complex ML models, like deep neural networks.</p><p>Explainable AI refers to the collection of methods and techniques which enable a human to understand why a model is giving specific results. Model explainability is a key topic we teach to our customers at Google Cloud’s <a href="https://cloud.google.com/asl">Advanced Solutions Lab</a>, and in this post, we show how to use Google Cloud’s <a href="https://cloud.google.com/explainable-ai/">Explainable AI</a> to deploy interpretable and inclusive machine learning models.</p><p>All of the code used in this post is available <a href="https://github.com/GoogleCloudPlatform/training-data-analyst/tree/master/blogs/explainable_ai">here</a>.</p><h3>Taxonomy of Explainability methods</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/544/0*rgH1H-dEciw3nTUe" /></figure><p><em>Overview of (post hoc) model explainability methods and examples of different techniques.</em></p><p>The majority of explainability methods can be separated along three axes [<a href="https://christophm.github.io/interpretable-ml-book/">source</a>]:</p><p><strong>Intrinsic vs post-hoc. </strong>By intrinsic, we mean models which are inherently interpretable. That is, they are simple enough in structure that we can understand how the model is making predictions by simply looking at the model itself. For example, the learned weights of a linear model or the splits that are learned with a decision tree can be used to interpret why a model makes the predictions it does.</p><p>Post-hoc methods involve using the trained model and data to understand why certain predictions are being made. In some cases, post-hoc methods can be applied to models which have intrinsic explainability as well.</p><p>In this post, we’ll focus on post-hoc model explainability as many advanced methods, such as gradient boosting and neural networks, are best understood using such methods.</p><p><strong>Model-agnostic vs model-specific. </strong>Model-agnostic means the explainability method can be applied to <em>any</em> model while a model-specific method can only be used with certain model types. For example, if the method that would only work with neural networks it would be considered model-specific. In contrast, if an explainability method treats the trained model as a black box then it would be considered model-agnostic.</p><p><strong>Local vs global</strong>: Local explainability methods aim to explain a single data point or prediction while a global method tries to provide a comprehensive explanation of how the model performs on the whole. All local methods can be turned into global techniques by using aggregations of the local results.</p><h3>Deploying interpretable models on Google Cloud</h3><p>You can deploy interpretable models on GCP using Explainable AI and make predictions using the gcloud beta ai-platform explain command.</p><p>The steps to train and predict with explanations are</p><ol><li>Train a model and deploy it on GCP.</li><li>Upload a JSON file that contains baseline feature values (more on this below) to a Cloud Storage Bucket.</li><li>Create a version of the model using this JSON file and specify the explanation-method.</li><li>Call gcloud beta ai-platform explain to get the explanations.</li></ol><p>Below we will show these steps in more detail.</p><p>To start, you’ll need a trained and deployed model on Google Cloud AI Platform (CAIP). We’ll be looking at the <a href="https://bigquery.cloud.google.com/table/nyc-tlc:yellow.trips?tab=details&amp;pli=1">New York City taxicab dataset</a>. You can check out <a href="https://towardsdatascience.com/how-to-train-machine-learning-models-in-the-cloud-using-cloud-ml-engine-3f0d935294b3">this blog post</a> to see how to easily train a model on CAIP. At the time of writing, AI Explanations only supports TensorFlow 1.x, so whatever model you build make sure you use TensorFlow 1.x. Once you have your model in the <a href="https://www.google.com/search?q=tensorflow+model+format&amp;oq=tensorflow+model+format&amp;aqs=chrome.0.0l2j69i61j69i65j69i60l4.2431j0j7&amp;sourceid=chrome&amp;ie=UTF-8">SavedModel</a> format, we’ll create a new model on CAIP:</p><pre>gcloud ai-platform models create taxifare</pre><p>Before we can deploy our model, we must configure an explanations_metadata.json file and copy it to the model directory. In this JSON file, we need to tell AI Explanations the names of the input and output tensors that our model is expecting.</p><p>Additionally, in this file we need to set the input_baselines, which tells the explanations service what the baseline input should be for our model. Understanding baselines is important for effective use of many model explanation techniques. Both supported techniques, Sampled Shapley and Integrated Gradients, compare predictions against baseline feature values and predictions. Choosing appropriate baselines is important because essentially you’re comparing how a model’s prediction compares to the baseline values. To read more about baselines, check out the <a href="https://storage.googleapis.com/cloud-ai-whitepapers/AI%20Explainability%20Whitepaper.pdf">Explainable AI Whitepaper.</a></p><p>Generally speaking, for numeric data we recommend that you choose a simple baseline such as the average or median to start. For this example, we will use the median for each feature — this means the baseline prediction for this model will be the taxi fare our model predicts using the median of each feature in our dataset.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9255ef7403f23b3acae0a2ae134d2fb9/href">https://medium.com/media/9255ef7403f23b3acae0a2ae134d2fb9/href</a></iframe><p>We can write out this Python dictionary to a JSON file :</p><pre># Write the json to a local file<br>with open(‘explanation_metadata.json’, ‘w’) as output_file:<br>    json.dump(explanation_metadata, output_file)<br>    # Copy the json to the model directory.</pre><p>Then in bash, we copy the JSON file to your model’s directory using gsutil.</p><pre>$ gsutil cp explanation_metadata.json $model_dir</pre><p>Now that we have created our explanations_metadata.json file, we will deploy a new version of the model. This code is very similar to the usual process for creating a model version using gcloud but with some additional flags:</p><pre>gcloud beta ai-platform versions create $VERSION_IG \<br>  -- model $MODEL \<br>  --origin $model_dir \<br>  --runtime-version 1.15 \<br>  --framework TENSORFLOW \<br>  --python-version 3.5 \<br>  --machine-type n1-standard-4 \<br>  <strong>--explanation-method ‘integrated-gradients’ \<br>  --num-integral-steps 25</strong></pre><p>Using the `<strong>explanation-method`</strong> flag, you specify the explanation method — currently both <a href="https://github.com/ankurtaly/Integrated-Gradients">integrated-gradients</a> and <a href="https://christophm.github.io/interpretable-ml-book/shapley.html">sampled-shapley</a> are supported.</p><p><em>Note: When deciding between Integrated Gradients and Sampled Shapley, we quote the </em><a href="https://storage.googleapis.com/cloud-ai-whitepapers/AI%20Explainability%20Whitepaper.pdf"><em>whitepaper</em></a><em>:</em></p><blockquote>Integrated Gradients is recommended for neural networks and differentiable models in general. It offers computational advantages especially for large input feature spaces (e.g. images with thousands of input pixels). Sampled Shapley is recommended for non-differentiable models, which is the case in AutoML Tables models consisting of meta-ensembles of trees and neural networks.</blockquote><p><em>Additionally, for those wondering how the Sampled Shapley approach differs from the popular </em><a href="https://github.com/slundberg/shap"><em>SHAP</em></a><em> library, we also quote the whitepaper:</em></p><blockquote>There are many ways to apply the Shapley values that differ in how they reference the model, the training data, and the explanation context. This gives rise to a multiplicity of Shapley values used in explaining a model’s prediction,<br>which is somewhat unfortunate considering Shapley’s uniqueness property. A thorough discussion of this topic can be found in <a href="https://arxiv.org/pdf/1908.08474.pdf">The many Shapley values for model explanation</a> by Mukund Sundararajan and Amir Najmi. Our methods fall into the Baseline Shapley category and support multiple simultaneous baselines across various input data modalities.</blockquote><p>Now that our model is deployed, we can get the local attributions from a Jupyter notebook:</p><pre>resp_obj = !gcloud beta ai-platform explain — model $MODEL \<br>           — version $VERSION_IG — json-instances=’taxi-data.txt’<br>response_IG = json.loads(resp_obj.s)</pre><pre># Analyze individual example.<br>explanations_IG = response_IG[‘explanations’][0][‘attributions_by_label’][0]</pre><p>We can load these into a Pandas DataFrame and plot the attributions for individual examples:</p><pre>df = pd.DataFrame(explanations_IG)<br>df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Ees37aLibLBUvcuH" /></figure><pre>row = df.iloc[0] # First example.<br>row.plot(kind=’barh’)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/410/0*AcszE5eINR9shYOo" /></figure><p>Finally, we can get global model explainability by aggregating the local attributions:</p><pre>df.mean(axis=0).plot(kind=’barh’, color=’orange’)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/410/0*f-L625uR2m9uPdoe" /></figure><p>For more insight in using Sampled Shapley global attributions, please refer to <a href="https://arxiv.org/pdf/1908.08474.pdf">this paper</a>.</p><h3>Conclusion</h3><p>That’s it! In this post we have shown how to deploy explainable models on Google Cloud Platform using Explainable AI. The Explainable AI tool allows users to get local explanations from a deployed model. These explanations can be combined together to provide global interpretability. In addition to the steps above, you can also check out the <a href="https://pair-code.github.io/what-if-tool/index.html#features">What-If tool</a> to inspect and interpret your model.</p><h3>Additional Resources</h3><ul><li><a href="https://cloud.google.com/blog/products/ai-machine-learning/explaining-model-predictions-structured-data">Explaining Model Predictions with Google AI Platform</a></li><li><a href="https://sararobinson.dev/2020/01/15/fraud-detection-tensorflow.html">Fraud Detection and Explainable AI</a></li><li><a href="https://storage.googleapis.com/cloud-ai-whitepapers/AI%20Explainability%20Whitepaper.pdf">Google AI Explainability Whitepaper</a></li><li><a href="https://christophm.github.io/interpretable-ml-book/index.html">Christopher Molnar’s Interpretable ML</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8da93f2e131d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/how-to-deploy-interpretable-models-on-google-cloud-platform-8da93f2e131d">How to deploy interpretable models on Google Cloud Platform</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to normalize features in TensorFlow]]></title>
            <link>https://medium.com/data-science/how-to-normalize-features-in-tensorflow-5b7b0e3a4177?source=rss-7b27ba77e1ee------2</link>
            <guid isPermaLink="false">https://medium.com/p/5b7b0e3a4177</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Chris Rawles]]></dc:creator>
            <pubDate>Tue, 17 Jul 2018 21:24:30 GMT</pubDate>
            <atom:updated>2018-07-26T15:53:22.619Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*pPFhoV6d9ZYkQsXi" /><figcaption>Photo by <a href="https://unsplash.com/@inf1783?utm_source=medium&amp;utm_medium=referral">Karsten Würth (@inf1783)</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><strong><em>TL;DR<br></em></strong><em>When using </em><em>tf.estimator, use the </em><em>normalizer_fn argument in </em><em>tf.feature_column.numeric_feature to normalize using the same parameters (mean, std, etc.) for training, evaluation, and serving.</em></p><pre>def zscore(col):<br>  mean = 3.04<br>  std = 1.2<br>  return (col — mean)/std</pre><pre>feature_name = ‘total_bedrooms’<br>normalized_feature = tf.feature_column.numeric_column(<br>  feature_name,<br>  normalizer_fn=zscore)</pre><h4><strong>Normalization for the </strong><strong>tf.estimator API</strong></h4><p>Neural network activations generally like their inputs to be normalized. Normalizing inputs to nodes in a network helps prevent the so-called vanishing (and exploding) gradients. <a href="https://towardsdatascience.com/how-to-use-batch-normalization-with-tensorflow-and-tf-keras-to-train-deep-neural-networks-faster-60ba4d054b73">Batch normalization</a> is the most comprehensive approach for normalization, but it incurs an extra cost and may be overkill for your problem. Thus you may just want to normalize your inputs.</p><p>There are two approaches to normalizing inputs when using the tf.estimator API (which is the easiest way to build a TensorFlow model): inside the input_fn and while creating a feature_column. I will show you an example to perform the ladder, then I will show you to train multiple models using <a href="https://cloud.google.com/ml-engine/">ML Engine</a>.</p><h4><strong>Boilerplate code: Normalize using the </strong>`normalizer_fn`<strong> argument</strong></h4><p>Check out <a href="https://github.com/GoogleCloudPlatform/training-data-analyst/tree/master/blogs/feature_column_normalization">this notebook</a> for a full working example.</p><p>Normalizing <a href="https://stackoverflow.com/a/46480180">inside the input_fn</a> allows for more flexibility (you can also perform feature engineering here), but I find using the normalizer_fn with tf.feature_column.numeric_column to be more elegant. Here is a basic example:</p><pre><em>def zscore(col):<br>  mean = 3.04<br>  std = 1.2<br>  return (col — mean)/std</em></pre><pre><em>feature_name = ‘total_bedrooms’<br>normalized_feature = tf.feature_column.numeric_column(<br>  feature_name,<br>  normalizer_fn=zscore)</em></pre><p>Below, I will show an end-to-end example, to get the normalization parameters, then normalize all numeric columns in my dataset.</p><p>You should compute the normalization parameters ahead of time on the training set. In this case, I’m using Pandas to get the mean and standard deviation for each numeric column:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4b26906c11082695baa2a77c0b753d3a/href">https://medium.com/media/4b26906c11082695baa2a77c0b753d3a/href</a></iframe><p>Alternatively, <a href="https://github.com/tensorflow/transform">TensorFlow Transform</a> provides a scalable approach for normalizing inputs and is particularly suitable for larger datasets. Check out an <a href="https://github.com/GoogleCloudPlatform/training-data-analyst/blob/master/courses/machine_learning/deepdive/06_structured/4_preproc_tft.ipynb">example here</a>.</p><p>After running the above, we return parameters for each columns:</p><pre>{‘households’: {‘mean’: 501.34073416222617, ‘std’: 382.81658748493305},<br>‘housing_median_age’: {‘mean’: 28.5441089402013, ‘std’: 12.610144469735635},<br>‘median_income’: {‘mean’: 3.8814239564831365, ‘std’: 1.9061255708184284},<br>‘population’: {‘mean’: 1428.941163410302, ‘std’: 1150.5106244960523},<br>‘total_bedrooms’: {‘mean’: 539.6057578448787, ‘std’: 418.76075045523334},<br>‘total_rooms’: {‘mean’: 2642.2929988158676, ‘std’: 2162.649970020439}}</pre><p>Now you can create the feature columns, using the training means and standard deviations computed above.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/be9e3d91be722622ca50fba56cb78c23/href">https://medium.com/media/be9e3d91be722622ca50fba56cb78c23/href</a></iframe><pre>NUMERIC_FEATURES = [‘housing_median_age’, ‘total_rooms’, ‘total_bedrooms’, ‘population’, ‘households’, ‘median_income’]</pre><pre>feature_columns = create_feature_cols(NUMERIC_FEATURES, use_normalization=True)</pre><p>And finally you can build the estimator using the feature columns:</p><pre>model = tf.estimator.DNNRegressor(hidden_units=[10,4], <br>                                  model_dir = outdir,<br>                                  feature_columns = feature_columns)</pre><h4>Training multiple models in the cloud using ML Engine</h4><p>Normalization is a hyperparameter, and in practice, it would be useful to evaluate different normalization schemes. For example you may want to try training a model just normalizing your features and comparing it to also normalizing inputs to hidden layers using batch normalization.</p><p>You can fire off multiple experiments in parallel using Cloud ML Engine. This will be particularly valuable for larger datasets, where you want to leverage the cloud to scale model training. To train with ML Engine, you need to <a href="https://towardsdatascience.com/how-to-train-machine-learning-models-in-the-cloud-using-cloud-ml-engine-3f0d935294b3">package up the model code</a>, creating task.py and model.py model files. See <a href="https://github.com/GoogleCloudPlatform/training-data-analyst/tree/master/blogs/feature_column_normalization">the repo</a> for an example.</p><p>It’s best practice to test the model package locally first to ensure there are no syntactic or semantic errors:</p><pre>OUTPUT_DIR=’trained_model’<br>export PYTHONPATH=${PYTHONPATH}:${PWD}/model_code<br>python -m trainer.task — outdir $OUTPUT_DIR — normalize_input 1</pre><p>After, you can submit a job to ML Engine using `gcloud ml-engine jobs submit training`:</p><pre>JOBNAME=my_ml_job_$(date -u +%y%m%d_%H%M%S)<br>REGION=’us-central1&#39;<br>BUCKET=’gs://crawles-sandbox’<br>OUTPUT_DIR=$BUCKET/’housing_trained_model’<br>PACKAGE_PATH=$PWD/model_code/trainer</pre><pre>gcloud ml-engine jobs submit training $JOBNAME \<br>  -- package-path=$PACKAGE_PATH \<br>  -- module-name=trainer.task \<br>  -- region=$REGION \<br>  --staging-bucket=$BUCKET\<br>  -- scale-tier=BASIC \<br>  -- runtime-version=1.8 \<br>  -- \<br>  -- outdir=$OUTPUT_DIR\<br>  -- normalize_input=0</pre><p>In my case, I trained with and without computing the z-score for the features:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*L55EwdYftTCKi6I4" /></figure><p>As expected, normalizing the inputs improves the final model performance.</p><p>That’s it! If you found this post helpful, please give it a clap so others can find it.</p><p><strong>Additional Resources</strong></p><ul><li>Normalize using <a href="https://stackoverflow.com/a/46480180">input_fn</a></li><li><a href="https://towardsdatascience.com/how-to-use-batch-normalization-with-tensorflow-and-tf-keras-to-train-deep-neural-networks-faster-60ba4d054b73">Batch Normalization</a> using tf.keras</li><li><a href="https://en.wikipedia.org/wiki/Standard_score">Standard score</a> (z-score)</li><li><a href="https://github.com/GoogleCloudPlatform/training-data-analyst/tree/master/blogs/feature_column_normalization">Repo with code from this </a>post</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5b7b0e3a4177" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/how-to-normalize-features-in-tensorflow-5b7b0e3a4177">How to normalize features in TensorFlow</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to train Machine Learning models in the cloud using Cloud ML Engine]]></title>
            <link>https://medium.com/data-science/how-to-train-machine-learning-models-in-the-cloud-using-cloud-ml-engine-3f0d935294b3?source=rss-7b27ba77e1ee------2</link>
            <guid isPermaLink="false">https://medium.com/p/3f0d935294b3</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <dc:creator><![CDATA[Chris Rawles]]></dc:creator>
            <pubDate>Wed, 23 May 2018 21:26:35 GMT</pubDate>
            <atom:updated>2018-06-04T14:20:02.969Z</atom:updated>
            <content:encoded><![CDATA[<p>And how to artfully write a task.py using the docopt package</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*3_oq3tEmuUBgvp1W." /></figure><p>Training ML models in the cloud makes a lot of sense. Why? Among many reasons, it allows you to train on large amounts of data with plentiful compute and perhaps train many models in parallel. Plus it’s not hard to do! On Google Cloud Platform, you can use Cloud ML Engine to train machine learning models in TensorFlow and other Python ML libraries (such as scikit-learn) without having to manage any infrastructure. In order to do this, you will need to put your code into a Python package (i.e. add setup.py and __init__.py files). In addition, it is a best practice to organize your code into a model.py and task.py. In this blog post, I will step you through what this involves.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2185PkdnkS-e-jNWGC3Hpw.png" /><figcaption><em>Submitting a job to ML Engine. The file </em><em>task.py is the file actually executed by ML Engine and it references the model logic located in </em><em>model.py.</em></figcaption></figure><p><strong>The </strong><strong>task.py file</strong></p><p>As a teacher, one of the first things I see students, particularly those newer to Python, get hung up on is creating a task.py file. Although it’s technically optional (see below), it’s highly recommended because it allows you to separate hyperparameters from the model logic (located in model.py). It’s usually the actual file that is called by ML engine and its purpose is twofold:</p><ol><li>Reads and parses model parameters, like location of the training data and output model, # hidden layers, batch size, etc.</li><li>Calls the model training logic located in model.py with said parameters</li></ol><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e064db22d5932db62c5caca63f321b81/href">https://medium.com/media/e064db22d5932db62c5caca63f321b81/href</a></iframe><p>There are many different ways you can write a task.py file — there are even different names you can give it. In fact the task.py and model.py convention is merely that — a convention. We could have called task.py aReallyCoolArgument_parser.py and model.py very_deeeep_model.py.</p><p>We could even combine these two entities into a single file that does argument parsing and trains a model. ML Engine doesn’t care as long as you arrange your code into a Python package (i.e. it <a href="https://python-packaging.readthedocs.io/en/latest/minimal.html">must contain</a> setup.py and __init__.py). But, stick with the convention of two files named task.py and model.py inside a trainer folder (more details below) that house your argument parsing and model logic, respectively.</p><p>Check out the <a href="https://github.com/GoogleCloudPlatform/cloudml-samples">Cloud ML samples</a> and <a href="https://github.com/GoogleCloudPlatform/training-data-analyst">Cloud ML training</a> repos for full examples of using Cloud ML Engine and examples of model.py and task.py files.</p><p><strong>Writing clean </strong><strong>task.py files using docopt</strong></p><p>Although many people use argparse, the standard Python library for parsing command-line arguments, I prefer to write my task.py files using the <a href="https://github.com/docopt/docopt">docopt</a> package. Why? Because it’s the most concise way to write a <em>documented </em>task.py. In fact, pretty much the only thing you write is your program’s usage message (i.e. the help message) and docopt takes care of the rest. Based on the usage message you write in the module’s doc string (Python will call this __doc__), you call docopt(__doc__), which generates an argument parser for you based on the format you specify in the doc string. Here is the above example using docopt:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bf3fcc7a8baf355f02416f9808f269fa/href">https://medium.com/media/bf3fcc7a8baf355f02416f9808f269fa/href</a></iframe><p>Pretty nice, right? Let me break it down. The first block of code is the usage for your task.py. If you call it with no arguments or incorrectly call task.py this will display to the user.</p><p>The line arguments = docopt(__doc__) parses the usage pattern (“Usage: …”) and option descriptions (lines starting with dash “-”) from the help string and ensures that the program invocation matches the usage pattern.</p><p>The final section assigns these parameters to model.py variables and then executes the training and evaluation.</p><p>Let’s run a job. Remember the task.py is part of a family of files, called a Python package. In practice you will spend the bulk of your time writing the model.py file, a little time creating the task.py file, and the rest is basically boilerplate.</p><pre>training_example # root directory<br>   setup.py # says how to install your package<br>   trainer # package name, “trainer” is convention<br>      model.py<br>      task.py<br>      __init__.py # Python convention indicating this is a package</pre><p>Because we are using docopt, which is not part of the standard library, we must add it to setup.py, so we insert an additional line into setup.py:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5c694d30177b9716038fac785bb234ab/href">https://medium.com/media/5c694d30177b9716038fac785bb234ab/href</a></iframe><p>This will tell Cloud ML Engine to install docopt by running pip install docopt when we submit a job to the cloud.</p><p>Finally once we have our files in the above structure we can submit a job to ML Engine. Before we do that, let’s first test our package locally using python -m and then ml-engine local predict. These two steps, while optional, can help you debug and test your packages functionality before submitting to the cloud. You typically do this on a tiny data set or just a very limited number of training steps.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0543316332bc14b9f7cdc41b1e68998c/href">https://medium.com/media/0543316332bc14b9f7cdc41b1e68998c/href</a></iframe><p>Once we have tested our model locally we will submit our job to ML Engine using gcloud ml-engine jobs submit training</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5c41728e39b726e05d649cbefc27e21f/href">https://medium.com/media/5c41728e39b726e05d649cbefc27e21f/href</a></iframe><p>These two lines are relevant to our discussion:</p><pre>— package-path=$(pwd)/my_model_package/trainer \<br>— module-name trainer.task</pre><p>The first line indicates the location of our package name, which we always call trainer (a convention). The second line indicates, in the trainer package, that we will call the task module (task.py) in the trainer package.</p><p><strong>Conclusion</strong></p><p>By building a task.py we can process hyperparameters as command line arguments, which allows us to decouple our model logic from hyperparameters. A key benefit is this allows us to easily fire off multiple jobs in parallel using different parameters to determine an optimal hyperparameter set (we can even use the <a href="https://cloud.google.com/ml-engine/docs/tensorflow/using-hyperparameter-tuning">built in hyperparameter tuning</a> service!). Finally, the docopt package automatically generates a parser for our task.py file, based on the usage string that the user writes.</p><p>That’s it! I hope this makes it clear how to submit a ML Engine job and build a task.py. Please leave a clap if you found this helpful so others can find it.</p><p><strong>Additional Resources</strong></p><ul><li><a href="https://github.com/GoogleCloudPlatform/cloudml-samples">Google Cloud ML samples</a></li><li><a href="https://github.com/GoogleCloudPlatform/training-data-analyst">Google Cloud machine learning training</a></li><li><a href="https://cloud.google.com/ml-engine/docs/tensorflow/technical-overview">Cloud ML Engine documentation</a></li><li><a href="https://github.com/docopt/docopt">docopt documentation</a></li><li><a href="http://python-packaging.readthedocs.io/en/latest/minimal.html">Packaging a Python module</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3f0d935294b3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/how-to-train-machine-learning-models-in-the-cloud-using-cloud-ml-engine-3f0d935294b3">How to train Machine Learning models in the cloud using Cloud ML Engine</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[tf.keras and TensorFlow: Batch Normalization to train deep neural networks faster]]></title>
            <link>https://medium.com/data-science/how-to-use-batch-normalization-with-tensorflow-and-tf-keras-to-train-deep-neural-networks-faster-60ba4d054b73?source=rss-7b27ba77e1ee------2</link>
            <guid isPermaLink="false">https://medium.com/p/60ba4d054b73</guid>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[towards-data-science]]></category>
            <dc:creator><![CDATA[Chris Rawles]]></dc:creator>
            <pubDate>Mon, 26 Feb 2018 14:28:34 GMT</pubDate>
            <atom:updated>2019-11-12T14:47:23.777Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IWfDIqaziRXxAqhCCK0Ssg.jpeg" /></figure><p>Training deep neural networks can be time consuming. In particular, training can be significantly impeded by vanishing gradients, which occurs when a network stops updating because the gradients, particularly in earlier layers, have approached zero values. Incorporating Xavier weight-initialization and ReLu activation functions helps counter the vanishing gradient problem. These techniques also help with the opposite, yet closely related issue of exploding gradients, where the gradients become extremely large preventing the model from updating.</p><p>Perhaps the most powerful tool for combatting the vanishing and exploding gradients issue is Batch Normalization. Batch Normalization works like this: for each unit in a given layer, first compute the z score, and then apply a linear transformation using two trained variables 𝛾 and 𝛽. Batch Normalization is typically done prior to the non-linear activation function (see below figure), however applying it after the activation function can also be beneficial. Check out <a href="http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture6.pdf">this lecture</a> for more detail of how the technique works.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*AxqT3EOPEThyRqfK." /><figcaption><em>During backpropagation gradients tend to get smaller at lower layers, slowing down weight updates and thus training. Batch Normalization helps combat the so-called vanishing gradients.</em></figcaption></figure><p>Batch Normalization can be implemented in three ways in TensorFlow. Using:</p><ol><li>tf.keras.layers.BatchNormalization</li><li>tf.layers.batch_normalization</li><li>tf.nn.batch_normalization</li></ol><p><em>08/18/2018 update: The </em><a href="https://www.tensorflow.org/api_docs/python/tf/estimator/DNNClassifier"><em>DNNClassifier</em></a><em> and </em><a href="https://www.tensorflow.org/api_docs/python/tf/estimator/DNNRegressor"><em>DNNRegressor</em></a><em> now have a </em><em>batch_norm parameter, which makes it possible and easy to do batch normalization with a canned estimator.</em></p><p>11/12/2019 <em>update: This has gotten even easier with TF 2.0 using tf.keras, you can simply add in a </em><em>BatchNormalization layer and do not need to worry about control_dependencies.</em></p><p>The tf.keras module became part of the core TensorFlow API in version 1.4. and provides a high level API for building TensorFlow models; so I will show you how to do it in Keras. The tf.layers.batch_normalization function has similar functionality, but Keras often proves to be an easier way to write model functions in TensorFlow.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e560b7c1b86a2b990f9ff04c993fef6d/href">https://medium.com/media/e560b7c1b86a2b990f9ff04c993fef6d/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lhOJCNNaSCCf3CjV." /><figcaption><em>In TensorFlow, Batch Normalization can be implemented as an additional layer using tf.keras.layers.</em></figcaption></figure><p>The second code block with tf.GraphKeys.UPDATE_OPS is important. Using tf.keras.layers.BatchNormalization, for each unit in the network, TensorFlow continually estimates the mean and variance of the weights over the training dataset. These are then stored in the tf.GraphKeys.UPDATE_OPS variable. After training, these stored values are used to apply Batch Normalization at prediction time. The training set mean and variance from each unit can be observed by printing extra_ops, which contains a list for each layer in the network:</p><pre>print(extra_ops)</pre><pre>[&lt;tf.Tensor ‘batch_normalization/AssignMovingAvg:0’ shape=(500,) dtype=float32_ref&gt;, # layer 1 mean values<br> &lt;tf.Tensor ‘batch_normalization/AssignMovingAvg_1:0’ shape=(500,) dtype=float32_ref&gt;, # layer 1 variances ...]</pre><p>While Batch Normalization is also available in the tf.nn module, it requires extra bookkeeping, as the mean and variance are required arguments for the function. Thus the user has to manually compute mean and variance at both the batch level and training set level. It is, thus, a lower abstraction level than tf.keras.layers or tf.layers; avoid the tf.nn implementation.</p><h3>Batch Normalization on MNIST</h3><p>Below, I apply Batch Normalization to the prominent MNIST dataset using TensorFlow. Check out the code <a href="https://github.com/GoogleCloudPlatform/training-data-analyst/tree/master/blogs/batch_normalization">here</a>. MNIST is an easy dataset to analyze and doesn’t require many layers to achieve low classification error. However, we can still build a deep network and observe how Batch Normalization affects convergence.</p><p>Let’s build a custom estimator using the <a href="https://www.tensorflow.org/get_started/custom_estimators">tf.estimator</a> API. First we build the model:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9f300996e4bf6fe7d0ad7a9daf7cc4c8/href">https://medium.com/media/9f300996e4bf6fe7d0ad7a9daf7cc4c8/href</a></iframe><p>After we define our model function, let’s build the custom estimator and train and evaluate our model:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/733aa589321fc92ded854fccae3dbe5d/href">https://medium.com/media/733aa589321fc92ded854fccae3dbe5d/href</a></iframe><p>Let’s test how Batch Normalization impacts models of varying depths. After we wrap our code into a Python package, we can fire off multiple experiments in parallel using <a href="https://cloud.google.com/ml-engine/docs/technical-overview">Cloud ML Engine</a>:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/123cfa36b4ab15846f6ece212c4066c4/href">https://medium.com/media/123cfa36b4ab15846f6ece212c4066c4/href</a></iframe><p>The below plot show the number of training iterations (1 iteration contains a batch size of 500) required to reach 90% testing accuracy — an easy target — as a function of network depth. It’s evident that Batch Normalization significantly speeds up training for the deeper networks. Without Batch Normalization, the number of training steps increases with each subsequent layer, but with it, the number of training steps is near constant. And in practice, on more difficult datasets, more layers is a prerequisite for success.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/938/0*fHofBSXjndh4BMCh." /><figcaption><em>Without Batch Normalization the number of training iterations required to hit 90% accuracy increases with the number of layers, likely due to the vanishing gradient effect.</em></figcaption></figure><p>Similarly, as shown below, for a fully connected network with 7 hidden layers, the convergence time without Batch Normalization is significantly slower.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/983/0*VZGXLr-RQbcdcuGR." /></figure><p>The above experiments utilize the commonly used ReLu activation function. Though obviously not immune to the vanishing gradient effect as shown above, the ReLu activation fares much better than the sigmoid or tanh activation functions. The vulnerability of the sigmoid activation function to vanishing gradients is rather intuitive to understand. At larger magnitude (very positive or negative) values, the sigmoid function “saturates” — i.e. the derivative of the sigmoid function approaches zero. And when many nodes saturate, the number of updates decreases, and network stops training.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/731/0*K5rfrS3lIXmDh6p6." /></figure><p>The same 7-layer network trains significantly slower using sigmoid activation functions without using Batch Normalization. With Batch Normalization, the network converges in a similar number of iterations when using ReLu.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/983/0*pq4eZ8J4PJJCnct_." /></figure><p>On the other hand, other activation functions, such as the exponential ReLu or leaky ReLu functions, can help combat the vanishing gradient issue as they have non-zero derivatives for both positive and negative large numbers.</p><p>Finally, it is important to note that Batch Normalization incurs an extra time cost to training. Though Batch Normalization typically decreases the number of training steps to reach convergence, it brings an extra time cost because it introduces an additional operation and also introduces two new trained parameters per unit.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/839/0*C-jJJwnqPtixZLEy." /><figcaption><em>For the MNIST classification problem (using a 1080 GTX GPU), Batch Normalization converges in (top) fewer iterations, however the time per iteration is slower. Ultimately, the Batch Normalization version still converges faster (bottom), but the improvement is less pronounced when incorporating total training time.</em></figcaption></figure><p>Incorporating <a href="https://www.tensorflow.org/performance/xla/">XLA</a> and fused Batch Normalization (fused argument in tf.layers.batch_normalization) could help speed up the Batch Normalization operation by combining several individual operations into a single kernel.</p><p>Regardless, Batch Normalization can be a very valuable tool for speeding the training of deep neural networks. As always with training deep neural networks, the best way to figure out if an approach will help for your problem is to try it!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=60ba4d054b73" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/how-to-use-batch-normalization-with-tensorflow-and-tf-keras-to-train-deep-neural-networks-faster-60ba4d054b73">tf.keras and TensorFlow: Batch Normalization to train deep neural networks faster</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Curry Ball Will Impact March Madness Brackets]]></title>
            <link>https://medium.com/built-to-adapt/how-curry-ball-will-impact-march-madness-brackets-e2789981ba52?source=rss-7b27ba77e1ee------2</link>
            <guid isPermaLink="false">https://medium.com/p/e2789981ba52</guid>
            <category><![CDATA[culture]]></category>
            <category><![CDATA[sports]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[nba]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Chris Rawles]]></dc:creator>
            <pubDate>Thu, 09 Mar 2017 22:37:37 GMT</pubDate>
            <atom:updated>2017-04-06T19:26:55.964Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lnSYHLb9objuE0HTit9M9A.png" /></figure><h3>402.</h3><p>It was April 13, 2016: the third time Stephen Curry had set the record for 3-pointers made in a single season, in just four years. He and his teammates on the Golden State Warriors were collectively making more 3-pointers than any other team ever before, and it was working: they were bound for the NBA Championship.</p><p>The numbers show that other NBA teams are following the Warriors’ lead: Houston Rockets coach Mike D’Antoni <a href="http://www.houstonchronicle.com/sports/rockets/article/Coach-thinks-Rockets-can-average-50-3-point-10958599.php">suggested his team</a> could eventually average more than 50 3-point attempts a game (they’re currently <a href="http://www.basketball-reference.com/teams/HOU/2017.html">just over 40</a>). It begs the question on the eve of Selection Sunday: is this transition playing out at all levels of the game?</p><p><a href="https://content.pivotal.io/blog/how-data-science-assists-sports">I recently wrote</a> about how Curry’s 3-point prowess has impacted NBA shooting overall, using a similar data tool that I expect many professional teams are leveraging to inform their coaching strategies. Accordingly, I was curious to see how the college game has changed as a result. My assumption was that college teams were mirroring the Warriors’ strategy, but I also wanted to know if teams shot more 3-pointers, would they perform better in the March Madness tournament?</p><h4>Overview</h4><p>Using data from <a href="http://www.sports-reference.com/">Sports Reference</a>, I looked specifically at:</p><ul><li>Team statistics for all Division I teams from 1993–2017</li><li>NCAA tournament results for the past ten years</li><li>Current AP Top 25 teams (as of March 6th, 2017)</li></ul><p>My hypotheses were:</p><ul><li>Shooting more 3-point shots results in higher shooting efficiency</li><li>High 3-point accuracy is necessary to go deep into the tournament</li><li>Successful underdogs shoot more 3’s due to the <a href="http://nyloncalculus.com/2015/07/20/a-random-walk-through-nba-variance/">high variance nature</a> of 3-point shots</li></ul><p>To test these, I compiled statistics from the last ten years of the March Madness Tournament from 2007–2016.</p><h4>TL;DR</h4><p>Now, I realize some of you probably want to learn the impact of this modeling on your March Madness bracket, so at a snapshot glance, here’s what I discovered:</p><ul><li>College teams, like those in the NBA, are shooting more 3-pointers</li><li>In the past ten years, high 3-point shooting volume and elite accuracy have not been an indicator for March Madness success (e.g. making it to the Final Four or better)</li><li>However, Cinderella teams have taken more 3-pointers than their higher seeded counterparts</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/718/1*BEYmJg_MBYmHSM9BVN4ccQ.gif" /><figcaption>Splash! 🏀 💦</figcaption></figure><h4>It’s Raining 3s!</h4><p>College teams are shooting more 3-point shots than ever, with the 2016–17 season on pace to see a record-breaking 36 percent of total shots coming from downtown. Three point attempts have been on the rise nearly every year in college basketball, with a notable exception of the 2008 season when the NCAA moved the college line out one foot to 20 feet 9 inches.</p><p>This trend is particularly interesting relative to the NBA, which has also observed a year-over-year increase in total attempts. Beginning with the NBA, both leagues have observed a sharp uptick in the past five years, arguably due to the success of Curry and the Warriors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Ve-IfBIkoYoR7-8F." /><figcaption><em>Figure 1. Comparison of the percentage of 3-point attempts in the NBA to NCAA Division I</em></figcaption></figure><p>A driving factor behind the recent upsurge in 3-point attempts is shot efficiency. As Dan D’Antoni, Marshall’s basketball head coach and brother of Mike D’Antoni, <a href="https://twitter.com/CraigMeyerPG/status/814343009244483585">explained in an interview</a>, “You can go get any computer and run what the best shots are and it will tell you the post-up is the worst shot in basketball. If you want to run down and try to get it [in the paint] to shoot over somebody, then you’re beating analytics. The best shot in basketball is that corner three. The next-best shot in basketball is any other three. Other than free throws, which we try to do, when you get to the foul line, you score 1.5 points every time you go to the foul line in the pros. It just trickles down. It’s the same thing for college kids…”</p><p>Accurate 3-point shooting has intrinsically a higher value compared to a contested, closer 2-point shot. Accordingly, teams use effective shooting percentage (eFG%) as a metric for a more complete picture of a team or player’s shooting value: because eFG% adjusts for the fact that while a 3-point shoot typically returns a lower field goal percentage (FG%), it should be valued higher because it’s worth 50 percent more than a 2-point shot. Although shooting a lot of 3-pointers isn’t the only way to achieve a high eFG% (high 3-point accuracy and close-to-the-basket 2-pointers help a lot), it is certainly correlated.</p><p>In the figure below, I fit a best fit line to show this relationship. Although I used all data for the regression model, for visualization purposes I’ve excluded from the plot three teams that shoot above 30 3-point shots a game, including Savannah State which is shooting a mind-boggling 40.5 3-pointers a game!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5LBUoXx9ED5YSe8B." /><figcaption><em>Figure 2. Effective field goal percentage vs. number of 3 point attempts/game. Taking more 3 pointers is correlated with effective field goal percentage.</em></figcaption></figure><h4>March Madness</h4><p>There’s no question that teams are shooting more and more 3-pointers, representing a fundamental shift in how the game is played. So how do 3-point shooting habits relate to a team’s success in the Tournament? The answer: it depends.</p><p>It shouldn’t come as a surprise that having a top eFG% is a strong indicator for a team’s success in the March Madness tournament. Five of the past 10 National Championship winners have ranked within the top 25 of all Division I teams in eFG% for their respective championship season.</p><p>Below is a chart highlighting that three of the top five teams today in eFG% rank among the top teams in college basketball.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fairtable.com%2Fembed%2FshrArUx6DxjmCs3qT&amp;url=https%3A%2F%2Fairtable.com%2FshrArUx6DxjmCs3qT&amp;image=https%3A%2F%2Fstatic.airtable.com%2Fimages%2Foembed%2Fairtable.png&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=airtable" width="800" height="533" frameborder="0" scrolling="no"><a href="https://medium.com/media/190e14c4ce31a4a8608714a301aa5f10/href">https://medium.com/media/190e14c4ce31a4a8608714a301aa5f10/href</a></iframe><p>While the trend has shown an increase in 3-point shooting across both leagues, of the past 10 winners of the Tournament, all of them have actually attempted just under the league average per game of 3-point shots. However, collectively their 3-point accuracy (38 percent) was above average compared to the the league (36 percent). This trends applies to Final Four participants as well, thus high 3-point shooting volume and accuracy are not necessarily indicators of a team’s likelihood to succeed in the tournament.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nUYx8CrwkttCvBog." /><figcaption>Figure 3 Overview of point shooting in the tournament over the last ten years along with the current AP Top 25 teams.</figcaption></figure><p>However, things get more interesting when we look at <strong>Cinderella teams who tend to shoot more 3-pointers than their higher seeded counterparts</strong>. Since the availability of our dataset from 1993 until present, 86 percent (166/192 teams) of teams that have made it to the Elite 8 round or better have been seeded sixth or lower, so for the sake of this post we’ve defined teams seeded sixth or higher as “Cinderellas.” Using the pace-invariant 3-point attempts over field goal attempts (3PA/FGA<strong>),</strong> we see these teams have shot 35% of total shots from beyond the arc, which is 5% higher than their top seeded counterparts who shot 30% (a statistically significant result at the 1% confidence level using both a <a href="https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test">Kolmogorov Smirnov test</a> and t<a href="https://en.wikipedia.org/wiki/Student%27s_t-test">-test</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*eYf9jzVYDB_V-GWb." /><figcaption>Figure 4 Cinderella teams shoot 3-pointers at a higher rate compared to other tournament teams.</figcaption></figure><p>For successful teams, we can also look at how 3-point shooting varies as a function of tournament seed. In the below figure, I have shown all of Elite 8 participants over the past 10 years and have fit a linear regression line for this trend. We can see a clear correlation between seeding and 3-point shooting. In short, successful lower seeded teams rely more on the 3-point shot compared to higher seeds.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*RG-DghMeoJPBLyIi." /><figcaption><em>Figure 5. NCAA Division I teams 3-point shots as percentage of total shots that continued in the tournament past the Sweet 16 round.</em></figcaption></figure><p>By definition, Cinderella teams are less common, but given past history, it is apparent that Cinderella teams take more 3-point shots. This is not a coincidence. There is a <a href="https://arxiv.org/pdf/1111.0693.pdf">strong foundation</a> suggesting that underdogs should shoot for 3-point shots due in part to their high variance nature. Accordingly, a hot 3-point shooting streak can be just the tool for propelling an undermatched team to the top. In essence, three-point shots, though more mathematically “risky” give underdogs their best chance to get lucky.</p><p>Of course, 3-point shooting is just one metric to analyze a team’s potential tournament success, and it’s important to also consider other offensive and defensive metrics for a more complete picture of the story. However, based on the analysis presented in this post, of the teams taking at least 25 3-pointers per game, six of them are likely to enter the tournament: <strong>Akron, UNC Wilmington, Princeton, Vanderbilt, Winthrop,</strong> and <strong>UT Arlington</strong>. If their shots are falling in the coming weeks, these prolific 3-point shooting teams are in a great position to go deep in the tournament.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2F2MblgY9v6Moqk%2Ftwitter%2Fiframe&amp;src_secure=1&amp;url=http%3A%2F%2Fgiphy.com%2Fgifs%2Fncaa-2MblgY9v6Moqk&amp;image=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2F2MblgY9v6Moqk%2Fgiphy.gif&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="244" frameborder="0" scrolling="no"><a href="https://medium.com/media/c744ef0fc17b99d26b0e2379ceca6dd8/href">https://medium.com/media/c744ef0fc17b99d26b0e2379ceca6dd8/href</a></iframe><h4>Data, changing the game</h4><p>Beyond the sheer spectacle of Curry’s 402 three-pointers, that late season game vs. the Memphis Grizzlies represented a new paradigm on the court: offensive big men dominating the paint were no longer the primary focus for building a championship team, rather a team of guards with high eFG% could propel a team to success.</p><p>While prolific 3-point shooting is not a tell-tale sign of tournament success for all teams, it can be a valuable tool for propelling Cinderellas deep into the tournament. Three-point shooting is on the rise for college basketball, and I expect teams will continue to follow the Warriors and Rockets’ lead, attempting more shots from downtown than ever before.</p><blockquote><em>Which college teams do you think have the 3-point skills to go deep in this year’s Tournament?</em></blockquote><figure><a href="http://builttoadapt.io/"><img alt="" src="https://cdn-images-1.medium.com/max/71/1*9fg8oHTkroGjylRUKyKQAw.png" /></a></figure><p><em>Change is the only constant, so individuals, institutions, and businesses must be</em> <a href="http://builttoadapt.io/"><em>Built to Adapt</em></a><em>. At</em> <a href="http://pivotal.io/"><em>Pivotal</em></a><em>, we believe change should be expected, embraced and incorporated continuously through development and innovation, because good software is never finished.</em></p><figure><a href="http://pivotal.io/"><img alt="" src="https://cdn-images-1.medium.com/max/300/1*5bAWyGkjY_V8h0kl1X_P-w.png" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e2789981ba52" width="1" height="1" alt=""><hr><p><a href="https://medium.com/built-to-adapt/how-curry-ball-will-impact-march-madness-brackets-e2789981ba52">How Curry Ball Will Impact March Madness Brackets</a> was originally published in <a href="https://medium.com/built-to-adapt">Built to Adapt</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>