<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Wes Swager on Medium]]></title>
        <description><![CDATA[Stories by Wes Swager on Medium]]></description>
        <link>https://medium.com/@WesSwager6?source=rss-18f13aeb73d6------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*ZRUGoGfnfyVjhYP1.png</url>
            <title>Stories by Wes Swager on Medium</title>
            <link>https://medium.com/@WesSwager6?source=rss-18f13aeb73d6------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 14:17:33 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@WesSwager6/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Women’s Soccer Expected Goals — Data Cleaning— Part-Five: Location Coordinates]]></title>
            <link>https://medium.com/@WesSwager6/womens-soccer-expected-goals-data-cleaning-part-five-location-coordinates-e5d54f939717?source=rss-18f13aeb73d6------2</link>
            <guid isPermaLink="false">https://medium.com/p/e5d54f939717</guid>
            <category><![CDATA[womens-soccer]]></category>
            <category><![CDATA[expected-goals]]></category>
            <category><![CDATA[data-cleaning]]></category>
            <category><![CDATA[soccer-analytics]]></category>
            <category><![CDATA[statsbomb]]></category>
            <dc:creator><![CDATA[Wes Swager]]></dc:creator>
            <pubDate>Tue, 16 Nov 2021 19:26:45 GMT</pubDate>
            <atom:updated>2021-12-02T15:40:19.895Z</atom:updated>
            <content:encoded><![CDATA[<h3>Women’s Soccer Expected Goals — Data Cleaning— Part-Five: Assessing Transforming Location Coordinates</h3><p>Part-five of a series for the data cleaning portion of the data science workflow creating a women’s club soccer expected goals (xG) classification model, explaining the process of assessing location-descriptive features and splitting them into individual coordinate features.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*wzAdV2PMlKhcy6Y5" /></figure><p><a href="https://github.com/wswager/womens_soccer_expected_goals_model">GitHub - wswager/womens_soccer_expected_goals_model: Classification model for expected goals (xG) in women&#39;s club soccer, predicting the likelihood that a shot will score using data extracted from StatsBomb.</a></p><h3>Background</h3><p>Continued from parts-one-through-four of the data cleaning process:</p><ul><li><a href="https://medium.com/@wswager/soccer-expected-goals-data-cleaning-part-one-defining-the-target-feature-9b26f9971ae1">Soccer Expected Goals — Data Cleaning — Part One: Defining the Target Feature</a></li><li><a href="https://medium.com/@wswager/soccer-expected-goals-data-cleaning-part-one-irrelevant-data-a4fb4692197b">Soccer Expected Goals — Data Cleaning — Part Two: Irrelevant Data</a></li><li><a href="https://medium.com/@wswager/soccer-expected-goals-data-cleaning-part-three-correcting-datatypes-3f80c0987370">Soccer Expected Goals — Data Cleaning — Part Three: Correcting Datatypes</a></li><li><a href="https://medium.com/@wswager/soccer-expected-goals-data-cleaning-part-four-processing-missing-values-c9af8b366605">Soccer Expected Goals — Data Cleaning — Part Four: Processing Missing Values</a></li></ul><h3>Introduction</h3><h4>Expected Goals (xG)</h4><p>xG is used to indicate the quality of a shot.</p><p>xG, as a metric, measures the likelihood that a shot will result in a goal based on the characteristics of the shot and the play preceding the shot.</p><p>xG is measured on a scale between zero and one, with one representing a goal. For example, a shot with a 0.5 xG indicates a shot having a 50% chance of being a goal.</p><p><a href="https://medium.com/@wswager/what-is-xg-f6f82053d2cc">What is xG?</a></p><h4>Classification Model</h4><p>The metric of expected goals is calculated through the use of a classification model.</p><p>A classification model refers to a predictive modeling problem where a class label is predicted for a given example of input data.</p><p>For this project a supervised approach was used with the training data for the model including which shots were goals.</p><h4>Data Cleaning</h4><blockquote>Data cleaning is the process of detecting and correcting corrupt or inaccurate records from a record set, table, or database and refers to identifying incomplete, incorrect, inaccurate, or irrelevant parts of the data and then replacing, modifying, or deleting the dirty or coarse data.</blockquote><p><a href="https://en.wikipedia.org/wiki/Data_cleansing"><em>Wikipedia</em></a></p><h4>The Data</h4><p>The data for this project was extracted from <a href="https://statsbomb.com/">StatsBomb</a>’s Open Data.</p><p>StatsBomb are a United Kingdom based football (soccer) data analytics company. StatsBomb provide free access to a segment of their proprietary dataset via GitHub: <a href="https://github.com/statsbomb/open-data">StatsBomb Open Data</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*6cs5aQCYc4GWCK-Z.jpeg" /></figure><p>For the purposes of this project the relevant data targeted was, primarily, characteristics of shots and, secondarily, characteristics of the plays creating those shots, from women’s club soccer matches.</p><p><em>Note: Assessment of plays creating shots is subjective and based on domain knowledge specific to the sport of soccer</em></p><h3>Data Cleaning</h3><h4>Working Data</h4><p>Previously, in <a href="https://medium.com/@wswager/womens-soccer-expected-goals-data-extraction-999f90d1f5a6">Women’s Soccer Expected Goals — Data Extraction</a>, shot event data and key pass features were extracted for target women’s club competitions, competition ids, and season ids within StatsBomb Open Data.</p><pre>extracted_data.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*3elFDNnP0i_T71o0.png" /></figure><pre>print(&quot;Total Women&#39;s Club Shot Events:&quot;, len(extracted_data))</pre><pre>Total Women&#39;s Club Competition Shot Events: 6114</pre><pre>print(&#39;Updated Shot w/ Key Pass Features:&#39;, extracted_data.shape[1])</pre><pre>Updated Shot w/ Key Pass Features: 81</pre><h4>Review from Parts-One-through-Four</h4><p>Previously, in <a href="https://medium.com/@wswager/soccer-expected-goals-data-cleaning-part-one-defining-the-target-feature-9b26f9971ae1">Women’s Soccer Expected Goals — Data Cleaning — Part One: Defining the Target Feature</a>, shot_outcome was updated from a categorical feature to the boolean feature goal, ‘True’ if a shot resulted in a goal, ‘False’ if not. The new feature, goal represents the target feature for the eventual supervised classification modeling.</p><pre>extracted_data[&#39;goal&#39;].value_counts()</pre><pre>False    5416<br>True      664<br>Name: goal, dtype: bool</pre><pre>print(&quot;goal Percent &#39;True&#39;:&quot;, round((((sum(extracted_data[&#39;goal&#39;])) / (len(extracted_data))) * 100), 2), &#39;%&#39;)</pre><pre>goal Percent &#39;True&#39;: 10.92 %</pre><p>Next, in <a href="https://medium.com/@wswager/soccer-expected-goals-data-cleaning-part-one-irrelevant-data-a4fb4692197b">Women’s Soccer Expected Goals — Data Cleaning — Part Two: Irrelevant Data</a>, irrelevant features were dropped:</p><ul><li>Features duplicated as a result of extracting and concatenating shot event and the pass event data</li><li>Features deemed as not characteristics of shots or plays creating shots</li></ul><pre>extracted_data.shape[1]</pre><pre>21</pre><pre>list(extracted_data.columns.values)</pre><pre>[&#39;period_x&#39;,<br> &#39;timestamp_x&#39;,<br> &#39;play_pattern_x&#39;,<br> &#39;location_x&#39;,<br> &#39;under_pressure_x&#39;,<br> &#39;shot_statsbomb_xg&#39;,<br> &#39;shot_end_location&#39;,<br> &#39;shot_technique&#39;,<br> &#39;goal&#39;,<br> &#39;shot_type&#39;,<br> &#39;shot_body_part&#39;,<br> &#39;shot_one_on_one&#39;,<br> &#39;shot_open_goal&#39;,<br> &#39;shot_first_time&#39;,<br> &#39;shot_redirect&#39;,<br> &#39;shot_deflected&#39;,<br> &#39;shot_follows_dribble&#39;,<br> &#39;pass_length&#39;,<br> &#39;pass_angle&#39;,<br> &#39;pass_height&#39;,<br> &#39;pass_type&#39;,<br> &#39;pass_switch&#39;,<br> &#39;pass_through_ball&#39;,<br> &#39;pass_technique&#39;,<br> &#39;pass_backheel&#39;,<br> &#39;pass_cross&#39;,<br> &#39;counterpress&#39;,<br> &#39;pass_cut_back&#39;,<br> &#39;pass_inswinging&#39;,<br> &#39;pass_straight&#39;,<br> &#39;pass_outswinging&#39;]</pre><p>Next, in <a href="https://medium.com/@wswager/soccer-expected-goals-data-cleaning-part-three-correcting-datatypes-3f80c0987370">Women’s Soccer Expected Goals — Data Cleaning — Part Three: Correcting Datatypes</a>, the data types of the features were assessed and corrected:</p><ul><li>Features identified as boolean in nature were updated from object to boolean datatype</li><li>timestamp_x was updated from object to datetime datatype</li></ul><pre>extracted_data.info()</pre><pre>&lt;class &#39;pandas.core.frame.DataFrame&#39;&gt;<br>Int64Index: 6080 entries, 0 to 6079<br>Data columns (total 31 columns):<br> #   Column                Non-Null Count  Dtype         <br>---  ------                --------------  -----         <br> 0   period_x              6080 non-null   int64         <br> 1   timestamp_x           6080 non-null   datetime64[ns]<br> 2   play_pattern_x        6080 non-null   object        <br> 3   location_x            6080 non-null   object        <br> 4   under_pressure_x      6080 non-null   bool          <br> 5   shot_statsbomb_xg     6080 non-null   float64       <br> 6   shot_end_location     6080 non-null   object        <br> 7   shot_technique        6080 non-null   object        <br> 8   goal                  6080 non-null   bool          <br> 9   shot_type             6080 non-null   object        <br> 10  shot_body_part        6080 non-null   object        <br> 11  shot_one_on_one       6080 non-null   bool          <br> 12  shot_open_goal        6080 non-null   bool          <br> 13  shot_first_time       6080 non-null   bool          <br> 14  shot_redirect         6080 non-null   bool          <br> 15  shot_deflected        6080 non-null   bool          <br> 16  shot_follows_dribble  6080 non-null   bool          <br> 17  pass_length           4138 non-null   float64       <br> 18  pass_angle            4138 non-null   float64       <br> 19  pass_height           4138 non-null   object        <br> 20  pass_type             960 non-null    object        <br> 21  pass_switch           6080 non-null   bool          <br> 22  pass_through_ball     6080 non-null   bool          <br> 23  pass_technique        355 non-null    object        <br> 24  pass_backheel         6080 non-null   bool          <br> 25  pass_cross            6080 non-null   bool          <br> 26  counterpress          6080 non-null   bool          <br> 27  pass_cut_back         6080 non-null   bool          <br> 28  pass_inswinging       6080 non-null   bool          <br> 29  pass_straight         6080 non-null   bool          <br> 30  pass_outswinging      6080 non-null   bool          <br>dtypes: bool(17), datetime64[ns](1), float64(3), int64(1), object(9)<br>memory usage: 813.4+ KB</pre><p>Next, in <a href="https://medium.com/@wswager/soccer-expected-goals-data-cleaning-part-four-processing-missing-values-c9af8b366605">Soccer Expected Goals — Data Cleaning — Part-Four: Processing Missing Values</a>, missing values were identified, assessed and processed.</p><p>pass_length, pass_angle, and pass_height each had 1,942 missing values. It was assumed that these shot events were not preceded by a pass, ‘no pass shots’:</p><pre>print(&#39;pass_length NA:&#39;, sum(extracted_data[&#39;pass_length&#39;].isna()), &#39;\n&#39;, &#39;pass_angle NA:&#39;, sum(extracted_data[&#39;pass_angle&#39;].isna()), &#39;\n&#39;, &#39;pass_height NA:&#39;, sum(extracted_data[&#39;pass_height&#39;].isna()))</pre><pre>pass_length NA: 1942 <br> pass_angle NA: 1942 <br> pass_height NA: 1942</pre><ul><li>Numerical pass-related features were replaced with 0</li><li>Categorical pass-related features were replaced with ‘No Pass’</li><li>Boolean pass-related features did not require filling as they already indicated ‘False’</li></ul><p>Additional missing values for pass_type were assumed to be passes from open-play and filled with ‘Open Play.’</p><pre>extracted_data[&#39;pass_type&#39;].value_counts(dropna = False)</pre><pre>Open Play       3177<br>No Pass         1943<br>Corner           400<br>Recovery         305<br>Free Kick        201<br>Throw-in          42<br>Interception      10<br>Kick Off           1<br>Goal Kick          1<br>Name: pass_type, dtype: int64</pre><p>Additional missing values for pass_technique were assumed to be standard passes and were filled with ‘Standard.’</p><pre>extracted_data[&#39;pass_technique&#39;].value_counts(dropna = False)</pre><pre>Standard        3782<br>No Pass         1943<br>Through Ball     198<br>Inswinging        76<br>Outswinging       55<br>Straight          26<br>Name: pass_technique, dtype: int64</pre><pre>extracted_data.isnull().sum()</pre><pre>period_x                0<br>timestamp_x             0<br>play_pattern_x          0<br>location_x              0<br>under_pressure_x        0<br>shot_statsbomb_xg       0<br>shot_end_location       0<br>shot_technique          0<br>goal                    0<br>shot_type               0<br>shot_body_part          0<br>shot_one_on_one         0<br>shot_open_goal          0<br>shot_first_time         0<br>shot_redirect           0<br>shot_deflected          0<br>shot_follows_dribble    0<br>pass_length             0<br>pass_angle              0<br>pass_height             0<br>pass_type               0<br>pass_switch             0<br>pass_through_ball       0<br>pass_technique          0<br>pass_backheel           0<br>pass_cross              0<br>counterpress            0<br>pass_cut_back           0<br>pass_inswinging         0<br>pass_straight           0<br>pass_outswinging        0<br>dtype: int64</pre><p>Working data prior to step-five:</p><pre>print(&quot;Total Women&#39;s Club Shot Events:&quot;, len(extracted_data))</pre><pre>Total Women&#39;s Club Competition Shot Events: 6114</pre><pre>print(&#39;Updated Shot w/ Key Pass Features:&#39;, extracted_data.shape[1])</pre><pre>Updated Shot w/ Key Pass Features: 31</pre><pre>extracted_data.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LfNKC4YqoC2W8igB-lY_PQ.png" /></figure><h3>Location Features</h3><p>The shot event data contains two location-descriptive features, both of which include multiple coordinates:</p><ul><li>location_x describes the location on the field from which the shot was taken and contains x and y-coordinates</li></ul><pre>extracted_data[&#39;location_x&#39;].head()</pre><pre>0    [109.0, 46.0]<br>1    [113.0, 35.0]<br>2     [94.0, 43.0]<br>3     [86.0, 34.0]<br>4     [94.0, 33.0]<br>Name: location_x, dtype: object</pre><ul><li>shot_end_location describes the location on the field where the shot’s path ended and contains x, y, and z-coordinates</li></ul><pre>extracted_data[&#39;shot_end_location&#39;].head()</pre><pre>0         [112.0, 45.0]<br>1    [120.0, 32.9, 0.4]<br>2    [120.0, 42.8, 0.5]<br>3    [119.0, 33.3, 0.5]<br>4    [120.0, 34.8, 0.5]<br>Name: shot_end_location, dtype: object</pre><h4>location_x</h4><p><em>Split location_x into separate x and y-coordinates</em></p><pre>shot_location_df = pd.DataFrame(extracted_data[&#39;location_x&#39;].tolist(), index = extracted_data.index)</pre><pre>shot_location_df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/116/1*nadBZM4V-zlJvKhzLePgsQ.png" /></figure><p><em>Replace location_x with shot_location_x and shot_location_y</em></p><pre>extracted_data.drop(&#39;location_x&#39;, axis = 1, inplace = True)</pre><pre>extracted_data[&#39;shot_location_y&#39;] = shot_location_df[0]<br>extracted_data[&#39;shot_location_x&#39;] = shot_location_df[1]</pre><pre>extracted_data[[&#39;shot_location_y&#39;, &#39;shot_location_x&#39;]].head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/302/1*Nfnb2ql7t2nHwMWVf2iG9g.png" /></figure><pre>extracted_data[&#39;shot_location_x&#39;].describe()</pre><pre>count    6043.000000<br>mean       40.286745<br>std         9.782598<br>min        12.400000<br>25%        33.000000<br>50%        40.100000<br>75%        47.000000<br>max        68.200000<br>Name: shot_location_x, dtype: float64</pre><pre>extracted_data[&#39;shot_location_y&#39;].describe()</pre><pre>count    6043.000000<br>mean      104.003690<br>std         8.819068<br>min        77.800000<br>25%        97.700000<br>50%       105.600000<br>75%       111.000000<br>max       120.000000<br>Name: shot_location_y, dtype: float64</pre><h4>shot_end_location</h4><p><em>Split shot_end_location into separate x and y-coordinates</em></p><pre>end_location_df = pd.DataFrame(extracted_data[&#39;shot_end_location&#39;].tolist(), index = extracted_data.index)</pre><pre>end_location_df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/151/1*TE4WaGBsjqdKH3AE4qbNcQ.png" /></figure><p><em>Assess the values of shot_end_location’s z-coordinate</em></p><pre>print(&#39;shot_end_location z-coordinate NA:&#39;, (sum(end_location_df[2].isna())), &#39;\n&#39;, &#39;Percent shot_end_location z-coordinate NA:&#39;, (round((((sum(end_location_df[2].isna())) / (len(extracted_data))) * 100), 2)), &#39;%&#39;)</pre><pre>shot_end_location z-coordinate NA: 1796 <br> Percent shot_end_location z-coordinate NA: 29.54 %</pre><p><em>Drop shot_end_location z-coordinate due to too many missing values</em></p><p><em>Replace shot_end_location with shot_end_location_x, shot_end_location_y, and shot_end_location_z</em></p><pre>extracted_data.drop(&#39;shot_end_location&#39;, axis = 1, inplace = True)</pre><pre>extracted_data[&#39;end_location_y&#39;] = end_location_df[0]<br>extracted_data[&#39;end_location_x&#39;] = end_location_df[1]</pre><pre>extracted_data[[&#39;end_location_y&#39;, &#39;end_location_x&#39;]].head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/284/1*SMFFvpYyvO3sDqCakz3B9Q.png" /></figure><pre>extracted_data[&#39;end_location_x&#39;].describe()</pre><pre>count    6043.000000<br>mean       40.155453<br>std         6.314044<br>min         0.100000<br>25%        36.400000<br>50%        40.000000<br>75%        43.800000<br>max        80.000000<br>Name: end_location_x, dtype: float64</pre><pre>extracted_data[&#39;end_location_y&#39;].describe()</pre><pre>count    6043.000000<br>mean      116.005527<br>std         6.252442<br>min        84.000000<br>25%       115.000000<br>50%       119.000000<br>75%       120.000000<br>max       120.000000<br>Name: end_location_y, dtype: float64</pre><h3>Results</h3><p>The values for the two location-descriptive features, location_x and shot_end_location, were split into separate coordinates. The location-descriptive features were then dropped and replaced with new numeric features for the separate coordinate values:</p><ul><li>shot_location_x</li><li>shot_location_y</li><li>end_location_x</li><li>end_location_y</li></ul><p><em>Note: The z-coordinate for shot_end_location was dropped.</em></p><p><em>Note: The previous location-descriptive features contained object datatypes. The new coordinate features float64 datatypes.</em></p><pre>print(&quot;Total Women&#39;s Club Shot Events:&quot;, len(extracted_data))</pre><pre>Total Women&#39;s Club Competition Shot Events: 6114</pre><pre>print(&#39;Updated Shot w/ Key Pass Features:&#39;, extracted_data.shape[1])</pre><pre>Updated Shot w/ Key Pass Features: 33</pre><pre>extracted_data.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LfNKC4YqoC2W8igB-lY_PQ.png" /></figure><pre>extracted_data.info()</pre><pre>&lt;class &#39;pandas.core.frame.DataFrame&#39;&gt;<br>Int64Index: 6080 entries, 0 to 6079<br>Data columns (total 33 columns):<br> #   Column                Non-Null Count  Dtype         <br>---  ------                --------------  -----         <br> 0   period_x              6080 non-null   int64         <br> 1   timestamp_x           6080 non-null   datetime64[ns]<br> 2   play_pattern_x        6080 non-null   object        <br> 3   under_pressure_x      6080 non-null   bool          <br> 4   shot_statsbomb_xg     6080 non-null   float64       <br> 5   shot_technique        6080 non-null   object        <br> 6   goal                  6080 non-null   bool          <br> 7   shot_type             6080 non-null   object        <br> 8   shot_body_part        6080 non-null   object        <br> 9   shot_one_on_one       6080 non-null   bool          <br> 10  shot_open_goal        6080 non-null   bool          <br> 11  shot_first_time       6080 non-null   bool          <br> 12  shot_redirect         6080 non-null   bool          <br> 13  shot_deflected        6080 non-null   bool          <br> 14  shot_follows_dribble  6080 non-null   bool          <br> 15  pass_length           6080 non-null   float64       <br> 16  pass_angle            6080 non-null   float64       <br> 17  pass_height           6080 non-null   object        <br> 18  pass_type             6080 non-null   object        <br> 19  pass_switch           6080 non-null   bool          <br> 20  pass_through_ball     6080 non-null   bool          <br> 21  pass_technique        6080 non-null   object        <br> 22  pass_backheel         6080 non-null   bool          <br> 23  pass_cross            6080 non-null   bool          <br> 24  counterpress          6080 non-null   bool          <br> 25  pass_cut_back         6080 non-null   bool          <br> 26  pass_inswinging       6080 non-null   bool          <br> 27  pass_straight         6080 non-null   bool          <br> 28  pass_outswinging      6080 non-null   bool          <br> 29  shot_location_y       6080 non-null   float64       <br> 30  shot_location_x       6080 non-null   float64       <br> 31  end_location_y        6080 non-null   float64       <br> 32  end_location_x        6080 non-null   float64       <br>dtypes: bool(17), datetime64[ns](1), float64(7), int64(1), object(7)<br>memory usage: 908.4+ KB</pre><h3>Continued</h3><p>Part-six continues the series for the data cleaning portion of the data science workflow creating a women’s club soccer expected goals (xG) classification model, explaining the process of identifying, assessing, and processing outliers.</p><h4>More</h4><p>If you liked this post, please give it an applause and follow me as I will be continuing with a series of posts for each process through the data science workflow of my <a href="https://github.com/wswager/womens_soccer_expected_goals_model">Women’s Soccer Expected Goals Model</a>:</p><p><a href="https://medium.com/@wswager">Wes Swager - Medium</a></p><p>Also, follow me on Twitter, where I post regularly about tactical observations for soccer:</p><p><a href="https://mobile.twitter.com/Wes_Swager_6">JavaScript is not available.</a></p><p>I would love to read any feedback you might have in the comments.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e5d54f939717" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Women’s Soccer Expected Goals — Data Extraction]]></title>
            <link>https://medium.com/@WesSwager6/womens-soccer-expected-goals-data-extraction-999f90d1f5a6?source=rss-18f13aeb73d6------2</link>
            <guid isPermaLink="false">https://medium.com/p/999f90d1f5a6</guid>
            <category><![CDATA[womens-soccer]]></category>
            <category><![CDATA[soccer-analytics]]></category>
            <category><![CDATA[expected-goals]]></category>
            <category><![CDATA[statsbomb]]></category>
            <category><![CDATA[data-extraction]]></category>
            <dc:creator><![CDATA[Wes Swager]]></dc:creator>
            <pubDate>Fri, 29 Oct 2021 21:17:13 GMT</pubDate>
            <atom:updated>2021-11-15T16:31:17.660Z</atom:updated>
            <content:encoded><![CDATA[<h3>Women’s Soccer Expected Goals — Data Extraction</h3><p>The data extraction process of the data science workflow for creating a women’s club soccer expected goals (xG) classification model, predicting the likelihood that a shot will score.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*aMCuPGPwzw9VX3G6" /></figure><p><em>Please Note: The following was previously published broken into individual posts per step of the data extraction process in:</em></p><ul><li><a href="https://medium.com/@wswager/using-statsbombpy-package-to-compile-statsbomb-data-c0f6734e5234">Soccer Expected Goals — Data Extraction — Part-One: Extracting Shots Data</a></li><li><a href="https://medium.com/@wswager/continued-extraction-of-stats-bomb-data-merging-key-pass-data-with-shot-event-data-5e1ca2faac11">Soccer Expected Goals — Data Extraction — Part-Two: Extracting Key Pass Features</a></li><li><a href="https://medium.com/@wswager/soccer-expected-goals-data-extraction-part-three-exploring-related-events-feature-5555a2231805">Soccer Expected Goals — Data Extraction — Part-Three: Exploring Related Events</a></li></ul><p><a href="https://github.com/wswager/womens_soccer_expected_goals_model">GitHub - wswager/womens_soccer_expected_goals_model: Classification model for expected goals (xG) in women&#39;s club soccer, predicting the likelihood that a shot will score using data extracted from StatsBomb.</a></p><h3>Introduction</h3><h4>Expected Goals (xG)</h4><p>xG is used to indicate the quality of a shot.</p><p>xG, as a metric, measures the likelihood that a shot will result in a goal based on the characteristics of the shot and the play preceding the shot.</p><p>xG is measured on a scale between zero and one, with one representing a goal. For example, a shot with a 0.5 xG indicates a shot having a 50% chance of being a goal.</p><p><a href="https://medium.com/@wswager/what-is-xg-f6f82053d2cc">What is xG?</a></p><h4>Classification Model</h4><p>The metric of expected goals is calculated through the use of a classification model.</p><p>A classification model refers to a predictive modeling problem where a class label is predicted for a given example of input data.</p><p>For this project a supervised approach was used with the training data for the model including which shots were goals.</p><h4>Data Extraction</h4><blockquote><em>Data extraction is the act or process of retrieving data out of…data sources for further data processing</em></blockquote><p><a href="https://en.wikipedia.org/wiki/Data_extraction"><em>Wikipedia</em></a></p><h4>The Data</h4><p>The data for this project was extracted from <a href="https://statsbomb.com/">StatsBomb</a>’s Open Data.</p><p>StatsBomb are a United Kingdom based football (soccer) data analytics company. StatsBomb provide free access to a segment of their proprietary dataset via GitHub: <a href="https://github.com/statsbomb/open-data">StatsBomb Open Data</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*6cs5aQCYc4GWCK-Z.jpeg" /></figure><p>StatsBomb Open Data is organized in JSON files:</p><p><strong>Matches</strong></p><ul><li>Folders are organized by competition (league or tournament)</li><li>Files within the folders are organized by season (year) ID</li><li>Files contain nested dictionaries with descriptive data for each individual match</li></ul><p><strong>Events</strong></p><ul><li>Files organized by match ID</li><li>Files contain nested dictionaries with descriptive data for each event within each individual match</li></ul><p>For the purposes of this project the relevant data targeted was, primarily, characteristics of shots and, secondarily, characteristics of the plays creating those shots, from women’s club soccer matches.</p><p><em>Note: Assessment of plays creating shots is subjective and based on domain knowledge specific to the sport of soccer</em></p><h4>statsbombpy</h4><p>The following processes of extracting data from the StatsBomb Open Data relies heavily on the use of <a href="https://github.com/statsbomb/statsbombpy">statsbombpy</a>.</p><p>statsbombpy is a Python package created by StatsBomb which streams StatsBomb data directly within python. The package allows access to StatsBomb Open Data for free or allows access to their API with use of log-in credentials.</p><h3>Data Extraction</h3><h4>Extract Target Matches Data</h4><p><em>Import statsbombpy for extracting StatsBomb data</em></p><pre>!pip install statsbombpy<br>from statsbombpy import sb</pre><p>View competitions available through StatsBomb Open Data</p><pre>competitions_df = sb.competitions()</pre><pre>print(&#39;Available Competitions:&#39;, competitions_df[&#39;competition_name&#39;].unique())</pre><pre>Available Competitions: [&#39;Champions League&#39; &quot;FA Women&#39;s Super League&quot; &#39;FIFA World Cup&#39; &#39;La Liga&#39;<br> &#39;NWSL&#39; &#39;Premier League&#39; &quot;Women&#39;s World Cup&quot;]</pre><p><em>Isolate and display target women’s competitions, competition ids, and season ids</em></p><pre>target_comp_df = competitions_df.loc[competitions_df[&#39;competition_gender&#39;] == &#39;female&#39;]</pre><pre>target_comp_ids = target_comp_df[&#39;competition_id&#39;].unique()</pre><pre>target_season_ids = target_comp_df[&#39;season_id&#39;].unique()</pre><pre>print(&quot;Women&#39;s Competitions:&quot;, target_comp_df[&#39;competition_name&#39;].unique(), &#39;\n&#39;, &quot;Women&#39;s competition_ids:&quot;, target_comp_ids, &#39;\n&#39;, &quot;Women&#39;s Competition season_ids:&quot;, target_season_ids)</pre><pre>Women&#39;s Competitions: [&quot;FA Women&#39;s Super League&quot; &#39;NWSL&#39;] <br> Women&#39;s competition_ids: [37 49] <br> Women&#39;s Competition season_ids: [42  4  3]</pre><p><em>Refine target competitions to women’s club competitions</em></p><pre>target_comp_df = competitions_df.loc[competitions_df[&#39;competition_id&#39;].isin([37, 49])]</pre><pre>target_comp_ids = target_comp_df[&#39;competition_id&#39;].unique()</pre><pre>target_season_ids = target_comp_df[&#39;season_id&#39;].unique()</pre><pre>print(&quot;Women&#39;s Club Competitions:&quot;, target_comp_df[&#39;competition_name&#39;].unique(), &#39;\n&#39;, &quot;Women&#39;s Club competition_ids:&quot;, target_comp_ids, &#39;\n&#39;, &quot;Women&#39;s Club Competition season_ids:&quot;, target_season_ids)</pre><pre>Women&#39;s Club Competitions: [&quot;FA Women&#39;s Super League&quot; &#39;NWSL&#39;] <br> Women&#39;s Club competition_ids: [37 49] <br> Women&#39;s Club Competition season_ids: [42  4  3]</pre><p><em>Create dataframes for the matches in each season of the target competitions</em></p><pre>matches_df_37_42 = sb.matches(competition_id = 37, season_id = 42)</pre><pre>matches_df_37_4 = sb.matches(competition_id = 37, season_id = 4)</pre><pre>matches_df_49_3 = sb.matches(competition_id = 49, season_id = 3)</pre><p><em>Concatenate target season dataframes into combined dataframe</em></p><pre>matches_df = pd.concat([matches_df_37_42, matches_df_37_4, matches_df_49_3], ignore_index = True)</pre><pre>matches_df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mnpEjTS8DVDipWRmNEFQTA.png" /></figure><pre>print(&#39;Number of Seasons:&#39;, len(target_season_ids))</pre><pre>Number of Women&#39;s Club Seasons: 3</pre><pre>print(&#39;Total Women&#39;s Club Matches:&#39;, len(matches_df))</pre><pre>Total Matches: 231</pre><h4>Extract Shot Events Data</h4><p><em>Create dataframes for shot events in each season of the target competitions</em></p><pre>shots_df_37_42 = sb.competition_events(country = &#39;England&#39;, division = &quot;FA Women&#39;s Super League&quot;, season = &#39;2018/2019&#39;, gender = &#39;female&#39;, split = True)[&#39;shots&#39;]</pre><pre>shots_df_37_4 = sb.competition_events(country = &#39;England&#39;, division = &quot;FA Women&#39;s Super League&quot;, season = &#39;2019/2020&#39;, gender = &#39;female&#39;, split = True)[&#39;shots&#39;]</pre><pre>shots_df_49_3 = sb.competition_events(country = &#39;United States of America&#39;, division = &#39;NWSL&#39;, season = &#39;2018&#39;, gender = &#39;female&#39;, split = True)[&#39;shots&#39;]</pre><p><em>Concatenate shot events dataframes into combined dataframe</em></p><pre>shots_df = pd.concat([shots_df_37_42, shots_df_37_4, shots_df_49_3], ignore_index = True)</pre><pre>shots_df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M7B6nF4iJPYotzbc9rBI2A.png" /></figure><pre>print(&quot;Total Women&#39;s Club Shot Events:&quot;, len(shots_df))</pre><pre>Total Women&#39;s Club Competition Shot Events: 6114</pre><pre>print(&#39;Total Shot Features:&#39;, shots_df.shape[1])</pre><pre>Total Shot Features: 37</pre><h4>Extract Key Pass Features</h4><p>As mentioned previously, characteristics of the play creating a shot can be valuable toward the potential quality of the shot. For this reason, features of other event types related to shot events can be valuable and worth assessment.</p><p>shot_key_pass_id is a feature for shot events which identifies specific pass events (indicating the event id feature value), as passes which lead directly to the shot (potential assist, if the shot were to score).</p><p><em>Create dataframes for pass events in each season of the target competitions</em></p><pre>passes_df_37_42 = sb.competition_events(country = &#39;England&#39;, division = &quot;FA Women&#39;s Super League&quot;, season = &#39;2018/2019&#39;, gender = &#39;female&#39;, split = True)[&#39;passes&#39;]</pre><pre>passes_df_37_4 = sb.competition_events(country = &#39;England&#39;, division = &quot;FA Women&#39;s Super League&quot;, season = &#39;2019/2020&#39;, gender = &#39;female&#39;, split = True)[&#39;passes&#39;]</pre><pre>passes_df_49_3 = sb.competition_events(country = &#39;United States of America&#39;, division = &#39;NWSL&#39;, season = &#39;2018&#39;, gender = &#39;female&#39;, split = True)[&#39;passes&#39;]</pre><p><em>Concatenate pass events dataframes into combined dataframe</em></p><pre>passes_df = pd.concat([passes_df_37_42, passes_df_37_4, passes_df_49_3], ignore_index = True)</pre><pre>passes_df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f-GEENl5WcijC-U3eTlqYQ.png" /></figure><pre>print(&quot;Total Women&#39;s Club Competition Pass Events:&quot;, len(passes_df))</pre><pre>Total Women&#39;s Club Competition Pass Events: 209122</pre><pre>print(&#39;Total Pass Features:&#39;, passes_df.shape[1])</pre><pre>Total Pass Features: 45</pre><p><em>Create a key_pass_events list from shot_key_pass_id values for shot events</em></p><pre>key_pass_events = list(shots_df[‘shot_key_pass_id’])</pre><p><em>Search pass events for key passes</em></p><pre>passes_to_shots_df = passes_df[passes_df[&#39;id&#39;].isin(key_pass_events)]</pre><pre>print(&quot;Pass Events Identified as &#39;shot_key_pass_id&#39; for Shot Events:&quot;, len(passes_to_shots_df))</pre><pre>Pass Events Identified as &#39;shot_key_pass_id&#39; for Shot Events: 4164</pre><p><em>Concatenate pass data from passes_df for passes identified as shot event key passes with shots_df</em></p><pre>passes_df2 = passes_df.rename(columns = {&#39;id&#39;: &#39;shot_key_pass_id&#39;})</pre><pre>extracted_data = pd.merge(shots_df, passes_df2, on = [&#39;shot_key_pass_id&#39;], how = &#39;left&#39;)</pre><pre>extracted_data.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LfNKC4YqoC2W8igB-lY_PQ.png" /></figure><pre>print(&#39;Updated Shot w/ Key Pass Features:&#39;, extracted_data.shape[1])</pre><pre>Updated Shot w/ Key Pass Features: 81</pre><h4>Extract Related Events Features</h4><p>related_events is a feature for shot events which identifies specific events other than shot or pass events (indicating the event id feature value), as events significantly connected with leading to the shot.</p><p>In exploring the types of events available in the StatsBomb data, <strong>dribble </strong>and <strong>carry </strong>events appear to be the event types most likely to provide valuable data toward describing the quality of the shot.</p><p><em>Note: Assessment of plays creating shots is subjective and based on domain knowledge specific to the sport of soccer</em></p><p><em>Create a list from related_events values for shot events</em></p><pre>related_events = list(shots_df[&#39;related_events&#39;])</pre><p><strong>Dribble Events</strong></p><p><em>Create dataframes for dribble events in each season of the target competitions</em></p><pre>dribbless_df_37_42 = sb.competition_events(country = &#39;England&#39;, division = &quot;FA Women&#39;s Super League&quot;, season = &#39;2018/2019&#39;, gender = &#39;female&#39;, split = True)[&#39;dribbles&#39;]</pre><pre>dribbles_df_37_4 = sb.competition_events(country = &#39;England&#39;, division = &quot;FA Women&#39;s Super League&quot;, season = &#39;2019/2020&#39;, gender = &#39;female&#39;, split = True)[&#39;dribbles&#39;]</pre><pre>dribbles_df_49_3 = sb.competition_events(country = &#39;United States of America&#39;, division = &#39;NWSL&#39;, season = &#39;2018&#39;, gender = &#39;female&#39;, split = True)[&#39;dribbles&#39;]</pre><p><em>Concatenate dribble event dataframes into a combined dataframe</em></p><pre>dribbles_df = pd.concat([dribbles_df_37_42, dribbles_df_37_4, dribbles_df_49_3], ignore_index = True)</pre><pre>dribbles_df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kCH3R5faQMeR-hFx3HVirg.png" /></figure><pre>print(&quot;Total Women&#39;s Club Competition Dribble Events:&quot;, len(dribbles_df))</pre><pre>Total Women&#39;s Club Competition Dribble Events: 8187</pre><pre>print(&#39;Total Dribble Features:&#39;, dribbles_df.shape[1])</pre><pre>Total Dribble Features: 23</pre><p><em>Search dribble events for related_events</em></p><pre>dribbles_to_shots_df = dribbles_df[dribbles_df[‘id’].isin(related_events)]</pre><pre>print(“Dribble Events Identified as ‘related_events’ for Shot Events:”, len(dribbles_to_shots_df))</pre><pre>Dribble Events Identified as &#39;related_events&#39; for Shot Events: 0</pre><p>No related_events values for shot events match the id feature values of dribble events, therefore no additional data can be extracted or added.</p><p><strong>Carry Events</strong></p><p><em>Create dataframes for carry events in each season of the target competitions</em></p><pre>carrys_df_37_42 = sb.competition_events(country = &#39;England&#39;, division = &quot;FA Women&#39;s Super League&quot;, season = &#39;2018/2019&#39;, gender = &#39;female&#39;, split = True)[&#39;carrys&#39;]</pre><pre>carrys_df_37_4 = sb.competition_events(country = &#39;England&#39;, division = &quot;FA Women&#39;s Super League&quot;, season = &#39;2019/2020&#39;, gender = &#39;female&#39;, split = True)[&#39;carrys&#39;]</pre><pre>carrys_df_49_3 = sb.competition_events(country = &#39;United States of America&#39;, division = &#39;NWSL&#39;, season = &#39;2018&#39;, gender = &#39;female&#39;, split = True)[&#39;carrys&#39;]</pre><p><em>Concatenate carry event dataframes into a combined dataframe</em></p><pre>carrys_df = pd.concat([carrys_df_37_42, carrys_df_37_4, carrys_df_49_3], ignore_index = True)</pre><pre>carrys_df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FtstBQBlvWj-5vk4Z9Dtqg.png" /></figure><pre>print(&quot;Total Women&#39;s Club Competition Carry Events:&quot;, len(carrys_df))</pre><pre>Total Women&#39;s Club Competition Carry Events: 168439</pre><pre>print(&#39;Total Carry Features:&#39;, carrys_df.shape[1])</pre><pre>Total Carry Features: 19</pre><p><em>Search carry events for related_events</em></p><pre>carrys_to_shots_df = carrys_df[carrys_df[‘id’].isin(related_events)]</pre><pre>print(“Carry Events Identified as ‘related_events’ for Shot Events:”, len(carrys_to_shots_df))</pre><pre>Carry Events Identified as &#39;related_events&#39; for Shot Events: 0</pre><p>No related_events values for shot events match the id feature values of carry events, therefore no additional data can be extracted or added.</p><h3>Review</h3><ul><li>Target matches, women’s club compeitions, were identified and their data extracted</li><li>Shot event data was extracted from each of the target competition seasons and concatenated into a combined dataframe</li><li>Key passes were identified and their data extracted and concatenated with the shot events</li><li>Related events were explored but it was concluded no additional valuable data was available</li></ul><h3>Results</h3><pre>print(&#39;Total Extracted Events:&#39;, len(extracted_data))</pre><pre>Total Extracted Events: 6114</pre><pre>print(&#39;Total Extracted Features:&#39;, extracted_data.shape[1])</pre><pre>Total ExtractedFeatures: 81</pre><p>The extraction process yielded 6,144 shot events with 81 features, describing the shots or the play preceding the shots.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*viMHvfObzsF5bYajcWnWJA.png" /></figure><h4>More</h4><p>If you liked this post, please give it an applause and follow me as I will be continuing with a series of posts for each process through the data science workflow of my <a href="https://github.com/wswager/womens_soccer_expected_goals_model">Women’s Soccer Expected Goals Model</a>:</p><p><a href="https://medium.com/@wswager">Wes Swager - Medium</a></p><p>Also, follow me on Twitter, where I post regularly about tactical observations for soccer:</p><p><a href="https://twitter.com/Wes_Swager_6">JavaScript is not available.</a></p><p>I would love to read any feedback you might have in the comments.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=999f90d1f5a6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Soccer Expected Goals — Data Cleaning — Part Four: Processing Missing Values]]></title>
            <link>https://medium.com/@WesSwager6/soccer-expected-goals-data-cleaning-part-four-processing-missing-values-c9af8b366605?source=rss-18f13aeb73d6------2</link>
            <guid isPermaLink="false">https://medium.com/p/c9af8b366605</guid>
            <category><![CDATA[statsbomb]]></category>
            <category><![CDATA[expected-goals]]></category>
            <category><![CDATA[soccer-analytics]]></category>
            <category><![CDATA[data-cleaning]]></category>
            <dc:creator><![CDATA[Wes Swager]]></dc:creator>
            <pubDate>Wed, 27 Oct 2021 20:11:43 GMT</pubDate>
            <atom:updated>2021-11-16T19:27:16.343Z</atom:updated>
            <content:encoded><![CDATA[<h3>Soccer Expected Goals — Data Cleaning — Part-Four: Processing Missing Values</h3><p>Part-four for the data cleaning portion of the data science workflow creating a soccer expected goals classification model, explaining the process of identifying, assessing, and processing missing values.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*vCRPZr-7keo33olE" /></figure><h3>Background</h3><h4>Previous Steps</h4><p>The data extraction process was previously explained in a series of posts:</p><ul><li><a href="https://medium.com/@wswager/using-statsbombpy-package-to-compile-statsbomb-data-c0f6734e5234?source=your_stories_page-------------------------------------">Part-One: Using statsbombpy to Extract StatsBomb Data</a></li><li><a href="https://medium.com/@wswager/continued-extraction-of-stats-bomb-data-merging-key-pass-data-with-shot-event-data-5e1ca2faac11?source=your_stories_page-------------------------------------">Part-Two: Key Pass Features</a></li><li><a href="https://medium.com/@wswager/soccer-expected-goals-data-extraction-part-three-exploring-related-events-feature-5555a2231805?source=your_stories_page-------------------------------------">Part-Three: Exploring Related Events Feature</a></li></ul><p>Previous processes of data cleaning were explained in:</p><ul><li><a href="https://medium.com/@wswager/soccer-expected-goals-data-cleaning-part-one-defining-the-target-feature-9b26f9971ae1">Soccer Expected Goals — Data Cleaning — Part One: Defining the Target Feature</a></li><li><a href="https://medium.com/@wswager/soccer-expected-goals-data-cleaning-part-one-irrelevant-data-a4fb4692197b">Soccer Expected Goals — Data Cleaning — Part Two: Irrelevant Data</a></li><li><a href="https://medium.com/@wswager/soccer-expected-goals-data-cleaning-part-one-irrelevant-data-a4fb4692197b">Soccer Expected Goals — Data Cleaning — Part Two: Irrelevant Data</a></li><li><a href="https://medium.com/@wswager/soccer-expected-goals-data-cleaning-part-three-correcting-datatypes-3f80c0987370">Soccer Expected Goals — Data Cleaning — Part Three: Correcting Datatypes</a></li></ul><h4>Expected Goals (xG)</h4><p><strong>xG measures the quality of a shot.</strong></p><p>xG, as a metric, indicates the likelihood that a shot will result in a goal based on the characteristics of the shot and the play preceding the shot.</p><p>xG is measured on a scale between zero and one, with one representing a goal. For example, a shot with a 0.5 xG indicates a shot having a 50% chance of being a goal.</p><p><a href="https://medium.com/@wswager/what-is-xg-f6f82053d2cc">What is xG?</a></p><h4>Classification Model</h4><p>The metric of expected goals is calculated through the use of a classification model.</p><p>A classification model refers to a predictive modeling problem where a class label is predicted for a given example of input data.</p><p>For this project a supervised approach was used with the training data for the model including which shots were goals.</p><h4>The Data</h4><p>The data for this project was extracted from <a href="https://statsbomb.com/">StatsBomb</a>’s Open Data.</p><p>StatsBomb are a United Kingdom based football (soccer) data analytics company. StatsBomb provide free access to a segment of their proprietary dataset via GitHub: <a href="https://github.com/statsbomb/open-data">StatsBomb Open Data</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*-iOC409lqvW7syhD.jpeg" /></figure><h4>Data Cleaning</h4><blockquote>Data cleaning is the process of detecting and correcting corrupt or inaccurate records from a record set, table, or database and refers to identifying incomplete, incorrect, inaccurate, or irrelevant parts of the data and then replacing, modifying, or deleting the dirty or coarse data.</blockquote><p><a href="https://en.wikipedia.org/wiki/Data_cleansing"><em>Wikipedia</em></a></p><h3>Data Cleaning: Identifying Missing Values</h3><p>Shot events data was previously extracted as part of the data extraction process (see Background — Previous Steps above).</p><p><em>Note: extracted_data contains 6,080 events with 81 features</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*s1Gncb79n32KYg7y.png" /></figure><p><em>Search current value counts</em></p><pre>extracted_data.info()</pre><pre>&lt;class &#39;pandas.core.frame.DataFrame&#39;&gt;<br>Int64Index: 6080 entries, 0 to 6079<br>Data columns (total 31 columns):<br> #   Column                Non-Null Count  Dtype         <br>---  ------                --------------  -----         <br> 0   period_x              6080 non-null   int64         <br> 1   timestamp_x           6080 non-null   datetime64[ns]<br> 2   play_pattern_x        6080 non-null   object        <br> 3   location_x            6080 non-null   object        <br> 4   under_pressure_x      6080 non-null   bool          <br> 5   shot_statsbomb_xg     6080 non-null   float64       <br> 6   shot_end_location     6080 non-null   object        <br> 7   shot_technique        6080 non-null   object        <br> 8   goal                  6080 non-null   bool          <br> 9   shot_type             6080 non-null   object        <br> 10  shot_body_part        6080 non-null   object        <br> 11  shot_one_on_one       6080 non-null   bool          <br> 12  shot_open_goal        6080 non-null   bool          <br> 13  shot_first_time       6080 non-null   bool          <br> 14  shot_redirect         6080 non-null   bool          <br> 15  shot_deflected        6080 non-null   bool          <br> 16  shot_follows_dribble  6080 non-null   bool          <br> 17  pass_length           4138 non-null   float64       <br> 18  pass_angle            4138 non-null   float64       <br> 19  pass_height           4138 non-null   object        <br> 20  pass_type             960 non-null    object        <br> 21  pass_switch           6080 non-null   bool          <br> 22  pass_through_ball     6080 non-null   bool          <br> 23  pass_technique        355 non-null    object        <br> 24  pass_backheel         6080 non-null   bool          <br> 25  pass_cross            6080 non-null   bool          <br> 26  counterpress          6080 non-null   bool          <br> 27  pass_cut_back         6080 non-null   bool          <br> 28  pass_inswinging       6080 non-null   bool          <br> 29  pass_straight         6080 non-null   bool          <br> 30  pass_outswinging      6080 non-null   bool          <br>dtypes: bool(17), datetime64[ns](1), float64(3), int64(1), object(9)<br>memory usage: 813.4+ KB</pre><p>Upon initial review, all of the features with missing values are pass-related (features extracted and concatenated from pass events which were identified as key passes for shot events, passes immediately preceding the shot (assist if the shot resulted in a goal)(previously explained in <a href="https://medium.com/@wswager/continued-extraction-of-stats-bomb-data-merging-key-pass-data-with-shot-event-data-5e1ca2faac11?source=your_stories_page-------------------------------------">Soccer Expected Goals — Data Extraction — Part-Two: Key Pass Features</a>).</p><h4>No Pass Shots</h4><p><em>Note: A ‘no pass shot’ will refer to a shot event which has been deemed to have not been preceded by a pass (unassisted if the shot were to result in a goal).</em></p><pre>print(&#39;pass_length NA:&#39;, sum(extracted_data[&#39;pass_length&#39;].isna()), &#39;\n&#39;, &#39;pass_angle NA:&#39;, sum(extracted_data[&#39;pass_angle&#39;].isna()), &#39;\n&#39;, &#39;pass_height NA:&#39;, sum(extracted_data[&#39;pass_height&#39;].isna()))</pre><pre>pass_length NA: 1942 <br> pass_angle NA: 1942 <br> pass_height NA: 1942</pre><p>The first observation is that pass_length, pass_angle, and pass_height each have the same number of missing values, 1,942.</p><pre>print(&#39;pass_length NA =/= pass_angle NA:&#39;, (1942 - (sum(extracted_data.loc[extracted_data[&#39;pass_length&#39;].isna()][&#39;pass_angle&#39;].isna()))), &#39;\n&#39;, &#39;pass_length NA =/= pass_height NA:&#39;, (1942 - (sum(extracted_data.loc[extracted_data[&#39;pass_length&#39;].isna()][&#39;pass_height&#39;].isna()))), &#39;\n&#39;, &#39;pass_angle NA =/= pass_height NA:&#39;, (1942 - (sum(extracted_data.loc[extracted_data[&#39;pass_angle&#39;].isna()][&#39;pass_height&#39;].isna()))))</pre><pre>pass_length NA =/= pass_angle NA: 0 <br> pass_length NA =/= pass_height NA: 0 <br> pass_angle NA =/= pass_height NA: 0</pre><p>Upon further investigation, the 1,942 missing values for pass_length, pass_angle, and pass_height are the same events.</p><pre>print(&#39;Percent shot events w/ no pass:&#39;, (round((((sum(extracted_data[&#39;pass_length&#39;].isna())) / (len(extracted_data))) * 100), 2)), &#39;%&#39;)</pre><pre>Percent shot events w/ no pass: 31.94 %</pre><p>The 1,942 missing values for pass_length, pass_angle, and pass_height represent 31.94% of the events, which is too significant to drop.</p><p>It can be assumed these shot events are no pass shots.</p><p><em>Note: This observation is subjective, based on domain knowledge specific to the sport of soccer.</em></p><p>A shot not being preceded by a pass is significantly descriptive of the play preceding the shot in a matter which adds valuable context regarding the potential quality of the shot. This further supports that the values should not be dropped</p><p>The features, pass_length, pass_angle, and pass_height, are significantly descriptive of the play preceding the shot in a matter which adds valuable context regarding the potential quality of the shot. Therefore, these features should not be dropped.</p><p>Because the values and features should not be dropped, the values will need to be replaced.</p><p><strong>Numerical No Pass Shots</strong></p><p>pass_length and pass_angle are numerical, and will therefore require numerical filling.</p><p>A 0 in these features indicates that the ball was not passed any length, at any angle to reach the point of the shot, implying no preceding pass.</p><p><em>Replace pass_length and pass_angle missing values with 0</em></p><pre>extracted_data[&#39;pass_length&#39;].fillna(0, inplace = True)<br>extracted_data[&#39;pass_angle&#39;].fillna(0, inplace = True)</pre><p><strong>Categorical No Pass Shots</strong></p><p><em>Compare shot events with no pass v pass-related categorical feature values</em></p><pre>extracted_data.loc[extracted_data[&#39;pass_length&#39;] == 0][&#39;pass_type&#39;].value_counts(dropna = False)</pre><pre>NaN    1943<br>Name: pass_type, dtype: int64</pre><pre>extracted_data.loc[extracted_data[&#39;pass_length&#39;] == 0][&#39;pass_technique&#39;].value_counts(dropna = False)</pre><pre>NaN    1943<br>Name: pass_type, dtype: int64</pre><p><em>Note: It appears one shot event already had values of 0 for pass_length and pass_angle, indicating this shot event was not preceded by a pass but was recorded differently, with 0 values instead of NaN values. Therefore the count for no pass events has increased to 1,943.</em></p><p>All of the shot events previously identified as no pass shots are also missing values for pass_height, pass_type and pass_technique. Because these features are categorical, the missing values of the no pass events will be filled with the value ‘No Pass.’</p><p><em>Note: This will not replace all missing values for these features, just previously identified no pass shots.</em></p><pre>extracted_data.loc[extracted_data[&#39;pass_length&#39;] == 0, [&#39;pass_height&#39;, &#39;pass_type&#39;, &#39;pass_technique&#39;]] = &#39;No Pass&#39;</pre><p><strong>Boolean No Pass Shots</strong></p><p>The following are pass-related features with boolean values:</p><ul><li>pass_switch</li><li>pass_through_ball</li><li>pass_backheel</li><li>pass_cross</li><li>pass_cut_back</li><li>pass_inswinging</li><li>pass_straight</li><li>pass_outswinging’</li></ul><p><em>Assess values of boolean features no pass shots</em></p><pre>boolean_pass_features = [&#39;pass_switch&#39;, &#39;pass_through_ball&#39;, &#39;pass_backheel&#39;, &#39;pass_cross&#39;, &#39;pass_cut_back&#39;, &#39;pass_inswinging&#39;, &#39;pass_straight&#39;, &#39;pass_outswinging&#39;]</pre><pre>extracted_data.loc[extracted_data[&#39;pass_length&#39;] == 0][boolean_pass_features].value_counts()</pre><pre>pass_switch  pass_through_ball  pass_backheel  pass_cross<br>False        False              False          False<br>pass_cut_back  pass_inswinging  pass_straight  pass_outswinging<br>False          False            False          False               </pre><pre>1943<br>dtype: int64</pre><p>All of the shot events previously identified as no pass shots currently have the value ‘False’ for the pass-related boolean features. This is technically accurate as an event with no pass would not have the characteristics specified by these features. For this reason, no filling is required for these features.</p><h4>Additional Missing Values</h4><pre>extracted_data.info()</pre><pre>&lt;class &#39;pandas.core.frame.DataFrame&#39;&gt;<br>Int64Index: 6080 entries, 0 to 6079<br>Data columns (total 31 columns):<br> #   Column                Non-Null Count  Dtype         <br>---  ------                --------------  -----         <br> 0   period_x              6080 non-null   int64         <br> 1   timestamp_x           6080 non-null   datetime64[ns]<br> 2   play_pattern_x        6080 non-null   object        <br> 3   location_x            6080 non-null   object        <br> 4   under_pressure_x      6080 non-null   bool          <br> 5   shot_statsbomb_xg     6080 non-null   float64       <br> 6   shot_end_location     6080 non-null   object        <br> 7   shot_technique        6080 non-null   object        <br> 8   goal                  6080 non-null   bool          <br> 9   shot_type             6080 non-null   object        <br> 10  shot_body_part        6080 non-null   object        <br> 11  shot_one_on_one       6080 non-null   bool          <br> 12  shot_open_goal        6080 non-null   bool          <br> 13  shot_first_time       6080 non-null   bool          <br> 14  shot_redirect         6080 non-null   bool          <br> 15  shot_deflected        6080 non-null   bool          <br> 16  shot_follows_dribble  6080 non-null   bool          <br> 17  pass_length           6080 non-null   float64       <br> 18  pass_angle            6080 non-null   float64       <br> 19  pass_height           6080 non-null   object        <br> 20  pass_type             2903 non-null   object        <br> 21  pass_switch           6080 non-null   bool          <br> 22  pass_through_ball     6080 non-null   bool          <br> 23  pass_technique        2298 non-null   object        <br> 24  pass_backheel         6080 non-null   bool          <br> 25  pass_cross            6080 non-null   bool          <br> 26  counterpress          6080 non-null   bool          <br> 27  pass_cut_back         6080 non-null   bool          <br> 28  pass_inswinging       6080 non-null   bool          <br> 29  pass_straight         6080 non-null   bool          <br> 30  pass_outswinging      6080 non-null   bool          <br>dtypes: bool(17), datetime64[ns](1), float64(3), int64(1), object(9)<br>memory usage: 813.4+ KB</pre><p>pass_type and pass_technique have remaining missing values after filling no pass shots.</p><p><strong>pass_type</strong></p><pre>extracted_data[&#39;pass_type&#39;].value_counts(dropna = False)</pre><pre>NaN             3177<br>No Pass         1943<br>Corner           400<br>Recovery         305<br>Free Kick        201<br>Throw-in          42<br>Interception      10<br>Kick Off           1<br>Goal Kick          1<br>Name: pass_type, dtype: int64</pre><p>The values defined for pass_type are either:</p><p>Set-pieces<strong>: </strong>Plays restarting play from a stoppage</p><ul><li>‘Corner’</li><li>‘Free Kick’</li><li>‘Throw-In’</li><li>‘Kick Off’</li><li>‘Goal Kick’</li></ul><p>-or-</p><p>Defensive Recoveries<strong>: </strong>Plays directly resulting from and immediately preceeded by the in-possession team having won possession from the opposition-team</p><ul><li>‘Recovery’</li><li>‘Interception’</li></ul><p>It can be assumed the missing values were shots in which the preceding pass was neither a set-piece nor a defensive recovery, and can simply be considered as from open-play. Therefore the additional missing values will be filled with the categorical value ‘Open Play.’</p><p><em>Note: This observation is subjective, based on domain knowledge specific to the sport of soccer.</em></p><p><em>Replace remaining pass_type missing values with ‘Open Play’</em></p><pre>extracted_data[&#39;pass_type&#39;].fillna(&#39;Open Play&#39;, inplace = True)</pre><p><strong>pass_technique</strong></p><pre>extracted_data[&#39;pass_technique&#39;].value_counts(dropna = False)<br>NaN             3782<br>No Pass         1943<br>Through Ball     198<br>Inswinging        76<br>Outswinging       55<br>Straight          26<br>Name: pass_technique, dtype: int64</pre><p>It can be assumed the missing values were shots in which the preceding pass was not a specialized type of pass, and can simply be considered a standard pass. Therefore the additional missing values will be filled with the categorical value ‘Standard Pass.’</p><p><em>Note: This observation is subjective, based on domain knowledge specific to the sport of soccer.</em></p><p><em>Replace remaining pass_technique missing values with ‘Standard Pass’</em></p><pre>extracted_data[&#39;pass_technique&#39;].fillna(&#39;Standard&#39;, inplace = True)</pre><h4>Results</h4><pre>extracted_data.info()</pre><pre>&lt;class &#39;pandas.core.frame.DataFrame&#39;&gt;<br>Int64Index: 6080 entries, 0 to 6079<br>Data columns (total 31 columns):<br> #   Column                Non-Null Count  Dtype         <br>---  ------                --------------  -----         <br> 0   period_x              6080 non-null   int64         <br> 1   timestamp_x           6080 non-null   datetime64[ns]<br> 2   play_pattern_x        6080 non-null   object        <br> 3   location_x            6080 non-null   object        <br> 4   under_pressure_x      6080 non-null   bool          <br> 5   shot_statsbomb_xg     6080 non-null   float64       <br> 6   shot_end_location     6080 non-null   object        <br> 7   shot_technique        6080 non-null   object        <br> 8   goal                  6080 non-null   bool          <br> 9   shot_type             6080 non-null   object        <br> 10  shot_body_part        6080 non-null   object        <br> 11  shot_one_on_one       6080 non-null   bool          <br> 12  shot_open_goal        6080 non-null   bool          <br> 13  shot_first_time       6080 non-null   bool          <br> 14  shot_redirect         6080 non-null   bool          <br> 15  shot_deflected        6080 non-null   bool          <br> 16  shot_follows_dribble  6080 non-null   bool          <br> 17  pass_length           6080 non-null   float64       <br> 18  pass_angle            6080 non-null   float64       <br> 19  pass_height           6080 non-null   object        <br> 20  pass_type             6080 non-null   object        <br> 21  pass_switch           6080 non-null   bool          <br> 22  pass_through_ball     6080 non-null   bool          <br> 23  pass_technique        6080 non-null   object        <br> 24  pass_backheel         6080 non-null   bool          <br> 25  pass_cross            6080 non-null   bool          <br> 26  counterpress          6080 non-null   bool          <br> 27  pass_cut_back         6080 non-null   bool          <br> 28  pass_inswinging       6080 non-null   bool          <br> 29  pass_straight         6080 non-null   bool          <br> 30  pass_outswinging      6080 non-null   bool          <br>dtypes: bool(17), datetime64[ns](1), float64(3), int64(1), object(9)<br>memory usage: 973.4+ KB</pre><h3>Conclusion</h3><p>As part-four for the data cleaning portion of the data science workflow creating a soccer expected goals classification model, missing values were identified, assessed, and processed.</p><p>First, missing values were identified throughout the dataset. All of the features with missing values were pass-related.</p><p>In assessing the missing values, it was noted that pass_length, pass_angle, and pass_height each had 1,942 missing values, and, after further investigation, it was assumed that these shot events were not preceded by a pass, ‘no pass shots’:</p><ul><li>Numerical pass-related features for no pass shots were replaced with the value 0.</li><li>Categorical pass-related features for no pass shots were replaced with the value ‘No Pass.’</li><li>Boolean pass-related features for no pass shots did not require filling.</li></ul><p>Following the filling of no pass shots, pass_type and pass_technique had additional missing values:</p><ul><li>pass_type missing values were assumed to be passes from open-play and were therefore filled with ‘Open Play.’</li><li>pass_technique missing values were assumed to be standard passes and were therefore filled with ‘Standard.’</li></ul><p>The final data contains no missing values, which will allow for better interpretation during the eventual modeling process.</p><h3>Continued</h3><p>This series explaining the processes within the data science workflow creating a soccer expected goals classification model, will be continued in part-five of the data cleaning process, explaining the process of splitting the location feature coordinates:</p><p><a href="https://medium.com/@wswager/womens-soccer-expected-goals-data-cleaning-part-five-location-coordinates-e5d54f939717">Women’s Soccer Expected Goals — Data Cleaning— Part-Five: Location Coordinates</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c9af8b366605" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Soccer Expected Goals — Data Cleaning — Part Three: Correcting Datatypes]]></title>
            <link>https://medium.com/@WesSwager6/soccer-expected-goals-data-cleaning-part-three-correcting-datatypes-3f80c0987370?source=rss-18f13aeb73d6------2</link>
            <guid isPermaLink="false">https://medium.com/p/3f80c0987370</guid>
            <category><![CDATA[expected-goals]]></category>
            <category><![CDATA[soccer-analytics]]></category>
            <category><![CDATA[data-cleaning]]></category>
            <category><![CDATA[statsbomb]]></category>
            <dc:creator><![CDATA[Wes Swager]]></dc:creator>
            <pubDate>Fri, 22 Oct 2021 17:13:33 GMT</pubDate>
            <atom:updated>2021-10-27T20:12:06.758Z</atom:updated>
            <content:encoded><![CDATA[<h3>Soccer Expected Goals — Data Cleaning — Part Three: Correcting Datatypes</h3><p>Part-three for the data cleaning portion of the data science workflow creating a soccer expected goals classification model, explaining the process of correcting the datatypes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*vHIPP81BmJd2PZYl" /></figure><h3>Background</h3><h4>Previous Steps</h4><p>The data extraction process was previously explained in a series of posts:</p><ul><li><a href="https://medium.com/@wswager/using-statsbombpy-package-to-compile-statsbomb-data-c0f6734e5234?source=your_stories_page-------------------------------------">Part-One: Using statsbombpy to Extract StatsBomb Data</a></li><li><a href="https://medium.com/@wswager/continued-extraction-of-stats-bomb-data-merging-key-pass-data-with-shot-event-data-5e1ca2faac11?source=your_stories_page-------------------------------------">Part-Two: Key Pass Features</a></li><li><a href="https://medium.com/@wswager/soccer-expected-goals-data-extraction-part-three-exploring-related-events-feature-5555a2231805?source=your_stories_page-------------------------------------">Part-Three: Exploring Related Events Feature</a></li></ul><p>Previous processes of data cleaning were explained in:</p><ul><li><a href="https://medium.com/@wswager/soccer-expected-goals-data-cleaning-part-one-defining-the-target-feature-9b26f9971ae1">Soccer Expected Goals — Data Cleaning — Part One: Defining the Target Feature</a></li><li><a href="https://medium.com/@wswager/soccer-expected-goals-data-cleaning-part-one-irrelevant-data-a4fb4692197b">Soccer Expected Goals — Data Cleaning — Part Two: Irrelevant Data</a></li></ul><h4>Expected Goals (xG)</h4><p><strong>xG measures the quality of a shot.</strong></p><p>xG, as a metric, indicates the likelihood that a shot will result in a goal based on the characteristics of the shot and the play preceding the shot.</p><p>xG is measured on a scale between zero and one, with one representing a goal. For example, a shot with a 0.5 xG indicates a shot having a 50% chance of being a goal.</p><p><a href="https://medium.com/@wswager/what-is-xg-f6f82053d2cc">What is xG?</a></p><h4>Classification Model</h4><p>The metric of expected goals is calculated through the use of a classification model.</p><p>A classification model refers to a predictive modeling problem where a class label is predicted for a given example of input data.</p><p>For this project a supervised approached was used with the training data for the model including which shots were goals.</p><h4>The Data</h4><p>The data for this project was extracted from <a href="https://statsbomb.com/">StatsBomb</a>’s Open Data.</p><p>StatsBomb are a United Kingdom based football (soccer) data analytics company. StatsBomb provide free access to a segment of their proprietary dataset via GitHub: <a href="https://github.com/statsbomb/open-data">StatsBomb Open Data</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*C9-sgP4ebppkhnMx.jpeg" /></figure><h4>Data Cleaning</h4><blockquote><em>Data cleaning is the process of detecting and correcting corrupt or inaccurate records from a record set, table, or database and refers to identifying incomplete, incorrect, inaccurate, or irrelevant parts of the data and then replacing, modifying, or deleting the dirty or coarse data.</em></blockquote><p><a href="https://en.wikipedia.org/wiki/Data_cleansing"><em>Wikipedia</em></a></p><h3>Data Cleaning: Correcting Datatypes</h3><p>Shot events data was previously extracted as part of the data extraction process (see Background — Previous Steps above).</p><p><em>Note: extracted_data contains 6,080 events with 81 features</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*N1kA9GFRAYIoFAjf.png" /></figure><p><em>Search current datatypes</em></p><pre>extracted_data.info()</pre><pre>&lt;class &#39;pandas.core.frame.DataFrame&#39;&gt;<br>Int64Index: 6080 entries, 0 to 6079<br>Data columns (total 31 columns):<br> #   Column                Non-Null Count  Dtype  <br>---  ------                --------------  -----  <br> 0   period_x              6080 non-null   int64  <br> 1   timestamp_x           6080 non-null   object <br> 2   play_pattern_x        6080 non-null   object <br> 3   location_x            6080 non-null   object <br> 4   under_pressure_x      1045 non-null   object <br> 5   shot_statsbomb_xg     6080 non-null   float64<br> 6   shot_end_location     6080 non-null   object <br> 7   shot_technique        6080 non-null   object <br> 8   goal                  6080 non-null   bool   <br> 9   shot_type             6080 non-null   object <br> 10  shot_body_part        6080 non-null   object <br> 11  shot_one_on_one       330 non-null    object <br> 12  shot_open_goal        70 non-null     object <br> 13  shot_first_time       1290 non-null   object <br> 14  shot_redirect         22 non-null     object <br> 15  shot_deflected        64 non-null     object <br> 16  shot_follows_dribble  3 non-null      object <br> 17  pass_length           4138 non-null   float64<br> 18  pass_angle            4138 non-null   float64<br> 19  pass_height           4138 non-null   object <br> 20  pass_type             960 non-null    object <br> 21  pass_switch           323 non-null    object <br> 22  pass_through_ball     198 non-null    object <br> 23  pass_technique        355 non-null    object <br> 24  pass_backheel         14 non-null     object <br> 25  pass_cross            754 non-null    object <br> 26  counterpress          5 non-null      object <br> 27  pass_cut_back         108 non-null    object <br> 28  pass_inswinging       76 non-null     object <br> 29  pass_straight         26 non-null     object <br> 30  pass_outswinging      55 non-null     object <br>dtypes: bool(1), float64(3), int64(1), object(26)<br>memory usage: 1.4+ MB</pre><h4>Boolean Features</h4><p>Upon initial review it is apparent that a number of the features extracted are boolean in nature, but are currently object datatypes.</p><p><em>Define boolean features</em></p><pre>boolean_features = [&#39;shot_one_on_one&#39;,<br>&#39;shot_open_goal&#39;,<br>&#39;shot_first_time&#39;,<br>&#39;shot_redirect&#39;,<br>&#39;shot_deflected&#39;,<br>&#39;shot_follows_dribble&#39;,<br>&#39;under_pressure_x&#39;,<br>&#39;counterpress&#39;,<br>&#39;pass_switch&#39;,<br>&#39;pass_through_ball&#39;,<br>&#39;pass_backheel&#39;,<br>&#39;pass_cross&#39;,<br>&#39;pass_cut_back&#39;,<br>&#39;pass_inswinging&#39;,<br>&#39;pass_straight&#39;,<br>&#39;pass_outswinging&#39;]</pre><p><em>Convert boolean features to boolean datatype</em></p><pre>extracted_data[boolean_features] = extracted_data[boolean_features].astype(bool)</pre><h4>Datetime</h4><p>The second observation is that timestamp_x is currently object datatype, but should be datetime.</p><p><em>Convert timestamp_x datatype to datetime</em></p><pre>extracted_data[&#39;timestamp_x&#39;] = extracted_data[&#39;timestamp_x&#39;].astype(str)</pre><pre>extracted_data[&#39;timestamp_x&#39;] = pd.to_datetime(extracted_data[&#39;timestamp_x&#39;])</pre><h4>Results</h4><pre>extracted_data.info()</pre><pre>&lt;class &#39;pandas.core.frame.DataFrame&#39;&gt;<br>Int64Index: 6080 entries, 0 to 6079<br>Data columns (total 31 columns):<br> #   Column                Non-Null Count  Dtype         <br>---  ------                --------------  -----         <br> 0   period_x              6080 non-null   int64         <br> 1   timestamp_x           6080 non-null   datetime64[ns]<br> 2   play_pattern_x        6080 non-null   object        <br> 3   location_x            6080 non-null   object        <br> 4   under_pressure_x      6080 non-null   bool          <br> 5   shot_statsbomb_xg     6080 non-null   float64       <br> 6   shot_end_location     6080 non-null   object        <br> 7   shot_technique        6080 non-null   object        <br> 8   goal                  6080 non-null   bool          <br> 9   shot_type             6080 non-null   object        <br> 10  shot_body_part        6080 non-null   object        <br> 11  shot_one_on_one       6080 non-null   bool          <br> 12  shot_open_goal        6080 non-null   bool          <br> 13  shot_first_time       6080 non-null   bool          <br> 14  shot_redirect         6080 non-null   bool          <br> 15  shot_deflected        6080 non-null   bool          <br> 16  shot_follows_dribble  6080 non-null   bool          <br> 17  pass_length           4138 non-null   float64       <br> 18  pass_angle            4138 non-null   float64       <br> 19  pass_height           4138 non-null   object        <br> 20  pass_type             960 non-null    object        <br> 21  pass_switch           6080 non-null   bool          <br> 22  pass_through_ball     6080 non-null   bool          <br> 23  pass_technique        355 non-null    object        <br> 24  pass_backheel         6080 non-null   bool          <br> 25  pass_cross            6080 non-null   bool          <br> 26  counterpress          6080 non-null   bool          <br> 27  pass_cut_back         6080 non-null   bool          <br> 28  pass_inswinging       6080 non-null   bool          <br> 29  pass_straight         6080 non-null   bool          <br> 30  pass_outswinging      6080 non-null   bool          <br>dtypes: bool(17), datetime64[ns](1), float64(3), int64(1), object(9)<br>memory usage: 813.4+ KB</pre><h3>Conclusion</h3><p>As part-three for the data cleaning portion of the data science workflow creating a soccer expected goals classification model, datatypes were corrected.</p><p>First, features which are boolean in nature were changed from object to boolean datatype.</p><p>Second, timestamp_x was changed from object to datetime datatype.</p><p>The corrected data will allow for better interpretation during the eventual modeling process.</p><h3>Continued</h3><p>This series explaining the processes within the data science workflow creating a soccer expected goals classification model, will be continued in part-four of the data cleaning process, explaining the process of identifying and replacing missing values:</p><p><a href="https://medium.com/@wswager/soccer-expected-goals-data-cleaning-part-four-processing-missing-values-c9af8b366605">Soccer Expected Goals — Data Cleaning — Part Four: Processing Missing Values</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3f80c0987370" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Soccer Expected Goals — Data Cleaning — Part One: Defining the Target Feature]]></title>
            <link>https://medium.com/@WesSwager6/soccer-expected-goals-data-cleaning-part-one-defining-the-target-feature-9b26f9971ae1?source=rss-18f13aeb73d6------2</link>
            <guid isPermaLink="false">https://medium.com/p/9b26f9971ae1</guid>
            <category><![CDATA[statsbomb]]></category>
            <category><![CDATA[soccer-analytics]]></category>
            <category><![CDATA[data-cleaning]]></category>
            <category><![CDATA[expected-goals]]></category>
            <dc:creator><![CDATA[Wes Swager]]></dc:creator>
            <pubDate>Fri, 22 Oct 2021 16:18:32 GMT</pubDate>
            <atom:updated>2021-10-22T16:32:30.220Z</atom:updated>
            <content:encoded><![CDATA[<h3>Soccer Expected Goals — Data Cleaning — Part One: Defining the Target Feature</h3><p>Part-one for the data cleaning portion of the data science workflow creating a soccer expected goals classification model, explaining the process of defining the target feature for the eventual model, goal.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*57DRofojI40jJZfX" /></figure><h3>Background</h3><h4>Previous Steps</h4><p>The data extraction process was previously explained in a series of posts:</p><ul><li><a href="https://medium.com/@wswager/using-statsbombpy-package-to-compile-statsbomb-data-c0f6734e5234?source=your_stories_page-------------------------------------">Part-One: Using statsbombpy to Extract StatsBomb Data</a></li><li><a href="https://medium.com/@wswager/continued-extraction-of-stats-bomb-data-merging-key-pass-data-with-shot-event-data-5e1ca2faac11?source=your_stories_page-------------------------------------">Part-Two: Key Pass Features</a></li><li><a href="https://medium.com/@wswager/soccer-expected-goals-data-extraction-part-three-exploring-related-events-feature-5555a2231805?source=your_stories_page-------------------------------------">Part-Three: Exploring Related Events Feature</a></li></ul><h4>Expected Goals (xG)</h4><p><strong>xG measures the quality of a shot.</strong></p><p>xG, as a metric, indicates the likelihood that a shot will result in a goal based on the characteristics of the shot and the play preceding the shot.</p><p>xG is measured on a scale between zero and one, with one representing a goal. For example, a shot with a 0.5 xG indicates a shot having a 50% chance of being a goal.</p><p><a href="https://medium.com/@wswager/what-is-xg-f6f82053d2cc">What is xG?</a></p><h4>Classification Model</h4><p>The metric of expected goals is calculated through the use of a classification model.</p><p>A classification model refers to a predictive modeling problem where a class label is predicted for a given example of input data.</p><p>For this project a supervised approached was used with the training data for the model including which shots were goals.</p><h4>The Data</h4><p>The data for this project was extracted from <a href="https://statsbomb.com/">StatsBomb</a>’s Open Data.</p><p>StatsBomb are a United Kingdom based football (soccer) data analytics company. StatsBomb provide free access to a segment of their proprietary dataset via GitHub: <a href="https://github.com/statsbomb/open-data">StatsBomb Open Data</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*EPtNbe2bTuTt4aJx.jpeg" /></figure><h4>Data Cleaning</h4><blockquote><em>Data cleaning is the process of detecting and correcting corrupt or inaccurate records from a record set, table, or database and refers to identifying incomplete, incorrect, inaccurate, or irrelevant parts of the data and then replacing, modifying, or deleting the dirty or coarse data.</em></blockquote><p><a href="https://en.wikipedia.org/wiki/Data_cleansing"><em>Wikipedia</em></a></p><h3>Data Cleaning: Target Feature</h3><p>Because xG indicates the likelihood that a shot will result in a goal, the target feature for the model is goal, as a boolean feature, with ‘True’ if a shot event resulted in a goal, ‘False’ if a shot event did not result in a goal.</p><p>Because the eventual modelling process will be a supervised classification model, the shot events will be labeled with the target feature, goal, to train the models, using the features of the shot events, so that they can then predict goals.</p><p>Shot events data was previously extracted as part of the data extraction process (see Background — Previous Steps above).</p><p><em>Note: extracted_data contains 6,080 events with 81 features</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g7spWMT8APmNI9jr5nSgIw.png" /></figure><p><em>extracted_data currently contains the feature shot_outcome which includes categorical values</em></p><pre>extracted_data[&#39;shot_outcome&#39;].value_counts(dropna = False)</pre><pre>Off T               1912<br>Saved               1531<br>Blocked             1460<br>Goal                 664<br>Wayward              336<br>Post                 136<br>Saved Off Target      24<br>Saved to Post         17<br>Name: shot_outcome, dtype: int64</pre><p><em>Update shot_outcome values to boolean for goal or not a goal</em></p><pre>extracted_data[&#39;shot_outcome&#39;] = extracted_data[&#39;shot_outcome&#39;].apply(lambda x: 1 if x == &#39;Goal&#39; else 0)</pre><pre>extracted_data[&#39;goal&#39;] = extracted_data[&#39;shot_outcome&#39;].astype(bool)</pre><p><em>Update shot_outcome column name to goal</em></p><pre>extracted_data.rename(columns = {&#39;shot_outcome&#39; : &#39;goal&#39;}, inplace = True)</pre><p><em>Results</em></p><pre>extracted_data[&#39;goal&#39;].value_counts()</pre><pre>False    5416<br>True      664<br>Name: goal, dtype: bool</pre><pre>print(&quot;goal Percent &#39;True&#39;:&quot;, round((((sum(extracted_data[&#39;goal&#39;])) / (len(extracted_data))) * 100), 2), &#39;%&#39;)</pre><pre>goal Percent &#39;True&#39;: 10.92 %</pre><h3>Conclusion</h3><p>In the process of defining the target feature for the eventual model, as part of the data cleaning portion of the data science workflow creating a soccer expected goals classification model, shot_outcome was converted from categorical features to the boolean feature goal, with ‘True’ if a shot event resulted in a goal, ‘False’ if a shot event did not result in a goal.</p><h3>Continued</h3><p>This series explaining the processes within the data science workflow creating a soccer expected goals classification model, is continued in part-two of the data cleaning process, explaining the process of dropping irrelevant data:</p><p><a href="https://medium.com/@wswager/soccer-expected-goals-data-cleaning-part-one-irrelevant-data-a4fb4692197b">Soccer Expected Goals — Data Cleaning — Part Two: Irrelevant Data</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9b26f9971ae1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Soccer Expected Goals — Data Extraction — Part-Three: Exploring Related Events]]></title>
            <link>https://medium.com/@WesSwager6/soccer-expected-goals-data-extraction-part-three-exploring-related-events-feature-5555a2231805?source=rss-18f13aeb73d6------2</link>
            <guid isPermaLink="false">https://medium.com/p/5555a2231805</guid>
            <category><![CDATA[expected-goals]]></category>
            <category><![CDATA[soccer-analytics]]></category>
            <category><![CDATA[data-extraction]]></category>
            <category><![CDATA[statsbomb]]></category>
            <dc:creator><![CDATA[Wes Swager]]></dc:creator>
            <pubDate>Mon, 18 Oct 2021 20:20:22 GMT</pubDate>
            <atom:updated>2021-11-15T16:25:29.050Z</atom:updated>
            <content:encoded><![CDATA[<h3>Women’s Soccer Expected Goals — Data Extraction — Part-Three: Exploring Related Events</h3><p>Part-two of a series for the data science workflow creating a women’s club soccer expected goals (xG) classification model, explaining the process of exploring related events and their features.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8yEyQWygnJcSQhU1.jpg" /></figure><p><a href="https://github.com/wswager/womens_soccer_expected_goals_model">GitHub - wswager/womens_soccer_expected_goals_model: Classification model for expected goals (xG) in women&#39;s club soccer, predicting the likelihood that a shot will score using data extracted from StatsBomb.</a></p><h3>Background</h3><p>Continued from parts-one and two, explaining the processes of extracting shot event data and key pass features:</p><ul><li><a href="https://medium.com/@wswager/using-statsbombpy-package-to-compile-statsbomb-data-c0f6734e5234">Soccer Expected Goals — Data Extraction — Part-One: Extracting Shots Data</a></li><li><a href="https://medium.com/@wswager/continued-extraction-of-stats-bomb-data-merging-key-pass-data-with-shot-event-data-5e1ca2faac11">Soccer Expected Goals — Data Extraction — Part-Two: Extracting Key Pass Features</a></li></ul><h3>Introduction</h3><h4>Expected Goals (xG)</h4><p>xG is used to indicate the quality of a shot.</p><p>xG, as a metric, measures the likelihood that a shot will result in a goal based on the characteristics of the shot and the play preceding the shot.</p><p>xG is measured on a scale between zero and one, with one representing a goal. For example, a shot with a 0.5 xG indicates a shot having a 50% chance of being a goal.</p><p><a href="https://medium.com/@wswager/what-is-xg-f6f82053d2cc">What is xG?</a></p><h4>Classification Model</h4><p>The metric of expected goals is calculated through the use of a classification model.</p><p>A classification model refers to a predictive modeling problem where a class label is predicted for a given example of input data.</p><p>For this project a supervised approach was used with the training data for the model including which shots were goals.</p><h4>Data Extraction</h4><blockquote><em>Data extraction is the act or process of retrieving data out of…data sources for further data processing</em></blockquote><p><a href="https://en.wikipedia.org/wiki/Data_extraction"><em>Wikipedia</em></a></p><h4>The Data</h4><p>The data for this project was extracted from <a href="https://statsbomb.com/">StatsBomb</a>’s Open Data.</p><p>StatsBomb are a United Kingdom based football (soccer) data analytics company. StatsBomb provide free access to a segment of their proprietary dataset via GitHub: <a href="https://github.com/statsbomb/open-data">StatsBomb Open Data</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*6cs5aQCYc4GWCK-Z.jpeg" /></figure><p>StatsBomb Open Data is organized in JSON files:</p><p><strong>Matches</strong></p><ul><li>Folders are organized by competition (league or tournament)</li><li>Files within the folders are organized by season (year) ID</li><li>Files contain nested dictionaries with descriptive data for each individual match</li></ul><p><strong>Events</strong></p><ul><li>Files organized by match ID</li><li>Files contain nested dictionaries with descriptive data for each event within each individual match</li></ul><p>For the purposes of this project the relevant data targeted was, primarily, characteristics of shots and, secondarily, characteristics of the plays creating those shots, from women’s club soccer matches.</p><p><em>Note: Assessment of plays creating shots is subjective and based on domain knowledge specific to the sport of soccer</em></p><h4>statsbombpy</h4><p>The following processes of extracting data from the StatsBomb Open Data relies heavily on the use of <a href="https://github.com/statsbomb/statsbombpy">statsbombpy</a>.</p><p>statsbombpy is a Python package created by StatsBomb which streams StatsBomb data directly within python. The package allows access to StatsBomb Open Data for free or allows access to their API with use of log-in credentials.</p><h3>Data Extraction</h3><p><em>Import statsbombpy for extracting StatsBomb data</em></p><pre>!pip install statsbombpy<br>from statsbombpy import sb</pre><h4>Review from Parts-One and Two</h4><p>Previously, in <a href="https://medium.com/@wswager/using-statsbombpy-package-to-compile-statsbomb-data-c0f6734e5234">Soccer Expected Goals — Data Extraction — Part-One: Extracting Shots Data</a>, target women’s club competitions, competition ids, and season ids were identified and isolated from StatsBomb matches data.</p><pre>matches_df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mnpEjTS8DVDipWRmNEFQTA.png" /></figure><pre>print(&#39;Number of Seasons:&#39;, len(target_season_ids))</pre><pre>Number of Women&#39;s Club Seasons: 3</pre><pre>print(&#39;Total Women&#39;s Club Matches:&#39;, len(matches_df))</pre><pre>Total Matches: 231</pre><p>Then, shot event data was extracted from StatsBomb events data.</p><pre>shots_df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M7B6nF4iJPYotzbc9rBI2A.png" /></figure><pre>print(&quot;Total Women&#39;s Club Shot Events:&quot;, len(shots_df))</pre><pre>Total Women&#39;s Club Competition Shot Events: 6114</pre><pre>print(&#39;Total Shot Features:&#39;, shots_df.shape[1])</pre><pre>Total Shot Features: 37</pre><p>Next, in <a href="https://medium.com/@wswager/continued-extraction-of-stats-bomb-data-merging-key-pass-data-with-shot-event-data-5e1ca2faac11">Soccer Expected Goals — Data Extraction — Part-Two: Extracting Key Pass Features</a>, the features for pass events identified by the shot events’ shot_key_pass_id feature were extracted and concatenated with their shot events.</p><pre>extracted_data.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*3elFDNnP0i_T71o0.png" /></figure><pre>print(&quot;Total Women&#39;s Club Shot Events:&quot;, len(extracted_data))</pre><pre>Total Women&#39;s Club Competition Shot Events: 6114</pre><pre>print(&#39;Updated Shot w/ Key Pass Features:&#39;, extracted_data.shape[1])</pre><pre>Updated Shot w/ Key Pass Features: 81</pre><h3>Related Events</h3><p>As mentioned previously, characteristics of the play creating a shot can be valuable toward the potential quality of the shot. For this reason, features of other event types related to shot events can be valuable and worth assessment.</p><p>related_events is a feature for shot events which identifies specific events other than shot or pass events (indicating the event id feature value), as events significantly connected with leading to the shot.</p><p>In exploring the types of events available in the StatsBomb data, <strong>dribble </strong>and <strong>carry </strong>events appear to be the event types most likely to provide valuable data toward describing the quality of the shot.</p><p><em>Note: Assessment of plays creating shots is subjective and based on domain knowledge specific to the sport of soccer</em></p><p><em>Create a list from related_events values for shot events</em></p><pre>related_events = list(shots_df[&#39;related_events&#39;])</pre><p><strong>Dribble Events</strong></p><p><em>Create dataframes for dribble events in each season of the target competitions</em></p><pre>dribbless_df_37_42 = sb.competition_events(country = &#39;England&#39;, division = &quot;FA Women&#39;s Super League&quot;, season = &#39;2018/2019&#39;, gender = &#39;female&#39;, split = True)[&#39;dribbles&#39;]</pre><pre>dribbles_df_37_4 = sb.competition_events(country = &#39;England&#39;, division = &quot;FA Women&#39;s Super League&quot;, season = &#39;2019/2020&#39;, gender = &#39;female&#39;, split = True)[&#39;dribbles&#39;]</pre><pre>dribbles_df_49_3 = sb.competition_events(country = &#39;United States of America&#39;, division = &#39;NWSL&#39;, season = &#39;2018&#39;, gender = &#39;female&#39;, split = True)[&#39;dribbles&#39;]</pre><p><em>Concatenate dribble event dataframes into a combined dataframe</em></p><pre>dribbles_df = pd.concat([dribbles_df_37_42, dribbles_df_37_4, dribbles_df_49_3], ignore_index = True)</pre><pre>dribbles_df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kCH3R5faQMeR-hFx3HVirg.png" /></figure><pre>print(&quot;Total Women&#39;s Club Competition Dribble Events:&quot;, len(dribbles_df))</pre><pre>Total Women&#39;s Club Competition Dribble Events: 8187</pre><pre>print(&#39;Total Dribble Features:&#39;, dribbles_df.shape[1])</pre><pre>Total Dribble Features: 23</pre><p><em>Search dribble events for related_events</em></p><pre>dribbles_to_shots_df = dribbles_df[dribbles_df[‘id’].isin(related_events)]</pre><pre>print(“Dribble Events Identified as ‘related_events’ for Shot Events:”, len(dribbles_to_shots_df))</pre><pre>Dribble Events Identified as &#39;related_events&#39; for Shot Events: 0</pre><p>No related_events values for shot events match the id feature values of dribble events, therefore no additional data can be extracted or added.</p><p><strong>Carry Events</strong></p><p><em>Create dataframes for carry events in each season of the target competitions</em></p><pre>carrys_df_37_42 = sb.competition_events(country = &#39;England&#39;, division = &quot;FA Women&#39;s Super League&quot;, season = &#39;2018/2019&#39;, gender = &#39;female&#39;, split = True)[&#39;carrys&#39;]</pre><pre>carrys_df_37_4 = sb.competition_events(country = &#39;England&#39;, division = &quot;FA Women&#39;s Super League&quot;, season = &#39;2019/2020&#39;, gender = &#39;female&#39;, split = True)[&#39;carrys&#39;]</pre><pre>carrys_df_49_3 = sb.competition_events(country = &#39;United States of America&#39;, division = &#39;NWSL&#39;, season = &#39;2018&#39;, gender = &#39;female&#39;, split = True)[&#39;carrys&#39;]</pre><p><em>Concatenate carry event dataframes into a combined dataframe</em></p><pre>carrys_df = pd.concat([carrys_df_37_42, carrys_df_37_4, carrys_df_49_3], ignore_index = True)</pre><pre>carrys_df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FtstBQBlvWj-5vk4Z9Dtqg.png" /></figure><pre>print(&quot;Total Women&#39;s Club Competition Carry Events:&quot;, len(carrys_df))</pre><pre>Total Women&#39;s Club Competition Carry Events: 168439</pre><pre>print(&#39;Total Carry Features:&#39;, carrys_df.shape[1])</pre><pre>Total Carry Features: 19</pre><p><em>Search carry events for related_events</em></p><pre>carrys_to_shots_df = carrys_df[carrys_df[‘id’].isin(related_events)]</pre><pre>print(“Carry Events Identified as ‘related_events’ for Shot Events:”, len(carrys_to_shots_df))</pre><pre>Carry Events Identified as &#39;related_events&#39; for Shot Events: 0</pre><p>No related_events values for shot events match the id feature values of carry events, therefore no additional data can be extracted or added.</p><h3>Conclusion</h3><p>In exploring related events and their features no additional features were found to be considered valuable toward the potential quality of the shot.</p><p>No related_events values for shot events match the id feature values of either dribble or carry events, therefore no additional data can be extracted or added.</p><h3>Results</h3><p><em>Note: No changes were made to extracted_data following the extraction of shot event data and key pass features as a result of assessing related events.</em></p><pre>print(&quot;Total Women&#39;s Club Shot Events:&quot;, len(extracted_data))</pre><pre>Total Women&#39;s Club Competition Shot Events: 6114</pre><pre>print(&#39;Updated Shot w/ Key Pass Features:&#39;, extracted_data.shape[1])</pre><pre>Updated Shot w/ Key Pass Features: 81</pre><p>The additional extraction of key pass features increased the number of features for the 6,144 shot events from 37 to 81.</p><pre>extracted_data.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LfNKC4YqoC2W8igB-lY_PQ.png" /></figure><h3>Continued</h3><p>Part-three completes data extraction portion for the data science workflow creating a women’s club soccer expected goals (xG) classification model.</p><p>Next the series will continue in the data cleaning portion, explaining the process of defining the target feature for modeling, goal.</p><p><a href="https://medium.com/@wswager/soccer-expected-goals-data-cleaning-part-one-defining-the-target-feature-9b26f9971ae1">Soccer Expected Goals — Data Cleaning — Part One: Defining the Target Feature</a></p><h4>More</h4><p>If you liked this post, please give it an applause and follow me as I will be continuing with a series of posts for each process through the data science workflow of my <a href="https://github.com/wswager/womens_soccer_expected_goals_model">Women’s Soccer Expected Goals Model</a>:</p><p><a href="https://medium.com/@wswager">Wes Swager - Medium</a></p><p>Also, follow me on Twitter, where I post regularly about tactical observations for soccer:</p><p><a href="https://mobile.twitter.com/Wes_Swager_6">JavaScript is not available.</a></p><p>I would love to read any feedback you might have in the comments.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5555a2231805" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Soccer Expected Goals — Data Cleaning — Part Two: Identifying and Dropping Irrelevant Data]]></title>
            <link>https://medium.com/@WesSwager6/soccer-expected-goals-data-cleaning-part-one-irrelevant-data-a4fb4692197b?source=rss-18f13aeb73d6------2</link>
            <guid isPermaLink="false">https://medium.com/p/a4fb4692197b</guid>
            <category><![CDATA[data-cleaning]]></category>
            <category><![CDATA[statsbomb]]></category>
            <category><![CDATA[expected-goals]]></category>
            <category><![CDATA[soccer]]></category>
            <dc:creator><![CDATA[Wes Swager]]></dc:creator>
            <pubDate>Mon, 11 Oct 2021 23:13:29 GMT</pubDate>
            <atom:updated>2021-12-02T15:33:53.104Z</atom:updated>
            <content:encoded><![CDATA[<h3>Soccer Expected Goals — Data Cleaning — Part Two: Identifying and Dropping Irrelevant Data</h3><p>Part-two for the data cleaning portion of the data science workflow creating a soccer expected goals classification model, explaining the process of identifying and dropping irrelevant data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*M709oRt_2r7yi0Y_" /></figure><h3>Background</h3><h4>Previous Steps</h4><p>The data extraction process was previously explained in a series of posts:</p><ul><li><a href="https://medium.com/@wswager/using-statsbombpy-package-to-compile-statsbomb-data-c0f6734e5234?source=your_stories_page-------------------------------------">Part-One: Using statsbombpy to Extract StatsBomb Data</a></li><li><a href="https://medium.com/@wswager/continued-extraction-of-stats-bomb-data-merging-key-pass-data-with-shot-event-data-5e1ca2faac11?source=your_stories_page-------------------------------------">Part-Two: Key Pass Features</a></li><li><a href="https://medium.com/@wswager/soccer-expected-goals-data-extraction-part-three-exploring-related-events-feature-5555a2231805?source=your_stories_page-------------------------------------">Part-Three: Exploring Related Events Feature</a></li></ul><p>Previously, the process of defining the target feature, part-one of data cleaning, was explained in:</p><p><a href="https://medium.com/@wswager/soccer-expected-goals-data-cleaning-part-one-defining-the-target-feature-9b26f9971ae1">Soccer Expected Goals — Data Cleaning — Part One: Defining the Target Feature</a></p><h4>Expected Goals (xG)</h4><p><strong>xG measures the quality of a shot.</strong></p><p>xG, as a metric, indicates the likelihood that a shot will result in a goal based on the characteristics of the shot and the play preceding the shot.</p><p>xG is measured on a scale between zero and one, with one representing a goal. For example, a shot with a 0.5 xG indicates a shot having a 50% chance of being a goal.</p><p><a href="https://medium.com/@wswager/what-is-xg-f6f82053d2cc">What is xG?</a></p><h4>Classification Model</h4><p>The metric of expected goals is calculated through the use of a classification model.</p><p>A classification model refers to a predictive modeling problem where a class label is predicted for a given example of input data.</p><p>For this project a supervised approached was used with the training data for the model including which shots were goals.</p><h4>The Data</h4><p>The data for this project was extracted from <a href="https://statsbomb.com/">StatsBomb</a>’s Open Data.</p><p>StatsBomb are a United Kingdom based football (soccer) data analytics company. StatsBomb provide free access to a segment of their proprietary dataset via GitHub: <a href="https://github.com/statsbomb/open-data">StatsBomb Open Data</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y8nmraLwqvc4Y6yLm3BPMw.jpeg" /></figure><h4>Data Cleaning</h4><blockquote>Data cleaning is the process of detecting and correcting corrupt or inaccurate records from a record set, table, or database and refers to identifying incomplete, incorrect, inaccurate, or irrelevant parts of the data and then replacing, modifying, or deleting the dirty or coarse data.</blockquote><p><a href="https://en.wikipedia.org/wiki/Data_cleansing"><em>Wikipedia</em></a></p><h3>Data Cleaning: Irrelevant Data</h3><p>Shot events data was previously extracted as part of the data extraction process (see Background — Previous Steps above).</p><p><em>Note: extracted_data contains 6,080 events with 81 features</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*-bv-G08qp2aD_gvY.png" /></figure><h4>Initial Features</h4><p>The data included the following 81 features following the data extraction process:</p><pre>extracted_data.shape[1]</pre><pre>81</pre><pre><br>list(extracted_data.columns.values)</pre><pre>&#39;id&#39;,<br> &#39;index_x&#39;,<br> &#39;period_x&#39;,<br> &#39;timestamp_x&#39;,<br> &#39;minute_x&#39;,<br> &#39;second_x&#39;,<br> &#39;type_x&#39;,<br> &#39;possession_x&#39;,<br> &#39;possession_team_x&#39;,<br> &#39;play_pattern_x&#39;,<br> &#39;team_x&#39;,<br> &#39;player_x&#39;,<br> &#39;position_x&#39;,<br> &#39;location_x&#39;,<br> &#39;duration_x&#39;,<br> &#39;under_pressure_x&#39;,<br> &#39;related_events_x&#39;,<br> &#39;match_id_x&#39;,<br> &#39;shot_statsbomb_xg&#39;,<br> &#39;shot_end_location&#39;,<br> &#39;shot_key_pass_id&#39;,<br> &#39;shot_technique&#39;,<br> &#39;goal&#39;,<br> &#39;shot_type&#39;,<br> &#39;shot_body_part&#39;,<br> &#39;shot_freeze_frame&#39;,<br> &#39;shot_one_on_one&#39;,<br> &#39;shot_aerial_won&#39;,<br> &#39;shot_open_goal&#39;,<br> &#39;shot_first_time&#39;,<br> &#39;out_x&#39;,<br> &#39;shot_redirect&#39;,<br> &#39;shot_deflected&#39;,<br> &#39;off_camera_x&#39;,<br> &#39;shot_saved_off_target&#39;,<br> &#39;shot_saved_to_post&#39;,<br> &#39;shot_follows_dribble&#39;,<br> &#39;index_y&#39;,<br> &#39;period_y&#39;,<br> &#39;timestamp_y&#39;,<br> &#39;minute_y&#39;,<br> &#39;second_y&#39;,<br> &#39;type_y&#39;,<br> &#39;possession_y&#39;,<br> &#39;possession_team_y&#39;,<br> &#39;play_pattern_y&#39;,<br> &#39;team_y&#39;,<br> &#39;player_y&#39;,<br> &#39;position_y&#39;,<br> &#39;location_y&#39;,<br> &#39;duration_y&#39;,<br> &#39;related_events_y&#39;,<br> &#39;match_id_y&#39;,<br> &#39;pass_recipient&#39;,<br> &#39;pass_length&#39;,<br> &#39;pass_angle&#39;,<br> &#39;pass_height&#39;,<br> &#39;pass_end_location&#39;,<br> &#39;pass_body_part&#39;,<br> &#39;pass_type&#39;,<br> &#39;under_pressure_y&#39;,<br> &#39;pass_outcome&#39;,<br> &#39;pass_aerial_won&#39;,<br> &#39;pass_assisted_shot_id&#39;,<br> &#39;pass_shot_assist&#39;,<br> &#39;off_camera_y&#39;,<br> &#39;pass_switch&#39;,<br> &#39;pass_through_ball&#39;,<br> &#39;pass_technique&#39;,<br> &#39;pass_backheel&#39;,<br> &#39;pass_cross&#39;,<br> &#39;counterpress&#39;,<br> &#39;pass_cut_back&#39;,<br> &#39;pass_deflected&#39;,<br> &#39;pass_goal_assist&#39;,<br> &#39;pass_miscommunication&#39;,<br> &#39;pass_inswinging&#39;,<br> &#39;pass_straight&#39;,<br> &#39;pass_outswinging&#39;,<br> &#39;pass_no_touch&#39;,<br> &#39;out_y&#39;</pre><h4>Duplicate Features</h4><p>Upon initial review it is apparent that a number of the features extracted were duplicated as a result of extracting data for both shot events and the pass events preceding the shot events.</p><p><em>Note: Conveniently, while concatenating the two datasets, pandas helpfully indicated columns with duplicate names with x and y suffixes.</em></p><p>The first-step in cleaning the irrelevant features is dropping the duplicate features:</p><pre>duplicate_features = [&#39;shot_saved_off_target&#39;,<br> &#39;shot_saved_to_post&#39;,<br> &#39;pass_outcome&#39;,<br> &#39;pass_assisted_shot_id&#39;,<br> &#39;pass_shot_assist&#39;,<br> &#39;pass_goal_assist&#39;,<br> &#39;pass_end_location&#39;,<br> &#39;index_y&#39;,<br> &#39;period_y&#39;,<br> &#39;timestamp_y&#39;,<br> &#39;minute_x&#39;,<br> &#39;second_x&#39;,<br> &#39;minute_y&#39;,<br> &#39;second_y&#39;,<br> &#39;type_y&#39;,<br> &#39;possession_y&#39;,<br> &#39;possession_team_y&#39;,<br> &#39;play_pattern_y&#39;,<br> &#39;team_y&#39;,<br> &#39;player_y&#39;,<br> &#39;position_y&#39;,<br> &#39;location_y&#39;,<br> &#39;duration_y&#39;,<br> &#39;related_events_y&#39;,<br> &#39;match_id_y&#39;,<br> &#39;under_pressure_y&#39;,<br> &#39;off_camera_y&#39;,<br> &#39;out_y&#39;]</pre><pre>extracted_data.drop(duplicate_features, axis = 1, inplace = True)</pre><h4>Non-Shot-Specific Features</h4><p>The second observation is that a number of features are not descriptive characteristics of the actual shot and/or not descriptive of the play preceding the shot in a matter which adds valuable context regarding the potential quality of the shot. Therefore, these features will not be beneficial toward the development of the eventual model.</p><p><em>Note: These observations are subjective, based on domain knowledge specific to the sport of soccer.</em></p><p><strong>Features Indicative of the Data’s Organizational Structure:</strong></p><ul><li>id: A unique identification number for the event.</li><li>match_id_x: A unique identification number for the match in which the event took place.</li><li>index_x: An identifier indicating the events’ location within StatsBombs’ nested data structure.</li><li>type_x: Categorizes various event types. (Only shot events have been extracted, therefore, the only event type is ‘shot.’)</li></ul><p>These features are useful toward navigating and sorting the larger dataset. However, these features are not descriptive characteristics of the actual shot and/or not descriptive of the play preceding the shot in a matter which adds valuable context regarding the potential quality of the shot.</p><p><strong>Features Referencing Video Recording</strong></p><ul><li>shot_freeze_frame: Describes how to locate the event in video recording.</li><li>off_camera_x: Indicates events which occurred off-camera</li></ul><p>These features are useful toward connecting the data associated with an event with video recordings of the event for further analysis. However, these features are not descriptive characteristics of the actual shot and/or not descriptive of the play preceding the shot in a matter which adds valuable context regarding the potential quality of the shot.</p><p><em>Note: These features could potentially have benefit toward adding additional features not currently recorded by StatsBomb through analysis of the video recording.</em></p><p><strong>Features Referencing Other Events</strong></p><ul><li>shot_key_pass_id: Indicates the id values of pass events for passes which directly preceded the shot event (potential assist if the shot were a goal). (<em>Note: During the data extraction process, the values of this feature were used to extract data for the specified pass events which was concatenated with the shot events in order to provide more data specific to the play preceding the shot.</em>)</li><li>related_events_x: Indicates other events significantly connected to the shot event. (<em>Note: During the data extraction process, the values of this feature were compared with dribble event ids, however, no matches were found. No other event types appear significant toward adding valuable context regarding the potential quality of the shot.</em>)</li></ul><p>These features were useful toward extracting (or attempting to extract) additional data regarding the play preceding the shot event during the data extraction process. However, these features are not descriptive characteristics of the actual shot and/or not descriptive of the play preceding the shot in a matter which adds valuable context regarding the potential quality of the shot.</p><p><strong>Features Descriptive of the Game-State</strong></p><ul><li>possession_x: Indicates a uniquely numbered possession within the game which the event was a part of. A possession is a period of time in which a team is in-possession of the ball. Within the context of the data, possessions begin when a team wins or receives the ball and end when the opposite team wins or receives the ball with all events occurring between assigned that possession value.</li><li>possession_team_x: Indicates which of the two teams was in-possession of the ball during the event.</li><li>team_x: Indicates the team which executed the event. (Because all of the events are shot events, the team executing the event would be the same as possession_team_x.)</li><li>player_x: Indicates the player who executed the event.</li><li>position_x: Indicates the position the player who executed the event plays.</li></ul><p>These features are useful toward describing the state of the game when the event occurred. However, these features are not descriptive characteristics of the actual shot and/or not descriptive of the play preceding the shot in a matter which adds valuable context regarding the potential quality of the shot.</p><p><strong>Features Specific to the Pass Preceding the Shot, but not the Shot</strong></p><ul><li>pass_recipient: Indicates the player who received the pass of a pass event. (Because pass event data was concatenated with the shot events they preceded, the pass_recipient would be the same as player_x.)</li><li>pass_body_part: Indicates the body part used to make the pass.</li><li>pass_aerial_won: Indicates if the pass was made as part of the action of winning the ball out of the air.</li><li>pass_deflected: Indicates if the pass was deflected while traveling between the passer and intended receiver.</li><li>pass_miscommunication: Indicates if there was a miscommunication between the passer and intended receiver.</li><li>pass_no_touch: Indicates if the pass was made without the passer taking a preceding touch or touches/the pass was made with the passers first and only touch.</li></ul><p>These features are descriptive of the pass preceding the shot, but in ways which do not add valuable context regarding the potential quality of the shot.</p><p><strong>Features Descriptive of the Shot, but not with Useful Data</strong></p><ul><li>duration_x: Indicates how much time the event required.</li><li>out_x: Indicates if the shot ended out of bounds. (This is redundant compared with the class label of ‘goal,’ a shot ending out of bounds does not provide additional information v a shot being classified as not a goal.)</li></ul><p>These features are descriptive characteristics of the actual shot, but will not be beneficial toward the development of the eventual model.</p><p>The second-step in cleaning the irrelevant features is dropping the non-shot-specific features:</p><pre>non_shot_specific_features = [&#39;id&#39;,<br> &#39;index_x&#39;,<br> &#39;type_x&#39;,<br> &#39;possession_x&#39;,<br> &#39;possession_team_x&#39;,<br> &#39;team_x&#39;,<br> &#39;player_x&#39;,<br> &#39;position_x&#39;,<br> &#39;duration_x&#39;,<br> &#39;related_events_x&#39;,<br> &#39;match_id_x&#39;,<br> &#39;shot_key_pass_id&#39;,<br> &#39;shot_freeze_frame&#39;,<br> &#39;out_x&#39;,<br> &#39;off_camera_x&#39;,<br> &#39;shot_aerial_won&#39;,<br> &#39;pass_recipient&#39;,<br> &#39;pass_body_part&#39;,<br> &#39;pass_aerial_won&#39;,<br> &#39;pass_deflected&#39;,<br> &#39;pass_miscommunication&#39;,<br> &#39;pass_no_touch&#39;]</pre><pre>extracted_data.drop(unrelated_features, axis = 1, inplace = True)</pre><h4>Final Set of Features</h4><p>After dropping the duplicate and non-shot-specific the following list of 21 features remains:</p><pre>extracted_data.shape[1]</pre><pre>21<br></pre><pre>list(extracted_data.columns.values)</pre><pre>[&#39;period_x&#39;,<br> &#39;timestamp_x&#39;,<br> &#39;play_pattern_x&#39;,<br> &#39;location_x&#39;,<br> &#39;under_pressure_x&#39;,<br> &#39;shot_statsbomb_xg&#39;,<br> &#39;shot_end_location&#39;,<br> &#39;shot_technique&#39;,<br> &#39;goal&#39;,<br> &#39;shot_type&#39;,<br> &#39;shot_body_part&#39;,<br> &#39;shot_one_on_one&#39;,<br> &#39;shot_open_goal&#39;,<br> &#39;shot_first_time&#39;,<br> &#39;shot_redirect&#39;,<br> &#39;shot_deflected&#39;,<br> &#39;shot_follows_dribble&#39;,<br> &#39;pass_length&#39;,<br> &#39;pass_angle&#39;,<br> &#39;pass_height&#39;,<br> &#39;pass_type&#39;,<br> &#39;pass_switch&#39;,<br> &#39;pass_through_ball&#39;,<br> &#39;pass_technique&#39;,<br> &#39;pass_backheel&#39;,<br> &#39;pass_cross&#39;,<br> &#39;counterpress&#39;,<br> &#39;pass_cut_back&#39;,<br> &#39;pass_inswinging&#39;,<br> &#39;pass_straight&#39;,<br> &#39;pass_outswinging&#39;]</pre><h3>Conclusion</h3><p>As part-two for the data cleaning portion of the data science workflow creating a Soccer Expected Goals Classification Model, irrelevant data was dropped from the dataframe.</p><p>First, features duplicated as a result of extracting data for both shot events and the pass events preceding the shot events were identified and dropped.</p><p>Second, features which are not descriptive characteristics of the actual shot and/or are not descriptive of the play preceding the shot in a matter which adds valuable context regarding the potential quality of the shot, and therefore, not beneficial toward the development of the eventual model, were subjectively rationalized and dropped.</p><p>The final set of data is significantly streamlined and concise toward the eventual goal of the model.</p><h3>Continued</h3><p>This series explaining the processes within the data science workflow creating a soccer expected goals classification model, will be continued in part-three of the data cleaning process, explaining the process of correcting the datatypes:</p><p><a href="https://medium.com/@wswager/soccer-expected-goals-data-cleaning-part-three-correcting-datatypes-3f80c0987370">Soccer Expected Goals — Data Cleaning — Part Three: Correcting Datatypes</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a4fb4692197b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Women’s Soccer Expected Goals — Data Extraction — Part-Two: Extracting Key Pass Features]]></title>
            <link>https://medium.com/@WesSwager6/continued-extraction-of-stats-bomb-data-merging-key-pass-data-with-shot-event-data-5e1ca2faac11?source=rss-18f13aeb73d6------2</link>
            <guid isPermaLink="false">https://medium.com/p/5e1ca2faac11</guid>
            <category><![CDATA[statsbomb]]></category>
            <category><![CDATA[statsbombpy]]></category>
            <category><![CDATA[soccer-analytics]]></category>
            <category><![CDATA[expected-goals]]></category>
            <category><![CDATA[data-extraction]]></category>
            <dc:creator><![CDATA[Wes Swager]]></dc:creator>
            <pubDate>Thu, 09 Sep 2021 21:05:37 GMT</pubDate>
            <atom:updated>2021-11-15T16:27:32.665Z</atom:updated>
            <content:encoded><![CDATA[<h3>Women’s Soccer Expected Goals — Data Extraction — Part-Two: Extracting Key Pass Features</h3><p>Part-two of a series for the data science workflow creating a women’s club soccer expected goals (xG) classification model, explaining the process of using the statsbombpy package to extract shot event key pass features from StatsBomb.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*WmSrXHtLw99elpa8.jpg" /></figure><p><a href="https://github.com/wswager/womens_soccer_expected_goals_model">GitHub - wswager/womens_soccer_expected_goals_model: Classification model for expected goals (xG) in women&#39;s club soccer, predicting the likelihood that a shot will score using data extracted from StatsBomb.</a></p><h3>Background</h3><p>Continued from part-one, explaining the process of extracting shot event data:</p><p><a href="https://medium.com/@wswager/using-statsbombpy-package-to-compile-statsbomb-data-c0f6734e5234">Soccer Expected Goals — Data Extraction — Part-One: Extracting Shots Data</a></p><h3>Introduction</h3><h4>Expected Goals (xG)</h4><p>xG is used to indicate the quality of a shot.</p><p>xG, as a metric, measures the likelihood that a shot will result in a goal based on the characteristics of the shot and the play preceding the shot.</p><p>xG is measured on a scale between zero and one, with one representing a goal. For example, a shot with a 0.5 xG indicates a shot having a 50% chance of being a goal.</p><p><a href="https://medium.com/@wswager/what-is-xg-f6f82053d2cc">What is xG?</a></p><h4>Classification Model</h4><p>The metric of expected goals is calculated through the use of a classification model.</p><p>A classification model refers to a predictive modeling problem where a class label is predicted for a given example of input data.</p><p>For this project a supervised approach was used with the training data for the model including which shots were goals.</p><h4>Data Extraction</h4><blockquote><em>Data extraction is the act or process of retrieving data out of…data sources for further data processing</em></blockquote><p><a href="https://en.wikipedia.org/wiki/Data_extraction"><em>Wikipedia</em></a></p><h4>The Data</h4><p>The data for this project was extracted from <a href="https://statsbomb.com/">StatsBomb</a>’s Open Data.</p><p>StatsBomb are a United Kingdom based football (soccer) data analytics company. StatsBomb provide free access to a segment of their proprietary dataset via GitHub: <a href="https://github.com/statsbomb/open-data">StatsBomb Open Data</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*6cs5aQCYc4GWCK-Z.jpeg" /></figure><p>StatsBomb Open Data is organized in JSON files:</p><p><strong>Matches</strong></p><ul><li>Folders are organized by competition (league or tournament)</li><li>Files within the folders are organized by season (year) ID</li><li>Files contain nested dictionaries with descriptive data for each individual match</li></ul><p><strong>Events</strong></p><ul><li>Files organized by match ID</li><li>Files contain nested dictionaries with descriptive data for each event within each individual match</li></ul><p>For the purposes of this project the relevant data targeted was, primarily, characteristics of shots and, secondarily, characteristics of the plays creating those shots, from women’s club soccer matches.</p><p><em>Note: Assessment of plays creating shots is subjective and based on domain knowledge specific to the sport of soccer</em></p><h4>statsbombpy</h4><p>The following processes of extracting data from the StatsBomb Open Data relies heavily on the use of <a href="https://github.com/statsbomb/statsbombpy">statsbombpy</a>.</p><p>statsbombpy is a Python package created by StatsBomb which streams StatsBomb data directly within python. The package allows access to StatsBomb Open Data for free or allows access to their API with use of log-in credentials.</p><h3>Data Extraction</h3><p><em>Import statsbombpy for extracting StatsBomb data</em></p><pre>!pip install statsbombpy<br>from statsbombpy import sb</pre><h4>Review from Part-One</h4><p>Previously, in <a href="https://medium.com/@wswager/using-statsbombpy-package-to-compile-statsbomb-data-c0f6734e5234">Soccer Expected Goals — Data Extraction — Part-One: Extracting Shots Data</a>, target women’s club competitions, competition ids, and season ids were identified and isolated from StatsBomb matches data.</p><pre>matches_df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mnpEjTS8DVDipWRmNEFQTA.png" /></figure><pre>print(&#39;Number of Seasons:&#39;, len(target_season_ids))</pre><pre>Number of Women&#39;s Club Seasons: 3</pre><pre>print(&#39;Total Women&#39;s Club Matches:&#39;, len(matches_df))</pre><pre>Total Matches: 231</pre><p>Then, shot event data was extracted from StatsBomb events data.</p><pre>shots_df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M7B6nF4iJPYotzbc9rBI2A.png" /></figure><pre>print(&quot;Total Women&#39;s Club Shot Events:&quot;, len(shots_df))</pre><pre>Total Women&#39;s Club Competition Shot Events: 6114</pre><pre>print(&#39;Total Shot Features:&#39;, shots_df.shape[1])</pre><pre>Total Shot Features: 37</pre><h4>Extract Key Pass Features</h4><p>As mentioned previously, characteristics of the play creating a shot can be valuable toward the potential quality of the shot. For this reason, features of other event types related to shot events can be valuable and worth assessment.</p><p>shot_key_pass_id is a feature for shot events which identifies specific pass events (indicating the event id feature value), as passes which lead directly to the shot (potential assist, if the shot were to score).</p><p><em>Create dataframes for pass events in each season of the target competitions</em></p><pre>passes_df_37_42 = sb.competition_events(country = &#39;England&#39;, division = &quot;FA Women&#39;s Super League&quot;, season = &#39;2018/2019&#39;, gender = &#39;female&#39;, split = True)[&#39;passes&#39;]</pre><pre>passes_df_37_4 = sb.competition_events(country = &#39;England&#39;, division = &quot;FA Women&#39;s Super League&quot;, season = &#39;2019/2020&#39;, gender = &#39;female&#39;, split = True)[&#39;passes&#39;]</pre><pre>passes_df_49_3 = sb.competition_events(country = &#39;United States of America&#39;, division = &#39;NWSL&#39;, season = &#39;2018&#39;, gender = &#39;female&#39;, split = True)[&#39;passes&#39;]</pre><p><em>Concatenate pass events dataframes into combined dataframe</em></p><pre>passes_df = pd.concat([passes_df_37_42, passes_df_37_4, passes_df_49_3], ignore_index = True)</pre><pre>passes_df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f-GEENl5WcijC-U3eTlqYQ.png" /></figure><pre>print(&quot;Total Women&#39;s Club Competition Pass Events:&quot;, len(passes_df))</pre><pre>Total Women&#39;s Club Competition Pass Events: 209122</pre><pre>print(&#39;Total Pass Features:&#39;, passes_df.shape[1])</pre><pre>Total Pass Features: 45</pre><p><em>Create a key_pass_events list from shot_key_pass_id values for shot events</em></p><pre>key_pass_events = list(shots_df[‘shot_key_pass_id’])</pre><p><em>Search pass events for key passes</em></p><pre>passes_to_shots_df = passes_df[passes_df[&#39;id&#39;].isin(key_pass_events)]</pre><pre>print(&quot;Pass Events Identified as &#39;shot_key_pass_id&#39; for Shot Events:&quot;, len(passes_to_shots_df))</pre><pre>Pass Events Identified as &#39;shot_key_pass_id&#39; for Shot Events: 4164</pre><p><em>Concatenate pass data from passes_df for passes identified as shot event key passes with shots_df</em></p><pre>passes_df2 = passes_df.rename(columns = {&#39;id&#39;: &#39;shot_key_pass_id&#39;})</pre><pre>extracted_data = pd.merge(shots_df, passes_df2, on = [&#39;shot_key_pass_id&#39;], how = &#39;left&#39;)</pre><h3>Results</h3><pre>print(&quot;Total Women&#39;s Club Shot Events:&quot;, len(extracted_data))</pre><pre>Total Women&#39;s Club Competition Shot Events: 6114</pre><pre>print(&#39;Updated Shot w/ Key Pass Features:&#39;, extracted_data.shape[1])</pre><pre>Updated Shot w/ Key Pass Features: 81</pre><p>The additional extraction of key pass features increased the number of features for the 6,144 shot events from 37 to 81.</p><pre>extracted_data.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LfNKC4YqoC2W8igB-lY_PQ.png" /></figure><h3>Continued</h3><p>Part-three continues the series for the data science workflow creating a women’s club soccer expected goals (xG) classification model, explaining the process of exploring other related events and their features.</p><p><a href="https://medium.com/@wswager/soccer-expected-goals-data-extraction-part-three-exploring-related-events-feature-5555a2231805">Soccer Expected Goals — Data Extraction — Part-Three: Exploring Related Events Feature</a></p><h4>More</h4><p>If you liked this post, please give it an applause and follow me as I will be continuing with a series of posts for each process through the data science workflow of my <a href="https://github.com/wswager/womens_soccer_expected_goals_model">Women’s Soccer Expected Goals Model</a>:</p><p><a href="https://medium.com/@wswager">Wes Swager - Medium</a></p><p>Also, follow me on Twitter, where I post regularly about tactical observations for soccer:</p><p><a href="https://twitter.com/Wes_Swager_6">JavaScript is not available.</a></p><p>I would love to read any feedback you might have in the comments.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5e1ca2faac11" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is xG?]]></title>
            <link>https://medium.com/@WesSwager6/what-is-xg-f6f82053d2cc?source=rss-18f13aeb73d6------2</link>
            <guid isPermaLink="false">https://medium.com/p/f6f82053d2cc</guid>
            <category><![CDATA[soccer]]></category>
            <category><![CDATA[soccer-analytics]]></category>
            <category><![CDATA[expected-goals]]></category>
            <dc:creator><![CDATA[Wes Swager]]></dc:creator>
            <pubDate>Fri, 03 Sep 2021 23:39:48 GMT</pubDate>
            <atom:updated>2021-09-03T23:39:48.204Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/733/1*6Yhc4YDf3x75ErIMsabwMQ.jpeg" /></figure><h3>Definition</h3><blockquote><em>‘The most difficult thing in [soccer] is to score a goal’</em></blockquote><blockquote><em>-Pep Guardiola</em></blockquote><p><strong>xG measures the quality of a shot.</strong></p><p>xG calculates the likelihood that a shot will be scored based on the characteristics of the shot and the play preceding the shot.</p><p>xG is measured on a scale between zero and one, with one representing a goal. For example, a shot with a 0.5 xG is calculated to have a 50% chance of scoring.</p><h3>xG History</h3><blockquote><em>‘Quality without results is pointless, results without quality is boring’</em></blockquote><blockquote><em>-Johan Cruyff</em></blockquote><p>In 2002, ‘<a href="https://en.wikipedia.org/wiki/Moneyball">moneyball</a>’ transformed baseball, when the Oakland Athletics utilized ‘<a href="https://en.wikipedia.org/wiki/Sabermetrics">sabermetrics</a>’ models to overperform what their budget suggested they were capable of through the use of data.</p><p>In much the same way, data science has begun to penetrate the highest levels of soccer, largely exemplified by the popularization of the xG metric.</p><p>In 2004, Jake Ensum, Richard Pollard, and Samuel Taylor conducted a study of matches from the 2002 World Cup ‘to investigate and quantify factors that might affect the success of a shot,’ and, using logistic regression, identified five factors which had a significant effect:</p><ul><li>Distance from Goal</li><li>Angle from Goal</li><li>Distance from Nearest the Defender</li><li>If the Assist was a Cross</li><li>Number of Players Between the Shot and Goal</li></ul><p>In 2012, Sam Green, of <a href="https://www.optasports.com/">Opta</a>, a British sports analytics company, introduced the term ‘expected goals’ (xG) in regards to soccer. He described the metric calculated by Opta as a means to:</p><ul><li>‘Accurately and effectively increase…chances of scoring and therefore winning matches’</li><li>‘Use data from a defensive perspective to limit the better chances by defending key areas of the pitch’</li><li>‘Look at each player’s shots and tally up the probability of each of them being a goal to give a [cumulative] expected goal (xG) value’</li></ul><p>Early adoption by betting markets has since helped xG become a regular feature of mainstream soccer broadcasts and media analysis.</p><h3>xG Calculation?</h3><blockquote><em>You study and think and discuss, and you come up with a model of playing.</em></blockquote><blockquote><em>Carlo Ancelloti</em></blockquote><p>xG is not a standardized metric, and, therefore, many different proprietary variants exist amongst various sports analytics organizations, betting companies, soccer clubs, etc.</p><p>Generally, xG is built using a classifier model with input from a large historical dataset of shot event data.</p><p>Some of the most commonly included features are:</p><ul><li>Distance to the Goal</li><li>Angle to the Goal​</li><li>Opponent Players Between the Shot and Goal</li><li>Total Players Between the Shot and Goal</li><li>Bodypart Used to Shoot</li><li>Type of Assist</li><li>Pattern of Play</li></ul><p>A shot’s xG is measured by comparing the shot to data from previous shots with similar features and calculating the quantity scored.</p><p>xG does not consider the quality the players or teams.</p><h3>xG Application?</h3><blockquote><em>‘I tell the players that the bus is moving. This club has to progress. And the bus wouldn’t wait for them. I tell them to get on board.’</em></blockquote><blockquote><em>Sir Alex Ferguson</em></blockquote><p>xG can add context beyond traditional statistics through assigning a measurement to the quality of shots.</p><p>The most direct application of xG is in the assessment of an individual shot, if a miss was a high quality opportunity or how difficult a goal was to score.</p><p>Assessment of individual shots can help in providing recommendations using using features significant to xG. For example, a training staff can use an example of a shot taken 20-yards from goal, and show using video replay that there was an opportunity to take an additional dribble, then provide a comparison between the xG of the actual shot and the xG of a shot with the same features but 15-yards from goal in order to help quantify the value of the difference.</p><p>The more common application of xG is in measuring xG cumulatively (for example, the xG for a team’s shots across the length of a match would indicate how many goals they were likely to score given the characteristics of those shots).</p><p>The cumulative xG of a player across the course of a game or set of games can be compared with actual goals to assess over or underperformance. For example, if a player had a cumulative xG of 3.5 across five games, but only scored 1 goal, they may be assessed as underperforming, and training staff may then assess the player’s shots across that period come to a conclusion that the player requires additional technical training to help improve their finishing.</p><p>Cumulative xG for players can also be used as a metric for comparison. For example, if a scouting staff are assessing two players and the players scored a similar number of goals from a similar number of shots, they then can use xG for a deeper comparison, such as, concluding the player with more actual goals v xG is the superior finisher or concluding the player with smaller difference of goals v xG is more likely to be consistent and maintain their current rate of scoring (v regressing to their xG mean).</p><p>Cumulative xG can also be measured for a team. For example, for a sports betting company, in assessing a team which has won nine of their previous ten matches, the assumption may be that they are likely to win their next match. However, if that team has also significantly overperformed their xG while their opponents have significantly underperformed their xG, then the expectation may change as a regression toward their xG means is likely.</p><p>Cumulative xG can also be measured across the course of a match using timestamps of shots. For example, a media company may create a visual of a match indicating each team’s cumulative xG across the length of the match as a means of demonstrating which team was more threatening toward scoring at various points.</p><p>Because soccer is a relatively low scoring sport, the ability to measure the likelihood of a goal being scored can add context toward quantifying the flow and story.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f6f82053d2cc" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Soccer Expected Goals — Data Extraction — Part-One: Extracting Shots Data]]></title>
            <link>https://medium.com/@WesSwager6/using-statsbombpy-package-to-compile-statsbomb-data-c0f6734e5234?source=rss-18f13aeb73d6------2</link>
            <guid isPermaLink="false">https://medium.com/p/c0f6734e5234</guid>
            <category><![CDATA[statsbomb]]></category>
            <category><![CDATA[statsbombpy]]></category>
            <category><![CDATA[expected-goals]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-extraction]]></category>
            <dc:creator><![CDATA[Wes Swager]]></dc:creator>
            <pubDate>Fri, 27 Aug 2021 20:37:18 GMT</pubDate>
            <atom:updated>2021-11-15T16:28:12.390Z</atom:updated>
            <content:encoded><![CDATA[<h3>Women’s Soccer Expected Goals — Data Extraction — Part-One: Extracting Shots Data</h3><p>Part-one of a series for the data science workflow creating a women’s club soccer expected goals (xG) classification model, explaining the process of using the statsbombpy package to extract shot event data from StatsBomb.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*xrXyZbRuz5cmKkH2.jpg" /></figure><p><a href="https://github.com/wswager/womens_soccer_expected_goals_model">GitHub - wswager/womens_soccer_expected_goals_model: Classification model for expected goals (xG) in women&#39;s club soccer, predicting the likelihood that a shot will score using data extracted from StatsBomb.</a></p><h3>Introduction</h3><h4>Expected Goals (xG)</h4><p>xG is used to indicate the quality of a shot.</p><p>xG, as a metric, measures the likelihood that a shot will result in a goal based on the characteristics of the shot and the play preceding the shot.</p><p>xG is measured on a scale between zero and one, with one representing a goal. For example, a shot with a 0.5 xG indicates a shot having a 50% chance of being a goal.</p><p><a href="https://medium.com/@wswager/what-is-xg-f6f82053d2cc">What is xG?</a></p><h4>Classification Model</h4><p>The metric of expected goals is calculated through the use of a classification model.</p><p>A classification model refers to a predictive modeling problem where a class label is predicted for a given example of input data.</p><p>For this project a supervised approach was used with the training data for the model including which shots were goals.</p><h4>Data Extraction</h4><blockquote><em>Data extraction is the act or process of retrieving data out of…data sources for further data processing</em></blockquote><p><a href="https://en.wikipedia.org/wiki/Data_extraction"><em>Wikipedia</em></a></p><h4>The Data</h4><p>The data for this project was extracted from <a href="https://statsbomb.com/">StatsBomb</a>’s Open Data.</p><p>StatsBomb are a United Kingdom based football (soccer) data analytics company. StatsBomb provide free access to a segment of their proprietary dataset via GitHub: <a href="https://github.com/statsbomb/open-data">StatsBomb Open Data</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*6cs5aQCYc4GWCK-Z.jpeg" /></figure><p>StatsBomb Open Data is organized in JSON files:</p><p><strong>Matches</strong></p><ul><li>Folders are organized by competition (league or tournament)</li><li>Files within the folders are organized by season (year) ID</li><li>Files contain nested dictionaries with descriptive data for each individual match</li></ul><p><strong>Events</strong></p><ul><li>Files organized by match ID</li><li>Files contain nested dictionaries with descriptive data for each event within each individual match</li></ul><p>For the purposes of this project the relevant data targeted was, primarily, characteristics of shots and, secondarily, characteristics of the plays creating those shots, from women’s club soccer matches.</p><p><em>Note: Assessment of plays creating shots is subjective and based on domain knowledge specific to the sport of soccer</em></p><h4>statsbombpy</h4><p>The following processes of extracting data from the StatsBomb Open Data relies heavily on the use of <a href="https://github.com/statsbomb/statsbombpy">statsbombpy</a>.</p><p>statsbombpy is a Python package created by StatsBomb which streams StatsBomb data directly within python. The package allows access to StatsBomb Open Data for free or allows access to their API with use of log-in credentials.</p><h3>Data Extraction</h3><h4>Extract Target Matches Data</h4><p><em>Import statsbombpy for extracting StatsBomb data</em></p><pre>!pip install statsbombpy<br>from statsbombpy import sb</pre><p>View competitions available through StatsBomb Open Data</p><pre>competitions_df = sb.competitions()</pre><pre>print(&#39;Available Competitions:&#39;, competitions_df[&#39;competition_name&#39;].unique())</pre><pre>Available Competitions: [&#39;Champions League&#39; &quot;FA Women&#39;s Super League&quot; &#39;FIFA World Cup&#39; &#39;La Liga&#39;<br> &#39;NWSL&#39; &#39;Premier League&#39; &quot;Women&#39;s World Cup&quot;]</pre><p><em>Isolate and display target women’s competitions, competition ids, and season ids</em></p><pre>target_comp_df = competitions_df.loc[competitions_df[&#39;competition_gender&#39;] == &#39;female&#39;]</pre><pre>target_comp_ids = target_comp_df[&#39;competition_id&#39;].unique()</pre><pre>target_season_ids = target_comp_df[&#39;season_id&#39;].unique()</pre><pre>print(&quot;Women&#39;s Competitions:&quot;, target_comp_df[&#39;competition_name&#39;].unique(), &#39;\n&#39;, &quot;Women&#39;s competition_ids:&quot;, target_comp_ids, &#39;\n&#39;, &quot;Women&#39;s Competition season_ids:&quot;, target_season_ids)</pre><pre>Women&#39;s Competitions: [&quot;FA Women&#39;s Super League&quot; &#39;NWSL&#39;] <br> Women&#39;s competition_ids: [37 49] <br> Women&#39;s Competition season_ids: [42  4  3]</pre><p><em>Refine target competitions to women’s club competitions</em></p><pre>target_comp_df = competitions_df.loc[competitions_df[&#39;competition_id&#39;].isin([37, 49])]</pre><pre>target_comp_ids = target_comp_df[&#39;competition_id&#39;].unique()</pre><pre>target_season_ids = target_comp_df[&#39;season_id&#39;].unique()</pre><pre>print(&quot;Women&#39;s Club Competitions:&quot;, target_comp_df[&#39;competition_name&#39;].unique(), &#39;\n&#39;, &quot;Women&#39;s Club competition_ids:&quot;, target_comp_ids, &#39;\n&#39;, &quot;Women&#39;s Club Competition season_ids:&quot;, target_season_ids)</pre><pre>Women&#39;s Club Competitions: [&quot;FA Women&#39;s Super League&quot; &#39;NWSL&#39;] <br> Women&#39;s Club competition_ids: [37 49] <br> Women&#39;s Club Competition season_ids: [42  4  3]</pre><p><em>Create dataframes for the matches in each season of the target competitions</em></p><pre>matches_df_37_42 = sb.matches(competition_id = 37, season_id = 42)</pre><pre>matches_df_37_4 = sb.matches(competition_id = 37, season_id = 4)</pre><pre>matches_df_49_3 = sb.matches(competition_id = 49, season_id = 3)</pre><p><em>Concatenate target season dataframes into combined dataframe</em></p><pre>matches_df = pd.concat([matches_df_37_42, matches_df_37_4, matches_df_49_3], ignore_index = True)</pre><pre>matches_df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mnpEjTS8DVDipWRmNEFQTA.png" /></figure><pre>print(&#39;Number of Seasons:&#39;, len(target_season_ids))</pre><pre>Number of Women&#39;s Club Seasons: 3</pre><pre>print(&#39;Total Women&#39;s Club Matches:&#39;, len(matches_df))</pre><pre>Total Matches: 231</pre><h4>Extract Shot Events Data</h4><p><em>Create dataframes for shot events in each season of the target competitions</em></p><pre>shots_df_37_42 = sb.competition_events(country = &#39;England&#39;, division = &quot;FA Women&#39;s Super League&quot;, season = &#39;2018/2019&#39;, gender = &#39;female&#39;, split = True)[&#39;shots&#39;]</pre><pre>shots_df_37_4 = sb.competition_events(country = &#39;England&#39;, division = &quot;FA Women&#39;s Super League&quot;, season = &#39;2019/2020&#39;, gender = &#39;female&#39;, split = True)[&#39;shots&#39;]</pre><pre>shots_df_49_3 = sb.competition_events(country = &#39;United States of America&#39;, division = &#39;NWSL&#39;, season = &#39;2018&#39;, gender = &#39;female&#39;, split = True)[&#39;shots&#39;]</pre><p><em>Concatenate shot events dataframes into combined dataframe</em></p><pre>shots_df = pd.concat([shots_df_37_42, shots_df_37_4, shots_df_49_3], ignore_index = True)</pre><h3>Results</h3><pre>print(&quot;Total Women&#39;s Club Shot Events:&quot;, len(shots_df))</pre><pre>Total Women&#39;s Club Competition Shot Events: 6114</pre><pre>print(&#39;Total Shot Features:&#39;, shots_df.shape[1])</pre><pre>Total Shot Features: 37</pre><p>The extraction process for shot event data yielded 6,144 shot events with 37 features.</p><pre>shots_df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M7B6nF4iJPYotzbc9rBI2A.png" /></figure><h3>Continued</h3><p>Part-two continues the series for the data science workflow creating a women’s club soccer expected goals (xG) classification model, explaining the process of using the statsbombpy package to extract shot event key pass features.</p><p><a href="https://medium.com/@wswager/continued-extraction-of-stats-bomb-data-merging-key-pass-data-with-shot-event-data-5e1ca2faac11">Soccer Expected Goals — Data Extraction — Part-Two: Extracting Key Pass Features</a></p><h4>More</h4><p>If you liked this post, please give it an applause and follow me as I will be continuing with a series of posts for each process through the data science workflow of my <a href="https://github.com/wswager/womens_soccer_expected_goals_model">Women’s Soccer Expected Goals Model</a>:</p><p><a href="https://medium.com/@wswager">Wes Swager - Medium</a></p><p>Also, follow me on Twitter, where I post regularly about tactical observations for soccer:</p><p><a href="https://twitter.com/Wes_Swager_6">JavaScript is not available.</a></p><p>I would love to read any feedback you might have in the comments.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c0f6734e5234" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>