<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Burak Arslan on Medium]]></title>
        <description><![CDATA[Stories by Burak Arslan on Medium]]></description>
        <link>https://medium.com/@arslanburak?source=rss-478f2a253ec5------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*9srnOpvuz3uNeRUym9L0TQ.png</url>
            <title>Stories by Burak Arslan on Medium</title>
            <link>https://medium.com/@arslanburak?source=rss-478f2a253ec5------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 03:20:10 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@arslanburak/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Is It Possible to Predict Football Players’ Value]]></title>
            <link>https://levelup.gitconnected.com/is-it-possible-to-predict-football-players-value-617eafe8669b?source=rss-478f2a253ec5------2</link>
            <guid isPermaLink="false">https://medium.com/p/617eafe8669b</guid>
            <category><![CDATA[web-scraping]]></category>
            <category><![CDATA[linear-regression]]></category>
            <category><![CDATA[football]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[regression]]></category>
            <dc:creator><![CDATA[Burak Arslan]]></dc:creator>
            <pubDate>Tue, 09 Feb 2021 04:07:56 GMT</pubDate>
            <atom:updated>2021-02-09T04:07:56.888Z</atom:updated>
            <content:encoded><![CDATA[<h4>A Brief Introduction of Web Scraping &amp; Regression Models</h4><p>For my second project at Data Science Bootcamp at Istanbul Data Science Academy; I tried to find out if it is possible to determine a player’s value by his 2019/2020 season statistics. This project is about getting hands-on experience on web scraping, regression models and cross validation. My target was to predict their values. I scraped data from famous football value and statistics site: transfermarkt.de. There was a problem here about the values. They were currently updated values based on their past and current performance. But the statistics that I used were from the previous season.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*-dSUeyyEuJOkMwBi" /><figcaption>Photo by <a href="https://unsplash.com/@marioklassen?utm_source=medium&amp;utm_medium=referral">Mario Klassen</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4>Data Fetching by Web Scraping</h4><p>I fetched player statistics from <a href="https://fbref.com/en/comps/Big5/2019-2020/stats/players/2019-2020-Big-5-European-Leagues-Stats">2019–2020 Big 5 European Leagues Stats (players) | FBref.com</a> and player values from transfermarkt.de. First I fetched data from multiple pages for various statistics including attacking, defending, creativity, passing, shooting and miscellaneous key performance indicators. With BeautifulSoup I parsed the data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*r6thANiojnAZNKU5" /><figcaption>Photo by <a href="https://unsplash.com/@valerysysoev?utm_source=medium&amp;utm_medium=referral">Valery Sysoev</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/44edc4ca213686f03d59e7895ba40b72/href">https://medium.com/media/44edc4ca213686f03d59e7895ba40b72/href</a></iframe><p>I repeated the code above and then merged all data into one frame. Here is a snapshot of the correlation of each item, that was created by seaborn heatmap function.</p><p>Features : Position, Age, Min, Goals, GoalsPerGame, Asists, AsistsPerGame, GoalAstPer90, 90s, SCA, SCA90, ShootPass,ShootDrib, GCA, GCA90, AssistPass, AssistDrib,CompletedPasses, PassCompRatio, TotDistPass, ProgDistPass,Crosses, PenaltyWon, AerialWonRate, ShootOnTarget, ShootPer90,GoalPerShoot, TacklesWon, TklDribbling, Blocks,SuccPressRatio, MatchPlayed, PointPerMatch, TouchPenArea,SuccDrib, DribLegs, PlayerType, League</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/831/1*zPRIbqBj3F9wCex2gOIW_A.png" /></figure><p>As target I fetched data from transfermarkt.de. Those were estimated values so they were all rounded. The players’ values had a skewed distribution. We know from theory that the symmetrically distributed data will help us got better results when we are building our models. Logarithmic transformation help us to transform skewed player values into a more normalized distribution. You can see how the distribution change after transformation is applied.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/392/1*88lhLEyTElx43VnwOn3Zmg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/386/1*yBgZGbGAAPgrrezxBNlaaQ.png" /><figcaption>Distribution of Player Values</figcaption></figure><h4>New Features for Better Fitting</h4><p><strong>Age groups instead of Age</strong>: I binned the data into groups that would correlate with the player&#39;s value.</p><p><strong>Winner</strong>: Playing in a big club is always positive for the players’ market value. To determine this, I created a kpi called winner that calculated as the following formula:</p><p>winner = “Point per Match Played” ^ 2 * Played Match count</p><p><strong>Goal Change Creator</strong>: I created a kpi to determine players that had an effect on teams’ scoring. Instead of ‘Assist’, ‘Shoot on Target’, ‘Dribbling lead to assist’, ‘pass lead to assist’, ‘Penalty won’, ‘Successful pressing at 3rd Area’.</p><p>Features that were converted to categorical variable into dummy/indicator variables: League and Player Position. Some Leagues are more challenging than the others, so the same statistics are more valuable there. And from data set, we know player value is highly correlated with the position he plays.</p><p>I also removed features that had a high correlation between one another. For example, players that had touched the ball in the penalty area scored more goals than other players.</p><p>I also had to remove players that substituted mostly; because they had no sufficient statistics. Prediction performance of our models heavily depends on data; so removing data means a decrease in model performance.</p><p>For modeling, I used various regression models from scikit-learn library. For better fitting, I created separate models for different positions. I trained different models for forward, midfield, and defense players. Because for each position the importance of the features can be different. For example, for a forward position, the number of touching the ball in the opponent penalty area is very correlated with the player’s value, but for a defense player it is not.</p><p>Cross validation is key to success. For training the data and testing I highly recommend using cross validation. By having different slices of data and training it, you will be more prepared for real life challenges. <a href="https://scikit-learn.org/stable/modules/cross_validation.html">Kfold</a> is a great function to automate this process, please check it. You can see a simple code for cross validation using kfold below.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/870a8ee7cd8c2b3166dc3eee451c8c05/href">https://medium.com/media/870a8ee7cd8c2b3166dc3eee451c8c05/href</a></iframe><p>After cross validation and fitting multiple models, Linear Regression is the best one-having min MSE- above the other regression methods. So I used it for training my data.</p><h4>Result</h4><p>Here is the prediction of some players vs their Transfermarkt values. I retransformed the value back in order to show more human friendly numbers. Here are some top players and their Transfermarkt values versus my prediction values (million Euro) and also a chart that compare player values vs prediction.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/545/1*-mt3S4lVByFmUEbQU0ak4Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/651/1*wF9xcX0GxTEUrFUNdsRe7w.jpeg" /></figure><h4>Future Work</h4><p>There are some issues that can not be figured out from statistics. As future work, we need to figure out how to overcome these issues.</p><ul><li>Team members may affect each other’s statistics.</li><li>Substitute players has missing statistics, so we are losing data.</li><li>Some players are injury-prone. Clubs avoid to sign a player who was frequently injured.</li><li>Players’ popularity in their country is important if the they have a few elite players. Keisuke Honda is a good example of this. Clubs can pay then their actual value to gain popularity in that regions.</li><li>Some people can handle stress, others can’t. Big clubs need players that can handle stress and can play big matches.</li><li>Player leadership, work rate, and ethics are essential. Talent is not enough.</li><li>The economic strength and prestige of the club.</li></ul><p>Please share your insights about how these issues can be handle. For presentation &amp; codes please check my <a href="https://github.com/burakarslan/DSbootcamp-Project2/blob/main/Project%202.pdf">github</a>.</p><p>Thanks for reading.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=617eafe8669b" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/is-it-possible-to-predict-football-players-value-617eafe8669b">Is It Possible to Predict Football Players’ Value</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploratory Data Analysis of NY MTA]]></title>
            <link>https://medium.com/i%CC%87stanbuldatascienceacademy/exploratory-data-analysis-of-ny-mta-96ceba3d82c1?source=rss-478f2a253ec5------2</link>
            <guid isPermaLink="false">https://medium.com/p/96ceba3d82c1</guid>
            <category><![CDATA[projects]]></category>
            <category><![CDATA[datascience-training]]></category>
            <category><![CDATA[exploratory-data-analysis]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[pandas]]></category>
            <dc:creator><![CDATA[Burak Arslan]]></dc:creator>
            <pubDate>Sun, 13 Dec 2020 14:50:23 GMT</pubDate>
            <atom:updated>2021-08-19T14:33:34.517Z</atom:updated>
            <content:encoded><![CDATA[<p>Last week I completed my first project at <a href="http://istdatascience.com/">Istanbul Data Science Academy</a> Data Science Bootcamp. The general purpose of the project is to be able to apply what we have learned in the first two weeks and to gain hands-on experience about these topics. By the way, I have to mention there are lots of practice and I really enjoy how things are going at the DS Bootcamp.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*HSHlohLJEp1UcHgo" /><figcaption>Photo by <a href="https://unsplash.com/@gcoppa?utm_source=medium&amp;utm_medium=referral">Guido Coppa</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Introduction</h3><p>The project starts with a fictional email that comes from WTWY International. They have an upcoming gala in the summer, and they want to increase interest, attendance, and awareness of their organization and reach key people that can contribute to their cause. They planned to collect emails at the entrances of subway stations in New York, so they need our analysis to place street teams at entrances to subway stations.</p><p>For Exploratory Data Analysis, the following Python libraries are essential: Pandas, Numpy and Matplotlib, and Seaborn.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/885/1*cJ2JugmLHWD5uarhlF_ZWQ.png" /></figure><h3>Data Analysis</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/962/1*EG1eAc43G3B00zw5CxFXiQ.png" /></figure><p>I used the data that is published weekly by the New York Metropolitan Transportation Authority.</p><p>Data Source: <a href="http://web.mta.info/developers/turnstile.html">http://web.mta.info/developers/turnstile.html</a></p><p>For a summer gala, I think it is good to collect emails in March so that there will be enough time to connect via email and proceed. I wanted to work with pre-pandemic data so I chose 4 data files-813096 rows- that match with March 2019.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/870/1*zQXOCupaFfxHhBLdyXGf1w.png" /></figure><pre>Field Description</pre><pre>C/A,UNIT,SCP,STATION,LINENAME,DIVISION,DATE,TIME,DESC,ENTRIES,EXITS<br></pre><pre>C/A      = Control Area (A002)<br>UNIT     = Remote Unit for a station (R051)<br>SCP      = Subunit Channel Position represents an specific address for a device (02-00-00)<br>STATION  = Represents the station name the device is located at<br>LINENAME = Represents all train lines that can be boarded at this station<br>           Normally lines are represented by one character.  LINENAME 456NQR repersents train server for 4, 5, 6, N, Q, and R trains.<br>DIVISION = Represents the Line originally the station belonged to BMT, IRT, or IND   <br>DATE     = Represents the date (MM-DD-YY)<br>TIME     = Represents the time (hh:mm:ss) for a scheduled audit event<br>DESc     = Represent the &quot;REGULAR&quot; scheduled audit event (Normally occurs every 4 hours)<br>           1. Audits may occur more that 4 hours due to planning, or troubleshooting activities. <br>           2. Additionally, there may be a &quot;RECOVR AUD&quot; entry: This refers to a missed audit that was recovered. <br>ENTRIES  = The comulative entry register value for a device<br>EXIST    = The cumulative exit register value for a device</pre><p><a href="http://web.mta.info/developers/resources/nyct/turnstile/ts_Field_Description.txt">Field description</a> is a great resource to start analyzing the data. Unfortunately, The time resolution of the data is not hourly. Instead, each row contains data for 4 hours. This was not ideal at all as the start and end time of 4-hour time slots varied from station to station. In the image below, you can easily spot there is no standard at the data for start and end times of time-slot. Entries and exits columns do not have information about how many people turn the turnstile at that time period. They are counters. ( With my domain expertise at Telco, I used to work with counters. )</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C18jVoErbz-NHRkiRxsktA.png" /></figure><p>While I tried to find out how many passengers per time slot by subtracting each turnstile record from the previous one, I understood how great Pandas was. First I grouped the data by Turnstile then I tried to find the difference between records by loops, but the performance was suffering. Then I discovered that Pandas had a <a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.core.groupby.DataFrameGroupBy.diff.html">diff </a>function that could work with grouping the data. It worked like a magic, fast and easy. Now I had the information of how many times a turnstile turn, instead of counters</p><p>There are 7297 rows of negative values for entries. As you can see from the next Image, some counters are counting down instead of down. In this example, a turnstile at EASTCHSTER/DYRE station is counting entries in the wrong way. But it’s working correctly for counting the people that exit the station.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aq8cs9OC7KIWEmH4si2Aog.png" /><figcaption>Data of Station EASTCHSTER/DYRE’ for date:03/29/2019 and turnstile ==’R730|R431|00–00–04&#39;</figcaption></figure><p>To overcome that problem, I took the absolute value of both entries and exits differences. Then I spotted there were lots of huge numbers that could not be True. There were 75 records that mean a minimum of twelve people were passing the turnstile per minute for that 4 hour time period. I had never been in New York so I watched a few videos from Youtube to understand New York Turnstile system. There were full height turnstiles that were hard to push and pass, and also I noticed some people struggled a lot when they swiped their metro cards. So I decided beyond 12 people per minute was wrong data. I rounded the values that are above 2880 to 2880.</p><p>Exits were easy to pass. I thought 2 people could pass in 1.5 seconds apart. There were 28 records that mean min 40 persons per minute. I rounded them down to 9660, too.</p><p>The next challenge was the time resolution. Here is a chart of total traffic. X-axis you can see the Hour the counter’s time period done and sent the data.</p><p>total traffic = entries+ exists</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/632/1*J6BTrq7-s3ANedIQJkLSsw.png" /></figure><p>Next, I compared Traffic per Days of the Week :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/649/1*SCIPEyE_buDXf8g9NwONUQ.png" /><figcaption>Daily Total Traffic ( All Stations )</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/515/1*X5MzVuqQmVFcoITGDOuRrA.png" /><figcaption>Daily Total Traffic for Top 10 Busy Station</figcaption></figure><p>Both Charts tell the same story. Traffic is high on weekdays.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/348/1*Qv4DzlSNj7vNcxy0n_1BrA.png" /></figure><p>Next, I wanted to eliminate a few from my top 10 busy stations. So I took a look at them on google maps.</p><p>The red ones are very close to each other on the map. I think it’s better to suggest to place street teams between these stations instead of the entrance of them. So they will be more likely to get a combination of foot traffic of both stations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/849/1*v5e-f8gBUgn7B_10H4Iu3Q.png" /></figure><h3>Result</h3><p>I decided to recommend this list to the customer. I will also suggest they have a survey with the attendees at the Gala. Next year that survey will be a key to analyze how to reach more connection.</p><p><strong>Stations<br></strong>*34 Street Penn Station<br>*Grand Central–42nd Street<br>*14th Street–Union Square<br>*42nd Street/Port Authority<br>*Fulton Street.</p><p><strong>Time</strong><br>*Only Weekdays to reach white-collar workers. This will fit the customer. They have a limited budget. they will not pay overtime to their street teams. <br>*It’s enough to collect between 11 AM — 20 PM.</p><h3>Appendix 1: Assumptions &amp; Challenges :</h3><ul><li>Some counters were counting downwards instead of upwards. I assumed they count correct but wrong direction.</li><li>Some stations may have entrances that are far from each of them that can mislead.</li><li>Some stations are hub stations that people just use for transferring to another line. I assume that all entries and exits will cause foot traffic for the client street team.</li></ul><h3>Appendix 2: Future Work :</h3><ul><li>Tourists may mislead the results. By comparing peak season and off-season, we can normalize our data according to tourist attractions.</li><li>With Restaurant prices and location data, we can spot where high-income people prefer to go for lunch and choose spots near there.</li><li>If there is a region where technology companies are concentrated, we should find and include company location data</li><li>If there is a station used for transfer between lines, we should handle it, too.</li></ul><p>Thanks for reading.</p><p>Burak</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=96ceba3d82c1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/i%CC%87stanbuldatascienceacademy/exploratory-data-analysis-of-ny-mta-96ceba3d82c1">Exploratory Data Analysis of NY MTA</a> was originally published in <a href="https://medium.com/i%CC%87stanbuldatascienceacademy">İstanbul Data Science Academy</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>