<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Yanan Wu on Medium]]></title>
        <description><![CDATA[Stories by Yanan Wu on Medium]]></description>
        <link>https://medium.com/@ywu120766?source=rss-3122a7fdf33------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*K-rbfLFnsuOp7tVrWO6Mvg.jpeg</url>
            <title>Stories by Yanan Wu on Medium</title>
            <link>https://medium.com/@ywu120766?source=rss-3122a7fdf33------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 06:33:41 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ywu120766/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Get Data using ArcGIS Server Services Directory REST API]]></title>
            <link>https://ywu120766.medium.com/get-data-using-arcgis-server-services-directory-rest-api-597bc2c43ebd?source=rss-3122a7fdf33------2</link>
            <guid isPermaLink="false">https://medium.com/p/597bc2c43ebd</guid>
            <category><![CDATA[arcgis]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[servers]]></category>
            <category><![CDATA[api]]></category>
            <category><![CDATA[rest-api]]></category>
            <dc:creator><![CDATA[Yanan Wu]]></dc:creator>
            <pubDate>Tue, 10 Dec 2024 16:44:57 GMT</pubDate>
            <atom:updated>2024-12-10T16:44:57.316Z</atom:updated>
            <content:encoded><![CDATA[<p>The <strong>ArcGIS Server Services Directory REST API</strong> is a web-based interface that allows users to interact with and access geographic services hosted on an ArcGIS Server. It provides a way to access maps, geoprocessing tasks, and other resources over the web, using a standard web browser or programming interfaces.</p><h3>What is it?</h3><p>The <strong>Services Directory</strong> is a RESTful API that exposes various ArcGIS Server resources as endpoints. These endpoints allow users to query data, retrieve maps, run analyses, and interact with GIS resources programmatically.</p><h3>Data Source</h3><p><a href="https://www.fs.usda.gov/">Forest Service U.S Department of Agriculture</a> has a <a href="https://apps.fs.usda.gov/arcx/rest/services/EDW/EDW_MTBS_01/MapServer">MapServer</a> that hosts many layers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dZSrzkMmnLKuH_546tuT8g.png" /></figure><h3>Layers</h3><p>Select <strong>2024 Burned Area Boundaries (82)</strong> as the sample data to browse the information</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Tx1933LAZ8sOfUri-O9AYg.png" /></figure><h3>Query Section</h3><p>Scroll down to the Query Section</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dWEh0jo064glZE0WUjeEnA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*71DewWM3tQz4r-EfuAhg-g.png" /></figure><h3>Get Data</h3><p>You can use the Query GUI to get data or use the code the get data.</p><pre>#copy url of the query page<br><br>url = &quot;https://apps.fs.usda.gov/arcx/rest/services/EDW/EDW_MTBS_01/MapServer/82/query&quot;</pre><h3>Define the query parameters</h3><pre>params = {<br>&quot;where&quot;: &quot;1=1&quot;, # Select all features<br>&quot;outFields&quot;: &quot;*&quot;, # Get all fields<br>&quot;returnGeometry&quot;: &quot;true&quot;, # Include geometry<br>&quot;f&quot;: &quot;geojson&quot; # Output format (GeoJSON, JSON, or PBF if supported)<br>}</pre><h3><strong>Send the GET request</strong></h3><pre>response = requests.get(url, params=params)</pre><h3>Save the response to a file</h3><pre>if response.status_code == 200:<br>  with open(&quot;burn.geojson&quot;, &quot;w&quot;) as file:<br>  file.write(response.text)<br>  print(&quot;Data downloaded successfully!&quot;)<br>else:<br>  print(&quot;Error:&quot;, response.status_code, response.text)</pre><p><strong><em>Thank you for reading until the end. :)</em></strong></p><h3>Colab:</h3><p><a href="https://colab.research.google.com/gist/gisynw/7068aa164ba59729911932f6622dcfe4/leafmap_geojson.ipynb">Google Colab</a></p><h3>Gist:</h3><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a62e2ec3dab0f95be5a75efc8c717581/href">https://medium.com/media/a62e2ec3dab0f95be5a75efc8c717581/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=597bc2c43ebd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Publish and update web map on ArcGIS Online Using ArcGIS API for Python]]></title>
            <link>https://ywu120766.medium.com/publish-a-web-map-on-arcgis-online-using-arcgis-api-for-python-a7b3dc639ed7?source=rss-3122a7fdf33------2</link>
            <guid isPermaLink="false">https://medium.com/p/a7b3dc639ed7</guid>
            <dc:creator><![CDATA[Yanan Wu]]></dc:creator>
            <pubDate>Thu, 22 Feb 2024 04:52:07 GMT</pubDate>
            <atom:updated>2025-03-17T23:27:22.712Z</atom:updated>
            <content:encoded><![CDATA[<p>This tutorial outlines the steps in collecting and updating police incident data from the API provided by the City of Dallas using the dynamic dashboard.</p><p><a href="https://www.arcgis.com/apps/dashboards/009d082a54e74019a1568b634869d773">Final Dashboard</a></p><p><strong>Please import the Python libraries.</strong></p><pre>import os<br>import arcpy<br>import pandas as pd<br>from arcgis.gis import GIS,User<br>from arcgis.mapping import WebMap<br>from IPython.display import display<br>from datetime import datetime, timedelta<br>from arcgis.features import FeatureLayerCollection</pre><p><strong>Connect to ArcGIS Online.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/716/1*L5LVSuFertbJcGYAMfPwGg.png" /></figure><p><strong>Publish a feature service from a shapefile</strong></p><p>To make a shapefile accessible, begin by uploading the zipped shapefile to the Portal as an item. Then, utilize the publish() method on the item to generate a web layer.</p><pre>shapefile_path = os.path.join(&#39;./update_Data/shapefile/tracts_28days.zip&#39;)<br>added_item = my_gis.content.add({}, data = shapefile_path,folder=&#39;Dallas Police Incidents&#39;)<br>published_item = added_item.publish()</pre><p>Next, you can use the same function to add other layers.</p><p><strong>Updating features in a feature layer</strong></p><p>As content publishers, you might need to maintain the accuracy of certain web layers over time. With the arrival of new data, tasks such as updating existing ones may become necessary.</p><p>Overwriting feature layers</p><p>1. Use search() to identify the existing feature that needs to be updated</p><p>2. Use fromitem() method returns the layer at the specified index</p><p>3. Overwrite the feature layer using the new file we just created</p><pre>updated_tracts_path = &#39;.\\update_Data\\shapefile\\tracts_28days.zip&#39;<br><br>search_results = my_gis.content.search(query=&quot;title:tracts_28days&quot;, item_type = &#39;Feature Layer&#39;)[0]<br>incts_collection = FeatureLayerCollection.fromitem(search_results)<br>incts_collection.manager.overwrite(updated_tracts_path)</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a7b3dc639ed7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Simple and Straightforward Classification with Extreme Gradient Boosting — XGboost]]></title>
            <link>https://ywu120766.medium.com/simple-and-straightforward-classification-with-extreme-gradient-boosting-xgboost-e88257dbe9a7?source=rss-3122a7fdf33------2</link>
            <guid isPermaLink="false">https://medium.com/p/e88257dbe9a7</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[supervised-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[gis]]></category>
            <category><![CDATA[classification]]></category>
            <dc:creator><![CDATA[Yanan Wu]]></dc:creator>
            <pubDate>Mon, 25 Sep 2023 21:24:33 GMT</pubDate>
            <atom:updated>2023-09-25T21:24:33.146Z</atom:updated>
            <content:encoded><![CDATA[<h3>Simple and Straightforward Classification with Extreme Gradient Boosting — XGboost</h3><h3>Basics of Extreme Gradient Boosting for Effective Classification</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*J2NF2mV0lDvdX0zb" /><figcaption>Photo by <a href="https://unsplash.com/@dulcineiadias?utm_source=medium&amp;utm_medium=referral">Dulcineia Dias</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Before you start this blog, I assume that you know the details of <a href="https://ywu120766.medium.com/gradient-boost-classification-bc1faf0a3dbe">Gradient Boosting.</a></p><p>XGboost is a tree-based algorithm in the supervised branch of Machine Learning. Unlike Gradient Boosting, XGBoost builds trees using the similarity score and Gain to determine the best node splits.</p><p><strong>Step 1:</strong> Input dataset {xᵢ, yᵢ} from i = 1 to n = 4.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/726/1*z08SywqhyunXIo9zWAIEig.png" /><figcaption>Table 1. Input Dataset</figcaption></figure><p><strong>Step 2.</strong> We assign the <em>initial prediction</em> as 0.5. <strong>0.5 is the default value</strong> in the XGBoost module. Thus, the <em>initial residual</em> is the difference between observing y and y_initial.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/924/1*quM8vQnejbQDtJfuiHCtBg.png" /></figure><h3>XGBoost builds trees based on residual and uses the best split to divide samples</h3><p><strong>Step 3.</strong> We use different thresholds to split the sample and then use the <em>Similarity Score</em> and <em>Gain</em> to measure the performance. The best split is the <strong><em>split with the highest gain</em></strong>. We have the <strong>regularization parameter λ</strong> in the formula. This is used to limit the model’s ability to learn and avoid overfitting. We temporarily set up it as 0. Now let’s measure the performance of two different splits in Figure 1 and Figure 2.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/967/1*MTQFvH1tPOi7YgJ8zQgqfA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/689/1*kA5LghXIic0sxTbLtL8R6g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/616/1*xNHwC17Ru6P5-DOdLi1s1w.png" /><figcaption>Figure 1. Threshold, Temperature &gt; 70</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/763/1*FFWzk79wCPFCEPHxzKUbFQ.png" /><figcaption>Figure 1–1. Measurement for Threshold &gt; 70</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/619/1*X2F-bT1cNS8ubRAr7KSqXg.png" /><figcaption>Figure 2. Threshold, Temperature &gt; 78</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/741/1*i0qLyQF97vtIa_R0TAkXRg.png" /><figcaption>Figure 2–1. Measurement for Threshold &gt; 78</figcaption></figure><p>As you can see in Figure 1–1 and Figure 2–1, the <strong>Threshold &gt; 70 has the highest gain</strong>, which means this split is better. So XGBoost will use this split to divide the samples. You can try to change the threshold for the first root, and then calculate the gain for Humility &gt; 30 and Humility &gt; 5 to see which one is best!</p><p><strong>Step 4：</strong>Prune the tree — using γ or λ</p><p>The goal of pruning the tree is to avoid overfitting. Overfitting: The model has good performance on the training dataset but bad performance on the testing dataset.</p><p>Let’s learn how to use <strong>γ</strong> to <strong>prune the tree</strong>.</p><p>γ means the <strong>minimum acceptable node gain. </strong>Take Figure 1 as an example, if we set up γ as 3. Then the difference between γ and gain is gain — γ = 2.67–3 = -0.43. In this case, we will remove the branch because the difference is negative. so the tree will look like Figure 3. In other words, if we set up λ as 2. The difference is 0.67. Then we will keep this branch.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/355/1*UArxxPtnyNiMDDEnjjVpgw.png" /><figcaption>Figure 3. Using λ=2 to prune the tree</figcaption></figure><p>λ is a regularization parameter, it is used to reduce the prediction’s sensitivity to isolated observations.</p><p>Say the λ is 1 and we have the new <em>Similarity</em> <em>Score </em>for each leaf and the new <em>Gain</em> for the node. From Figure 4, you can see that when λ = 1, the <em>Similarity</em> <em>Scores</em> and <em>Gain</em> are smaller.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/719/1*p1WtiTwAtiEFM17NEfxIbA.png" /><figcaption>Figure 4. New Similarity and New Gain when λ = 1</figcaption></figure><h3>Change in λ influences the rule about γ to prune the tree</h3><p>When we increase λ to 1, the new Gain is 0.73, which is smaller than γ = 2. So compared to the previous result, we need to remove the branch because now the difference is negative: 0.73–2 = -1.36. So this is a trick that when you tune the hyperparameter for XGBoost.</p><p>Congratulations! Now you have understood how the XGBoost uses the best split to divide the samples, and how the two hyperparameters λ and γ to prune the tree! :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e88257dbe9a7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Gradient Boost — Classification]]></title>
            <link>https://ywu120766.medium.com/gradient-boost-classification-bc1faf0a3dbe?source=rss-3122a7fdf33------2</link>
            <guid isPermaLink="false">https://medium.com/p/bc1faf0a3dbe</guid>
            <category><![CDATA[classification]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[supervised-learning]]></category>
            <dc:creator><![CDATA[Yanan Wu]]></dc:creator>
            <pubDate>Mon, 25 Sep 2023 01:33:19 GMT</pubDate>
            <atom:updated>2023-09-25T15:04:56.687Z</atom:updated>
            <content:encoded><![CDATA[<h3>Gradient Boost — Classification</h3><h3>Simple and Straightforward for beginner</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*s0O_x4Se97U3MLpL" /><figcaption>Photo by <a href="https://unsplash.com/@ikarishinjigao?utm_source=medium&amp;utm_medium=referral">Robbin Huang</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><strong>Input data:</strong></p><ol><li>(xᵢ,yᵢ) from i=1 to n, xᵢ is the x variable (Near school, Build Year, Parking lot) in the dataset, and yᵢ is the dependent variable (Price) in the dataset.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/689/1*PNf-1grv7Sc_q8wIpcBIIQ.png" /><figcaption>Table 1. House Price Sample Dataset</figcaption></figure><p><strong>Step 1:</strong> Gradient Boost construct each tree based on the relationship between residuals and x variables. So we must establish a <em>Pseudo Residual (initial prediction) </em>for the first constructed tree. In the classification task, it will be the log(odds) of the dependent variable (Figure 1). Subsequently, we convert the log(odds) to probability (Figure 2).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/748/1*UI6kGcLDllyeQyavo_pmGg.png" /><figcaption>Figure 1. Pseudo Residual in log(odds)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/569/1*Hfxb0xwYP-GSPDjd2LJOAQ.png" /><figcaption>Figure 2. Resudo Residual in probability</figcaption></figure><p><strong>Step 2.</strong> The whole processes from 2–1 to 2–4 are iterated <em>K</em> times. <em>K</em> denotes the number of trees you want to create. Below is an example of creating a single tree.</p><p><strong>Step 2–1</strong>. For each sample, compute the difference between the observed value and <em>pesudo residual</em>. The goal of this step is to update the residual.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1001/1*tHxPi_JExBQs-KHJExWZGA.png" /><figcaption>Table 1. Difference between y initial and r initial</figcaption></figure><p><strong>Step 2–2</strong>. Fit a tree for r₁. So we use ‘<em>Near school’, ‘Build Year’, </em>and<em> ‘Parking lot’</em> to predict <em>r₁ </em>and<em> </em>label the leaves.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/589/1*yfrDp9cU3-E9QnEt8-mqUQ.png" /><figcaption>Figure 3. First Classification Tree based on residual_initial</figcaption></figure><p><strong>Step 2–3</strong>. For each leaf in the First Classification tree, calculate the output value based on the formula 1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/790/1*I9fp__oR31cDDV495gndaw.png" /><figcaption>Formula 1. Calculate the output value for each leaf</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/503/1*00ZxAPew6h0SU-LuYEhkjw.png" /><figcaption>Figure 3. Output value for each leaf</figcaption></figure><p><strong>Step 2–4:</strong> Make a new prediction for each sample and update the residual. Notice that the new prediction is in log(odds), so we need to convert it to probability after we get it (Table 2). The learning rate is used to scale the contribution from the new tree. The learning rate here is 0.1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*co7aH1lY0HEb2ujdTEIRuw.png" /><figcaption>Figure 4. Predicted y for each sample</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SyUIvNj-R3C2_j8LJj16eA.png" /><figcaption>Table 2. New predict and residual from the first classification tree</figcaption></figure><p>From Table 2, you may notice that the residual for the first observation is worse, and the residual for the second and third observations is improved. This is one reason why we build a lot of trees, and not just one.</p><p>From steps 2–1 to 2–4, we create only one tree. In the GB algorithm, it repeatedly generates more trees based on the new residual by following all steps under Step 2. For example, the second new classification tree is built based on new residual r₁.</p><p>Thank you for your reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bc1faf0a3dbe" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Divide Linestring(sf) into Equal Segments with R]]></title>
            <link>https://ywu120766.medium.com/how-to-divide-linestring-sf-into-equal-segments-with-r-935a9ebaec7a?source=rss-3122a7fdf33------2</link>
            <guid isPermaLink="false">https://medium.com/p/935a9ebaec7a</guid>
            <category><![CDATA[r]]></category>
            <category><![CDATA[geospatial-data]]></category>
            <category><![CDATA[spatial-analysis]]></category>
            <category><![CDATA[gis]]></category>
            <dc:creator><![CDATA[Yanan Wu]]></dc:creator>
            <pubDate>Sun, 27 Aug 2023 00:21:32 GMT</pubDate>
            <atom:updated>2023-08-27T00:21:32.651Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*QbNjQatJlDsVk54V" /><figcaption>Photo by <a href="https://unsplash.com/@markusspiske?utm_source=medium&amp;utm_medium=referral">Markus Spiske</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Transforming Street Segments with Flexibility and Precision</h3><p>Begin by importing LINESTRING data into R using the <em>read_sf</em>.</p><p>The sf structure consists of 683 features, which can be understood as 683 LINESTRINGs, accompanied by one field (Shape_Len)</p><pre>str = read_sf(&#39;F:\\github file\\Equally divide the street segment\\street.shp&#39;)<br>str<br></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/843/1*65A_2YaRFomf6-MbqHVw6w.png" /><figcaption>Figure 1. sf structure</figcaption></figure><p>2. Select one feature, <em>str[198,]</em>, and establish an equal distance <em>max_length</em>.</p><pre>input_lines = str[198,]<br>input_lines$length = as.numeric(st_length(input_lines))<br>max_length = 100</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/556/1*jOO0MrTzDtb4j421nMM8Og.png" /><figcaption>Figure 2. Line length of the str[198,]</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/870/1*wJT0UqcqZOIxmvtmpWOEYg.png" /><figcaption>Figure 3. Structure of the selected features (str[198,])</figcaption></figure><p>3. Identify points at 100m intervals along the LINESTRING. The total length is 547.26. So there should be 5 points. Figure 4 shows the split location on the LINESTRING.</p><pre>pts_100 = rgeos::gInterpolate(sf::as_Spatial(input_lines), seq(0, input_lines$length, max_length))<br><br>plot(input_lines$geometry)<br>plot(pts_100, add=T, col = &#39;red&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/556/1*7u9klqghtrlv2hpVzkNVtA.png" /><figcaption>Figure 4. Read cross at 100M intervals.</figcaption></figure><p>4. Organize the coordinates of 100m intervals into a data frame.</p><pre>split_nodes = data.frame(pts_100@coords)<br>split_nodes = dplyr::mutate(split_nodes,len  = c(0, rep(max_length, (length(pts_100)-1))),cumlen = cumsum(len),newID = NA)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/321/1*N3LVrI71de_gi5kFHYlbJA.png" /><figcaption>Figure 5. Dataframe of 100m interval</figcaption></figure><p>5. Retrieve the original coordinates from <em>input_lines. </em>Figure 6 shows the original coordinates of the input lines. Next, calculate the length and cumulative length based on these coordinates (Figure 7).</p><pre>coords &lt;- dplyr::select(data.frame(sf::st_coordinates(input_lines)), X,Y) <br>sf_points &lt;- st_as_sf(coords, coords = c(&quot;X&quot;, &quot;Y&quot;), crs = st_crs(input_lines))<br>#plot input lines and its original coordinates<br>plot(sf_points$geometry, col = &#39;red&#39;)<br>plot(input_lines, add=T)<br><br>coords &lt;- dplyr::mutate(coords,len  = sqrt(((X - (lag(X)))^2) + (((Y - (lag(Y)))^2))))<br>coords[[&#39;len&#39;]] = ifelse(is.na(coords[[&#39;len&#39;]]), 0, coords[[&#39;len&#39;]])<br>coords = dplyr::mutate(coords, cumlen = cumsum(coords[[&#39;len&#39;]]), newID = &quot; &quot;)<br>  <br>colnames(coords) = c(&quot;x&quot;, &quot;y&quot;,&quot;len&quot;, &quot;cumlen&quot;, &quot;newID&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/541/1*y5I0S0kGOnJb2dhOzE58jg.png" /><figcaption>Figure 6. Original Coordinates of input_lines</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/392/1*m0kbMUX4hscQe31yamWGTg.png" /><figcaption>Figure 7. Length and cumulative length based on original coordinates of input_file</figcaption></figure><p>6. Combine the split_nodes and coords data frames using row binding. The merged data frame will be sorted based on cumulative length. Rows with &#39;NA&#39; values in the newID column represent the newly created coordinates based on the 100m interval. (Figure 8)</p><pre>  total_pts = data.frame(rbind(split_nodes[2:nrow(split_nodes),], coords))<br>  total_pts02 = total_pts[order(total_pts$cumlen),]<br>  <br>  rownames(total_pts02) = c(1:nrow(total_pts02))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/429/1*2paBN-vOrW7EAcaVBqLF-w.png" /><figcaption>Figure 8. Merged Dataframe</figcaption></figure><p>7. Split the merged data frame so that it records the start ID and node ID of the 100m street segment. Figure 9 shows that from 1 row to 3 row in merged data frame is our first new 100m street segment, and from 3 to 6 is our second new 100m street segment, etc.</p><pre>split_points = data.frame(start_nID = c(1,which(is.na(total_pts02$newID))), stop_nID = c(which(is.na(total_pts02$newID)), nrow(total_pts02)))<br>split_points$FID = c(1:nrow(split_points))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/219/1*FFz_5HDIbJEWywOtORAnbw.png" /><figcaption>Figure 9. Record ID of 100m street segment</figcaption></figure><p>8. Create 100m street segments based on start_nID and stop_nID. Figure 10 shows that now we have 6 new LINESTRINGs (original length is 547.26. so the first 5 new LINESTRINGs are 100m street segments, while the last one is 47.26m street segments)</p><pre>new_line &lt;- function(start_stop, coords) {<br>    return(sf::st_linestring(as.matrix(coords[start_stop[1]:start_stop[2], c(&quot;x&quot;, &quot;y&quot;)])))<br>  }<br>split_lines &lt;-plyr::alply(as.matrix(split_points[,c(&quot;start_nID&quot;, &quot;stop_nID&quot;)]),<br>                            .margins = 1, .fun = new_line, coords = total_pts02)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_mFyAP0G5MA24eA5T_M6LA.png" /><figcaption>Figure 10. new LINESTRINGs</figcaption></figure><p>9. Lastly, convert the LINESTRINGs to sf objects. Upon comparing Figure 11 with Figure 3, you can observe that the new sf object now comprises six features, all equally spaced.</p><pre>sf_lines = st_sf(geometry = st_sfc(split_lines, crs = sf::st_crs(str)))<br>attr_line = data.frame(ID = c(1,2), length = c(&#39;&#39;,&#39;&#39;))<br>lines_new = st_as_sf(cbind(attr_line, sf_lines))<br>lines_new$length = as.numeric(st_length(lines_new))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/607/1*lV2UPwUEtnfypVs9qaE4Ew.png" /><figcaption>Figure 11. New sf structure</figcaption></figure><p>If you have any questions, suggestions, or experiences to share, feel free to leave a comment below. Happy mapping!</p><p>Code please find on<a href="https://github.com/gisynw/Equally-divide-the-street-segment.git"> GitHub</a></p><p>The code presented in this blog post was inspired by a project on GitHub from approximately one year ago. However, regrettably, I am unable to locate the original source at this time. If you come across this blog and recognize any similarities between the code and your own work, please kindly let me know so that I can provide proper attribution and credit. Thanks! :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=935a9ebaec7a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Ipyleaflet & IpyWidgets: Interactive Map in Jupyter Notebook]]></title>
            <link>https://ywu120766.medium.com/ipyleaflet-ipywidgets-interactive-map-in-jupyter-notebook-a6ba76586cb5?source=rss-3122a7fdf33------2</link>
            <guid isPermaLink="false">https://medium.com/p/a6ba76586cb5</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[geospatial-data]]></category>
            <category><![CDATA[visualization]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-visualization]]></category>
            <dc:creator><![CDATA[Yanan Wu]]></dc:creator>
            <pubDate>Sat, 22 Jul 2023 14:22:19 GMT</pubDate>
            <atom:updated>2024-03-13T21:39:18.838Z</atom:updated>
            <content:encoded><![CDATA[<p>A beginner&#39;s guide on interactive Geospatial Data visualization with Ipyleaflet &amp; IpyWidget</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*rOnv-qZNOH6t8Kgj" /><figcaption>Photo by <a href="https://unsplash.com/@sadswim?utm_source=medium&amp;utm_medium=referral">ian dooley</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>After you read this, you can make an interactive map with the following features:</p><ol><li>Animation trajectory</li><li>Dropdown widget</li><li>Output widget</li><li>Marker Cluster</li><li>Search</li><li>Reset</li><li>Check-box</li><li>Legend control</li></ol><h4>1. Install packages</h4><pre>##install packges<br>!pip install geopandas<br>!pip install ipyleaflet<br>!jupyter nbextension enable --py --sys-prefix ipyleaflet<br>!pip install geemap<br>!jupyter nbextension list</pre><h4>2. Import packages</h4><pre>import geopandas as gpd<br>import os, json, geemap, csv<br>import ipywidgets<br>import ipyleaflet<br>from ipyleaflet import (Map, DrawControl, basemaps,GeoData,AwesomeIcon,LayersControl, ScaleControl, FullScreenControl, WidgetControl,Marker,Popup, MarkerCluster,SearchControl,GeoJSON, AntPath,LegendControl)<br>from ipywidgets import HTML, widgets,interact,AppLayout,Layout, VBox, HBox<br>import pandas as pd<br>import numpy as np<br>import geemap</pre><h4>3. Read data</h4><pre># read shapefile<br>gps = gpd.read_file(&quot;...&quot;)<br># to geographic coordinate system<br>gps_crs = gps.to_crs(epsg = &#39;4326&#39;)<br>pickup = gpd.read_file(&quot;...&quot;)<br>pickup_crs = pickup.to_crs(epsg = &#39;4326&#39;)</pre><h4>4. Authorize access needed by Earth Engine</h4><p>This step entails accessing the Earth Engine to load the base map, leading to an output akin to Figure 1. Subsequently, you will be prompted to input the automatically generated verification code.</p><pre>import ee<br>ee.Authenticate()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6u61tjDVv9of19b4-gHC1w.png" /><figcaption>Figure 1. Input the verification code to access the Earth Engine</figcaption></figure><p>4.1 Load Basemap, <a href="https://courses.geemap.org/geemap_intro/04_using_basemaps/">more Basemap please find here</a>.</p><pre>#Locad basemap, and zoom to your research area<br>Map = geemap.geemap.Map(basemap=&#39;CartoDB.Positron&#39;,<br>center = [32.7766, -96.7969], zoom = 11,scroll_wheel_zoom= True)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B3MmFttZFZeDvUqQNVdBLw.png" /><figcaption>Figure 2. ‘CartoDB.Positron’ Basemap and focus on the input center</figcaption></figure><h4>5. Convert Geodataframe (GPS data) to leaflet geodata</h4><p>Throughout the conversion process, you possess the flexibility to define the color, radius, or other attributes of your data. Upon completion of the conversion, you can effortlessly integrate the layer into the base map.</p><pre>## convert GPS data to leaflet geodata<br>geo_data = GeoData(geo_dataframe = gps_crs,style={&#39;color&#39;: &#39;light blue&#39;, &#39;radius&#39;:2, &#39;fillColor&#39;: &#39;#3366cc&#39;, &#39;opacity&#39;:0.5, &#39;weight&#39;:1.9, &#39;dashArray&#39;:&#39;2&#39;, &#39;fillOpacity&#39;:0.6},<br>    hover_style={&#39;fillColor&#39;: &#39;red&#39; , &#39;fillOpacity&#39;: 0.2},<br>    point_style={&#39;radius&#39;: 5, &#39;color&#39;: &#39;red&#39;, &#39;fillOpacity&#39;: 0.8, &#39;fillColor&#39;: &#39;blue&#39;, &#39;weight&#39;: 3},<br>    name = &#39;Taxi Trajectory&#39;)<br>## Add leaflet geodata to basemap<br>myMap.add_layer(geo_data)<br>myMap</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/937/1*7rTrcuvQQdClolF-qhOguQ.png" /><figcaption>Figure 3. Taxi GPS data</figcaption></figure><h4>6. Convert Geodataframe (Pickup locations) to the icon.</h4><p>6.1 Represent layer using icon</p><p>This conversion is well-suited for point data that denotes the locations of individual facilities, such as parks or universities. In this specific instance, I converted the layer of pickup locations to utilize the ‘taxi’ icon. <a href="https://fontawesome.com/v4/icons/">More icons please find here.</a></p><pre># add pickup location as the icon<br>icon1 = AwesomeIcon(name=&#39;taxi&#39;,marker_color=&#39;white&#39;,icon_color=&#39;red&#39;,spin=False<br>)</pre><p>6.2 Marker Cluster: a cluster of marker</p><p>On occasions when a large number of icons need to be displayed, it becomes imperative to adopt a visually pleasing approach. To accomplish this, I employ the MarkerCluster technique, which groups icons into distinct clusters (Figure 4). The yellow cluster, labeled as ‘20’, signifies the presence of 20 points within that particular area. As you zoom in, the cluster expands, revealing more detailed information. However, the pickup location in the Northwest remains distinct from other locations and, therefore, does not belong to any clusters.</p><pre>## Marker Cluster widget<br>maker_cluster = MarkerCluster(<br>    markers = [Marker(icon=icon1, location=([i.geometry.y, i.geometry.x]), draggable = False, title = i.Name) for i in pickup_crs.iloc()],<br>    name = &#39;Pickup location&#39;<br>)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/525/1*EUa8WxssMZku6K4vkOOo6g.png" /><figcaption>Figure 4. GPS data (Blue) and pick-up locations (yellow and green circles).</figcaption></figure><h4>7 Add widgets</h4><p>7.1 Add drop-down widget</p><pre>#dropdown widget about selecting the departments<br>dropdown_pickup = widgets.Dropdown(options=pickup_crs[&#39;Name&#39;],<br>    value=&#39;Pickup 1&#39;,description=&#39;Pick up location&#39;,<br>    disabled=False,layout=widgets.Layout(width=&#39;max-content&#39;),<br>    style = style)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/382/1*303ckcslhgNHhChblM8I0A.png" /><figcaption>Figure</figcaption></figure><p>7.2 Add output widget</p><pre>## output widget<br>output_pickup = widgets.Output(layout = {&#39;border&#39;: &#39;2px solid blue&#39;})<br>##output gps widget<br>output_gps = widgets.Output(layout = {&#39;border&#39;: &#39;2px solid black&#39;})<br>with output_gps:<br>  print(&#39;Passanger Info:&#39;)</pre><p>7.3 Add reset widget</p><pre>## rest widget<br>button_reset = widgets.Button(description = &quot;Reset Map&quot;)<br>default_bounds = [[32.58153501413734, -97.2688263424698],<br>[33.04320549616557, -96.32674870575104]]</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/223/1*IBLJYeWHHrS-H7jjjqqlpw.png" /></figure><p>7.4 Add check box widget</p><pre># add check box widget<br>checkbox = widgets.Checkbox(value=False,<br>    description=&#39;Please show animation path&#39;,indent=False)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/277/1*Btn-FOMO6Rxx3uDSyWpS2g.png" /></figure><p>7.5 Merged version of the four widgets (7.1–7.4) arranged vertically</p><pre>fire_total = VBox([dropdown_pickup, output_pickup,button_reset,checkbox])<br>control_fire = ipyleaflet.WidgetControl(widget = fire_total, position = &#39;topleft&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/375/1*oqNhA2KzfooCokF3JDAUTA.png" /></figure><p>7.6 Put the output widget to the bottom right</p><pre>control_gps = ipyleaflet.WidgetControl(widget = output_gps,<br> position = &#39;bottomright&#39;)</pre><p>7.7 Add legend control</p><pre>legend = LegendControl({&quot;GPS points&quot;:&#39;blue&#39;, &quot;Pick up location&quot;:&#39;red&#39;}, <br>name=&quot;Legend&quot;, position=&quot;topright&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/192/1*8tqDG43MVzcGXSbJDFZ_Qw.png" /></figure><p>8. Create connections between widgets and layers</p><p>8.1 Selection in the drop-down widget triggers alterations in the layers.</p><pre>def dropdown_taxi_eventhandle(change):<br>  ## when the selection in dropdown widget change<br>  if change[&#39;type&#39;] == &#39;change&#39; and change[&#39;name&#39;] == &#39;value&#39;:<br>    temp = pickup_crs.loc[pickup_crs[&quot;Name&quot;] == change[&#39;new&#39;]]<br>    address = list(temp[&#39;City&#39;])[0]<br>    add_zip = list(temp[&#39;Zip&#39;])[0]<br>    ## make change to gps_crs <br>    table = gps_crs[gps_crs[&quot;pick_up&quot;] == change[&#39;new&#39;]]<br>    succ_table = table[table[&quot;passanger&quot;] == &#39;1&#39;]<br><br>    ## change reflect in output widget <br>    with output_gps:<br>      output_gps.clear_output()<br>      print(&#39;Total number of Vehicles: &#39; + str(len(set(table[&#39;Vehicle&#39;]))))<br>      print(&#39;Total number of Passangers: &#39;, str(len(set(succ_table[&#39;passanger&#39;]))))<br>    <br>    with output_pickup:<br>      output_pickup.clear_output()<br>      print(&quot;City  : {:&lt;20}&quot;.format(address))<br>      print(&quot;Zip Code : {:&lt;20}&quot;.format(add_zip))<br>    table = gps_crs[gps_crs[&quot;pick_up&quot;] == change[&#39;new&#39;]]<br><br>    table_bound = table.geometry.bounds<br><br>    ## rebound<br>    if(table.shape[0] &gt; 0):<br>       minx,maxx = table_bound[&#39;minx&#39;].min(),table_bound[&#39;maxx&#39;].max()<br>       miny,maxy = table_bound[&#39;miny&#39;].min(),table_bound[&#39;maxy&#39;].max()<br>       myMap.fit_bounds([[miny-0.005,minx-0.005],[maxy+0.005,maxx+0.005]])<br><br>    ## substitute the geodata<br>       for item in myMap.layers:<br>          if isinstance(item,ipyleaflet.leaflet.GeoData):<br>            if &#39;pick_up&#39; in item.geo_dataframe.columns:<br>              old_GeoData = item<br>              new_GeoData = GeoData(geo_dataframe = table,<br>                                style={&#39;color&#39;: &#39;blue&#39;, &#39;radius&#39;:2, &#39;fillColor&#39;: &#39;#3366cc&#39;, &#39;opacity&#39;:0.5, &#39;weight&#39;:1.9, &#39;dashArray&#39;:&#39;2&#39;, &#39;fillOpacity&#39;:0.6},<br>                                hover_style={&#39;fillColor&#39;: &#39;blue&#39; , &#39;fillOpacity&#39;: 0.2},<br>                                point_style={&#39;radius&#39;: 5, &#39;color&#39;: &#39;blue&#39;, &#39;fillOpacity&#39;: 0.8, &#39;fillColor&#39;: &#39;blue&#39;, &#39;weight&#39;: 3},<br>                                name = &#39;EVs trajectory&#39;)<br>              myMap.substitute(old_GeoData,new_GeoData)<br><br>          ## substitute the marker<br>          if isinstance(item,(ipyleaflet.leaflet.MarkerCluster,ipyleaflet.leaflet.Marker)):<br>            old_marker= item<br>            locations = [list(temp[&#39;geometry&#39;].y)[0], list(temp[&#39;geometry&#39;].x)[0]]<br>            new_marker = Marker(icon=icon1, location=locations, draggable = False, titles = str(change[&#39;new&#39;]), name = &#39;taxi&#39;)<br>            myMap.substitute(old_marker,new_marker)<br>dropdown_pickup.observe(dropdown_taxi_eventhandle)</pre><p>As an illustration, upon selecting “Pickup 3” from the drop-down widget, the layer will dynamically update, triggering a zoom-in to the precise location of “Pickup 3” and concurrently displaying the associated GPS trajectories related to “Pickup 3.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QTZ9yW1o7hOALask7GKVAQ.gif" /></figure><p>8.2 Reset the bound</p><p>By clicking the ‘Reset Map’ button, the Map layer will revert to its original view, restoring the initial state prior to any user interactions.</p><pre>## reset the boundary<br>def onclick_reset_btn(self,style = style):<br>    myMap.fit_bounds(default_bounds)<br>    with output_gps:<br>      output_gps.clear_output()<br>      print(&#39;Passanger Info:&#39;)<br><br>    for item in myMap.layers:<br>        if isinstance(item, ipyleaflet.leaflet.GeoData):<br>            if &#39;pick_up&#39; in item.geo_dataframe.columns:<br>              old_GeoData = item<br>              myMap.substitute(old_GeoData,geo_data)<br>        if isinstance(item, ipyleaflet.leaflet.Marker):<br>            old_marker = item<br>            myMap.substitute(old_marker,maker_cluster)<br>            # break<br><br>button_reset.on_click(onclick_reset_btn)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9RTxcbedB5THyqyGxrP37g.gif" /></figure><p>8.3 Dynamic path</p><p>As the GPS points remain static, the directional movement of the vehicle cannot be ascertained. However, the ‘Animation path’ feature has the capability to generate a dynamic trajectory by connecting the locations based on the temporal sequence of GPS points.</p><pre>## show the dynamic path<br>def anpath_gps_trajectory(change):<br>  if change[&#39;type&#39;] == &#39;change&#39; and change[&#39;new&#39;] == True:<br>    table = gps_crs[gps_crs[&quot;pick_up&quot;] == dropdown_pickup.value]<br>    antpath_list = []<br>    for i in set(table[&#39;Vehicle&#39;]):<br>      single_run = table[table[&#39;Vehicle&#39;] == i].sort_values(by = &#39;Time&#39;)<br>      results = single_run[[&#39;Lat&#39;, &#39;Lon&#39;]].to_numpy().tolist()<br>      antpath_list.append(results)<br>    for item in myMap.layers:<br>      if isinstance(item,ipyleaflet.leaflet.AntPath):<br>        item.locations = antpath_list<br>  if change[&#39;type&#39;] == &#39;change&#39; and change[&#39;new&#39;] == False:<br>    for item in myMap.layers:<br>      if isinstance(item,ipyleaflet.leaflet.AntPath):<br>        item.locations = []<br><br>checkbox.observe(anpath_gps_trajectory)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pDnWpS7DPHacI_D1K2Oe3g.gif" /></figure><p>Complete code please find in <a href="https://github.com/gisynw/Interactive-Map-with-ipyleaflet-and-ipywidgets.git">GitHub</a></p><p>I hope you find this helpful. Encouragement is always motivating, and I’ll strive to provide even better examples in the future.</p><p>Thank you for the instructions in <a href="https://medium.com/pier31co/how-to-create-an-animated-image-for-your-medium-publication-e1a577423b69">‘How to create an animated image in Medium’</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a6ba76586cb5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Support Vector Machine in R: Hyperplane and Maximal Margin Classifier (Part 01)]]></title>
            <link>https://ywu120766.medium.com/support-vector-machine-in-r-hyperplane-and-maximal-margin-classifier-part-01-21c3f93f751d?source=rss-3122a7fdf33------2</link>
            <guid isPermaLink="false">https://medium.com/p/21c3f93f751d</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-scienc]]></category>
            <category><![CDATA[r]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Yanan Wu]]></dc:creator>
            <pubDate>Sat, 17 Jun 2023 20:18:33 GMT</pubDate>
            <atom:updated>2023-06-19T02:03:59.728Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o1xAa0kbvYaI5vEnsT4hrw.jpeg" /><figcaption>A boundary between river and rock. Taken at Yellowstone Park — 2017</figcaption></figure><p>Support vector machine (SVM) is a machine learning algorithm designed to construct an optimal ‘hyperplane’ to separate data into different classes.</p><ol><li>What is a hyperplane?</li></ol><p>In <em>n-dimension</em> space, a hyperplane is a flat affine subspace with an <em>n-1</em> dimension. For instance, in 2-dimension space, a hyperplane is a 1-dimensional line (Figure 1). The mathematical definition of a 2-dimension hyperplane is simple and defined by the equation (1.1)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/265/1*KEcuUsAgsr948cTKgUwA1A.png" /><figcaption>1.1 Equation. A line in 2 dimensions</figcaption></figure><p>In the p-dimensional space, the dimension of the hyperplane is p-1, as illustrated in equation (1.2)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/399/1*XNhLOTlG2FK721TrhROpcg.png" /><figcaption>1.2 Equation. A line in p dimensions</figcaption></figure><p>2. Classification using a hyperplane</p><p>A hyperplane (black solid line: 2+3 * x_1 + 2 * x_2 = 0 (Equ. 1.3)) in two-dimensional space is shown in Figure 1. We have two observations in two classes — that is y_1,…,y_n ∈ { 1,-1}. Green points and red points are the points that do not satisfy (1.3), for example,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/376/1*mSF1y7GzjjAd4OwxEIK7xQ.png" /><figcaption>1.4 Equation. Red: points in class 1; Green: points in class -1</figcaption></figure><pre>## user-defined function to draw 2-d hyperplane<br>hyper_func = function(intercept, slope1, slope2, min,max, length){<br>  data_ = list()<br>  data_[[1]] &lt;- seq(min, max, length.out = length)<br>  data_[[2]] &lt;- (intercept + slope1 * data_[[1]]) / -slope2<br>  return(data_)<br>}<br><br>## randomly generate points for a specific range<br>x1 = runif(200, min=-2, max=2)<br>x2 = runif(200, min=-2, max=2)<br>input_data = cbind(x1, x2)<br>plot(x1,x2, pch = 16)<br><br>plane_func = 2+3 * x1 + 2 * x2<br>input_data = cbind(input_data, plane_func)<br>## calculate the class for each points based on which side they locate.  <br>group_data = lapply(input_data[,3], function(x) if(x&gt;0){return(&#39;red&#39;)}else{<br>  return(&#39;green&#39;)})<br>input_data = cbind(input_data, groups = unlist(group_data))<br>plot(input_data[,&#39;x1&#39;],input_data[,&#39;x2&#39;],col = input_data[,&#39;groups&#39;], pch = 16, xlab = &#39;x1&#39;, ylab = &#39;x2&#39;)<br><br>##draw a 2-D hyperplane,beta_0 = 2, beta_1 = 3, beta_2 = 2. min &amp; max = 2. Length = 200  <br>pts_plane = hyper_func(2,3,2,-2,2,200)<br>lines(pts_plane[[1]], pts_plane[[2]], col = &quot;black&quot;, lwd = 2)</pre><figure><img alt="Figure 1." src="https://cdn-images-1.medium.com/max/860/1*LOrIesuJml1xXj5UQ9aK9w.png" /><figcaption>Figure 1. The hyperplane 2+3 * x1 + 2 * x2 = 0 is shown in the black line. Red points are corresponding to 2+3 * x1 + 2 * x2 &gt; 0. Green points are corresponding to 2+3 * x1 + 2 * x2 &lt; 0.</figcaption></figure><p>3. The Maximal Margin Classifier</p><p>We can have an infinite number of hyperplanes since even a slight adjustment to the hyperplane does not intersect any of the observations in the training set. For example, three possible hyperplanes as shown in Figure 2.</p><pre>#filter some points to illustrate this sample<br>sample_pts = input_data[which(abs(as.numeric(input_data[,&#39;plane_func&#39;])) &gt; 3),]<br>plot(sample_pts[,&#39;x1&#39;],sample_pts[,&#39;x2&#39;], col = sample_pts[,&#39;groups&#39;],pch = 16, xlab = &#39;x1&#39;, ylab = &#39;x2&#39;)<br><br># draw three possible hyperplane<br>pts_plane = hyper_func(2,3,2,-2,2,200)<br>lines(pts_plane[[1]], pts_plane[[2]], col = &quot;black&quot;, lwd = 2)<br><br>pts_plane = hyper_func(1.5,1.5,2,-2,2,200)<br>lines(pts_plane[[1]], pts_plane[[2]], col = &quot;black&quot;, lwd = 2)<br><br>pts_plane = hyper_func(2.2,2.2,2,-2,2,200)<br>lines(pts_plane[[1]], pts_plane[[2]], col = &quot;black&quot;, lwd = 2)</pre><figure><img alt="We have two group points, one is presented in red color that mainly distributed at right-upper corner, and secodn group is presented in green color that mainly distributed at left-bottome corner. We have three hyperplane that can separate the two group points. And neither of them interact with any points." src="https://cdn-images-1.medium.com/max/850/1*1WyE2DBigUhb_9uu039g9Q.png" /><figcaption>Figure 2. Three possible hyperplanes (solid black lines) that separate the points into two different classes</figcaption></figure><p>The rule we employ to choose the <em>optimal hyperplane</em>, known as the <em>Maximal Margin Hyperplane (MMH)</em>, is based on maximizing the margin. The <em>margin</em> refers to the minimum distance between the points in the training set and the hyperplane (d+ is the shortest distance from the hyperplane to the closest positive point, and d- is the shortest distance from the hyperplane to the closest negative point. So <em>margin = d+ plus d-</em>).</p><p>By seeking the hyperplane with the largest margin, we aim to find the one that has the farthest minimum distance to the training points. In essence, the MMH represents the linear hyperplane that optimally separates the data. Once the MMH is identified, we can classify the testing points by determining which side of the hyperplane they fall on. This approach is commonly referred to as the <em>Maximal Margin Classifier.</em></p><pre>##This example illustrate the classification for two classes based on svm. <br>##so x variable should be numeric format, y variable should be factor. <br>svm_data = data.frame(x1 = as.numeric(sample_pts[,&#39;x1&#39;]), x2 = as.numeric(sample_pts[,&#39;x2&#39;]),groups = as.factor(sample_pts[,&#39;groups&#39;])) <br>##using svm to find the optimal hyperplane. <br>svmfit = svm(groups~x1+x2, data = svm_data, kernel = &#39;linear&#39;, cost = 5, scale = FALSE)<br><br>##generate a regular grid dots<br>make.grid = function(x, n = 75) {<br>  grange = apply(x, 2, range)<br>  x1 = seq(from = grange[1,1], to = grange[2,1], length = n)<br>  x2 = seq(from = grange[1,2], to = grange[2,2], length = n)<br>  expand.grid(x1 = x1, x2 = x2)<br>}<br>##x is the x_1 and x_2. <br>x = svm_data[,c(1,2)]<br>xgrid = make.grid(svm_data[,c(1,2)])<br>##predict class for regular grid dots<br>ygrid = predict(svmfit, xgrid)<br>##plot regular grid dots<br>plot(xgrid, col = c(&quot;green&quot;,&quot;red&quot;)[as.numeric(ygrid)], pch = 20, cex = .2)<br>##plots points in two class<br>points(svm_data, col = as.character(svm_data[,3]), pch = 19)<br>##plot support vector, svmfit$index can provide index for support vector <br>points(x[svmfit$index,], pch = 5, cex = 2, col = &#39;blue&#39;)<br><br>## get coefficients of optimal hyperplane (beta_0 (intercept), beta_1 and beta_2)<br>coeffis = coef(svmfit)<br>## draw optimal hyperplane<br>pts_plane = hyper_func(coeffis[1],coeffis[2],coeffis[3],-2,2,200)<br>lines(pts_plane[[1]], pts_plane[[2]], col = &quot;black&quot;, lwd = 2)<br>## draw margin for negative class, so intercept plus 1. Why plus 1? See in Equation 1.5<br>pts_plane = hyper_func(coeffis[1]+1,coeffis[2],coeffis[3],-2,2,200)<br>lines(pts_plane[[1]], pts_plane[[2]], col = &quot;black&quot;, lwd = 2, lty = 3)<br>## draw margin for positive class, so intercept minus 1. Why minus 1? See in Equation 1.5<br>pts_plane = hyper_func(coeffis[1]-1,coeffis[2],coeffis[3],-2,2,200)<br>lines(pts_plane[[1]], pts_plane[[2]], col = &quot;black&quot;, lwd = 2, lty = 3)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/825/1*ZT9FaJ_gve0cypjaDJ4Y_w.png" /><figcaption>Figure 3. Optimal hyperplane, maximal margin, plane, and support vector</figcaption></figure><p>In Figure 3, we observe three points positioned at equal distances from the maximal margin hyperplane, aligning with the dashed lines. These three points are <em>support vectors</em> to ‘support’ the maximal margin hyperplane (each class must have at least one <em>support vector</em>). In other words, any minor change in the position of a <em>support vector</em> would consequently shift the optimal hyperplane, while changes in the other points would not affect the hyperplane</p><p>The two dashed lines (<em>plane</em>: H_1 and H_-1)are the boundary of the optimal hyperplane.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/319/1*m-c9P3j1UALgRXdBjpB07Q.png" /><figcaption>1.5 Equation. Plane for H_1 and H_-1</figcaption></figure><p>So all the points in the positive class satisfy</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/299/1*m5pVSfdVuEkaID41k8wfTw.png" /><figcaption>1.6 Equation. Points in Class 1</figcaption></figure><p>All the points in the negative class</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/323/1*gZgYa44JmPnUrywHlcUa8A.png" /><figcaption>1.7 Equation. Points in Class -1</figcaption></figure><p>Code on <a href="https://github.com/gisynw/Medium-Hyperplane-and-Maximal-Margin-Classifier/tree/main">GitHub</a></p><p>Coming soon…..</p><p>SVM in R: Support Vector Classifiers (Part 02)</p><p>SVM in R: Support Vector Machines (Part 03)</p><p>Thank you for reading!</p><p>Credits</p><p>Chapter 09 in Gareth, J., Daniela, W., Trevor, H., &amp; Robert, T. (2013). An introduction to statistical learning: with applications in R. Spinger.</p><p>Part of the code is from <a href="https://www.datacamp.com/tutorial/support-vector-machines-r">here</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=21c3f93f751d" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>