<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Thomson Comer on Medium]]></title>
        <description><![CDATA[Stories by Thomson Comer on Medium]]></description>
        <link>https://medium.com/@thomcom?source=rss-f68431a3f25a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Stories by Thomson Comer on Medium</title>
            <link>https://medium.com/@thomcom?source=rss-f68431a3f25a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 23:12:13 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@thomcom/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Accelerating Geographic Information Systems (GIS) Data Science with RAPIDS cuSpatial and GPUs]]></title>
            <link>https://medium.com/rapids-ai/acclerating-gis-data-science-with-rapids-cuspatial-and-gpus-fd012b27af0a?source=rss-f68431a3f25a------2</link>
            <guid isPermaLink="false">https://medium.com/p/fd012b27af0a</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[gis]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[open-source-software]]></category>
            <category><![CDATA[spatial]]></category>
            <dc:creator><![CDATA[Thomson Comer]]></dc:creator>
            <pubDate>Tue, 21 Jul 2020 20:20:59 GMT</pubDate>
            <atom:updated>2020-07-22T22:17:08.113Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*015rU_lQzWd-DLzS" /></figure><p>The economic impact of Geographic Information Systems (GIS) has <a href="https://www.geospatialworld.net/blogs/gis-and-spatial-analytics-market/">doubled since 2013</a>. <a href="https://opendata.cityofnewyork.us/">Civic data</a>, <a href="https://github.com/datasets/covid-19">pandemic information</a>, and <a href="https://waymo.com/open/">autonomous vehicle information</a> are creating vast databases in need of understanding. Our goal at NVIDIA is to bring GPU-accelerated solutions to the most often used aspects of GIS data science. For the last nine months, we’ve been working on a GIS project called <a href="https://github.com/rapidsai/cuspatial">RAPIDS cuSpatial</a> to bring massive performance to these kinds of problems.</p><p>cuSpatial is an efficient C++ library accelerated on GPUs using <a href="https://developer.nvidia.com/cuda-zone">NVIDIA CUDA</a> and <a href="https://github.com/rapidsai/cudf">RAPIDS cuDF</a>, an open-source GPU DataFrame library that includes Python bindings for use by the data science community. cuSpatial provides significant GPU-acceleration to common spatial and spatiotemporal operations such as point-in-polygon tests, distances between trajectories, and trajectory clustering. Speed-ups in comparison with CPU-based implementations range from 500x to 10,000x depending on the operation.</p><p>A few more important use cases of GIS processing include <a href="https://github.com/rapidsai/plotly-dash-rapids-census-demo">epidemiology and contact tracing</a>, autonomous vehicles, civil planning, and research. During the COVID-19 pandemic, substantial new data sets are being generated based on location, position tracking, and contact lists. Producing important epidemiological information from these data sets depends upon many of the functions that cuSpatial provides, such as proximal distance clustering, trajectory grouping, geofencing, and point in polygon. Scientists and physicians are providing epidemiological GIS data to GPU researchers, producing meaningful insights with a critically faster turnaround.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/0*Nm-D7wCnzQ-pxMaY" /><figcaption>Figure 1: New York State Epidemiology</figcaption></figure><h3>Latest cuSpatial Features</h3><p>Previously we reduced the clustering time of over 10,000 trajectories with the <a href="https://medium.com/rapids-ai/releasing-cuspatial-to-accelerate-geospatial-and-spatiotemporal-processing-b686d8b32a9">release of cuSpatial</a>. We’ve been hard at work, accelerating new problems and refining old ones. This blog introduces the addition of cubic spline fitting to trajectories and other curves, quadtree indexing, and refinements and updates to cuSpatial Python APIs.</p><h3>Spatial Distance and Inclusion Testing</h3><p>cuSpatial builds on top of the <a href="https://github.com/rapidsai/cudf">RAPIDS cuDF </a>data frame library. Put your data into cudf.DataFrame or cudf.Series to load it onto the GPU before calling cuSpatial functions. The following functions demonstrate some of the most common use cases of cuSpatial and the performance gains you can expect.</p><ul><li>Point in Polygon Testing</li><li>Haversine Distance</li><li>Directed Hausdorff Distance</li><li>Cubic Splines for Trajectory Interpolation​</li><li>Quadtree Indexing</li></ul><h3>Point in Polygon Testing</h3><p>With cuSpatial 0.14, performance gains of up to 10,000x over CPU processing can be found in the point_in_polygon function, which can test up to 30 billion points per second for inclusion in multiple polygons on an NVIDIA GV100 GPU. Future performance metrics apply to this hardware.</p><p>The easiest way to load polygon data for point_in_polygon is to use cuspatial.read_polygon_shapefile. Given an input shapefile containing only polygon features, the resulting tuple poly_offsets, poly_ring_offsets, poly_points perfectly matches the input requirements of point_in_polygon. See <a href="https://github.com/rapidsai/cuspatial/blob/branch-0.14/notebooks/nyc_taxi_years_correlation.ipynb">this demo notebook</a> for more details.</p><pre>cuspatial.point_in_polygon(points_x, points_y,<br>                           poly_offsets,poly_ring_offsets,<br>                           poly_points_x,poly_points_y)</pre><p>The return value of point_in_polygon is a cudf.DataFrame with a column per polygon. Each column contains a Boolean mask of each input coordinate pair from points_coords representing whether or not that coordinate is inside the corresponding polygon.</p><h3>Haversine Distance</h3><p>cuSpatial can compute <a href="https://en.wikipedia.org/wiki/Great-circle_distance">great circle distances</a> on enormous datasets with full parallelism, which enables a straightforward calculation of haversine distance between datasets of 10 billion lat/lon coordinate pairs per second using a single GPU. This is limited only by GPU memory. Or increase throughput arbitrarily using <a href="https://github.com/rapidsai/dask-cudf">dask-cudf</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/220/0*p94YpEu1uSH0Eypo" /><figcaption>Figure 2: Haversine distance</figcaption></figure><pre>cuspatial.haversine_distance(<br>    cudf.Series([0, 0, 0]), #longitude p0<br>    cudf.Series([0, 0, 0]), #latitude p0<br>    cudf.Series([45, 46, 47]), #longitude p1<br>    cudf.Series([45, 46, 47]), #latitude P1<br>)</pre><h3>Directed Hausdorff Distance</h3><p>Hausdorff distance is the <a href="https://en.wikipedia.org/wiki/Hausdorff_distance">maximum minimum distance between two spaces</a>. It is an essential metric in path similarity, image processing, and computer graphics. Hausdorff Distance and <a href="https://arxiv.org/abs/1508.04904">Symmetric Segment Path-Distance</a> (SSPD) are useful for comparing the similarity of sets of trajectories and geometries. cuSpatial’s implementation of this algorithm enables path comparisons on previously intractable set sizes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/349/0*PTSb7s3eOXNrYQvs" /><figcaption>Figure 3: Hausdorff Distance</figcaption></figure><pre>cuspatial.directed_hausdorff_distance(<br>    cudf.Series([0, 0, 0, 1, 1, 1, 1]), # points x<br>    cudf.Series([1, 1, 1, 2, 2, 2, 2]), # points y<br>    cudf.Series([3, 4]), # size of each set<br>)</pre><h3>Cubic splines for trajectory interpolation​</h3><p>cuSpatial accelerates cubic spline interpolation over large sets of splines with extremely high performance on the GPU. This can be used to interpolate cubic splines based on four control point curves that pass through each start and end control point, through the two interior control points, and maintain a smooth second derivative. It is particularly useful for computing smooth vehicular trajectories for prediction, position estimation, and autonomous vehicle control.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/378/0*0Fioe9QfpkFnyJpI" /><figcaption>Figure 4: Uninterpolated curve samples</figcaption></figure><pre>Trajectory t = {{0, 3}, {1, 2}, {2, 3}, {3, 4}, {4, 3}}​</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/378/0*kYSAtNNHL7RuC8FX" /><figcaption>Figure 5: Interpolated curve from Figure 4 samples.</figcaption></figure><p>Here’s some example code that interpolates two trajectories of length 5 using cuSpatial. This code extends easily out to 1 million curves with a minimal increase in runtime. The input values soa_t and soa_x represent the source sample points in Figure 4. The final result smoothed is the smooth curve for both curves after interpolating the input values sequence_to_interpolate.</p><pre>import cudf<br>import cupy as cp<br>import cuspatial</pre><pre>length = 5<br>number = 2<br>soa_t = cudf.Series(cp.tile(cp.arange(length), number))<br>soa_x = cudf.Series(cp.tile(cp.random.random(length), number))<br>cuspatial_splines = cuspatial.interpolate.CubicSpline(soa_t, soa_x,<br>    size=length)<br>sequence_to_interpolate = cudf.Series(cp.tile(cp.linspace(<br>    0, length, 10), number))<br>curve_ids = cudf.Series(cp.repeat(cp.arange(number), 10))<br>smoothed = cuspatial_splines(sequence_to_interpolate, curve_ids)</pre><p>In the future, we aim to extend support to mixed splines of arbitrary length.</p><h3>Quadtree Indexing</h3><p>cuSpatial is in the process of building support for high-performance quadtree spatial indexing. cuSpatial 0.14 adds support for building the quadtree structure on position data, and future releases will use this data structure for spatial indexing and spatial refinement for large numbers of points. This will support points that may lie inside multiple polygons, and it substantially improves performance over cuSpatial’s current point_in_polygon function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*YYlWaYCb0ghhmwTc" /><figcaption>Figure 6: Dividing groups of coordinates into quadrants.</figcaption></figure><h4>cuSpatial’s quadtree spatial indexing uses a three-phase approach:</h4><ol><li>Compute all non-empty quadrants at the finest level​.</li><li>Construct the quadtree by removing non-qualified nodes​.</li><li>Perform polygon or coordinate indexing into the quadtree in log(n) time.</li></ol><p>Spatial indexing with a quadtree allows point-in-polygon testing to be performed only on point/polygon pairs that fall within the same bounding boxes, providing much higher efficiency for spatial queries.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/727/0*Sk1PMGBZvog8xLAs" /><figcaption>Figure 7: Quadtree-accelerated point-in-polygon</figcaption></figure><blockquote>See: <a href="http://www.adms-conf.org/2019-camera-ready/zhang_adms19.pdf">Efficient Quadtree Construction for Indexing Large-Scale Point Data on GPUs: Bottom-Up vs. Top-Down</a></blockquote><h3>Conclusion</h3><p>cuSpatial accelerates spatial distance and similarity functions, indexing, and trajectory smoothing. Combined with dataframe join methods provided by cuDF, forms of spatial indexing like range and grid search are trivial. If these methods are part of your workflow, using cuSpatial will give immediate results. GPU acceleration can turn day-long runtimes into minutes and hours into seconds.</p><p>Stay tuned as cuSpatial gets faster, easier to use, and more relevant for computing solutions in complex spatial datasets. We’re already in the works on <a href="https://arxiv.org/abs/1508.04904">SSPD distance</a> functions, even faster indexing, and seamless I/O compatibility with geoPandas.</p><p>Try out the <a href="https://github.com/rapidsai/cuspatial">notebooks and demos</a>, and get started using cuSpatial today. If you’d like to request a new feature, please <a href="https://github.com/rapidsai/cuspatial/issues/new/choose">file an issue</a> or hit us up at <a href="https://join.slack.com/t/rapids-goai/shared_invite/zt-2qmkjvzl-K3rVHb1rZYuFeczoR9e4EA">RAPIDS-GoAI</a> today.</p><h3>The cuSpatial Team</h3><p>There are numerous engineers part of the RAPIDS community contributing to the cuSpatial project. Below are some of the top contributors. Thank you all for your hard work and dedication.</p><ul><li><a href="https://medium.com/u/f68431a3f25a">Thomson Comer</a><em>, Sr. System Software Engineer for RAPIDS, NVIDIA</em></li><li><em>Dr. </em><a href="https://medium.com/u/cef041e1ddd2">Jianting Zhang</a><em>, Associate Professor in GIS and Computer Science, City College of New York. Visiting faculty member, NVIDIA</em></li><li><em>Christopher Harris, Sr. System Software Engineer for RAPIDS, NVIDIA</em></li><li><em>Paul Taylor, Sr. System Software Engineer for RAPIDS, NVIDIA</em></li><li><a href="https://medium.com/u/e87badfef277"><em>Mark Harris</em></a><em>, Distinguished Engineer, NVIDIA</em></li><li><a href="https://medium.com/u/92f7c6758a3a">Keith Kraus</a><em>, Sr. System Software Engineering Manager for RAPIDS, NVIDIA</em></li><li><a href="https://medium.com/u/89249218693f">Sujit Biswas</a><em>, Principal Engineer for IVA, NVIDIA</em></li><li><em>Dr. Milind </em><a href="https://medium.com/u/65fe9db3444a"><em>Naphade</em></a><em>, CTO for IVA and AI Cities, NVIDIA</em></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fd012b27af0a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/rapids-ai/acclerating-gis-data-science-with-rapids-cuspatial-and-gpus-fd012b27af0a">Accelerating Geographic Information Systems (GIS) Data Science with RAPIDS cuSpatial and GPUs</a> was originally published in <a href="https://medium.com/rapids-ai">RAPIDS AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tutorial for Machine Learning Reporting]]></title>
            <link>https://medium.com/@thomcom/tutorial-for-machine-learning-reporting-185cc74f197f?source=rss-f68431a3f25a------2</link>
            <guid isPermaLink="false">https://medium.com/p/185cc74f197f</guid>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Thomson Comer]]></dc:creator>
            <pubDate>Wed, 25 Apr 2018 03:33:30 GMT</pubDate>
            <atom:updated>2018-04-25T16:22:08.238Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/470/1*wYMnC-aY1CksH8HyekZe3g.png" /></figure><p>Start by storytelling. Who are you and what are you doing. Why is what you are doing interesting?</p><h3>ML Research Reporting</h3><p>I learned/researched/discovered/implemented something new in my education/study/curiousity about Machine Learning this week. I found it interesting because of this and that.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4fttlugzZEWT1SBGQFO1gA.png" /><figcaption>By Kenneth Jensen [CC BY-SA 3.0 (<a href="https://creativecommons.org/licenses/by-sa/3.0">https://creativecommons.org/licenses/by-sa/3.0</a>)], via Wikimedia Commons</figcaption></figure><p>I started implementing it this way.</p><pre># LAMBDA SCHOOL<br>#<br># MACHINE LEARNING<br>#<br># MIT LICENSE</pre><pre>code.sample(&#39;data&#39;)</pre><p>Here are some interesting anecdotes I encountered along the way: Example of what makes programming hard/tool useful/implementation worked first time or had bugs. Another anecdote.</p><pre># Great code, leading up to</pre><pre>plt.plot(x,y)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6hKg-hKG0plkYH3m52i18g.png" /><figcaption>Figure 1: Overlay of data and function fit.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/130/1*Ofm7QwOt6clLIXrr6R3o9w.png" /><figcaption>Equation 1: g = x^{1/\sqrt(3)}</figcaption></figure><h3>Conclusion</h3><p>I love studying Machine Learning. I’m learning about X and Y and looking forward to changing the world in the domain of A, B, and C.</p><h3>References and Links</h3><ul><li><a href="http://datavizcatalogue.com/">Data Viz Catalogue</a></li><li><a href="http://datavizproject.com/">datavisproject.com</a></li><li><a href="https://python-graph-gallery.com/">python-graph-gallery.com</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=185cc74f197f" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>