• cuDF pandas accelerator mode now allows you to process larger-than-GPU memory workloads, as well as supports columns with two billion rows of strings.
• RAPIDS cuVS achieves better search accuracy by ensuring connected graphs, and maintains the accuracy as you add additional vectors. cuVS CPU/GPU interoperability has also been improved by providing a C++ API within cuVS for graph conversion.
• cuML now accelerates dimensionality reduction of much larger datasets with UMAP by up to 300x.

Read on for more details!

Process workloads larger than GPU memory with cuDF pandas accelerator mode

cuDF’s pandas accelerator mode — the zero-code-change accelerator for pandas workflows — can now speed up processing of much larger datasets and workloads that exceed available GPU memory. To achieve this, cudf.pandas now uses CUDA Unified Memory (managed memory) by default. Unified Memory enables CUDA programs to oversubscribe GPU memory by automatically migrating memory pages between CPU and GPU memory. To ensure high performance, cuDF now also automatically prefetches managed memory to the GPU before accessing it.

In Figure 1, the benchmark for accelerated pandas, running data processing workloads with a 10 GB dataset using cuDF achieved up to 30x speedups compared to CPU-only pandas for data joins on an NVIDIA T4 GPU with 16 GB memory. It is important to note that queries shown in Figure 1 require allocating more memory than available on the GPU. Before cuDF 24.08, this would have caused fallback to CPU-only pandas.

For more information about these benchmark results and how to reproduce them, see the cuDF benchmarks guide.

The second big improvement introduced in cuDF 24.08 is support for large string columns. Previously, dataframes could only contain 2.1 billion characters per column because indexes were based on 32-bit integers. Now cuDF dynamically switches between 32-bit and 64-bit indices when a string column’s data exceeds 2.1B characters. This allows cuDF to keep a lower memory footprint and higher processing speed and still support efficient processing for large string columns. Together, these two new features enable you to process much larger datasets with much more textual data.

Accelerating UMAP in cuML with faster knn graph building

You can now speed up dimensionality reduction and visualization of massive datasets using UMAP with the addition of a faster algorithm for building the initial knn graph. This feature adds a new `build_algo` argument to the UMAP estimator in cuML, and by default this is set to `auto` so most users don’t need to do anything to take advantage of this new capability. To force the faster graph building, `build_algo` can be set to use `nn_descent` ANN algorithm. Of course `brute_force` can still be used for great performance on the GPU when the original exact knn graph build algorithm is preferred. To start using this algorithm, you just need to set the `build_algo` parameter to `nn_descent` like below.

from cuml.manifold.umap import UMAP
data = generate_data()
umap_nnd = UMAP(build_algo = "nn_descent") # defaults to "auto"
Embedding = umap_nnd.fit_transform(data)

As shown in the above graph, the speed up can be up to 300x for a 30GB dataset on an H100 GPU (GPU Memory: 80GB) with no material impact to the embedding quality as seen by the trustworthiness score. In the upcoming release, UMAP will also be capable of processing datasets larger than GPU memory through a novel batching process. Stay tuned for that!

Seamless CPU / GPU index interoperability and improved vector search with cuVS

cuVS, the RAPIDS vector search library, has now further improved seamless execution between CPU and GPU, both for vector index building and search tasks. While CAGRA, the GPU-native graph-based approximate nearest neighbors (ANN) index, tends to have lower latency than the CPU overall, this interoperability between GPU and CPU offers a significant cost reduction by offloading the most compute-intensive parts to the GPU. For your library development work, CuVS now has a C++ API that allows converting CAGRA indexes which were built on the GPU to HNSW (loaded via HNSW API wrapper in cuVS) on CPU. You will also be able to use the C and Python-based APIs to perform the above index conversion in the future releases. The code snippet below shows a sample C++ code that you can use to convert the indexes.

#include <raft/core/device_resources.hpp>
#include <cuvs/neighbors/cagra.hpp>
using namespace cuvs::neighbors;
raft::device_resources res;
cagra::index_params index_params;
auto index_vectors = raft::make_device_matrix<float, int64_t>(res, n_vecs, dim);
// … populate index vectors
auto cagra_index = cagra::build(res, index_params, index_vectors);
auto hnsw_index = hnsw::from_cagra(res, cagra_index);

You can now also add vectors to an existing CAGRA index, another highly requested feature. Adding vectors to the CAGRA graph while maintaining search accuracy has been a challenge. Now, you can add 20% additional vectors without seeing a significant degradation in search accuracy. See the below example demonstrating how the C++ based API can be used to add vectors to an existing CAGRA index.

#include <raft/core/device_resources.hpp>
#include <cuvs/neighbors/cagra.hpp>
using namespace cuvs::neighbors;
raft::device_resources res;
cagra::index_params index_params;
auto index_vectors = raft::make_device_matrix<float, int64_t>(res, n_vecs, dim);
// … populate index vectors
auto index = cagra::build(res, index_params, index_vectors);
auto new_vectors = raft::make_host_matrix<float, int64_t>(res, n_new_vecs, dim);
// … populate new vectors
cagra::extend_params extend_params;
cagra::extend(res, extend_params, raft::make_const_mdspan(new_vectors.view()), index);

Lastly, you can improve the accuracy of the vector search by ensuring that the CAGRA graph is connected. Now, you can enable the build parameter option to guarantee that the graph is connected, ensuring comprehensive traversal across nodes and improving the search performance. See the below example of how you can set the `guarantee_connectivity` parameter in the CAGRA API as true to ensure connected graphs.

#include <raft/core/device_resources.hpp>
#include <cuvs/neighbors/cagra.hpp>
using namespace cuvs::neighbors;
raft::device_resources res;
cagra::index_params index_params;
index_params.guarantee_connectivity = true;
auto index_vectors = raft::make_device_matrix<float, int64_t>(res, n_vecs, dim);
// … populate index vectors
auto index = cagra::build(res, index_params, index_vectors);

Enhanced CUDA 12.5 support and adjustments to dependency versions

RAPIDS has a few dependency support changes that users should be aware of. For more, please see our RAPIDS Notices page.

1. RAPIDS added CUDA 12.5 support in the 24.08 release. So RAPIDS 24.08 CUDA 12 packages are now installable and usable with CUDA 12.0–12.5. Docker containers for CUDA 12.5 are also included.
2. RAPIDS deprecated support for CUDA 12.2 in its Docker containers in the 24.08 release; RAPIDS 24.08 will be the last version to release CUDA 12.2 Docker containers. More details can be found in the RAPIDS Support Notice (RSN) here.
3. RAPIDS is dropping support for Python 3.9 in its upcoming 24.10 release. RAPIDS will simultaneously add support for Python 3.12. More details can be found in the RAPIDS Support Notice (RSN) here.

Conclusion

The RAPIDS 24.08 release takes another step forward in our mission to make accelerated computing more accessible to data scientists and engineers. We can’t wait to see what people do with these new capabilities.

To get started using RAPIDS, visit the Quick Start Guide.

RAPIDS 24.08: Better scalability, performance, and CPU/GPU interoperability was originally published in RAPIDS AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Table of Contents

• Bringing accelerated vector search to everyone with cuVS
• Expanded zero code change acceleration for pandas and NetworkX code
• Dask Expressions / Query planning now supports cuDF (experimental)
• RAPIDS-wide support for Python 3.11 and pandas 2
• Free on-demand video deep dives into Data Science from GTC

Bringing accelerated vector search to everyone with cuVS

To better serve the needs of the vector search and database community, we’ve released cuVS, a new library dedicated to vector search.

cuVS contains state-of-the-art implementations of several algorithms for running approximate nearest neighbors and clustering on the GPU. It’s derived from the core algorithms in RAPIDS RAFT and purpose built to be easily embeddable in the world’s vector databases or used as a standalone tool.

As part of our work to simplify tapping into accelerated vector search, cuVS now includes both C and Rust APIs in addition to the existing C++ and Python APIs.

The example below shows how to build a vector search index with cuVS’ new C API and DLPack:

#include <cuvs/neighbors/cagra.h>

cuvsResources_t res;
cuvsCagraIndexParams_t index_params;
cuvsCagraIndex_t index;

DLManagedTensor *dataset;
load_dataset(dataset);

cuvsResourcesCreate(&res);
cuvsCagraIndexParamsCreate(&index_params);
cuvsCagraIndexCreate(&index);

cuvsCagraBuild(res, index_params, dataset, index);

cuvsCagraIndexDestroy(index);
cuvsCagraIndexParamsDestroy(index_params);
cuvsResourcesDestroy(res);

For those who prefer Rust, use the following example of building a vector search index with cuVS’ new Rust API:

use cuvs::cagra::{Index, IndexParams, SearchParams};
use cuvs::{ManagedTensor, Resources, Result};

use ndarray::s;
use ndarray_rand::rand_distr::Uniform;
use ndarray_rand::RandomExt;

/// Example showing how to index and search data with CAGRA
fn cagra_example() -> Result<()> {
    let res = Resources::new()?;

    // Create a new random dataset to index
    let n_datapoints = 65536;
    let n_features = 512;
    let dataset =
        ndarray::Array::<f32, _>::random((n_datapoints, n_features), Uniform::new(0., 1.0));

    // build the CAGRA index
    let build_params = IndexParams::new()?;
    let index = Index::build(&res, &build_params, &dataset)?;
    println!(
        "Indexed {}x{} datapoints into cagra index",
        n_datapoints, n_features
    );

    Ok(())
}

With these new C and Rust bindings, you can now integrate CAGRA and all of its goodness into nearly any codebase.

For example, the team at SearchScale have been working on bringing cuVS into Apache Lucene via the new C API to help power an ecosystem built on Java. We’re excited to see this work progress and are excited to help bring accelerated vector search to every platform.

Stay tuned for more information about cuVS in an upcoming blog.

Expanding Zero Code Change Acceleration for pandas and NetworkX code

We’re continuing to invest in making accelerated computing more accessible to data scientists and engineers who may be working with their favorite CPU-based tools and frameworks day-to-day.

cudf.pandas is now GA
cuDF’s pandas accelerator mode, the zero-code change accelerator for pandas workflows, has graduated from Beta and is now Generally Available (GA). Over the past six months, we’ve fixed dozens of bugs, improved the profiler, and expanded our comprehensive unit and integration tests suite.

Now that cudf.pandas is GA it’s fully supported by NVIDIA AI Enterprise — enabling every organization to go to production confidently. You can learn more in the cudf.pandas section of the cuDF documentation.

cuGraph Backend for NetworkX
With the 24.04 release, the cuGraph backend for NetworkX (nx-cugraph) now supports more than 60 algorithms. We continue to collaborate with the NetworkX community to both solidify the backend dispatching infrastructure for all backends and ensure that the user experience for GPU-accelerated NetworkX is smooth.

In line with that goal, NetworkX 3.3 includes support for graph caching to backends. In prior versions, each time NetworkX dispatched to cuGraph (or any other backend) the underlying graph data structure needed to be converted into something the backend could operate on (e.g., a GPU object).

Now, this graph conversion only occurs once — bringing significant performance gains to workflows that involve running multiple graph algorithms on the same underlying data.

If you’re curious about whether your favorite algorithm is supported by the cuGraph backend, the latest NetworkX documentation indicates whether an algorithm is supported by alternative backends.

To learn more about NetworkX backend dispatching in general, visit the NetworkX Backends documentation.

Dask-cuDF Now Supports Query Planning [Experimental]

We’ve been working with the Dask community to bring automatic query planning and optimization to Dask DataFrames through the dask-expr project. Beginning with the 24.04 release, Dask cuDF now supports automatic query optimization as an experimental, optional configuration.

Query optimization can improve performance and reduce peak memory requirements for a given workflow by avoiding unnecessary computation and choosing the most efficient execution path.

With query planning enabled in the example query below, both the equality filter and column selection will be pushed down into the read_parquet call, avoiding significant unnecessary IO and computation. Without query planning, you’d need to manually optimize your code to achieve the same behavior (which can get quite complex in larger workflows).

We see this play out in practice when running various analytics workflows on a 100GB dataset. Query optimization provides benefits for both single and multi-GPU workflows, resulting in both faster performance and the ability to complete larger workflows on a single-GPU system without running out of memory.

In the RAPIDS v24.06 release, we anticipate Dask cuDF will support query optimization by default rather than as an experimental, opt-in configuration.

RAPIDS-wide support for Python 3.11 and pandas 2

Python 3.11
The 24.04 release brings Python 3.11 support to RAPIDS with packages and containers now available. The RAPIDS Quick Start guide now uses Python 3.11 by default, but you can always use the Release Selector to choose your specific combination of platform dependencies.

Pandas 2
Version 2 was a major milestone for pandas, bringing significant updates like expanded PyArrow support, copy-on-write enhancements, nullable data types, and more.

Beginning with the 24.04 release, cuDF and all RAPIDS libraries now support (and require) pandas 2. As part of this upgrade, cuDF now contains numerous feature improvements, API optimizations and long-standing deprecation enforcements that simplify data analysis workflows. In addition to providing a better experience for cuDF and cudf.pandas users, these enhancements propagate to downstream libraries like cuML and cuGraph that are designed to accept both cuDF and pandas inputs.

For those interested in a full list of breaking changes compared to prior versions of cuDF tied to pandas 1.x, you can learn more in the breaking changes section of the cuDF user guide.

Free on demand video deep dives into Data Science from GTC

NVIDIA GTC took place last month and highlighted some of the incredible work going on in the world of accelerated data science and data processing. In case you missed it, you can still register for free and watch sessions about:

• Accelerated vector search and databases
• Scaling graph neural networks to trillion-edge sized graphs
• Reducing Apache Spark cloud costs with Spark RAPIDS
• Best practices for GPU memory management in Python and C++
• And so much more

You can search all of the data science and data processing sessions here.

Conclusion

The RAPIDS 24.04 release takes another step forward in our mission to make accelerated computing more accessible to data scientists and engineers. We can’t wait to see what people do with these new capabilities.

To get started using RAPIDS, visit the Quick Start Guide.

RAPIDS 24.04 Release was originally published in RAPIDS AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

RAPIDS just released its 24.02 version, which brings with it a major expansion to the new zero-code-change experience for NetworkX users, improvements to XGBoost 2.0, and improvements to cuDF’s parquet and JSON readers. Additionally, RAFT now supports more flexible vector search deployments on CPU, RMM now supports pinned host memory, and RAPIDS now works on Databricks with Dask.

24.02 also brought with it a number of changes to platform and support dependencies (including the end of Pascal architecture support) — make sure to understand if any of these impact you!

Finally, we’d be remiss not to mention the return of NVIDIA’s GTC conference in-person this year! Data science has many talks and events–take a look at the data science talks here, and we hope to see you there!

Table of Contents

• Zero code change NetworkX
• cuML + XGBoost 2.0
• cuDF Improvements
• RAFT + Vector Search
• RMM Memory Manager
• RAPIDS on Databricks with Dask
• Infrastructure and Support Updates
• NVIDIA GTC
• Conclusion

Zero code change NetworkX

nx-cugraph is a backend for NetworkX that uses cugraph to GPU accelerate NetworkX workflows, and for release 24.02, nx-cugraph expanded the number of algorithms, bringing the total to 37, with an additional 41 functions for creating test graphs. NetworkX users can install nx-cugraph to experience GPU acceleration for algorithms such as ancestors, descendants, BFS, reciprocity, triangle counting, weakly connected components, and others, all without changing their code.

cuML + XGBoost 2.0

XGBoost 2.0 is now available in our conda channel! This brings all the improvements of XGBoost 2.0 including: a simpler and consistent API for CUDA execution, multi-target trees with vector-leaf outputs, GPU acceleration of the approximate tree method, rewritten learning to rank implementation, quantile regression among many other features. This release also includes RMM integration which enables using XGBoost 2.0 with memory pools that are shared with other RMM enabled libraries (including RAPIDS, PyTorch, CuPy and others), speeding up applications and allowing better use of GPU memory across complex workflows. Alongside FIL and Triton, we continue working to improve and accelerate both training and inference of gradient boosted models with GPUs.

cuML also now fully supports multi node-multi gpu Logistic Regression with dask as well as Spark. Using cuml.dask on a single node with 4 H100, this allows processing a 60GB dataset 3.5x faster than a single H100, and also allows processing of much larger datasets without needing to spill to system memory. For example, a single T4 (with 16GB of GPU memory) can process datasets of up to 12 GB, but 4 T4s elevate that to 46GBs. Not only does this allow scaling up for bigger datasets, it enables flexibility for users doing training of Logistic Regression models even with GPUs that have lower memory footprints or very diverse configurations. Sparse data processing will be available within the next few releases.

cuDF improvements

RAPIDS 24.02 includes improvements to libcudf’s chunked parquet reader, making it possible to reduce the memory footprint of parquet reads. For 536 MB tables, the chunked reader drops memory footprint from ~2.2 to ~1.4x peak memory footprint with a 200MB chunk size. Using results from the libcudf microbenchmarks, the chunked parquet reader uses 40% less peak memory while maintaining >85% of the data throughput. Chunked parquet reading is available in the C++ API, and we’re planning to add support to cuDF-python in the future (see more in this issue).

libcudf’s JSON reader also added new options for reading mixed data types in the same key and supporting single-quoted JSON variants. Previously cuDF-python would raise an exception if a JSON key had values of varying data types:

>>> j = '{"product":{"id":12}}\n{"product":[6,9]}'
>>> b = BytesIO(j.encode('utf-8'))
>>> pd.read_json(b, lines=True)
product
0 {'id': 12}
1 [6, 9]
# pandas returns python objects
>>> cudf.read_json(b, lines=True)
RuntimeError: CUDF failure at: /nfs/repo/cudf24.02pqchunk/cpp/src/io/json/json_column.cu:629: A mix of lists and structs within the same column is not supported
# cudf raises an exception
>>> cudf.read_json(b, lines=True, mixed_types_as_string=True)
product
0 {"id":12}
1 [6,9]
# cudf returns unparsed strings, instead of crashing or losing data

RAFT + Vector Search

The graph-based approximate nearest neighbors (ANN) index CAGRA can now be trained on the GPU and moved over to host (RAM) memory to be searched on the CPU with the popular HNSW algorithm. While the CAGRA index tends to have lower latency than the CPU overall, this interoperability between GPU and CPU offers a significant cost reduction by offloading the most compute-intensive parts to the GPU.

Below is a code example using RAFT’s Python API to build a CAGRA index on the GPU and load it into an HNSW index for searching.

import cupy as cp
from pylibraft.common import DeviceResources
from pylibraft.neighbors import cagra, hnsw
n_samples = 50000
n_features = 50
dataset = cp.random.random_sample((n_samples, n_features), dtype=cp.float32)
handle = DeviceResources()
index = cagra.build(cagra.IndexParams(), dataset, handle=handle)
hnsw_index = hnsw.from_cagra(index, handle=handle)

Benchmarking index build times for RAFT’s graph-based CAGRA algorithm against the industry-standard HNSW algorithm on the CPU. Benchmarking was done using all available cores of an Intel Sapphire Rapids CPU and an Nivida H100 GPU.

HNSW has become an industry standard for vector search because it is simple to use, works out of the box without minimal parameter tuning, and its low-latency search is often fast enough for many use-cases like retrieval augmented generation (RAG) and recommender systems. Unfortunately, while building the index can often be done in parallel, the resulting performance leaves something to be desired.

The chart below shows the average build times for indexes that performed at various levels of recall. It takes significantly less time to build the CAGRA algorithm on the GPU than building the HNSWlib algorithm on the CPU.

RMM Memory Manager Library

When copying data between GPU (device) and CPU (host), using pinned host memory results in much higher bandwidth and reduced synchronization. While RMM supported a host_memory_resource for allocating pinned memory in the past, this could not be used as the upstream memory resource (MR) for an RMM memory pool. RMM 24.02 includes a new host_pinned_memory_resource that allocates pinned host memory and can be used as the upstream MR for a pool_memory_resource. Pinned host pools are available in the RMM 24.02 C++ API and will be used in the near future to improve performance of Parquet reading and dataframe spilling in cuDF.

In this release RMM also moved further into a significant refactoring to use the new cuda::memory_resource concepts now included in libcu++. cuda::memory_resource and related features provides a C++ interface for heterogeneous, stream-ordered memory allocation tailored to the needs of CUDA C++ developers. The design builds on the success of RMM and evolves the design based on lessons learned. <cuda/memory_resource> is not intended to replace RMM, but instead moves the definition of the memory allocation interface to a more centralized home in the CUDA Core C++ Libraries (CCCL). RMM will remain as a collection of implementations of the cuda::mr interfaces. This refactoring will continue over the next few releases and include changes to other RAPIDS libraries.

With the new memory_resource concepts, instead of implementing a memory resource as a class that derives from rmm::mr::device_memory_resource and implements the pure virtual methods of the base class, now memory resource implementations only need to implement the cuda::mr::memory_resource and/or cuda::mr::async_memory_resource concepts. This simply means that the class must implement the appropriate allocate/deallocate and/or allocate_async/deallocate_async functions and comparison operators, as in the following (trivial) example. Please see the docs for more examples.

struct example_memory_resource {
  void* allocate(std::size_t, std::size_t) { return nullptr; }
  void deallocate(void*, std::size_t, std::size_t) {}
  void* allocate_async(std::size_t, std::size_t, cuda::stream_ref) { return nullptr; }
  void deallocate_async(void*, std::size_t, std::size_t, cuda::stream_ref) {}
  bool operator==(const valid_resource&) const { return true; }
  bool operator!=(const valid_resource&) const { return false; }
};
static_assert(cuda::mr::async_resource<valid_resource>, "");

A library can easily decide whether the memory resource supports the async (stream-ordered) interface:

template<class MemoryResource>
  requires cuda::mr::resource<MemoryResource>
void* maybe_allocate_async(MemoryResource& resource, std::size_t size, std::size_t align, cuda::stream_ref stream) {
  if constexpr(cuda::mr::async_resource<MemoryResource>) {
    return resource.allocate_async(size, align, stream);
} else {
  return resource.allocate(size, align);
  }
}

Note the function above uses the requires constraint from C++20 concepts, but it can be converted to work with earlier versions of C++ by using SFINAE instead of concepts.

RAPIDS on Databricks with Dask

RAPIDS has a long history of accelerating workloads on Databricks and with 23.12 that integration expanded to include leveraging multi-node GPU clusters with Dask.

If you’re running Spark workloads on Databricks you can use the RAPIDS Accelerator for Spark to speed up your existing workloads. But now with the new dask-databricks package you can also launch a Dask cluster alongside Spark on the same Databricks infrastructure. This is extremely useful if you want to try out packages like xgboost.dask on GPUs but use infrastructure you are already familiar with.

Adding Dask to your cluster is as quick as adding an init script that starts the Dask components with the RAPIDS dask-cuda package.

# Start the Dask cluster with dask-cuda workers
dask databricks run - cuda
Then you can get a Dask client from your Databricks notebook.
import dask_databricks
client = dask_databricks.get_client()

Check out the RAPIDS Deployment documentation to learn more.

Infrastructure and support updates

RAPIDS 24.02 brought with it a number of changes to platforms and dependencies that RAPIDS supports. Specifically:

• RAPIDS 24.02 brought an end to support for Pascal GPUs (see more here). Effective this release, use of a Pascal GPU will either fail or return invalid results.
• RAPIDS also no longer publishes containers supporting CUDA 11.2 (see more here). We continue to publish containers for CUDA 11.8 and 12.0. Users who wish to continue using CUDA 11.2 can do so via conda installation.
• Starting with the RAPIDS 24.04 release, we are going to deprecate our containers for CentOS 7. These containers will no longer be published starting RAPIDS 24.06. See more here.
• Finally, RAPIDS will stop supporting Pandas 1.x effective with the 24.04 release (see more here). Effective RAPIDS 24.04, only Pandas 2.x series will be supported.

NVIDIA GTC

The in-person GTC experience is back this year! Come connect with a dream team of industry luminaries, developers, researchers, and business strategists helping shape what’s next in AI and accelerated computing.

Data science topics, including RAPIDS, will be heavily represented. Data science talks will include:

• RAPIDS in 2024: Accelerated Data Science Everywhere
• XGBoost is All You Need
• Accelerating Pandas with Zero Code Change using RAPIDS cuDF
• Large-Scale Graph GNN Training Accelerated With cuGraph
• And more!

See the full list of data science topics here. We hope to see you there!

Conclusion

We’re excited to get RAPIDS 24.02 out into the wild and to see what people build with it. We hope to see you at GTC!

To get started using RAPIDS, visit the Quick Start Guide.

RAPIDS 24.02 release was originally published in RAPIDS AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

The RAPIDS 23.10 release takes a huge step forward in breaking down barriers to bringing accelerated computing to the data science community. RAPIDS now enables a zero code change CPU/GPU user experience for dataframes, graph analytics, and machine learning. This release also introduces XGBoost 2.0 and makes substantial improvements to accelerated vector search and text processing for LLMs.

Table of Contents

• RAPIDS Goes CPU/GPU
• XGBoost 2.0
• Accelerated Vector Search and Text Processing

RAPIDS Goes CPU/GPU

On November 8th, NVIDIA hosted the AI and Data Science Virtual Summit, bringing together experts from across the community to discuss how accelerated computing is advancing data science.

At the Summit, we announced major enhancements to RAPIDS cuDF, cuGraph, and cuML that bring a unified, zero code change CPU/GPU experience to dataframe, graph analytics, and machine learning workflows. Below, we summarize these key enhancements. For those interested in learning more, we encourage you to register for free for the Summit to watch the session replays and stay tuned for in-depth blogs in the future.

cudf.pandas

Pandas is the quintessential tool for data scientists today. With 9.5 million users, it’s the most popular library in Python for working with tabular data. But, it becomes slow as dataset sizes grow into the gigabytes.

At the Summit, we announced that cuDF’s new pandas accelerator mode (cudf.pandas) solves this problem, bringing the speed of cuDF to every pandas workflow with zero code change required. It works with most third-party libraries that operate on pandas objects and will accelerate pandas operations within these libraries, too. Just load cudf.pandas to accelerate your workflow on the GPU, with automatic CPU fallback if needed.

This new mode is available in the standard cuDF package. To accelerate IPython or Jupyter Notebooks, use the magic:

%load_ext cudf.pandas
import pandas as pd
…

To accelerate a Python script, use the Python module flag on the command line:

python -m cudf.pandas script.py

cudf.pandas is designed to accelerate workflows where pandas struggles with performance, so the 5 GB scale DuckDB/H2O.ai Database-like Ops Benchmark perfectly illustrates the impact of GPU-accelerating your pandas code. The benchmark requires executing a variety of common merge and groupby operations.

By just “flipping the switch”, we can turn minutes of processing into just one or two seconds.

You can learn more about this benchmark in the cuDF documentation. And you can learn more about how cudf.pandas works at rapids.ai/cudf-pandas and test-drive the introductory notebook in a free GPU-enabled notebook environment on Google Colab.

nx-cuGraph

With more than 27 million downloads per month, NetworkX is the go-to library for graph analytics in Python thanks to its ease of use, wide selection of algorithms, and fantastic community.

But as datasets and graphs problem sizes grow, performance of NetworkX’s pure Python implementation becomes a significant hurdle, forcing users to stop using their favorite library or wait potentially hours for results. Over the past year, we’ve been collaborating with the NetworkX community to develop backend dispatching capabilities that can address these challenges.

We’re excited to share that cuGraph can now be used as a backend for NetworkX through our nx-cugraph package, enabling NetworkX users to GPU-accelerate their workflows with zero code change.

Just set an environment variable and your workflow will use cuGraph if it’s available and the algorithm is supported, falling back to standard CPU-based NetworkX otherwise.

export NETWORKX_AUTOMATIC_BACKENDS=cugraph
python my_nx_app.py

That’s it! Your NetworkX code will now use the GPU for all supported algorithms, enabling you to access up to 600x speedups when processing graphs like the US patent citation network dataset with 3.7 million nodes and 16.5 million edges.

nx-cugraph currently includes support for three algorithms and we aim to support 12 algorithms in the 23.12 release.

You can install nx-cugraph using either conda or pip:

pip install nx-cugraph-cu11 — extra-index-url https://pypi.nvidia.com
conda install -c rapidsai -c conda-forge -c nvidia nx-cugraph

To learn more about nx-cugraph and this benchmark, please visit this in-depth blog.

cuML CPU/GPU

In this release, we’ve significantly expanded cuML’s CPU/GPU capabilities. The majority of cuML estimators now support both GPU-based and CPU-based execution capabilities with zero code change required to switch between them. A subset of estimators even support exporting models across hardware, enabling you to train and run inference on different hardware.

Now, you can prototype using cuML in your workflows even on systems without access to a GPU. When you’re ready, take the same code and run it on a GPU-enabled system to tap into the power of accelerated computing:

import cuml # no change is needed even for the import!
from cuml.manifold.umap import UMAP

X, y = …

# define the cuml UMAP model and use fit_transform function to obtain the
# low dimensional representation of the input dataset
embeddings = UMAP(
  n_neighbors=10, min_dist=0.01, init=”random”
)

transformed_embeddings = embeddings.fit_transform(X)

To get started prototyping with cuML on CPU-only machines, you can install via conda:

conda install -c rapidsai -c nvidia -c conda-forge cuml-cpu=23.10

To learn more about these capabilities, please visit the cuML on GPU and CPU documentation.

XGBoost 2.0

In partnership with the XGBoost community, we released XGBoost 2.0 in September! This is a huge milestone for the project and a testament to the incredible group of contributors and users driving the project forward over the past few years. Today, XGBoost is downloaded more than 2.5 million times per week and is used across nearly every industry.

XGBoost 2.0 is chock full of huge improvements to both performance and user experience, but we’ll spotlight several below.

Unified GPU interface with a single device parameter

Using a GPU for different tasks within XGBoost tasks historically required setting a variety of different parameters like gpu_hist, tree_method, gpu_predictor, gpu_id, and more. Now, all of these capabilities are configurable with a single, simple device parameter that controls CPU or GPU execution.

Quantile Regression

XGBoost now supports quantile regression, a popular technique used for probabilistic forecasting scenarios in which you care about parts of the distribution beyond just the conditional mean. This may be particularly valuable for use cases in which outcomes in the tails of the distribution are particularly impactful or important, such as in supply chain forecasting.

The quantile loss (also known as pinball loss) is supported on both CPUs and GPUs.

PySpark Interface

The official XGBoost PySpark interface is now much more mature and ready for wider use! With support for GPU-based training and predictions, improved logging, performance optimizations, computing SHAP-based feature contributions, and more, we’re excited to see what the Apache Spark community creates!

To learn more, visit the XGBoost Python documentation.

Accelerated Vector Search and Text Processing

Accelerated Vector Search with RAFT

23.10 brings substantial performance and functional enhancements to CAGRA, the GPU-accelerated graph-based approximate nearest neighbors technique with world class performance for large batch queries, single queries, and graph construction time.

Pre-Filtering

Pre-filtering allows removing irrelevant records before querying our vector database index to ensure we return high quality results. As the team at Pinecone noted, “Pre-filtering is excellent for returning relevant results, but significantly slows-down our search…”

With support for pre-filtering now available, CAGRA can address these CPU-based performance challenges and enable returning high quality results.

Nearest Neighbor Descent

Building vector indexes is a computationally challenging problem well suited for GPUs and nearest neighbor descent is a state-of-the-art technique for iteratively constructing a k-nearest neighbors graph. Our graph-based CAGRA algorithm can also use nn-descent to improve the initial graph-construction performance over 10x compared to using IVF-PQ to construct the graph.

Faster JSON parsing with cuDF and Dask

Training large language models can require processing terabytes of text data, often representing essentially the entire internet! cuDF and Dask has emerged as a great combination to efficiently process the documents in these training pipelines.

In the 23.10 release, we’ve made algorithmic enhancements to the cuDF JSON reader that improve performance for reading the types of files common in large-language model training pipelines. In our benchmarks, we now observe end-to-end read throughput up to 70% of the theoretical limit with H100 GPUs reading from local NVME drives.

If you’re using cuDF as part of your LLM training pipeline, you can expect to immediately see performance gains.

Summary

At the AI and Data Science Summit, we put forward a commitment to meeting data scientists where they are today. The RAPIDS 23.10 release is a major milestone in bringing accelerated computing into the day-to-day workflows of data scientists and engineers.

Pandas and NetworkX users can now GPU-accelerate their code with zero code change. cuML now provides a large suite of CPU/GPU capabilities. XGBoost 2.0 dramatically simplifies using GPUs. And we’re continuing to improve vector search and text processing to empower emerging technologies like LLMs.

To get started using RAPIDS, visit the Quick Start Guide.

RAPIDS 23.10 Release was originally published in RAPIDS AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

This is the second chapter in our blog series on GPU-accelerated graph analysis using RAPIDS cuGraph. In the first chapter, we introduced the basics of graphs and analyzed them using a few of the many algorithms in cuGraph. That blog answered some practical questions using a simple dataset and visually verified those results. In this blog, we look at a category of graph algorithms broadly called similarity. If you have not read the first blog, we encourage you to start with the introduction chapter.

Application

Let’s start by answering the question of why and how similarity is important and dive into what type of application could use similarity. It needs to be noted that we are focused on vertex similarity and not graph similarity. Vertex similarity is a metric of how equivalent one vertex is to another vertex. The score is typically a percentage where 0.00 (0%) means no similarity and 1.0 (100%) means an exact match.

Graph similarity on the other hand is a measure of how close one graph (vertices and edges) matches another graph. This is also known as graph edit distance.

Note: Graph similarity is a topic for a future blog once we have graph similarity algorithms in cuGraph.

Similarity algorithms give insight into:

Missing relationships — these algorithms are often called “link prediction”. Incomplete datasets lead to incorrect conclusions. Running similarity algorithms on a graph can reveal likely links that can be either corroborated through other means or used in follow-up calculations to test hypotheses.

Predictions in dynamic data — adding or removing nodes or edges will change how a community behaves. By finding similar nodes, we can identify actors who might emerge or decline as the graph changes.

Making recommendations — while recommender systems can become hugely complex, a simplified model could be to find users (vertices) with “similar” behavior and then recommend to User A those items that User B has that are different. Consider the bipartite graph in figure 1, where we are comparing User 1 (U1) to User 2 (U2). The system would then recommend product P3 to User 1.

Similarity Algorithms

RAPIDS cuGraph implements several algorithms that have their origin in set comparisons from mathematics. When used for node similarity in a graph, these algorithms compare the sets of nodes, which generally comprise the neighbors where Set A = Neighbors(A).

cuGraph currently supports 3 similarity algorithms: Jaccard, Overlap, and Sorensen which we briefly compare in table 1 below. All three algorithms are normalized to values between zero and one. However, since they normalize on different terms, the exact values don’t equate.

Notes: |A| and |B| are the sizes of each set, see Figure 2 below for how this pertains to graph comparisons in a one hop example. Jaccard is marginally slower than the other two algorithms.

Consider this simple graph to illustrate the three similarity algorithms implemented in cuGraph. The calculations are all based on this graph:

Jaccard Similarity

The Jaccard similarity measure was developed by botanist Paul Jaccard who used the measure to compare plant species. His work popularized the measure’s use in in other fields as well. It sees great use in recommender systems particularly in collaborative filters which identify users with similar preferences.

Jaccard similarity always results in a value of zero to one. This set comparison algorithm is used in a variety of fields including:

• Plagiarism analysis
• Social network analysis
• Machine learning
• Recommendation systems

Using the graph in figure 2 above, the similarity between nodes 1 and 2 would be.

Size of |A ꓵ B| = 5

Size of |A ∪ B| = 8

Jaccard Similarity of (A, B) = 5/8 = 0.625

Overlap Similarity

This measure is also known as the Szymkiewicz-Simpson coefficient.

The main difference from Jaccard is that Overlap considers only the size of the smaller of the two compared sets.
Overlap finds frequent use in:

• Graphs with a larger degree distribution
• Topic Modeling
• Binary classification applications

Again, calculating based on figure 2.

|A ꓵ B| = 5

min(|A|, |B|) = 6

Overlap Coefficient of (A, B) = 5/6 = 0.83

Figure 5 illustrates that as the size of set B (node 2’s community) grows, the Jaccard coefficient stays the same while the overlap coefficient changes. Therefore, Jaccard can miss community changes in dynamic datasets. This example comes from an excellent blog discussing the differences and advantages of Jaccard versus Overlap.

Sørensen-Dice Coefficient

Like Jaccard, Sørensen considers both set sizes, however it doubles the intersection size and does not use the size of the union. While Sørensen results in a zero to one score like the other two algorithms, even small intersections skew to higher values.

While Sørensen is easily applied to graphs and neighbors, it is better known for use in semantic comparisons like:

• N-grams
• Fingerprint segments
• Genome distances

Again, going through the calculation based on figure 2:

Size of |A ꓵ B| = 5

Sum of (|A|+ |B|) = 13

Sørensen Coefficient of (A, B) = 10/13 = 0.77

Unlike Overlap, the Sørensen value will be reduced by growth of either node’s neighborhood instead of just the smaller one.

Calculating the similarities using RAPIDS cuGraph

The following code sample shows how easy it is in cuGraph to compute the similarity scores.

import cugraph
from cugraph.datasets import dining_prefs
gdf = dining_prefs.get_edgelist(download=True)
G = cugraph.from_cudf_edgelist(gdf, source='src',destination='dst')
jdf = cugraph.jaccard(G)
odf = cugraph.overlap(G)
sdf = cugraph.sorensen(G)
print(jdf.sort_values(by='jaccard_coeff', ascending=False).head())
print(odf.sort_values(by='overlap_coeff', ascending=False).head())
print(sdf.sort_values(by='sorensen_coeff', ascending=False).head())

By default, running these algorithms uses every connected pair of nodes, which is referred to as 1-hop neighbors. While this gives a representative answer, it can produce more results than needed.

Doing the calculation on all adjacent pairs might be impossible due to the memory requirements of processing extremely large graphs. Additionally, considering adjacent nodes only, leaves out a valuable source of candidates, namely two-hop connections. In figure 2, nodes one and two are adjacent. However, their shared neighbors would be missed had the two not been directly connected as seen in figure 8. The highlighted two-hop relationships are often a great source of similarity between nodes not directly connected. Consider a communication graph where a person has two phones, they could share contacts, but the two numbers logically wouldn’t call each other. This is a common use of similarity to find an alias in a graph.

The cuGraph implementations of these algorithms allow for comparing 2-hop connections or any customized set of node pairs, by providing the second optional argument specifying the node pairs to examine. This is especially useful in graphs with two unconnected node types called bipartite graphs. A classic bipartite example is a sales graph containing customers and products with edges describing purchases. Code examples in figures 9 and 10 below, extract all node pairs within two hops of each other and look for similarities among those only.

The cuGraph get_two_hop_neighbors function returns all pairs within two hops of each other. An example of calling this on the dormitory data used later is shown in figure 10.

We will now use the familiar New York Dormitory dining preference data (from the intro BLOG) and will use similarity algorithms to answer several questions beyond their two favorite dining partners. Figure 12 highlights some of the similarity calculations from table 2 as they appear in the visualization graph visualization.

Using the RAPIDS cuGraph similarity notebook, we calculate the similarity of each pair with the highest values shown in the table below.

In this small dataset, the rankings from all three algorithms are nearly identical as shown in table 2. This will not normally be the case, but it does allow a general demonstration of what similarity reveals here.

How do the results reflect what we can infer from the diagram above?

Missing relationships — There are five residents in the graph who have transitive similarities (Louise -> Lena -> Marion -> Frances -> Adele) This cluster is revealed by similarity scores. It not as obvious in the graph since each resident is connection-limited by the sampling technique. The similarity series points to a possible hierarchy or clique. In a larger graph, a set of similar nodes often points to a cohesive community within the graph.

Predictions in dynamic data — What happens if someone leaves the dorm? In this case, if Lena or Louise left the dorm, it is logical the other would inherit their preferences. In a larger graph, we could run the algorithms after manipulating the data to see if new similarities reveal changes in the groupings.

Making recommendations — Frances and Lena are similar to Marion but are not connected to each other. If Marion were to leave the dorm, either could be recommended as a partner in her place. A larger graph with higher edge density would provide abundant alternatives to recommend.

Wrap-up

Similarity algorithms are simple to understand but immensely useful in fields as disparate as community detection, recommender systems, and Natural Language Processing (NLP). As shown, RAPIDS cuGraph enables running these in just a few lines of python code and yet leverages the parallelism of GPU-computing to scale over multiple GPU’s or even multiple nodes with multiple GPU’s.

The next blog in this series will focus on centrality functionality in cuGraph. It will explore various measures of node and edge importance focusing on why they are central to graph analytics. There will be more code examples to quickly run any of those measures on even the largest datasets.

How to get started with cuGraph

• Setup and installation instructions. https://docs.rapids.ai/install
• Latest stable version of cuGraph docs — https://docs.rapids.ai/api/cugraph/stable/
• Using containers and Google Colab are the easiest paths to get started with cuGraph. https://hub.docker.com/r/rapidsai/notebooks — the available container to run cuGraph notebooks.

References:

• The first blog in this graph analytics series: https://medium.com/rapids-ai/introduction-to-graph-analysis-using-cugraph-a9dc2fbc3c5e
• Calculations in this blog were done using this notebook: https://github.com/rapidsai/cugraph/blob/main/notebooks/algorithms/link_prediction/similarity_combined.ipynb
• Earlier NVIDIA blog comparing Jaccard and Overlap: https://medium.com/p/610e083b877d

Intro to Graph Analysis using cuGraph: Similarity Algorithms was originally published in RAPIDS AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Author: Vibhu Jawa

Graph neural networks (GNNs) have emerged as a powerful tool for a variety of machine learning tasks on graph-structured data. These tasks range from node classification and link prediction to graph classification. They also cover a wide range of applications such as social network analysis, drug discovery in healthcare, fraud detection in financial services, and molecular chemistry.

In this post, I introduce how to use cuGraph-DGL, a GPU-accelerated library for graph computations. It extends Deep Graph Library (DGL), a popular framework for GNNs that enables large-scale applications.

Basics of graph neural networks

Before I dive into cuGraph-DGL, I want to establish some basics. GNNs are a special kind of neural network designed to work with data structured as graphs. Unlike traditional neural networks that assume independence between samples, which doesn’t fit well with graph data, GNNs effectively exploit the rich and complex interconnections within graph data.

In a nutshell, GNNs work by propagating and transforming node features across the graph structure in multiple steps, often referred to as layers (Figure 1). Each layer updates the features of each node based on its own features and the features of its neighbors.

In Figure 1, the first step “prepares” a message composed of information from an edge and its connected nodes and then “passes” the message to the node. This process enables the model to learn high-level representations of nodes, edges, and the graph as a whole, which can be used for various downstream tasks like node classification, link prediction, and graph classification.

Figure 2 shows how a 2-layer GNN is supposed to compute the output of node 5.

Bottlenecks when handling large-scale graphs

The bottleneck in GNN sampling and training is the lack of an existing implementation that can scale to handle billions or even trillions of edges, a scale often seen in real-world graph problems. For example, if you’re handling a graph with trillions of edges, you must be able to run DGL-based GNN workflows quickly.

One solution is to use RAPIDS, which already possesses the foundational elements capable of scaling to trillions of edges using GPUs.

What is RAPIDS cuGraph?

cuGraph is a part of the RAPIDS AI ecosystem, an open-source suite of software libraries for executing end-to-end data science and analytics pipelines entirely on GPUs. The cuGraph library provides a simple, flexible, and powerful API for graph analytics, enabling you to perform computations on graph data at scale and speed.

What is DGL?

Deep Graph Library (DGL) is a Python library designed to simplify the implementation of graph neural networks (GNNs) by providing intuitive interfaces and high-performance computation.

DGL supports a broad array of graph operations and structures, enhancing the modeling of complex systems and relationships. It also integrates with popular deep learning frameworks like PyTorch and TensorFlow, fostering seamless development and deployment of GNNs.

What is cuGraph-DGL?

cuGraph-DGL is an extension of cuGraph that integrates with the Deep Graph Library (DGL) to leverage the power of GPUs to run DGL-based GNN workflows at unprecedented speed. This library is a collaborative effort between DGL developers and cuGraph developers.

In addition to cuGraph-DGL, cuGraph also provides the cugraph-ops library, which enables DGL users to get performance boosts using CuGraphSAGEConv, CuGraphGATConv, and CuGraphRelGraphConv in place of the default SAGEConv, GATConv, and RelGraphConv models. You can also import the SAGEConv, GATConv, and RelGraphConv models directly from the cugraph_dgl library.

In GNN sampling and training, the major challenge is the absence of an implementation that can manage real-world graph problems with billions or trillions of edges. To address this, use cuGraph-DGL, with its inherent capability to scale to trillions of edges using GPUs.

Setting up cuGraph-DGL

Before you dive into the code, make sure that you have cuGraph and DGL installed in your Python environment. To install the cuGraph-DGL-enabled environment, run the following command:

conda install mamba -c conda-forge

mamba create -n cugraph_dgl_23_06 -c pytorch -c dglteam/label/cu118 -c rapidsai-nightly -c nvidia -c conda-forge dgl cugraph-dgl=23.10 pylibcugraphops=23.10 cudatoolkit=11.8 torchmetrics ogb

Implementing a GNN with cuGraph-DGL

With your environment set up, put cuGraph-DGL into action and construct a simple GNN for node classification. Converting an existing DGL workflow to a cuGraph-DGL workflow has the following steps:

1. Use cuGraph-ops models such as CuGraphSAGECon, in place of the native DGL model (SAGEConv).
2. Create a CuGraphGraph object from a DGL graph.
3. Use the cuGraph data loader in place of the native DGL Dataloader.

Using cugraph-dgl on a 3.2 billion-edge graph, we observed a 3x speedup when using eight GPUs for sampling and training, compared to a single GPU UVA DGL setup. Additionally, we saw a 2x speedup when using eight GPUs for sampling and one GPU for training.

Create a cuGraph-DGL graph

To create a cugraph_dgl graph directly from a DGL graph, run the following code example.

import dgl
import cugraph_dgl

dataset = dgl.data.CoraGraphDataset()
dgl_g = dataset[0]
# Add self loops as cugraph
# does not support isolated vertices yet
dgl_g = dgl.add_self_loop(dgl_g)
cugraph_g = cugraph_dgl.convert.cugraph_storage_from_heterograph(dgl_g, single_gpu=True)

For more information about creating a cuGraph storage object, see CuGraphStorage.

Create a cuGraph-Ops-based model

In this step, the only modification to make is the importation of cugraph_ops-based models. These models are drop-in replacements for upstream models like dgl.nn.SAGECon.

# Drop in replacement for dgl.nn.SAGEConv
from dgl.nn import CuGraphSAGEConv as SAGEConv

import torch.nn as nn
import torch.nn.functional as F

class SAGE(nn.Module):
    def __init__(self, in_size, hid_size, out_size):
        super().__init__()
        self.layers = nn.ModuleList()
        # three-layer GraphSAGE-mean
        self.layers.append(SAGEConv(in_size, hid_size, "mean"))
        self.layers.append(SAGEConv(hid_size, hid_size, "mean"))
        self.layers.append(SAGEConv(hid_size, out_size, "mean"))
        self.dropout = nn.Dropout(0.5)
        self.hid_size = hid_size
        self.out_size = out_size

    def forward(self, blocks, x):
        h = x
        for l_id, (layer, block) in enumerate(zip(self.layers, blocks)):
            h = layer(block, h)
            if l_id != len(self.layers) - 1:
                h = F.relu(h)
                h = self.dropout(h)
        return h


# Create the model with given dimensions
feat_size = cugraph_g.ndata["feat"]["_N"].shape[1]
model = SAGE(feat_size, 256, dataset.num_classes).to("cuda")

Train the model

In this step, you opt to use cugraph_dgl.dataloading.NeighborSampler and cugraph_dgl.dataloading.DataLoader, replacing the conventional data loaders of upstream DGL.

# Drop in replacement for dgl.nn.SAGEConv
from dgl.nn import CuGraphSAGEConv as SAGEConv

import torch.nn as nn
import torch.nn.functional as F

class SAGE(nn.Module):
    def __init__(self, in_size, hid_size, out_size):
        super().__init__()
        self.layers = nn.ModuleList()
        # three-layer GraphSAGE-mean
        self.layers.append(SAGEConv(in_size, hid_size, "mean"))
        self.layers.append(SAGEConv(hid_size, hid_size, "mean"))
        self.layers.append(SAGEConv(hid_size, out_size, "mean"))
        self.dropout = nn.Dropout(0.5)
        self.hid_size = hid_size
        self.out_size = out_size

    def forward(self, blocks, x):
        h = x
        for l_id, (layer, block) in enumerate(zip(self.layers, blocks)):
            h = layer(block, h)
            if l_id != len(self.layers) - 1:
                h = F.relu(h)
                h = self.dropout(h)
        return h


# Create the model with given dimensions
feat_size = cugraph_g.ndata["feat"]["_N"].shape[1]
model = SAGE(feat_size, 256, dataset.num_classes).to("cuda")
                        num_workers=0,
                        sampling_output_dir=temp_dir_name)


      acc = evaluate(model, features, labels, dataloader)
      print("Epoch {:05d} | Acc {:.4f} | Loss {:.4f} ".format(epoch, acc, total_loss))


def evaluate(model, features, labels, dataloader):
    with torch.no_grad():
        model.eval()
        ys = []
        y_hats = []
        for it, (in_nodes, out_nodes, blocks) in enumerate(dataloader):
            with torch.no_grad():
                x = features[in_nodes]
                ys.append(labels[out_nodes])
                y_hats.append(model(blocks, x))
        num_classes = y_hats[0].shape[1]
        return MF.accuracy(
            torch.cat(y_hats),
            torch.cat(ys),
            task="multiclass",
            num_classes=num_classes,
        )

train(cugraph_g, model)

Epoch 00000 | Acc 0.3401 | Loss 39.3890 
Epoch 00001 | Acc 0.7164 | Loss 27.8906 
Epoch 00002 | Acc 0.7888 | Loss 16.9441 
Epoch 00003 | Acc 0.8589 | Loss 12.5475 
Epoch 00004 | Acc 0.8863 | Loss 9.9894 
Epoch 00005 | Acc 0.8948 | Loss 9.0556 
Epoch 00006 | Acc 0.9029 | Loss 7.3637 
Epoch 00007 | Acc 0.9055 | Loss 7.2541 
Epoch 00008 | Acc 0.9132 | Loss 6.6912 
Epoch 00009 | Acc 0.9121 | Loss 7.0908

Conclusion

By combining the power of GPU-accelerated graph computations with the flexibility of DGL, cuGraph-DGL emerges as an invaluable tool for anyone dealing with graph data.

This post has only scratched the surface of what you can do with cuGraph-DGL. I encourage you to explore further, experiment with different GNN architectures, and discover how cuGraph-DGL can accelerate your graph-based, machine-learning tasks.

For details on how to implement GNNs in the cuGraph-PyG ecosystem, please see our post “Intro to Graph Neural Networks with cuGraph-PyG.”

For more graph analytics tutorials, see our other graph analytics posts.

Intro to Graph Neural Networks with cuGraph-DGL was originally published in RAPIDS AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

The RAPIDS 23.08 release was a big one, with new features to support large-scale text processing and LLMs in both cuDF and RAFT, major new functionality to cuXfilter and cuSpatial (and the new cuProj library for coordinate transforms), impressive performance numbers from cuGraph, and major changes to infrastructure and packaging (CUDA 12.0 conda packages, docker container changes). cuSignal is also being ported into CuPy (and will no longer be released as a standalone package) — we’ve seen how users frequently combine cuSignal and CuPy in workflows, and are excited to unify these tools.

Get started with the quick start installation instructions for conda, pip, or docker or read on to see more details on this month’s upgrades!

Table of Contents

• Docker Container Updates
• CUDA 12.0 and Conda packaging
• RAPIDS XGBoost Package
• cuDF — ETL and data processing
• RAFT and Vector Search
• cuProj: Blazing Fast Geographic Coordinate Transformations
• cuSpatial: Fast cartesian distance computation and binary predicate improvements
• Cuxfilter — accelerated cross-filtering viz
• cuGraph — graph analytics & GNNs
• KvikIO — Improved Zarr support & high-speed IO
• cuSignal — porting to CuPy
• Conclusion

Docker Container Updates

We’ve completely revamped our Docker containers to simplify and improve the experience. A quick summary below, but please note there are some breaking changes so for a full rundown please see this GitHub Issue describing the full impacts.

• All containers are now multi-arch (supporting both ARM and x86–64) so there’s no need to specify architecture
• To standardize installations, all containers (except for CUDA 11.2 containers) are based on Ubuntu 22.04. CUDA 11.2 is not supported in Ubuntu 22.04, and thus is still based on Ubuntu 20.04
• There are now only Base and Notebooks containers
• Images are now inheriting from nvidia/cuda’s base images, which contain only cuda-cudart & cuda-compat (no other libraries, headers, etc.)
• Everything else CUDA-related comes from Conda packages
• All images start with a non-root user, which should improve general usability
• Conda packages are installed in Conda’s base environment, which should simplify usage when activation is not an option
• The RAPIDS base image (Alt) starts in an ipython shell and includes what’s necessary to run RAPIDS
• The RAPIDS notebooks image (Alt) includes RAPIDS notebooks and starts a JupyterLab server and adds our standard suite of Jupyter Notebooks to sample a taste of what RAPIDS has to offer

CUDA 12.0 and Conda packaging

RAPIDS 23.08 adds support for CUDA 12.0 Conda packages! The team worked hard to create and merge new CUDA packages on conda-forge for CUDA 12.0 that allow for a much more flexible CUDA future. When building Conda packages using the new model, no libraries are pulled in outside of cuda-cudart unless explicitly specified in the requirements/host for CUDA 12. Also note that cuda-cudart is statically linked by default, if users want different behavior they need to add cuda-cudart-dev to requirements/host and possibly tweak the compiler flags accordingly. Currently conda-forge is in the process of migrating packages for CUDA 12 following this guidance, which is generally useful. Here’s an example of these changes:

To enable this improvement, the cuda-version package will supersede cudatoolkit as the way to specify the CUDA version. Note that the cuda-version works for both prior CUDA versions (like 11.x) and newer ones (like CUDA 12.0). Given the restructuring of the CUDA Conda packages, the cudatoolkit will no longer exist in the new Conda packaging model (starting with CUDA 12.0). An example installation of RAPIDS, using cuda-version, is shown in the figure below.

RAPIDS XGBoost Package

The RAPIDS 23.08 release also includes a revamped XGBoost package. This is now more in line with the conda-forge package (with many changes already upstreamed to conda-forge). The package is now maintained in this repo.

cuDF — ETL and data processing

cuDF 23.08 brings new tools for accelerating LLM pipelines and processing document data at scale. Thejaccard_index function enables 6x faster runtimes and hash_character_ngrams enables up to 5x larger batch sizes. These functions build on the >40x speedups from minhash (released in 23.06).

We also released Expression-based IO filtering in libcudf for the Parquet reader. Now you can efficiently target your file reads to a subset of interest — this is critical for network-attached and cloud storage.

In this release we also implemented a number of order-of-magnitude optimizations for the existing APIs. The ORC writer now writes files with string columns up to 10x faster. The ORC reader also got a dramatic performance boost — more than 20x faster runtimes when reading list columns with high row counts. Unbounded window functions are now 10–15x faster.

Finally we are introducing the pylibcudf subpackage and developer guide. pylibcudf is a lightweight Cython wrapper for direct and efficient libcudf usage, designed to be the new core of cuDF-python as well as performance-critical feature engineering.

RAFT and Vector Search

RAFT has added a new index to its vector search toolbelt, as the novel state-of-the-art graph-based approximate nearest neighbors algorithm, CAGRA, has been promoted from experimental status and is now ready to accelerate your vector similarity search applications. CAGRA demonstrates superior performance to other GPU-accelerated indexes when given large numbers of queries at a time but it has been purpose-built to increase GPU utilization even when searching for only a single vector at a time. CAGRA’s graph-based index is a low-latency and high-throughput alternative to both SCaNN and HNSW, which are the current state-of-the-art on the CPU.

In addition to faster search, CAGRA also reduces the amount of time spent building indexes. Version 23.08 of RAFT cuts index building time nearly in half compared to HNSW and this will be improved even further in upcoming releases.

CAGRA comes with both a C++ API and lightweight Python API and will soon be available in Milvus and Redis.

cuProj: Blazing Fast Geographic Coordinate Transformations

The cuSpatial 23.08 introduces a new subproject called cuProj. cuProj is a library and Python package for accelerated geographic and geodetic coordinate transformations. cuProj can transform billions of geospatial coordinates per second from one coordinate reference system (CRS) to another on GPUs.

cuProj provides a Python API that closely matches the popular PyProj API, and a header-only C++ API designed to implement many of the same transformations provided by the widely used Proj. Currently cuProj supports a subset of the Proj transformations and projections, specifically conversion of WGS84 (latitude/longitude as used by GPS) to and from the Universal Transverse Mercator (UTM) standard. Conda and Pip packages for cuProj are now available.

The chart in Figure 1 shows performance of cuProj compared to PyProj in a benchmark similar of WGS84 to UTM projection. cuProj running on an NVIDIA H100 GPU achieves a peak speedup of over 4100x vs. PyProj running on an Intel Xeon Platinum 8480C CPU. For perspective, cuProj can transform 1 billion points in about 30ms: interactive rates. PyProj takes nearly 2 minutes to do the same.

cuSpatial: Fast cartesian distance computation and binary predicate improvements

Aside from cuProj, multiple improvements went into cuSpatial 23.08. The GeoSeries.distance API has been added to compute the Cartesian distances between two GeoSeries on the GPU. Depending on the input GeoSeries type, this API supports any combinations of GeoSeries between Points, MultiPoint, Multilinestring and Multipolygon geoseries. It provides a GeoPandas-like interface (including the align argument). Given 10 million pairs of 20-segment linestrings and 38-sided polygons, cuSpatial achieves a 66x speedup compared to GeoPandas in computing the linestring-polygon distance.

Multiple bug fixes have gone into 23.08 to increase the stability and correctness of the .contains API. When a linestring shares a boundary point with a polygon, while none of its endpoints are in the polygon, it is now computed as “not contained” to match GeoPandas. In addition, multi*_range classes now have 100% test coverage of public APIs. As an abstraction to geometry objects shared across many vectorized ops, increased test coverage increases API stability. For more information about the .distance API benchmark, binary predicate improvements and test coverage, look for an upcoming cuSpatial 23.08 blog post soon.

Cuxfilter — accelerated cross-filtering viz

Cuxfilter development didn’t take a break this summer, adding several major features and UX updates. For this release, cuxfilter now supports CUDA 12 and is available through pip! Using pip will facilitate faster and simpler installation, especially on cloud services like Colab and Studio Lab. Keen observers will also note a more polished and integrated UI, thanks in part to better holoviews and bokeh 3.0 integration. This allows for better widget tools, easier theme switching, and more robust drag-and-drop layouts.

Speed and string support improvements in cuDF resulted in us removing our long-running precomputed datatile feature. We also removed the bokeh line chart, since the datashader version is better suited for our needs. As a result, we are now able to implement string columns without mapping and our favorite new feature: ghosted linked brushing. See below for an example.

Using d.app() will now show much better laid out dashboards inline within a notebook. Dashboard and individual chart heights can now be directly set. As such, we decided to remove the somewhat confusing d.preview() static preview functionality.

Finally, we cleaned up our documentation and are working on an advanced cuxfilter features guide notebook. And in case you missed it, we just released an extensive visualization user guide notebook, spanning, hvplot, datashader, plotly dash, cuDF cuGraph, cuML, cuSpatial, AND cuxfilter. It’s a good one.

You can try all the new cuxfilter features out in Colab and Studio Lab right now.

cuGraph — graph analytics & GNNs

One of the cuGraph goals is to ensure that all our algorithms scale, we are happy to announce that the Edge Betweenness Centrality algorithm now supports scaling from a single GPU to multiple GPUs and even multiple GPUs across multiple nodes (what we call MNMG).

Supporting Graph Neural Networks continues to be a big push for cuGraph. This release we made major improvements to the bulk sampling function and added the ability to do MFG (message flow graph) creation as part of sampling. We tested on multiple graphs that are multiples of the OGB paper100M dataset:

3 billion edges (ogb_paper100M x 2) with a batch size of 512:

• Old bulk sampler + post MFG creation: 77.283 seconds
• New sampler with MFG: 32.39 seconds

6 billion edges (ogb_paper100M x4) and batch size of 512

• Old bulk sampler + post MFG creation: 151.89 seconds
• New sampler with MFG: 77.33 seconds

Increasing the batch size further improved performance. On 6 billion edges with a batch size of 16,384 the sampling with MFG time drops to only 11.45 seconds.

Release 23.08 also has cuGraph support for CUDA 12, except the cugraph-dgl and cugraph-pyg packages which will be CUDA 12 in release 23.10 due to a dependency on PyTorch which will have CUDA 12 in September. The WholeGraph package has also been refactored and is now available via conda and pip.

KvikIO — Improved Zarr support & high-speed IO

The focus of this release is Zarr compatibility and support of on-the-fly compression/decompression when reading or writing Zarr files. It is now possible to use nvCOMP’s batch API when accessing Zarr files, which allows multiple buffers to be compressed/decompressed together. A demo can be found in this notebook.

Additionally, Dask-CUDA has switched from using cuCIM to KvikIO for handling GDS-based spilling, and has been updated to leverage vectored IO, which hands off all reads/writes to system calls in the CPU case or KvikIO in the GPU case.

cuSignal — porting to CuPy

The cuSignal project is in the process of porting to CuPy and will be deprecated and archived in RAPIDS. The 23.08 release is the last formal release of cuSignal. More details can be found in RSN 32. We’ve seen how users frequently combine cuSignal and CuPy in workflows, and we’re excited to unify these tools to improve the user experience.

The cuSignal team thanks RAPIDS users for the years of support and is looking forward to its future as a core CuPy module.

Conclusion

We’re proud of what we’ve been able to achieve in RAPIDS 23.08, and can’t wait to share with you what is coming for RAPIDS 23.10+! We’d like to extend our thanks to our contributors and users, and hope that you will give RAPIDS 23.08 a try:

• Quick Start Guide
• Full Install Guide
• Documentation, including complete release changelogs

RAPIDS 23.08 Release was originally published in RAPIDS AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

What is Graph Analysis?

When analyzing data, sometimes the relationships between the data elements are important along with the scope of the question being asked. In those cases, it is better to represent the data as a graph and apply graph analytics.

An example coming out of the COVID pandemic is that everyone became aware of the concept of contact tracing. That process involves collecting information about who someone was in contact with and who those folks were in contact with. Basically, building a social network where the nodes represent people and edges represent contact. This is a case where the relationship, who contacts whom, is the focus of the analysis more than the characteristics of the person in question. Graph analysis is the application of graph techniques and algorithms to answer questions related to the relationship between data objects.

This blog represents a brief introduction to graph analysis and is the kickoff to a series of blogs covering graph analysis using cuGraph. Later blogs will cover cuGraph in depth, with code, scale, and performance details.

The series will cover topics such as:

• Node Similarity
• Node and Edge Centrality
• Path Finding
• Community detection
• Sampling
• Graphs and types of graphs
• … and many more

Why should I care, and what questions can graphs answer?

A full discussion of graph analytics can easily morph into a complete book, see the great work by Stanley Wasserman and Katherine Faust for example, or a PhD level paper on a graph algorithm or performance. However, it is enlightening to first look at the questions graph analytics can solve and ignore the details of how algorithms work.

Graph algorithms help answer questions as diverse as:

• How are diseases spread in a community?
• How do international trade routes develop?
• What are the most important papers in a citation list?
• Which members are redundant or indispensable in an organization?
• Based on information flow, how might a department function if specific people(s) leave?
• How to get from point A to point B with the least cost — the classic traveling salesman problem

Basics of a graph

The next blog will dive into this in greater detail, but for now here are a few key graph parts:

• Nodes are actors in the data, also called vertices in graph theory.
• Edges indicate the interactions between nodes, also called links.
• Attributes are characteristics of nodes and edges, sometimes called features or properties.

Let us consider the following very simple graph that consists of two nodes, A and B, and a single edge that has a weight attribute of 3.

Introduction to RAPIDS cuGraph in 3 simple steps

RAPIDS cuGraph is part of NVIDIA’s suite of accelerated data science libraries that aims to accelerate most aspects of Data Science via GPU’s. RAPIDS cuGraph focuses on providing acceleration to graph algorithms and workflows. You do not need to be a Data Scientist or a GPU expert to benefit quickly from cuGraph. The easiest platform to use cuGraph is a Linux computer with an NVIDIA GPU (Pascal or later generation). RAPIDS cuGraph provides an intuitive and easy to use Python interface as well as a collection of Jupyter notebook examples.

Only three steps are needed to use cuGraph: (1) Load the graph data; (2) Create a Graph; and (3) Call an algorithm.

Okay, really four steps since the packages must be imported.

Let’s dive into a problem

Given a social dataset, the question we want to ask, and understand, is: who is the most popular and least popular person?

Consider a simple dataset which comes from work by JL Moreno in the 1940s looking at the dining-table partners in a dormitory at a New York State Training School. He collected data by asking all twenty-six girls in a dormitory their first and second choice of dining partners. The raw data structure is very simple. Each resident’s answer is a row in a comma-separated file. The ‘source’ is the actor answering the question, the ‘target’ is the actor they would like to dine with, and the weight is either a 1 for first choice or a 2 for second choice. From the sample data shown below, you can see that Ada would prefer to sit with Cora as her 1st choice and with Louise as her 2nd option. In the top left corner of figure 1 below , we can see edges from Ada to Cora and Louise with the corresponding edge attributes.

For determining who is the most important, we are going to look at five different centrality measurements:

In figure 4 above it is visually apparent that Eva, Marion, and Edna are the most popular dining partners. We can discern this by counting the number of edges pointing to them. It is not a surprise that the graph calculations in figure 5 below, validate our observations. In a larger dataset, visualization is more difficult or impossible.

As expected, both Eva and Marion topped all the measures here. The next question to ask is: How do their roles differ?

Betweenness and PageRank scores suggest that Eva is most important to lines of communication and social structure of the dormitory. Eigenvector and Katz scores indicate Marion wields the most influence over both the popular students and the entire population.

Conversely, the lowest scorers in these measures are also of interest.

Many residents have low connectedness as shown by the in-degree distribution chart, Figure 6 below. A total of 10 out of the 26 students were only chosen as a preferred partner once or not at all. The pool of potentially isolated residents is over one third of the population.

Let us sort the graph algorithm results, Figure 7 below, in ascending order (smallest at top) so that we can evaluate least popular actors.

Note: due to the small dataset size, degree centrality does not completely correlate to lowest degree.

We can ask a range of additional question beyond just popularity, like:

• Who in the dorm is the most isolated?
  - From the data, many students share the lowest degree centrality identifying Alice, Laura, Ella, and Cora as being among the most isolated,
• Who is vulnerable to bullying?
  - Those like Betty and Ruth who had the lowest PageRank and Eigenvector centrality scores lack influence and connections to approach others if they are bullied.
• Who would be at risk if the population changes?
  - Those with low degree to others with low degree like Ada and Cora could suddenly become very isolated if either one left the dorm.
• Whose voices are underrepresented in the group?
  - Laura has minimum links to only highly connected students. She seems to have no peer group and could be a bystander to the mutually connected groups. Her degree centrality is minimum but scores in the middle of connectedness measures.

While the questions above are answered by observing these measures, further analysis of this dataset could address answer questions like:

• Whose removal would most disrupt the social structure in the dorm?
• Who brokers social status within the dorm?
• Who would ascend in prominence if a particular person left the dorm?

What happens when the data set gets larger?

Let us look at a larger dataset, one that contains over 40 thousand edges, and is illustrated in figure 6 below. At that scale it starts becoming nearly impossible to visually discern the characteristics of individual nodes. Imagine what a graph with over a million edges would look like.

The figure depicts an email communication network from the 1990’s infamous Enron corporate scandal. For our analysis, we will duplicate the same measurements from the above simple set. However, we must rely on some additional external enrichment data to see if the answers make sense.

This dataset contains email metadata publicized during the Enron scandal trial.

Analysis of email traffic (a type of graph analysis) helped convict many of the conspirators by establishing hidden relationships, determining key actors, and isolating small groups within the full email corpus.

• The scandal was widely impactful:
  - led to the bankruptcy of Enron in 2001
  - Resulted in billions of dollars in lawsuits including a $40 billion suit from shareholders.
  - Arthur Anderson, a top five accounting firm, was dissolved.
  - The Sarbanes-Oxley Act, among other reforms, was passed to regulate financial record keeping.
  - Several Enron officers were sentenced to prison terms
• The data contains 40777 unique email connections (the edges) between 6187 different email addresses (the nodes).
• Weights containing the number of emails between addresses are included in each connection data but are not used in this analysis. (Later blogs will use the weights)

Note: Katz Centrality has been omitted from these calculations since it does not properly converge. More on that in a later blog.

Within a matter of seconds (depending on your GPUs), cuGraph can identify important actors in this graph and give clues to their role in the community it represents. In this example…

• Tana.jones@enron.com is connected to the most actors in the graph.
• Jeff.dasovich@enron.com has the highest Betweenness and PageRank. That email is the most influential in the core network and the most important to the graph’s structure. It facilitates the shortest paths between other nodes in the graph.
• Pete.davis@enron.com has the highest eigenvector centrality. That node had the most influence over the entirety of the graph rather than just the other important nodes.

What makes cuGraph special?

There are many platforms and tools to do graph analytics but cuGraph has a variety of significant benefits:

• There are no licensing fees needed to use cuGraph.
• RAPIDS cuGraph integrates with widely used open-source packages like DASK, PyTorch and scikit-learn with more on the way.
• It is part of the RAPIDS open-source GPU accelerated ecosystem allowing start to finish GPU acceleration.
• Abundant Support in the RAPIDS community and from NVIDIA via GitHub.
• It is easy to set up. This article is not focused on setup/installation, but links below contain instructions for running in several environments.
• Scalability and performance are peerless.
  - A single 32GB GPU can process graphs of up to 250 million edges.
  - NVIDIA engineers have scale tested cuGraph on multi-node, multi-GPU systems on graphs of over one trillion edges.
• User-friendly interface: There is a Python API similar to NetworkX, and for integrators there are both C and C++ APIs.
• RAPIDS cuGraph is available in Amazon AWS, Google CoLab, and Paperspace.

How to get started with cuGraph.

• Setup and installation instructions. https://docs.rapids.ai/install
• Latest stable version of cuGraph docs — https://docs.rapids.ai/api/cugraph/stable/
• The base notebook used to do these calculations https://github.com/rapidsai/cugraph/blob/main/notebooks/algorithms/centrality/Centrality.ipynb
• Using containers and Google Colab are the easiest paths to get started with cuGraph.
  https://hub.docker.com/r/rapidsai/rapidsai/ contains the available containers to run cugraph
  https://colab.research.google.com/drive/1cb40U3gdXZ7ASQsWZypzBFrrFOKpvnbB?usp=drive_open presents a tutorial for using cuGraph in in Google Colab

Where to find more cuGraph information.

RAPIDS cuGraph provides a full suite of documentation and examples including:

• Past blogs, presentations and white papers related to cuGraph here:https://docs.rapids.ai/api/cugraph/nightly/tutorials/cugraph_blogs/
• The full RAPIDS documentation collection: https://docs.rapids.ai/
• Thoroughly tested and maintained cuGraph Jupyter notebooks: https://github.com/rapidsai/cugraph/tree/main/notebooks
  - The centrality notebook used for the calculations presented here: https://github.com/rapidsai/cugraph/blob/main/notebooks/algorithms/centrality/Centrality.ipynb
  - Notebooks containing examples of each supported algorithm.
  - Some include DASK integration scaling to multiple GPUs.
  - All are updated with the latest cuGraph additions and fixes.
  - Demonstrate cuGraph algorithm visualization, integration, and benchmarking.

In conclusion

As mentioned in the introduction, this is the first in a series of blogs covering graph analysis using cuGraph. The next blog will explore the concept of similarity, followed by blogs on centrality, transversal and other classes of graph algorithms. The series will also dive into the notion of graph ETL, and the steps needed to process the data before creating a graph. Lastly, the series will include new relevant topics and requests accepted into the cuGraph GitHub issues list at https://github.com/rapidsai/cugraph/issues.

Additional References:

For additional read on the New York State Training S study by JL Moreno, his book “Who Shall Survive” is online at: https://archive.org/details/whoshallsurviven00jlmo/mode/2up

“Social Network Analysis: Methods and Applications” by Stanley Wasserman and Katherine Faust is a wonderful book.

Lastly, “Disrupting Dark Networks” by Sean Everton is an informative read on the application of network analysis.

Licensed Material:

The Enron image is from https://lucidmanager.org/data-science/analyse-enron-corpus, and is under a Creative Commons Attribution-ShareAlike 4.0 International License.

Introduction to Graph Analysis using cuGraph was originally published in RAPIDS AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

RAPIDS’ 23.06 release is now available! This release adds a number of features and improvements we are excited about, including:

• Major improvements to cuSpatial, including DE-9IM support, pip packages, performance improvements, and improved example notebooks
• RAFT integrations with Implicit (collaborative filtering) and Meta’s FAISS (similarity search), as well as improvements to pylibraft
• Performance and memory usage improvements in cuGraph
• Timezone support and numerous performance increases in cuDF
• CUDA 12 support in pip wheels

Get started with the quick start installation instructions for conda, pip, or docker or read on to see more details on this months’ upgrades!

Table of Contents

• Expanded spatial relationship and distance queries in cuSpatial
• Nearest neighbors and more in RAFT
• Better memory utilization and performance in cuGraph
• Timezone support and performance boosts in cuDF
• CUDA 12 pip packages now available!
• Conclusion

Expanded spatial relationship and distance queries in cuSpatial

23.06 is one of our biggest releases yet with the focus being on increasing our reach, making it easier than ever before to use cuSpatial and to keep the data and compute on the GPU so further processing and analysis is faster than ever. Major changes include:

• cuSpatial can now be installed with pip
• Cartesian distance between GeoSeries of any geometry type.
• Spatial Predicates for any two non-multigeometry columns
• Notebook Improvements including using cuML with cuSpatial and a cuDF accelerated WKT parser

Starting with 23.06, you can now install cuSpatial using pip alongside the other main RAPIDS libraries, hosted on the NVIDIA index. We support both CUDA 11.2–11.8 and 12.0 with pip.

# If using driver 525+, with support for CUDA Toolkit 12.0+

pip install --extra-index-url=https://pypi.nvidia.com cuspatial-cu12

# If using driver 450.80+, with support for CUDA Toolkit 11.2+

pip install --extra-index-url=https://pypi.nvidia.com cuspatial-cu11

This release also includes fully parallelized Cartesian distance between any two pairs of geometry. It’s easier and faster than ever to find the distance between objects in large datasets of (multi)points, (multi)polygons, and (multi)linestrings.

23.06 now supports the complete set of spatial binary predicates, also known as DE-9IM. Nine predicates are supported in this release: contains, geom_equals, intersects, covers, crosses, disjoint, overlaps, touches, and within. These predicates give you the ability to leverage GPU parallelization to determine spatial relationships between two GeoSeries.

import cuspatial
import geopandas
import numpy as np
from shapely import LineString, GeometryCollection
left = geopandas.GeoSeries([
    LineString([(-4, 1), (-2, 1), (0, 1), (2, 1), (4, 1)]),
    LineString([(-4, 2), (-1, 2), (-1, -2), (1, -2), (1, 2), (4, 2)]),
    LineString([(0, 4), (0, -4)])
])
right = geopandas.GeoSeries([
    LineString([(-4, 0), (-2, 0), (0, 0), (2, 0), (4, 0)]),
    LineString([(-4, 0), (-2, 0), (0, 0), (2, 0), (4, 0)]),
    LineString([(-4, 0), (-2, 0), (0, 0), (2, 0), (4, 0)]),
])

sample1 = np.random.randint(0, 3, 10_000_000)
sample2 = np.random.randint(0, 3, 10_000_000)
gpd_left_lines = left[sample1].reset_index(drop=True)
gpd_right_lines = right[sample2].reset_index(drop=True)
left_lines = cuspatial.from_geopandas(left)[sample1].reset_index(drop=True)
right_lines = cuspatial.from_geopandas(
    right
)[sample2].reset_index(drop=True)

This example demonstrates a 96x speedup using cuSpatial for LineString-LineString intersection tasks on an AMD EPYC 7642 (48 core) CPU and an NVIDIA A100 GPU.

In [2]: %time gpu_result = left_lines.intersects(right_lines)
   ...: 
CPU times: user 422 ms, sys: 59.4 ms, total: 482 ms
Wall time: 481 ms

In [3]: %time host_result = gpd_left_lines.intersects(gpd_rig
   ...: ht_lines)
CPU times: user 46.4 s, sys: 90.2 ms, total: 46.5 s
Wall time: 46.5 s

To get your wheels turning on ways to use cuSpatial and leverage the power of RAPIDS alongside it, we’ve updated the Hausdorff Clustering Notebook to use cuML, and the zip code stop sign comparison notebook now uses a cuDF-accelerated WKT parser that improves IO performance by up to 40x.

Just like with the 23.04 cuSpatial release, there’s too much to cover in this article, so we are planning another standalone cuSpatial 23.06 blog — stay tuned for more!

Nearest neighbors and more in RAFT

RAFT continues to provide important building blocks for building machine learning and data analytics applications and release 23.06 is no exception! pylibraft continues to expand, with 23.06 bringing RAFT’s powerful k-selection APIs to Python libraries.

Implicit, the popular library for collaborative filtering in Python, is officially using RAFT’s C++ k-selection primitives and found an instant 25% speedup over its prior implementation. Stay tuned while Implicit continues to integrate RAFT primitives, such as brute-force nearest neighbors, in coming releases. Meta’s FAISS library, which has long been known as the standard for approximate nearest neighbors (ANN) algorithms on the GPU, has also been integrating RAFT’s C++ brute-force nearest neighbors (aka Flat index) and will soon be integrating RAFT’s C++ IVF-Flat, IVF-PQ, and CAGRA implementations.

Nearest neighbors aside, all of RAFT’s public APIs now accept the new raft::resources object, which provides API-level environment and dependency agnosticism. As an example, CUDA math libraries are no longer required dependencies for users who aren’t invoking APIs that need them.

Better memory utilization and performance in cuGraph

Release 23.06 saw cuGraph addressing performance and expanding the list of algorithms that scale to Multi-Node Multi-GPU (MNMG). In 23.06 the team focused on reducing the memory overhead in creating a graph in MNMG environments. The memory overhead (what cuGraph uses versus the size of the input data) was reduced from:

• Directed Graph: overhead dropped from 5.9x to 3.07x
• Undirected Graph: overhead dropped from 12x to 6.14x for undirected graph

An Undirected graph requires that the data be symmetrized hence memory is 2x of directed.

By reducing unneeded data copies, the performance of graph creation improved between 1.7x and 4.2x depending on the size of the graph.

For release 23.06, cuGraph is pleased to announce the release of MNMG Leiden for community detection / clustering on large graphs. We also enhanced Node2Vec to support MNMG for GNN sampling on large graphs. Speaking of GNNs, we spent a lot of time optimizing and improving our sampling algorithms. The cugraph-pyg code was refactored so that the node loader now matches upstream PyG. Additionally, we released a blog discussing cugraph-pyg. And lastly, the RMAT generator was updated to support the creation of Bipartite graphs.

Timezone support and performance boosts in cuDF

cuDF 23.06 introduces support for timezone-aware datetime types. You can use the tz_localize method to create timezone-aware data:

>>> dti = cudf.date_range("2001-01-01", freq="3h", periods=10)
>>> dti
DatetimeIndex(['2001-01-01 00:00:00', '2001-01-01 03:00:00',
            '2001-01-01 06:00:00', '2001-01-01 09:00:00',
            '2001-01-01 12:00:00', '2001-01-01 15:00:00',
            '2001-01-01 18:00:00', '2001-01-01 21:00:00',
            '2001-01-02 00:00:00', '2001-01-02 03:00:00'],
           dtype='datetime64[ns]')

>>> dti_tz = dti.tz_localize("America/New_York")
>>> dti_tz
DatetimeIndex(['2001-01-01 00:00:00-05:00', '2001-01-01 03:00:00-05:00',
            '2001-01-01 06:00:00-05:00', '2001-01-01 09:00:00-05:00',
            '2001-01-01 12:00:00-05:00', '2001-01-01 15:00:00-05:00',
            '2001-01-01 18:00:00-05:00', '2001-01-01 21:00:00-05:00',
            '2001-01-02 00:00:00-05:00', '2001-01-02 03:00:00-05:00'],
           dtype='datetime64[ns, America/New_York]')

To convert between timezones, use tz_convert:

>>> dti_tz.tz_convert("America/Chicago")
DatetimeIndex(['2000-12-31 23:00:00-06:00', '2001-01-01 02:00:00-06:00',
            '2001-01-01 05:00:00-06:00', '2001-01-01 08:00:00-06:00',
            '2001-01-01 11:00:00-06:00', '2001-01-01 14:00:00-06:00',
            '2001-01-01 17:00:00-06:00', '2001-01-01 20:00:00-06:00',
            '2001-01-01 23:00:00-06:00', '2001-01-02 02:00:00-06:00'],
           dtype='datetime64[ns, America/Chicago]')

These functions are similar to their Pandas counterparts, but are much faster:

psr = pd.Series(pd.date_range("1970-01-01", "1980-01-01", freq="1T"))
sr = cudf.from_pandas(psr)

In [6]: %timeit psr.dt.tz_localize("America/New_York", ambiguous="NaT", nonexistent="NaT")
1.55 s ± 9.91 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [7]: %timeit sr.dt.tz_localize("America/New_York", ambiguous="NaT", nonexistent="NaT")
9.98 ms ± 246 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

libcudf — the C++ library that powers cuDF — implemented several important performance improvements and new features in 23.06:

• Released the first libcudf algorithm powered by cuco::static_set, delivering an additional 3–4x speedup to distinct_count (see left).
• Introduced the minhash function, designed for de-duplication in large-language model training
• Completed the removal of lazily-computed null counts, the last major blocker for custom stream support
• Implement Python drop_duplicates with cudf::stable_distinct, delivering 20–55% speedups
• Add row-wise filtering step to read parquet

CUDA 12 pip packages now available!

Last but not least, we are proud to announce that RAPIDS pip packages supporting CUDA 12 are now available via the NVIDIA pip package index. Give them a try via the install picker or try the instructions below:

# Will install cudf, cuml, cugraph, cuspatial–adjust packages as needed
pip install --extra-index-url=https://pypi.nvidia.com cudf-cu12 cuml-cu12 cugraph-cu12 cuspatial-cu12

Conclusion

We are proud of the updates that 23.06 brings, and hope that you will give it a try! For more information, you can check out:

• Quick Start Guide
• Full Install Guide
• Documentation, including complete release changelogs

RAPIDS 23.06 Release — Accelerating Data Science was originally published in RAPIDS AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Authors: Thomson Comer, Mark Harris, Ben Jarmak, Michael Wang

cuSpatial continues barreling towards complete spatial relationship predicate (DE-9IM) support and Cartesian distance between any two geometries. In support of these goals cuSpatial 23.04 includes multiple new features, as well as a new notebook that shows off the capabilities of cuSpatial. There are also some breaking changes in this release to ensure all features in the library meet our high standards.

New Features

• GeoSeries Standardization
• Linestring-linestring Intersection
• Point-Polygon Distance Computation
• Binary Predicates: Equals and Intersects
• Stop Sign Counting Notebook

GeoSeries Standardization

Past cuSpatial releases greatly enhanced the capabilities of cuSpatial’s geometry series object: GeoSeries. In this release, cuSpatial standardizes all API inputs with GeoSeries. Instead of passing custom offset and coordinate arrays like before, all cuSpatial APIs now accept GeoSeries as input. Constructing a cuSpatial GeoSeries from GeoPandas is as simple as:

import geopandas as gpd
gpds1 = gpd.GeoSeries([Point(0, 0), Point(1, 1)])
cus1 = cuspatial.GeoSeries(gpds)

You can also directly construct a GeoSeries from a list of shapely objects:

cus2 = cuspatial.GeoSeries([Point(2, 2), Point(3, 3)])

Using GeoSeries everywhere significantly simplifies cuSpatial API calls.

# cuspatial 23.02
old = cuspatial.haversine_distance(p1_lon, p1_lat, p2_lon, p2_lat)
# cuspatial 23.04
new = cuspatial.haversine_distance(p1, p2)

Interleaved x-y coordinates v.s. separate x-y coordinate column

GeoSeries assumes that its underlying coordinates are stored in a single interleaved x-y column. For data stored in separate x-y columns, cuSpatial provides the following new factory methods to construct a GeoSeries.

GeoSeries.from_points_xy
GeoSeries.from_multipoints_xy
GeoSeries.from_linestrings_xy
GeoSeries.from_polygons_xy

To construct a GeoSeries from separate coordinate columns, first call interleave_columns() from cuDF to construct an interleaved coordinate column, then construct the geoseries with from_points_xy.

coords = cudf.DataFrame({“x”: xcol, “y”: ycol}).interleave_columns()
points = GeoSeries.from_points_xy(coords)

Updating the API to accept GeoSeries input is a major breaking change. It’s a necessary change since there are many GIS data formats out there, while libcuspatial computation kernels have strong assumptions on inputs. By separating the data structure construction and computation, this design makes cuSpatial computation more efficient and robust.

Linestring-Linestring Intersection

Set operations allow users to identify and analyze overlapping spatial features, which is critical for tasks like route optimization, resource allocation, and conflict resolution in infrastructure planning. cuSpatial 23.04 introduces an API that computes the intersecting geometries between two columns of linestrings. The API can be called via cuspatial.pairwise_linestring_intersection. The following example shows how to use it.

Given two linestrings (ABCDE, FGHI), compute the intersections between them:

>>> from shapely.geometry import *
>>> from cuspatial.core.binops.intersection import pairwise_linestring_intersection
>>> l1 = cuspatial.GeoSeries([
… LineString([(0, 0), (2, 1), (4, 3), (2, 5), (0, 3)])])
>>> l2 = cuspatial.GeoSeries([
… LineString([(-1, 6), (2, 3), (3, 4), (1, 6)])])
>>> offsets, geoms, lookbacks = pairwise_linestring_intersection(l1, l2)
>>> offsets
<cudf.core.column.numerical.NumericalColumn object at 0x7f097c987bc0>
[
 0,
 2
]
dtype: int32
>>> geoms
0 POINT (1.00000 4.00000)
1 LINESTRING (2.00000 5.00000, 3.00000 4.00000)
dtype: geometry
>>> lookbacks
 lhs_linestring_id lhs_segment_id rhs_linestring_id rhs_segment_id
0 [0, 0] [3, 2] [0, 0] [0, 2]

The API returns a 3-tuple:

• offsets, a cuDF integer series, which includes the offsets to the first result geometries for each pair;
• geoms, a GeoSeries that contains the intersection results (points and linestrings);
• lookbacks, a 4-column dataframe that contains the source geometry IDs from which to search the result.

In this example, the offsets result of [0, 2] indicates that the intersection for the 0th pair starts at index 0 of the geoseries, and it has two results.

The geoseries (geoms), shows the result contains two values:

1. A point with coordinates (1, 4).
2. This is the point of intersection between the linestrings (D, E) and (F, G) in the image above.
3. A linestring with coordinates (2, 5)->(3, 4).
4. This is the intersecting segment (D, H) in the image above.

The 0th element in the 0th row of the lookback ids (lookbacks) indicates the position of the first result geometry in the first pair. From the example above, the first value — which corresponds with the first geometry in geoms — for each element in lookbacks is (0, 3, 0, 0).

(0, 3, 0, 0) means that POINT (1, 4) (the first (0) geometry returned in geoms) comes from the first (0) linestring (ABCDE), fourth (3) segment (2, 5) -> (0, 3) on the left hand side and first (0) linestring (FGHI), first (0) segment (-1, 6)->(2, 3) on the right hand side.

There is no multilinestring in the input, therefore all linestring lookback ids are 0.

We recognize that this adds complexity, but it’s because intersection is a powerful primitive. In a sequential API that takes a single pair of geometries for each intersection test, the calling code knows immediately which input geometries intersect. A parallel API that operates on arrays of data must provide details of which geometries intersect so that the caller can act on the results. We can’t wait to see what you create with pairwise_linestring_intersection!

Point-Polygon Distance Computation

Great strides were made to cuSpatial’s suite of Cartesian Distance computations (known as ST_Distance in PostGIS/MySQL/SpatiaLite parlance) in this release with the addition of point-polygon distance computation. To use this new feature, call the cuspatial.pairwise_point_polygon_distance API.

This example shows two pairs of inputs: Point E and Polygon ABC, and MultiPoint G, F and Polygon BCD. Use the API to compute the distances as in the following code.

>>> import cuspatial
>>> from shapely.geometry import *
>>> lhs = cuspatial.GeoSeries([MultiPoint([(0, 2)]), MultiPoint([(2 ,1), (2, 3)])])
>>> rhs = cuspatial.GeoSeries([Polygon([(0, 0), (2, 0), (1, 2), (0, 0)]),
… Polygon([(2, 0), (3, 2), (1, 2), (2, 0)])])
>>> cuspatial.pairwise_point_polygon_distance(lhs, rhs)
0 0.894427
1 0.000000
dtype: float64

Note that the second pair in the series contains a multipoint, where one of the points intersects with the polygon in the corresponding pair, leading to a distance of 0.

Binary Predicates: Equals and Intersects

cuSpatial 23.04 expands support for topological binary predicates by adding those that depend on point-to-point equality and/or intersection. This capability enables a large set of predicates for various geometry combinations.

points.geom_equals(points)
linestrings.geom_equals(linestrings)
polygons.geom_equals(linestrings)
multipoints.geom_equals(multipoints)

points.contains(points)
points.covers(points)
points.intersects(points)
points.within(points)
points.crosses(points)
points.overlaps(points)
points.intersects(points)

linestrings.intersects(points)
linestrings.intersects(multipoints)
linestrings.intersects(linestrings)

points.disjoint(linestring)
linestrings.disjoint(points)
linestrings.disjoint(linestrings)

linestrings.contains(points)
linestrings.covers(points
linestrings.crosses(points)
linestrings.crosses(linestrings)
linestrings.crosses(polygons)
linestrings.overlaps(points)
linestrings.overlaps(linestrings)
linestrings.overlaps(polygons)

The next release of cuSpatial aims for complete spatial predicate support.

Stop Sign Counting Notebook

cuSpatial adds a new notebook demonstrating spatial join capability with the point-in-polygon function. This notebook includes a custom QuadTree data structure to allow fast spatial joins on the GPU in just a few lines of code, for example:

# Build a quadtree with all stop signs in the US
stop_quadtree = QuadTree(d_stops, x_column=’x’, y_column=’y’)

# Pass CA zip code boundary polygons
stop_quadtree.set_polygon(CA_zipcode, poly_column=”WKT”)

# Join the stop signs and the zip code dataframe
stop_by_zipcode = stop_quadtree.point_left_join_polygon([“x”, “y”], [“ZCTA5CE10”])

This filters all stop signs that exist in every zip code in California. Finally, the notebook visualizes the count of the zipcodes on a map. Check out the notebook to see and interact with the map!

Other Improvements and Bug Fixes

In addition to the major features above, cuSpatial 23.04 includes a number of other improvements and bug fixes. The refactoring of cuSpatial’s C++ implementation is complete after about a year of effort, with a followup in 23.06 to improve header and source reorganization. Libcuspatial now comprises two layers: the generic header-only API where algorithms are implemented, and the cuDF column-based API which enables interoperability with libcudf.

The GDAL dependency has been removed, making the installation lighter weight, and paving the way to PIP installation support.

Documentation continues to improve, with updates and corrections to multiple sections of the User Guide.

Breaking Changes

In addition to changes to APIs to support GeoSeries, there are two significant API removals. The cubic spline interpolation function was not widely used and uncovered bugs in dependencies, limiting usability. This API also did not mesh well with the cuSpatial feature set, so it has been removed. The very limited shapefile reader has also been removed because of its heavy dependency on GDAL, limited parallelism, and limited feature set (only polygons and no other geometry types could be loaded). We plan to begin adding more capable and higher performance geospatial data I/O in a near future release.

Try cuSpatial 23.04 Today

Experience the new features and high performance of cuSpatial 23.04 and take your spatial analysis to new places. With the latest release, we have introduced several exciting features and enhancements that will revolutionize your spatial relationship computations. Try out the new features and new notebooks. Join our growing community of developers and researchers leveraging cuSpatial’s capabilities for efficient and robust spatial analytics.

cuSpatial User Guide

cuSpatial Discussions Forum

cuSpatial Github

cuSpatial 23.04 was originally published in RAPIDS AI on Medium, where people are continuing the conversation by highlighting and responding to this story.