<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Grant Uy on Medium]]></title>
        <description><![CDATA[Stories by Grant Uy on Medium]]></description>
        <link>https://medium.com/@grantuy?source=rss-85549627e07f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*IhuVelcb4nsMmDdX</url>
            <title>Stories by Grant Uy on Medium</title>
            <link>https://medium.com/@grantuy?source=rss-85549627e07f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 10:01:42 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@grantuy/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[A tour of PyG’s data loaders]]></title>
            <link>https://medium.com/stanford-cs224w/a-tour-of-pygs-data-loaders-9f2384e48f8f?source=rss-85549627e07f------2</link>
            <guid isPermaLink="false">https://medium.com/p/9f2384e48f8f</guid>
            <category><![CDATA[graph-machine-learning]]></category>
            <category><![CDATA[graph-neural-networks]]></category>
            <category><![CDATA[pyg]]></category>
            <category><![CDATA[graphml]]></category>
            <category><![CDATA[gnn]]></category>
            <dc:creator><![CDATA[Grant Uy]]></dc:creator>
            <pubDate>Mon, 15 May 2023 18:50:18 GMT</pubDate>
            <atom:updated>2023-05-15T18:50:18.992Z</atom:updated>
            <content:encoded><![CDATA[<p><em>By Grant Uy and Huijian Cai as part of the Stanford CS224W course project.</em></p><p>Graph neural networks (GNNs) are powerful tools with broad applicability to many domains because real-world networks, like social networks, are often well modeled by graphs.</p><p>However, real-world networks tend to be massive and most useful at scale. For example, imagine Amazon’s “product co-purchasing” graph, where edges connect products (nodes) that are bought together. This is useful at Amazon’s scale for recommending or classifying products, but with millions of products, the graph size makes applying ML difficult.</p><p>Such graphs are too large to fit into GPU memory, so when implementing a GNN, we have to <em>batch </em>the data. While this is a simple idea in principle, deciding on how to split up the graph is surprisingly nuanced and affects downstream performance, both training time and accuracy.</p><p>PyG, built on PyTorch, is a powerful GNN library that provides several different graph data loaders, which we’ll explore in this post!</p><h3>Loader basics</h3><p>PyG’s loaders are built on PyTorch DataLoaders. A DataLoader is an abstraction over a dataset that enables batching.</p><p>Each PyG loader accepts a Data object, which represents a graph, as well as additional parameters that control how nodes and edges are sampled into batches.</p><h4>RandomNodeLoader</h4><p>The most straightforward loader is RandomNodeLoader, which randomly samples batches of nodes from the input graph. For example, given a Data object data, we can construct one like this:</p><pre>from torch_geometric.loader import RandomNodeLoader<br><br>random_loader = RandomNodeLoader(data, num_parts=5, shuffle=True)</pre><p>num_parts controls the batch size. This RandomNodeLoader splits the graph into 5 subgraphs by partitioning the <em>N</em> nodes into 5 subsets. shuffle tells PyG to pick these subsets randomly. Otherwise, the first batch would always contain the first <em>N/5</em> nodes (by node ID), and so on. Generally, shuffling is good for training so that the model sees different batches for every epoch.</p><p>For example, here’s a toy graph batched with this loader:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/293/1*De1dS0wA_lODT17v5deFPw.png" /><figcaption>Each node is colored based on which of the 5 batches it’s in.</figcaption></figure><p>Notice that the subgraphs tend to be disconnected since the partitioning was random. While RandomNodeLoader is a useful introduction, we’ll see that more cleverly dividing up the graph can improve performance!</p><p><a href="https://colab.research.google.com/drive/1S9TVPALutxOHJiwD1u_gRpkxKESc1Vfv"><em>This Colab</em></a><em> contains a simple library we built to generate and visualize batches in toy graphs if you want to play around with these loaders! We used it for all of the visualizations in this post.</em></p><h4>Retrieving batches</h4><p>Given a loader, we can retrieve the batches by iterating through it, like for PyTorch DataLoaders. In a training loop, this might look like:</p><pre>for batch in random_loader:<br>  # &quot;batch&quot; is another Data instance representing the subgraph<br><br>  # ...<br><br>  # if the original Data had properties like &quot;x&quot; and &quot;edge_index&quot;,<br>  # batch will have relevant slices of them:<br>  out = model(batch.x, batch.edge_index)<br><br>  # ...</pre><p>len(random_loader) returns the total number of batches, and the batch indices can be retrieved in a loop via enumerate(random_loader).</p><h4>Data splits</h4><p>In practice, the graph is usually partitioned into training, validation, and test sets. To handle this, we typically stash train_mask, val_mask, and test_mask attributes in the original Data , and then those masks will be propagated to the batches.</p><p>For example, PyG’s <a href="https://pytorch-geometric.readthedocs.io/en/latest/generated/torch_geometric.transforms.RandomNodeSplit.html#torch-geometric-transforms-randomnodesplit">RandomNodeSplit</a> transform can be used to add these masks to the Data <em>before </em>passing it to the loader. (See <a href="https://pytorch-geometric.readthedocs.io/en/latest/get_started/introduction.html">PyG’s documentation</a> for details, which are beyond the scope of this post!)</p><p>Using this method, each batch will contain a mix of train, validation, and test nodes. Here’s one batch from above:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/293/1*mh6tOI-Dn41fm4HNGNie9g.png" /><figcaption>The red batch is itself split into training (same red as above), validation (dark red), and test (light red).</figcaption></figure><p>In this case, we’d use the same loader regardless of which split we’re iterating over (e.g., during the training loop vs. during model evaluation), but we’d use the masks to select the appropriate subset of nodes. For example, batch.x[batch.train_mask] would select only the x values for nodes in the training set.</p><p>Some loaders, such as NeighborLoader, specially handle splits and accept the split mask (e.g., train_mask) when the loader is constructed. In such cases, we’ll typically have multiple loaders: train_loader, val_loader, and test_loader. More on this below!</p><h4>Other useful parameters</h4><p>A couple more parameters that are the same as in PyTorch are:</p><ul><li>num_workers: enables multi-process data loading [<a href="https://pytorch.org/docs/stable/data.html#multi-process-data-loading">documentation</a>]</li><li>drop_last: skips the last batch if it’s a different size (e.g., when batch size doesn’t perfectly divide total size) [<a href="https://pytorch.org/docs/stable/data.html#automatic-batching-default">documentation</a>]</li></ul><h3>NeighborLoader</h3><p>NeighborLoader is another widely applicable node-based loader type. We’ll dive into it since it’s so foundational!</p><h4>Basics</h4><p>NeighborLoader partitions nodes into batches like RandomNodeLoader, but for every batch, NeighborLoader also adds neighboring nodes to the subgraph. Each node is a “root” (or “starter”) node in exactly 1 batch, but that node can appear in other batches if pulled in as a neighbor.</p><p>A NeighborLoader can be instantiated like:</p><pre>from torch_geometric.loader import NeighborLoader<br><br>neighbor_loader = NeighborLoader(data,<br>                                 num_neighbors=[3],<br>                                 batch_size=2,<br>                                 shuffle=True)</pre><p>Suppose data contains 40 nodes. Then, this will partition the 40 nodes into 20 batches of size 2. For each batch, it will add3 neighbors for each of the 2 “root” nodes. Hence, each batch will be a subgraph with 8 total nodes (not accounting for overlaps). For example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/293/1*25IxzoBcOt3_QC5Fs_D__w.png" /><figcaption>One batch from this NeighborLoader.</figcaption></figure><p>Unlike RandomNodeLoader, this results in a more connected subgraph!</p><h4>Parameterization</h4><p>By default, the resulting subgraph only contains edges that were “followed” when adding neighbors. To change this behavior, pass directed=False to the NeighborLoader. (Despite its name, directed isn’t used to distinguish directed and undirected graphs!) When directed=False, the full node-induced subgraph is used instead. In the example above, directed=False would add these 2 red edges:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/293/1*J3z8kfPWspG7aYuET2UfoA.png" /><figcaption>The red edges connect selected nodes but were not “followed” when adding neighbors.</figcaption></figure><p>Additionally, NeighborLoader supports multiple “hops” for adding neighbors. In the example above, num_neighbors=[3] indicates that we want 1 round of adding 3 neighbors.</p><p>Suppose we set num_neighbors=[3,2] (with batch_size=2). Any given batch still starts with 2 “root” nodes, with 3 neighbors added for each. But, for each of the 6 added neighbors, 2 neighbors of those neighbors are added. Here’s 1 color-coded batch for our graph:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/460/1*LqbtslgZmsdtH664XJ_osQ.png" /><figcaption>The 2 “root nodes” are red, the 3 added neighbors for each “root” node are in blue, and the neighbors-of-neighbors are in green. (There are also some overlaps, hence the green edge connecting blue nodes.)</figcaption></figure><p>Adding more hops increases the neighborhood sizes but exponentially increases the subgraph’s size! We’ll see later how this tradeoff affects performance.</p><h4>Data splits</h4><p>NeighborLoader accepts a split mask like train_mask via the parameter called input_nodes. For example:</p><pre>train_loader = NeighborLoader(data, input_nodes=data.train_mask,<br>                              num_neighbors=[3,2], batch_size=2,<br>                              directed=False, shuffle=True)<br><br>val_loader   = NeighborLoader(data, input_nodes=data.val_mask,<br>                              num_neighbors=[3,2], batch_size=2,<br>                              directed=False, shuffle=True)<br><br>test_loader  = NeighborLoader(data, input_nodes=data.test_mask,<br>                              num_neighbors=[3,2], batch_size=2,<br>                              directed=False, shuffle=True)</pre><p>When input_nodes is set to a mask, only matching nodes will be selected as “root nodes.” These loaders will be used separately; for example, we can iterate over onlytrain_loader during model training.</p><p>Each batch from train_loader can still contain nodes not in the training set! (Similar for val_loader and test_loader.) The mask only affects “root” nodes, not which nodes are added as neighbors. Hence, to avoid validation/test nodes during training, you still have to apply train_mask. For example, a training loop might look like:</p><pre>for batch in train_loader:<br>  # ...<br><br>  # Run the model on all nodes in the batch so that message passing<br>  # still works for non-training nodes<br>  out = model(batch.x, batch.edge_index)<br><br>  # Compute the loss only for training nodes.<br>  loss = loss_fn(out[batch.train_mask], batch.y[batch.train_mask])<br><br>  # ...</pre><p>For a more complete example, see <a href="https://colab.research.google.com/drive/1y2UYgco2Rcq4ru_UxSEbZbc0052MkW1-">our Colab</a>.</p><h4>Applications</h4><p>We recommend NeighborLoader as a good starting point for most node-based tasks. RandomNodeLoader is simpler but tends to perform poorly on large, sparse graphs (as most real-world graphs are!) due to the disconnectedness of batches. NeighborLoader also:</p><ul><li>produces more connected subgraphs but is still simple</li><li>conveniently handles data splits (e.g., won’t produce edge-case batches without training nodes)</li><li>supports heterogeneous graphs out-of-the-box!</li></ul><h4>Performance comparison</h4><p>We set up a basic ML pipeline for 2 <a href="https://ogb.stanford.edu/">Open Graph Benchmark (OGB)</a> [1] node classification datasets, <a href="https://ogb.stanford.edu/docs/nodeprop/#ogbn-products"><strong>ogbn-products</strong></a> and <a href="https://ogb.stanford.edu/docs/nodeprop/#ogbn-proteins"><strong>ogbn-proteins</strong></a>, and then varied NeighborLoader parameters to see the effects on downstream performance, namely accuracy and training time. You can follow along in <a href="https://colab.research.google.com/drive/1y2UYgco2Rcq4ru_UxSEbZbc0052MkW1-">our Colab</a>!</p><p>For simplicity, we used a 3-layer <a href="https://pytorch-geometric.readthedocs.io/en/2.0.1/modules/nn.html#torch_geometric.nn.models.GCN">GCN</a> provided by PyG, since our goal was not to optimize the rest of the pipeline.</p><p><a href="https://ogb.stanford.edu/docs/nodeprop/#ogbn-products"><strong>ogbn-products</strong></a> is a product co-purchasing network from Amazon [2], as described above, with 200K training nodes. The goal is to classify products into 47 categories. Our NeighborLoaders produced these results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/504/1*1o-TW5p21nOFYt6pRJ81hw.png" /><figcaption>Naming scheme: “neighbor-{num_neighbors}-{batch_size}-{directed}”</figcaption></figure><p>We had a few key takeaways:</p><ul><li>Models with smaller subgraphs (e.g., neighbor-1-256, which had num_neighbors=[1] and batch_size=256) trained faster but had worse accuracy.</li><li>directed=True (the default!) performed <em>significantly </em>worse than directed=False, since it reduces the connectivity of the subgraphs, and our 3-layer GCN considers 3-hop neighborhoods.</li><li>Loaders with more than 1 neighbor hop performed the best overall, but they also took significantly longer to train.</li></ul><p>Here’s the same data, sorted by training time to show the relationship with accuracy:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/1*jn3PHCH1wzQDe5mrQkF0Mg.png" /><figcaption>Longer training time tended to produce better accuracy, but the tradeoff isn’t that simple!</figcaption></figure><p>However, parameter effects vary wildly by use case (e.g., problem domain) due to different graph structures! We repeated the same experiment for <a href="https://ogb.stanford.edu/docs/nodeprop/#ogbn-proteins"><strong>ogbn-proteins</strong></a>, which encodes associations between proteins [3]. For a multi-label binary classification task for protein functions, we had the following results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/504/1*laIIpqaNjaF70TwnMoBfkg.png" /><figcaption>Naming scheme: “proteins-neighbor-{num_neighbors}-{batch_size}-{directed}”</figcaption></figure><p>For instance, notice that directed=True performed <em>better </em>for this case! Loader experimentation is important for optimizing any ML pipeline.</p><h3>Other node-based loaders</h3><h4>ClusterLoader</h4><p>ClusterLoader relies on ClusterData, which partitions the node into fixed clusters:</p><pre>from torch_geometric.loader import ClusterData, ClusterLoader<br><br># In practice, num_parts would be much higher than 4!<br>cluster_data = ClusterData(data, num_parts=4)<br>cluster_loader = ClusterLoader(cluster_data, batch_size=2, shuffle=True)</pre><p>num_parts is the desired number of clusters, like for RandomNodeLoader. batch_size is the number of clusters to include per batch. In this case, ClusterData forms 4 node clusters, and ClusterLoader randomly picks 2 of those clusters for each batch. The resulting batch represents the node-induced subgraph for all of the nodes in those 2 clusters. For example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/586/1*8BzDPbUgVPovZl6IFKwWhQ.png" /><figcaption>Left: clusters from ClusterData. Right: 1 batch from ClusterLoader.</figcaption></figure><p>Notably, this includes the edges between the two clusters.</p><p>ClusterLoader is useful when the underlying graph has “community-like” structures, like in our example graph or in a social network. ClusterData can handle “communities” of varying sizes, whereas NeighborLoaderalways picks subgraphs of roughly the same shape. Additionally, NeighborLoader can’t guarantee that an entire related cluster of nodes will make it into the same batch, especially when the cluster is large.</p><p>For ClusterLoader, every node appears in exactly one batch, unlike for NeighborLoader. Hence, ClusterLoader treats nodes more equally, whereas highly-connected nodes appear more frequently inNeighborLoader batches. Whether this is good depends on the use case: highly-connected nodes could be important (e.g., for product classification) or poorly representative of a population.</p><h4>GraphSAINTRandomWalkSampler</h4><p>GraphSAINTRandomWalkSampler, similar to NeighborLoader, picks batch_size “root” nodes for each batch and then adds surrounding nodes, but instead of adding direct neighbors, it starts a random walk and adds all encountered nodes.</p><p>Unlike NeighborLoader, the “root” nodes are not from a partition of the original set of nodes. Rather, they’re sampled with replacement and normalization. Check out the GraphSAINT paper [4] for details!</p><pre>saint_walk_sampler = GraphSAINTRandomWalkSampler(data,<br>                                                 batch_size=2,<br>                                                 walk_length=6,<br>                                                 num_steps=3)</pre><p>GraphSAINTRandomWalkSampler’s key parameters are:</p><ul><li>batch_size: the number of “root” nodes (not each batch’s final number of nodes!)</li><li>walk_length: how many steps to take on each random walk starting from a “root” node</li><li>num_steps: the number of batches to create (not to be confused with walk_length!)</li></ul><p>For example, the above parameters produce batches like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/293/1*zi8ojOSeuXigmaZwSMD0LA.png" /><figcaption>One batch from the GraphSAINTRandomWalkSampler.</figcaption></figure><p>This loader is useful when you care more about “depth” in the graph than “breadth.” NeighborLoader is similar to breadth-first exploration, whereas random walks can be more like depth-first exploration. This can allow more information to pass between nodes that are farther away, especially when you’re using a deep GNN with many layers.</p><p>That said, there are other methods (outside the scope of this post) to increase graph connectivity even when using other loaders, such as adding a “virtual node” connected to all nodes [5].</p><h4>Performance comparison</h4><p>Choosing the right loader type has meaningful performance implications, sometimes even more than parameterization! We repeated the NeighborLoader experiment on <strong>ogbn-products</strong> using different loader types (and fewer epochs):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/504/1*b3a1wS_JP05cqw-OxkR3cQ.png" /><figcaption>Results for 4 loader types on ogbn-products. “neighbor_best_5epochs” is the top-performing model from above, but we used 10 epochs in the last experiment.</figcaption></figure><p>A couple takeaways:</p><ul><li>RandomNodeLoader predictably performed poorly in terms of both accuracy and training time</li><li>GraphSAINTRandomWalkSampler achieved better accuracy than NeighborLoader in significantly less time</li></ul><p>Like parameterization, the choice of loader type also depends heavily on the use case and the graph structure. For <strong>ogbn-proteins</strong>, the same experiment had very different results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/504/1*Z6UmA3QizsJTBTZKyQDhMg.png" /><figcaption>Results for different loader types on ogbn-proteins, similar to above.</figcaption></figure><p>In particular, notice that the NeighborLoader outperformed the other types in this case!</p><p>While this analysis is neither rigorous nor perfect, it’ll still hopefully inspire you to try different loaders! See <a href="https://colab.research.google.com/drive/1y2UYgco2Rcq4ru_UxSEbZbc0052MkW1-">our Colab</a> for details.</p><h4>Even more loader types</h4><p>We couldn’t cover every PyG loader type in depth, but a few worth calling out are:</p><ul><li><strong>HGTLoader</strong>: useful when working with heterogeneous graphs because it samples different types of nodes in a balanced way</li><li><strong>ImbalancedLoader</strong>: useful for classification tasks with low-frequency labels, because it allows for oversampling based on the node class</li><li><strong>ShaDowKHopSampler</strong>: like a more advanced version of NeighborLoader that more carefully picks subgraphs and locally smooths them</li></ul><h3>LinkNeighborLoader</h3><p>PyG also supports loaders that edges instead of nodes. This is useful for edge-related tasks like link prediction. LinkNeighborLoader, NeighborLoader’s edge-based analog, is a great starting point!</p><h4>Negative sampling</h4><p>Link prediction is essentially binary classification, predicting whether an edge would exist. The graph’s existing edges are considered positive examples, but we also need to make predictions on negative examples, edges that are <em>not </em>actually in the graph. These “negative edges” need to be sampled too! Thankfully, PyG supports this out-of-the-box!</p><h4>Parameters</h4><p>LinkNeighborLoader is constructed similarly to NeighborLoader:</p><pre>link_neighbor_loader = LinkNeighborLoader(<br>    data,<br>    num_neighbors=[2],<br>    neg_sampling_ratio=1.0,<br>    batch_size=2,<br>    shuffle=True)</pre><p>For each batch, LinkNeighborLoader selects batch_size positive edges and neg_sampling_ratio*batch_size negative edges. Then, for each node (i.e., each edge endpoint), neighbors are added like for NeighborLoader. For example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/698/1*mBZX8q-HBzjW944Xmjpgrg.png" /><figcaption>One batch, with and without negative sampling. Red edges are positive edges; blue ones are negative edges. Pink edges are added “neighbor” edges.</figcaption></figure><p>The resulting batches have two edge indexes:</p><ul><li>batch.edge_index for edges for message passing</li><li>batch.edge_label_index for positive/negative edges to be labeled (with the ground truth in batch.edge_label)</li></ul><h4>Data splits</h4><p>LinkNeighborLoader is typically constructed in conjunction with RandomLinkSplit (instead of using masks) to handle data splits:</p><pre>transform = T.RandomLinkSplit(<br>    num_val=0.2,  # fraction of data held out for validation<br>    num_test=0.2, # fraction of data held out for test<br>    neg_sampling_ratio=2.0,<br>    add_negative_train_samples=False,<br>)<br>train_data, val_data, test_data = transform(data)<br><br>batch_size = 2<br>num_neighbors = [3,2]<br>train_loader = LinkNeighborLoader(train_data,<br>                                  num_neighbors=num_neighbors,<br>                                  batch_size=batch_size,<br>                                  # these are only set for train!<br>                                  neg_sampling_ratio=2.0,<br>                                  shuffle=True)<br>val_loader   = LinkNeighborLoader(val_data,<br>                                  num_neighbors=num_neighbors,<br>                                  batch_size=batch_size)<br>test_loader  = LinkNeighborLoader(test_data,<br>                                  num_neighbors=num_neighbors,<br>                                  batch_size=batch_size)</pre><p>RandomLinkSplit adds negative edges to val_data and test_data (but not for train_data due to add_negative_train_samples=False) when they’re constructed so that these negative edges remain constant for every run through the validation/test data. Conversely, negative edges for training are added by LinkNeighborLoader to each <em>batch</em> so that the negative edges differ for every training epoch. Hence, neg_sampling_ratio is only set for train_loader!</p><h4>Applications</h4><p>Unlike node-based loaders, LinkNeighborLoader is a more obvious choice for edge-related tasks like link prediction and edge property prediction. There aren’t many options for edge-based loaders, and LinkNeighborLoader is the only one that supports on-the-fly negative sampling.</p><h4>Performance comparison</h4><p>We created <a href="https://colab.research.google.com/drive/1GrAxHyZCZ13jpTkMy9vVO_v_U9nHDdvB">this Colab</a> to demonstrate how to apply an edge-level loader to a heterogenous graph, largely following <a href="https://medium.com/@pytorch_geometric/link-prediction-on-heterogeneous-graphs-with-pyg-6d5c29677c70">PyG’s blog post</a>, which goes into more detail! Similar to above, our focus was to demonstrate how LinkNeighborLoader parameters affect downstream performance.</p><p>We used the <a href="https://pytorch-geometric.readthedocs.io/en/latest/generated/torch_geometric.datasets.MovieLens.html"><strong>MovieLens</strong></a> dataset [6], a heterogeneous graph containing ratings (edges) from users for movies (both nodes), for link prediction. Predicting missing ratings can be useful in recommending movies to users.</p><p>Using a model similar to the one <a href="https://medium.com/@pytorch_geometric/link-prediction-on-heterogeneous-graphs-with-pyg-6d5c29677c70">in the blog post</a>, we got the following results from several LinkNeighborLoader parameterizations:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I38y9dhhiOfMnJ95LV8dYQ.png" /><figcaption>Naming scheme: “linkneighbor-{neg_sampling_ratio}-{num_neighbors}-{batch_size}-{directed}”</figcaption></figure><p>The main takeaway is that even seemingly minor parameter changes can significantly affect performance, but a couple other call-outs are:</p><ul><li>large batch sizes significantly decreased overall training time since they reduced the total overhead from non-labeled message passing edges</li><li>adding more neighbors increased training time but didn’t necessarily improve performance</li></ul><p>See <a href="https://colab.research.google.com/drive/1GrAxHyZCZ13jpTkMy9vVO_v_U9nHDdvB">our Colab</a> and <a href="https://medium.com/@pytorch_geometric/link-prediction-on-heterogeneous-graphs-with-pyg-6d5c29677c70">the original blog post</a> for more details!</p><h3>Summary</h3><p>To recap, we recommend NeighborLoader as a starting point for node-based tasks and NeighborLinkLoader for edge-based tasks.</p><p>For node-based tasks, we’d suggest additionally looking into:</p><ul><li>ClusterLoader if your graph has many “community-like” structures</li><li>GraphSAINTRandomWalkSampler if “far away” nodes matter more (relative to local neighborhoods), especially for deeper GNNs</li><li>ShaDowKHopSampler for more advanced neighbor sampling</li><li>HGTLoader for handling heterogeneous nodes in a more balanced way</li><li>ImbalancedLoader for skewed node classification problems</li></ul><h3>Closing</h3><p>We hope this was a helpful tour of PyG’s loaders! If you want to dive into the code, here are all the Colabs we used in this post:</p><ul><li><a href="https://colab.research.google.com/drive/1S9TVPALutxOHJiwD1u_gRpkxKESc1Vfv">Toy graph generation and visualization</a></li><li><a href="https://colab.research.google.com/drive/1y2UYgco2Rcq4ru_UxSEbZbc0052MkW1-">Node classification pipeline for ogbn-products and ogbn-proteins</a></li><li><a href="https://colab.research.google.com/drive/1GrAxHyZCZ13jpTkMy9vVO_v_U9nHDdvB">Link prediction pipeline for MovieLens</a></li><li><a href="https://colab.research.google.com/drive/1EfXjPUPFR5TiZHSXxrD4yrpRifx-r9bY">Data visualization</a></li></ul><h4>Further reading</h4><ul><li><a href="https://pytorch-geometric.readthedocs.io/en/latest/modules/loader.html">PyG loader documentation</a></li><li><a href="https://pytorch.org/tutorials/beginner/basics/data_tutorial.html">PyTorch data loading tutorial</a></li><li><a href="https://medium.com/@pytorch_geometric/link-prediction-on-heterogeneous-graphs-with-pyg-6d5c29677c70">Link Prediction on Heterogeneous Graphs with PyG</a></li></ul><h4>References</h4><p>[1] Hu et al. 2020. <a href="https://arxiv.org/abs/2005.00687">https://arxiv.org/abs/2005.00687</a><br>[2] Bhatia et al. 2016. <a href="http://manikvarma.org/downloads/XC/XMLRepository.html">http://manikvarma.org/downloads/XC/XMLRepository.html</a><br>[3] Szklarczyk et al. 2019. <a href="https://doi.org/10.1093/nar/gky1131">https://doi.org/10.1093/nar/gky1131</a><br>[4] Zeng et al. 2020. <a href="https://arxiv.org/abs/1907.04931">https://arxiv.org/abs/1907.04931</a><br>[5] CS224W 2023 lecture 6, slide 18: <a href="http://web.stanford.edu/class/cs224w/slides/06-GNN3.pdf">http://web.stanford.edu/class/cs224w/slides/06-GNN3.pdf</a><br>[6] Harper et al. 2015. <a href="https://doi.org/10.1145/2827872">https://doi.org/10.1145/2827872</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9f2384e48f8f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/stanford-cs224w/a-tour-of-pygs-data-loaders-9f2384e48f8f">A tour of PyG’s data loaders</a> was originally published in <a href="https://medium.com/stanford-cs224w">Stanford CS224W: Machine Learning with Graphs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>