<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Thileepan Beniel on Medium]]></title>
        <description><![CDATA[Stories by Thileepan Beniel on Medium]]></description>
        <link>https://medium.com/@thileepanbeniel?source=rss-c10bcff4077c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*lZLjkaliIHM44AQg.</url>
            <title>Stories by Thileepan Beniel on Medium</title>
            <link>https://medium.com/@thileepanbeniel?source=rss-c10bcff4077c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 00:40:58 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@thileepanbeniel/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Understanding Parallel Reduction with CUDA in modern GPUs]]></title>
            <link>https://medium.com/@thileepanbeniel/understanding-parallel-reduction-in-cuda-1a21701e09b1?source=rss-c10bcff4077c------2</link>
            <guid isPermaLink="false">https://medium.com/p/1a21701e09b1</guid>
            <category><![CDATA[nvidia]]></category>
            <category><![CDATA[cuda]]></category>
            <category><![CDATA[parallel-computing]]></category>
            <category><![CDATA[parallel-reduce]]></category>
            <category><![CDATA[h100]]></category>
            <dc:creator><![CDATA[Thileepan Beniel]]></dc:creator>
            <pubDate>Thu, 01 May 2025 19:16:41 GMT</pubDate>
            <atom:updated>2025-05-01T19:49:06.472Z</atom:updated>
            <content:encoded><![CDATA[<p>Reduction is a major primitive in the parallel coding patterns. It&#39;s a good place to start and understand, a step-by-step approach to get a more optimal way to solve this problem.</p><p>I have implemented versions with iterative improvements and tested them on modern H100. Some of the traditional optimization gains do not reflect in H100 as much compared to earlier architectures, and are discussed in this article. Let’s dive into details!</p><p>The code for the step-by-step implementations is available [6].</p><h3>What is a Reduction operation?</h3><p>Consider it as a sum of N numbers! A serial operation is very straightforward and can easily be done via a loop as given below.</p><pre>for (int i = 0; i &lt; N; i++){<br>    sum += arr[i];<br>}</pre><p>However, reduction is not limited to summation. Rather, any binary operation applied repeatedly to a set of numbers (generalized as elements) to produce a single result.</p><figure><img alt="Generalized reduction operation definition." src="https://cdn-images-1.medium.com/max/808/1*v07plll6erIG_KhusFSufw.png" /></figure><p>The operation will have a time complexity of O(N) for N numbers. For 1M float numbers, the following runtime and bandwidth on the AMD EPYC 9334 CPU are achieved.</p><pre>1M Numbers<br>Average Runtime: 33.2495 ms<br>Achieved Bandwidth: 0.117483 GB/s</pre><h3>Parallel Reduction</h3><p>Back to our original example of the addition of N numbers, we can convert the serial additions, distributed as independent additions executed in parallel, resulting in an execution tree as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*3oKxyHPBrlmUtfEej_K3dw.png" /><figcaption>Reduction addition operation graph of N=8 numbers.</figcaption></figure><p>When we approach this problem as a GPU problem, it falls into the category of a memory-bound problem [5], as for each number read, we are performing only one addition. Therefore, the most optimal implementation of reduction should reach the maximum bandwidth limits.</p><p>And the operation will have a time complexity of O(logN) for N numbers. However, each implementation is not always O(logN), as we need to consider Brent’s Theorem to get a more accurate time complexity. We are not going to touch it further theoretically here. I focus on using experimental evaluation for each implementation.</p><h3>Vanilla CUDA implementation</h3><p>The vanilla implementation reflects the above tree, where the total number of threads needed for N numbers to be added will be N/2. After the initial N/2 additions, the next level requires half of that, N/4. With that in mind, let&#39;s have a generic CUDA kernel to solve one level of addition. We can reuse this kernel to sum other levels until the final results are reached.</p><pre>#include &lt;reduction_kernels.h&gt;<br>__global__ void reduce_vanilla_kernel(float *arr, int m)<br>{<br>    int thread_id = threadIdx.x + blockIdx.x * blockDim.x;<br>    arr[thread_id] += arr[thread_id + m];<br>}<br><br>void reduce_vanilla(float* dev_arr, int arr_size)<br>{<br>    for (int i = arr_size / 2; i &gt; 0; i /= 2)<br>    {<br>        int threads = std::min(MAX_BLOCK_SIZE, i);<br>        int blocks = std::max(i/MAX_BLOCK_SIZE, 1);<br><br>        reduce_vanilla_kernel&lt;&lt;&lt;blocks, threads&gt;&gt;&gt;(dev_arr, i);<br>    }<br>    cudaDeviceSynchronize();<br>}</pre><h4>Results</h4><p>Here we call the CUDA kernel iteratively, folding the problem size in half each iteration. With testing 1M float numbers, on H100 following runtime and bandwidth are achieved.</p><pre>1M numbers<br>Average Runtime: 0.06841818 ms<br>Achieved Bandwidth: 57.11687 GB/s<br><br>4M numbers<br>Average Runtime: 0.07365079 ms<br>Achieved Bandwidth: 212.2103 GB/s<br><br>16M numbers<br>Average Runtime: 0.1576936 ms<br>Achieved Bandwidth: 396.3411 GB/s</pre><p>Vanilla GPU implementation achieved 486x performance improvement over linear reduction. However, this is not a fair comparison with CPU, as we can achieve decent parallelism in modern CPUs. The system I used has 128 threads in total and can have 16x parallel addition operations with the AVX-512 SIMD extensions (Check Discussion for more details).</p><h3>Thread Linear Addressing</h3><p>Can we improve this further? This question always arises, doesn&#39;t it? Let&#39;s analyze our previous implementation a bit deeper to find the bottleneck.</p><p>As we already discussed the memory-boundness of the problem, the kernel operation <strong>arr[thread_id] += arr[thread_id + m] </strong>reads two global memory addresses and then writes back to a global memory address. Secondly, on each iterative call of the kernel, half of all the addresses previously accessed are reaccessed! Can we reduce the number of global memory accesses?</p><p>The solution is fixing the grid and block sizes and accessing multiple array elements within a thread in a linear strided access.</p><pre>__global__ void reduce_thread_linear_kernel(float *arr, int arr_size)<br>{<br>    int thread_id = threadIdx.x + blockIdx.x * blockDim.x;<br>    float thread_sum = 0.0f;<br><br>    for (int i = thread_id; i &lt; arr_size; i += blockDim.x * gridDim.x)<br>    {<br>        thread_sum += arr[i];<br>    }<br><br>    arr[thread_id] = thread_sum;<br>}<br><br>float reduce_thread_linear(thrust::device_vector&lt;float&gt; dev_arr, int arr_size)<br>{<br>    reduce_thread_linear_kernel&lt;&lt;&lt;MAX_GRID_SIZE, MAX_BLOCK_SIZE&gt;&gt;&gt;(dev_arr.data().get(), arr_size);<br>    reduce_thread_linear_kernel&lt;&lt;&lt;1, MAX_BLOCK_SIZE&gt;&gt;&gt;(dev_arr.data().get(), MAX_BLOCK_SIZE*MAX_GRID_SIZE);<br>    reduce_thread_linear_kernel&lt;&lt;&lt;1, 1&gt;&gt;&gt;(dev_arr.data().get(), MAX_BLOCK_SIZE);<br>    float result = dev_arr[0];<br>    return result;<br>}</pre><p>Here we are storing the partial sum into a register and reusing the global space to store the partial sum systematically, regardless of the total elements N. The first kernel call reads the global array and stores it in <em>GRID_SIZE x BLOCK_SIZE </em>partial sums. Then the second kernel call reduces the partial sum to <em>BLOCK_SIZE</em>, and the final call reduces it to a single value. With an optimization of global memory accesses, we achieved the following runtime and bandwidth:</p><h4>Results</h4><pre>1M numbers<br>Average Runtime: 0.03751488 ms<br>Achieved Bandwidth: 104.2406 GB/s<br><br>4M numbers<br>Average Runtime: 0.04066029 ms<br>Achieved Bandwidth: 384.710291 GB/s<br><br>16M numbers<br>Average Runtime: 0.09951186 ms<br>Achieved Bandwidth: 628.1227 GB/s</pre><p>The results show that the performance improved 50–80% compared to the vanilla implementation. Though we target to reduce the number of global memory accesses, this version packs more positive changes. We&#39;ll be discussing it later.</p><h3>Shared Memory</h3><p>One of the biggest issues above code is using global memory to pass the intermediate partial sums to the next level. Additionally, the last kernel in the previous implementation only runs in a single thread, which is not good either.</p><p>To handle this, we are going to use the shared memory of the GPU, which has more than 40% higher bandwidth and is co-located in the same resource as L1 cache (we can trade off L1 cache for more shared memory, but not needed here). However, this is a very limited resource and we have to be carful to asses whether our problem is within available resources.</p><p>To know how much shared memory is available, you can refer to the spec of the Nvidia GPU you are using, or you can use the following code to get the numbers programmatically:</p><pre>int device_id;<br>cudaGetDevice(&amp;device_id);<br><br>cudaDeviceProp device_prop;<br>cudaGetDeviceProperties(&amp;device_prop, device_id);<br><br>int shared_mem_per_thread = device_prop.sharedMemPerBlock / <br>      device_prop.maxThreadsPerBlock;</pre><p>In H100, we have 48 KB shared memory per block with a maximum of 1024 threads. Therefore, we can have 48 bytes of shared memory per thread.</p><pre>__global__ void reduce_shared_kernel(float *arr, float *block_sum)<br>{<br>    extern __shared__ float shared_arr[];<br>    int local_id = threadIdx.x;<br>    int thread_id = threadIdx.x + blockIdx.x * blockDim.x;<br><br>    shared_arr[local_id] = arr[thread_id];<br>  <br>    __syncthreads();<br><br>    for (unsigned int i = 1; i &lt; blockDim.x; i *= 2) {<br>        if (local_id % (2 * i) == 0) {<br>            shared_arr[local_id] += shared_arr[local_id + i];<br>        }<br>        __syncthreads();<br>    }<br><br>    if (local_id == 0) {<br>        block_sum[blockIdx.x] = shared_arr[0];<br>    }<br>}<br><br>float reduce_shared(thrust::device_vector&lt;float&gt; dev_arr, <br>  thrust::device_vector&lt;float&gt;, int arr_size)<br>{<br>    // block_par_sum is size of <br>    //      (arr_size + MAX_BLOCK_SIZE - 1) / MAX_BLOCK_SIZE<br><br>    // First reduction pass<br>    reduce_shared_kernel&lt;&lt;&lt;grid_size, MAX_BLOCK_SIZE, MAX_BLOCK_SIZE * sizeof(float)&gt;&gt;&gt;(<br>        dev_arr.data().get(), block_par_sum.data().get());<br>    cudaDeviceSynchronize();<br><br>    // Second reduction passes<br>    while (grid_size &gt; MAX_BLOCK_SIZE) {<br>        int new_grid_size = (grid_size + MAX_BLOCK_SIZE - 1) / MAX_BLOCK_SIZE;<br>        reduce_shared_kernel&lt;&lt;&lt;new_grid_size, MAX_BLOCK_SIZE, MAX_BLOCK_SIZE * sizeof(float)&gt;&gt;&gt;(<br>            block_par_sum.data().get(), block_par_sum.data().get());<br>        cudaDeviceSynchronize();<br>        grid_size = new_grid_size;<br>    }<br>    // Final reduction pass<br>    reduce_shared_kernel&lt;&lt;&lt;1, grid_size, grid_size * sizeof(float)&gt;&gt;&gt;(<br>        block_par_sum.data().get(), dev_arr.data().get());<br>    cudaDeviceSynchronize();<br><br>    float result = dev_arr[0];<br>    return result;<br>}</pre><p>In the implementation, we are trying to get the block size calculated similarly to our Vanilla CUDA implementation. Each thread reads a single global value into shared memory. And reduction happens in shared memory in an interleaved manner.</p><p><strong>NOTE:</strong> The third kernel parameter represents the shared memory allocation. Here, we allocate 1 float element per thread.</p><h4>Result</h4><pre>1M numbers<br>Average Runtime: 0.03099144 ms<br>Achieved Bandwidth: 126.1401 GB/s<br><br>4M numbers<br>Average Runtime: 0.06853245 ms<br>Achieved Bandwidth: 228.065991 GB/s<br><br>16M numbers<br>Average Runtime: 0.2239382 ms<br>Achieved Bandwidth: 279.1155 GB/s</pre><p>We get mixed results from the previous step. Though shared memory versions are performing better than the vanilla implementation, it is underperforming than the linear addressing version for 4M and 16M problem sizes.</p><h3>Optimized Shared Memory with Linear Addressing</h3><p>The above shared memory implementation can be further improved by removing these bottlenecks:</p><ul><li>Kernel operation divergence at — <em>if (local_id % (2 * i) == 0).</em></li><li>The costly % operator is used.</li><li>GPU Shared memory bank conflict arises from interleaved addressing.</li><li>For larger problem sizes, the number of repeated kernel calls increases.</li></ul><p>We can fix this by mixing both shared memory and linear addressing techniques. This will improve global memory throughput.</p><p>In order to fix the interleaved addressing issue, we can make the shared memory reduction for the loop in a reverse manner, where the shared memory bank conflict will be eliminated as it is accessed thread linear, and the constant <em>mod</em> operation removed as well as a byproduct.</p><pre>__global__ void reduce_shared_linear_kernel(float *arr, float *block_sum, <br>      int arr_size)<br>{<br>    extern __shared__ float shared_arr[];<br>    int local_id = threadIdx.x;<br>    int thread_id = threadIdx.x + blockIdx.x * blockDim.x;<br>    int stride = blockDim.x * gridDim.x;<br>    float thread_sum = 0.0f;<br><br>    // Initialize shared memory + linear addressing<br>    for (int i = thread_id; i &lt; arr_size; i += stride) {<br>        thread_sum += arr[i];<br>    }<br>    shared_arr[local_id] = thread_sum;<br><br>    __syncthreads();<br><br>    // Reduction in shared memory<br>    for (int i = blockDim.x / 2; i &gt; 0; i /= 2) {<br>        if (local_id &lt; i) {<br>            shared_arr[local_id] += shared_arr[local_id + i];<br>        }<br>        __syncthreads();<br>    }<br><br>    if (local_id == 0) {<br>        block_sum[blockIdx.x] = shared_arr[0];<br>    }<br>}<br><br>float reduce_shared_linear(thrust::device_vector&lt;float&gt; dev_arr, <br>    thrust::device_vector&lt;float&gt; block_sum, int arr_size)<br>{<br>    // block_par_sum is size of MAX_GRID_SIZE<br><br>    reduce_shared_linear_kernel&lt;&lt;&lt;MAX_GRID_SIZE, MAX_BLOCK_SIZE, <br>        MAX_BLOCK_SIZE * sizeof(float)&gt;&gt;&gt;(dev_arr.data().get(),<br>        block_sum.data().get(),arr_size);<br><br>    reduce_shared_linear_kernel&lt;&lt;&lt;1, MAX_GRID_SIZE, <br>        MAX_GRID_SIZE * sizeof(float)&gt;&gt;&gt;(block_sum.data().get(), <br>        dev_arr.data().get(), MAX_GRID_SIZE);<br><br>    cudaDeviceSynchronize();<br><br>    float result = dev_arr[0];<br>    return result;<br>}</pre><h4>Results</h4><pre>1M numbers<br>Average Runtime: 0.0163517 ms<br>Achieved Bandwidth: 239.5355 GB/s<br><br>4M numbers<br>Average Runtime: 0.01993889 ms<br>Achieved Bandwidth: 784.200964 GB/s<br><br>16M numbers<br>Average Runtime: 0.07792045 ms<br>Achieved Bandwidth: 802.3276 GB/s</pre><p>The result showcases promising performance improvement compared to previous implementations.</p><h3>Warp unrolling</h3><p>One of the bottlenecks in the previous code is that in the reduction of shared memory, the active threads are reduced by half, each iteration. At the point it reaches 32, which is the warp size of an SM. Since instructions are synchronous inside warp, we can remove <em>__syncthreads()</em> and <em>if (local_id &lt; i)</em> checks.</p><p>Let&#39;s make a separate device function to unroll these last warp operations.</p><pre>__device__ inline void warp_reduce(volatile float *sdata, int tid)<br>{<br>    sdata[tid] += sdata[tid + 32];<br>    sdata[tid] += sdata[tid + 16];<br>    sdata[tid] += sdata[tid + 8];<br>    sdata[tid] += sdata[tid + 4];<br>    sdata[tid] += sdata[tid + 2];<br>    sdata[tid] += sdata[tid + 1];<br>}</pre><p>And the kernel changes added to incorporate this warp unroll function:</p><pre>__global__ void reduce_warp_unroll_kernel(float *arr, float *block_sum, int arr_size)<br>{<br>    extern __shared__ float shared_arr[];<br>    int local_id = threadIdx.x;<br>    int thread_id = threadIdx.x + blockIdx.x * blockDim.x;<br>    int stride = blockDim.x * gridDim.x;<br>    float thread_sum = 0.0f;<br><br>    // Initialize shared memory + linear addressing<br>    for (int i = thread_id; i &lt; arr_size; i += stride) {<br>        thread_sum += arr[i];<br>    }<br>    shared_arr[local_id] = thread_sum;<br><br>    __syncthreads();<br><br>    // Reduction in shared memory<br>    for (int i = blockDim.x / 2; i &gt; 32; i &gt;&gt;= 1) {<br>        if (local_id &lt; i) {<br>            shared_arr[local_id] += shared_arr[local_id + i];<br>        }<br>        __syncthreads();<br>    }<br><br>    if (local_id &lt; 32)<br>        warp_reduce(shared_arr, local_id);<br>    <br>    if (local_id == 0) {<br>        block_sum[blockIdx.x] = shared_arr[0];<br>    }<br>}</pre><h4>Results</h4><pre>1M numbers<br>Average Runtime: 0.01616397 ms<br>Achieved Bandwidth: 241.7591 GB/s<br><br>4M numbers<br>Average Runtime: 0.01954589 ms<br>Achieved Bandwidth: 800.091127 GB/s<br><br>16M numbers<br>Average Runtime: 0.07791206 ms<br>Achieved Bandwidth: 802.2152 GB/s</pre><h3>Fully Unrolling</h3><p>Though warp unrolling had a huge speedup in previous works [1], it has not improved much on modern GPU architecture as reflected in [3].</p><p>However, one of the ultimate steps is to optimize the reduction the fully unrolling the block. Here we templatize the kernel and will only call a particular template based on the block size.</p><p>First, we need to templatize the <em>warp_reduce</em> function. Since I&#39;m using C++17, using <strong><em>contexpr</em></strong> condition over the template parameter <em>blockSize</em> as this will reduce the number of instructions at compilation. If you&#39;re not using C++17, you can simply write them as normal conditions.</p><pre>template &lt;unsigned int blockSize&gt;<br>__device__ inline void warp_reduce(volatile float *sdata, int tid)<br>{<br>    if constexpr (blockSize &gt;= 64)<br>        sdata[tid] += sdata[tid + 32];<br>    if constexpr (blockSize &gt;= 32)<br>        sdata[tid] += sdata[tid + 16];<br>    if constexpr (blockSize &gt;= 16)<br>        sdata[tid] += sdata[tid + 8];<br>    if constexpr (blockSize &gt;= 8)<br>        sdata[tid] += sdata[tid + 4];<br>    if constexpr (blockSize &gt;= 4)<br>        sdata[tid] += sdata[tid + 2];<br>    if constexpr (blockSize &gt;= 2)<br>        sdata[tid] += sdata[tid + 1];<br>}</pre><p>We improve the previous kernel with template parameter <em>blockSize</em>, where we can explicitly unwrap based on the template used.</p><pre>template &lt;unsigned int blockSize&gt;<br>__global__ void reduce_full_unroll_kernel(float *arr, float *block_sum, <br>    int arr_size)<br>{<br>    extern __shared__ float shared_arr[];<br>    int local_id = threadIdx.x;<br>    int thread_id = threadIdx.x + blockIdx.x * blockDim.x;<br>    int stride = blockDim.x * gridDim.x;<br>    float thread_sum = 0.0f;<br><br>    // Initialize shared memory + linear addressing<br>    for (int i = thread_id; i &lt; arr_size; i += stride) {<br>        thread_sum += arr[i];<br>    }<br>    shared_arr[local_id] = thread_sum;<br><br>    __syncthreads();<br><br>    // Fully unrolling<br>    if constexpr (blockSize &gt;= 512){<br>        if (local_id &lt; 256) {  <br>          shared_arr[local_id] += shared_arr[local_id + 256];<br>        }<br>        __syncthreads();<br>    }<br>    if constexpr (blockSize &gt;= 256){<br>        if (local_id &lt; 128) {  <br>          shared_arr[local_id] += shared_arr[local_id + 128];<br>        }<br>        __syncthreads();<br>    }<br>    if constexpr (blockSize &gt;= 128){<br>        if (local_id &lt; 64) {  <br>          shared_arr[local_id] += shared_arr[local_id + 64];<br>        }<br>        __syncthreads();<br>    }<br><br>    // warp unroll <br>    if (local_id &lt; 32)<br>        warp_reduce&lt;blockSize&gt;(shared_arr, local_id);<br>    <br>    if (local_id == 0) {<br>        block_sum[blockIdx.x] = shared_arr[0];<br>    }<br>}</pre><h4>Results</h4><pre>1M numbers<br>Average Runtime: 0.01562337 ms<br>Achieved Bandwidth: 251.5402 GB/s<br><br>4M numbers<br>Average Runtime: 0.01894948 ms<br>Achieved Bandwidth: 828.944118 GB/s<br><br>16M numbers<br>Average Runtime: 0.07714151 ms<br>Achieved Bandwidth: 810.323 GB/s</pre><h3>Discussion</h3><h4>Setup</h4><p>The system I ran this experiment on is a H100 server with 2x AMD EPYC 9334 32-Core Processors with 2 hyper threads per core. All the experiments ran for a number of iterations (&gt;1000) for numerical accuracy.</p><h4>Result Summary</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CD3ZumPeNIevNaOrxDpj8Q.png" /><figcaption>1M Numbers — Summary</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8bRBVW8SBjbr46_AAgX6oA.png" /><figcaption>4M Numbers — Summary</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5fsPM63HKx3ZXx-frOXYcg.png" /><figcaption>16M Numbers — Summary</figcaption></figure><p>The result shows significant improvement over the vanilla implementation. However, the step speedup may not be as much as the previous works [1][3] mainly due to the more modern H100 performs better and is equipped with 3.9 TB/s global memory bandwidth [4] and the more optimal compilation of the modern CUDA toolchain.</p><p>On one hand, this gives better performance with a hastily written parallel design, and the justification for optimization needs to be recalibrated. Especially, the warp-divergent related and unroll optimizations are better handled by a hardware plus compiler combination. This is reflected in my implementation of the warp unroll and full unroll sections. The manual warp unroll, full unroll didn&#39;t make any tangible improvement, as the compiler pulls the work already over my manual shared memory and thread linear optimization.</p><p><strong>If you enjoyed this breakdown, follow me for more deep dives into the evolving landscape of HPC, including GPU programming, FPGA programming, and Scientific Computing. Thanks for Reading!</strong></p><h3>References</h3><p>[1] <a href="https://developer.download.nvidia.com/assets/cuda/files/reduction.pdf">https://developer.download.nvidia.com/assets/cuda/files/reduction.pdf</a></p><p>[2] R. Ansorge, <em>Programming in Parallel with CUDA: A Practical Guide</em>. Cambridge: Cambridge University Press, 2022.</p><p>[3] <a href="https://medium.com/@rimikadhara/7-step-optimization-of-parallel-reduction-with-cuda-33a3b2feafd8">https://medium.com/@rimikadhara/7-step-optimization-of-parallel-reduction-with-cuda-33a3b2feafd8</a></p><p>[4] <a href="https://www.nvidia.com/en-gb/data-center/h100/">https://www.nvidia.com/en-gb/data-center/h100/</a></p><p>[5] <a href="https://nanxiao.gitbooks.io/cuda-little-book/content/posts/compute-bound-and-memory-bound-kernels.html">https://nanxiao.gitbooks.io/cuda-little-book/content/posts/compute-bound-and-memory-bound-kernels.html</a></p><p>[6] <a href="https://github.com/benielT/CUDA_design_examples">https://github.com/benielT/CUDA_design_examples</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1a21701e09b1" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>