A fast communication-overlapping library for tensor/expert parallelism on GPUs.

🚀🚀🚀 This repository lists some awesome public CUDA, cuda-python, cuBLAS, cuDNN, CUTLASS, TensorRT, TensorRT-LLM, Triton, TVM, MLIR, PTX and High Performance Computing (HPC) projects.

Examples of CUDA implementations by Cutlass CuTe

CUTLASS and CuTe Examples

Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.

GEMM and Winograd based convolutions using CUTLASS

study of cutlass

Multiple GEMM operators are constructed with cutlass to support LLM inference.

Kernel Library Wheel for SGLang

A cutlass cute implementation of headdim-64 flashattentionv2 TensorRT plugin for LightGlue. Run on Jetson Orin NX 8GB with TensorRT 8.5.2.

Reading notes on the open source code of AI infrastructure (sglang, llm, cutlass, hpc, etc.)

pytorch implements block sparse

Lightweight and production level C++ Open source Library

Docs

This repository showcases common optimization techniques for kernels.