marlin
by IST-DASLab
FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.
AI summary
Matrix multiplier
An optimized FP16xINT4 matrix multiplication kernel for large language models
- stars
- 655
- forks
- 52
- watching
- 15
Similar projects
Found by comparing what the projects do, not just their names.
Inference accelerator
A Python library designed to accelerate model inference with high-throughput and low latency capabilities
Matrix calculator
A small size matrix handling module with linear algebra operations for MicroPython (Python3)
Handwritten digit recognizer
A Matlab implementation of a two-layer perceptron to recognize handwritten digits from the MNIST dataset.
google/gemmlowp1.8K
Matrix multiplier
A small C++ library for low-precision matrix multiplication
Matrix Driver
A Python driver for an 8x8 LED Matrix display using I2C communication
Matrix multiplication library
A library to support efficient mixed-precision matrix multiplications on GPUs for deep learning model deployment
HPCLib
A high-performance computing library providing optimized primitives for tensor and matrix operations
Array library
A Numpy-like library in Swift for multi-dimensional array and matrix operations
Matrix calculator
A library providing basic matrix arithmetic operations and functions for the MicroPython language.
Model optimizer
Tools and techniques for optimizing large language models on various frameworks and hardware platforms.
Matrix library
A fast and efficient matrix library for Elixir/Erlang with C implementation using CBLAS.
Matrix library
A linear algebra library with type-based static size checking for matrix operations.
Memory-efficient algorithms
A C++ library for compact data structures and algorithms optimized for memory efficiency and high performance
Matrix computation library
A Clojure library providing optimized native libraries for fast matrix and linear algebra computations on CPU and GPU.
Matrix arithmetic library
A toolbox implementing arithmetic operations for HODLR and HSS matrices in MATLAB.
