ktransformers
by kvcache-ai
A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations
AI summary
LLM optimizer
A flexible framework for LLM inference optimizations with support for multiple models and architectures
- stars
- 771
- forks
- 41
- watching
- 15
- awesome list
- 1
Similar projects
Found by comparing what the projects do, not just their names.
Model optimizer
Automates model building and deployment process by optimizing hyperparameters and compressing models for edge computing.
LLM optimizer
A tool for optimizing large language models by collecting feedback and metrics to improve their performance over time
Model optimizer
This project presents an optimization technique for large-scale image models to reduce computational requirements while maintaining performance.
brml/climin181
Optimizer
A Python package for gradient-based function optimization in machine learning
KFAC optimizer
Library providing an implementation of the K-FAC optimizer and curvature estimator for second-order optimization in neural networks.
Prompt Optimizer
A tool to reduce the complexity of text prompts to minimize API costs and model computations.
LLM benchmarker
A benchmarking framework for large language models
Language model optimizer
An optimization technique for large language models allowing them to run on limited hardware resources without significant performance loss.
LLM framework
A high-performance LLM written in Python/Jax for training and inference on Google Cloud TPUs and GPUs.
Memory optimizer
A JAX transform that simplifies the training of large language models by reducing memory usage through low-rank adaptation.
System optimizer
Toolset for optimizing and calibrating physical systems using machine learning and quantum computing
LLM engine
An LLM inference and serving engine with high performance, flexibility, and support for various hardware platforms.
Efficient LLM
A large language model with improved efficiency and performance compared to similar models
Deep learning optimizer
An open-source project providing hardware accelerated, batchable and differentiable optimizers in JAX for deep learning.
Conversational AI model
A large language model with 70 billion parameters designed for chatbot and conversational AI tasks
Featured in 1 awesome list
Each link jumps to the spot where the list mentions ktransformers.
