exllama
by turboderp
A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.
AI summary
GPU-based chat model
A re-implementation of Llama for efficient use with quantized weights on modern GPUs.
- stars
- 2.8K
- forks
- 220
- watching
- 37
- awesome list
- 1
Similar projects
Found by comparing what the projects do, not just their names.
LLM trainer
A native PyTorch library for training large language models using distributed parallelism and optimization techniques.
ggerganov/llama.cpp69.2K
LLM inference software
Enables LLM inference with minimal setup and high performance on various hardware platforms
nomic-ai/gpt4all71.2K
LLM client
An open-source Python client for running Large Language Models (LLMs) locally on any device.
haotian-liu/llava20.7K
Visual Instruction System
A system that uses large language and vision models to generate and process visual instructions
Model optimizer
A package for optimizing large language models for efficient inference on GPUs and other hardware platforms.
LLM toolkit
An open-source toolkit for pretraining and fine-tuning large language models
Language model trainer
Provides a framework for training large-scale language models on GPUs with advanced features and optimizations.
Instruction-following model tuner
An implementation of a method for fine-tuning language models to follow instructions with high efficiency and accuracy
Language Model
An implementation of a large language model using the nanoGPT architecture
plasma-umass/scalene12.3K
Profiler
A high-performance Python profiler that analyzes CPU, GPU, and memory usage, providing detailed information and AI-powered optimization suggestions.
GPU tensor library
A Python library providing tensors and dynamic neural networks with strong GPU acceleration
nvidia/megatron-lm10.8K
LLM trainer
A framework for training large language models using scalable and optimized GPU techniques
LLM Quantizer
An open-source software project that enables efficient and accurate low-bit weight quantization for large language models.
LLaMA Tuner
A tool for efficiently fine-tuning large language models across multiple architectures and methods.
LLM library
A PyTorch library for easily authoring and experimenting with large language models
Featured in 1 awesome list
Each link jumps to the spot where the list mentions exllama.
