llm-awq

lyogavin/airllm5.4K

Inference optimizer

Optimizes large language model inference on limited GPU resources

vllm-project/vllm32K

LLM server

An inference and serving engine for large language models

optimalscale/lmflow8.3K

LLM toolkit

A toolkit for fine-tuning and inferring large machine learning models

autogptq/autogptq4.6K

Model optimizer

A package for optimizing large language models for efficient inference on GPUs and other hardware platforms.

opengvlab/llama-adapter5.8K

Instruction-following model tuner

An implementation of a method for fine-tuning language models to follow instructions with high efficiency and accuracy

alpha-vllm/llama2-accessory2.7K

LLM toolkit

An open-source toolkit for pretraining and fine-tuning large language models

intel/neural-compressor2.3K

Model optimizer

Tools and techniques for optimizing large language models on various frameworks and hardware platforms.

internlm/lmdeploy4.9K

LLM toolkit

A toolkit for optimizing and serving large language models

linkedin/liger-kernel3.8K

LLM optimization kernels

A collection of optimized kernels and post-training loss functions for large language models

modeltc/lightllm2.7K

LLM server

A Python-based framework for serving large language models with low latency and high scalability.

microsoft/lmops3.7K

LLM booster

A research initiative focused on developing fundamental technology to improve the performance and efficiency of large language models.

haotian-liu/llava20.7K

Visual Instruction System

A system that uses large language and vision models to generate and process visual instructions

ggerganov/llama.cpp69.2K

LLM inference software

Enables LLM inference with minimal setup and high performance on various hardware platforms

mlc-ai/mlc-llm19.4K

LLM Deployment Engine

A machine learning compiler and deployment engine for large language models

nomic-ai/gpt4all71.2K

LLM client

An open-source Python client for running Large Language Models (LLMs) locally on any device.