airllm

modeltc/lightllm2.7K

LLM server

A Python-based framework for serving large language models with low latency and high scalability.

ggerganov/llama.cpp69.2K

LLM inference software

Enables LLM inference with minimal setup and high performance on various hardware platforms

mit-han-lab/llm-awq2.6K

LLM Quantizer

An open-source software project that enables efficient and accurate low-bit weight quantization for large language models.

internlm/lmdeploy4.9K

LLM toolkit

A toolkit for optimizing and serving large language models

opengvlab/llama-adapter5.8K

Instruction-following model tuner

An implementation of a method for fine-tuning language models to follow instructions with high efficiency and accuracy

vllm-project/vllm32K

LLM server

An inference and serving engine for large language models

optimalscale/lmflow8.3K

LLM toolkit

A toolkit for fine-tuning and inferring large machine learning models

fminference/flexllmgen9.2K

Batch processor

Generates large language model outputs in high-throughput mode on single GPUs

alpha-vllm/llama2-accessory2.7K

LLM toolkit

An open-source toolkit for pretraining and fine-tuning large language models

young-geng/easylm2.4K

LLM framework

A framework for training and serving large language models using JAX/Flax

nomic-ai/gpt4all71.2K

LLM client

An open-source Python client for running Large Language Models (LLMs) locally on any device.

haotian-liu/llava20.7K

Visual Instruction System

A system that uses large language and vision models to generate and process visual instructions

sjtu-ipads/powerinfer8K

LLM inference engine

An efficient Large Language Model inference engine leveraging consumer-grade GPUs on PCs

thudm/glm-130b7.7K

Bilingual Language Model

An open-source implementation of a large bilingual language model pre-trained on vast amounts of text data.

tloen/alpaca-lora18.7K

Language model tuning

Tuning a large language model on consumer hardware using low-rank adaptation