lmdeploy

vllm-project/vllm32K

LLM server

An inference and serving engine for large language models

internlm/internlm6.6K

Chatbot models

A collection of large language models designed to improve reasoning and tool use capabilities in chatbots.

sjtu-ipads/powerinfer8K

LLM inference engine

An efficient Large Language Model inference engine leveraging consumer-grade GPUs on PCs

lyogavin/airllm5.4K

Inference optimizer

Optimizes large language model inference on limited GPU resources

alpha-vllm/llama2-accessory2.7K

LLM toolkit

An open-source toolkit for pretraining and fine-tuning large language models

optimalscale/lmflow8.3K

LLM toolkit

A toolkit for fine-tuning and inferring large machine learning models

mit-han-lab/llm-awq2.6K

LLM Quantizer

An open-source software project that enables efficient and accurate low-bit weight quantization for large language models.

opengvlab/llama-adapter5.8K

Instruction-following model tuner

An implementation of a method for fine-tuning language models to follow instructions with high efficiency and accuracy

nomic-ai/gpt4all71.2K

LLM client

An open-source Python client for running Large Language Models (LLMs) locally on any device.

modeltc/lightllm2.7K

LLM server

A Python-based framework for serving large language models with low latency and high scalability.

opengvlab/internvl6.4K

Multimodal model builder

Develops large language models capable of processing multiple data types and modalities

young-geng/easylm2.4K

LLM framework

A framework for training and serving large language models using JAX/Flax

eleutherai/lm-evaluation-harness7.2K

Evaluation framework

Provides a unified framework to test generative language models on various evaluation tasks.

microsoft/deepspeed35.9K

Deep Learning Optimizer

A deep learning optimization library that simplifies distributed training and inference on modern computing hardware.

hiyouga/llama-factory36.2K

LLaMA Tuner

A tool for efficiently fine-tuning large language models across multiple architectures and methods.