lightllm

lyogavin/airllm5.4K

Inference optimizer

Optimizes large language model inference on limited GPU resources

mlabonne/llm-course40.1K

LLM tutorial

A comprehensive course and resource package on building and deploying Large Language Models (LLMs)

alpha-vllm/llama2-accessory2.7K

LLM toolkit

An open-source toolkit for pretraining and fine-tuning large language models

young-geng/easylm2.4K

LLM framework

A framework for training and serving large language models using JAX/Flax

meta-llama/codellama16.1K

Code generator

Provides inference code and tools for fine-tuning large language models, specifically designed for code generation tasks

nlpxucan/wizardlm9.3K

Instruction framework

Large pre-trained language models trained to follow complex instructions using an evolutionary instruction framework

optimalscale/lmflow8.3K

LLM toolkit

A toolkit for fine-tuning and inferring large machine learning models

sgl-project/sglang6.6K

Server framework

A fast serving framework for large language models and vision language models.

ericlbuehler/mistral.rs4.7K

LLM inference framework

A high-performance LLM inference framework written in Rust

ggerganov/llama.cpp69.2K

LLM inference software

Enables LLM inference with minimal setup and high performance on various hardware platforms

vllm-project/vllm32K

LLM server

An inference and serving engine for large language models

zilliztech/gptcache7.3K

LLM Cache

A semantic cache designed to reduce the cost and improve the speed of LLM API calls by storing responses.

mlc-ai/mlc-llm19.4K

LLM Deployment Engine

A machine learning compiler and deployment engine for large language models

fminference/flexllmgen9.2K

Batch processor

Generates large language model outputs in high-throughput mode on single GPUs

mooler0410/llmspracticalguide9.6K

LLM guide

A curated list of resources to help developers navigate the landscape of large language models and their applications in NLP