sglang
by sgl-project
SGLang is a fast serving framework for large language models and vision language models.
AI summary
Server framework
A fast serving framework for large language models and vision language models.
- stars
- 6.6K
- forks
- 579
- watching
- 60
- awesome list
- 1
Similar projects
Found by comparing what the projects do, not just their names.
Batch processor
Generates large language model outputs in high-throughput mode on single GPUs
modeltc/lightllm2.7K
LLM server
A Python-based framework for serving large language models with low latency and high scalability.
haotian-liu/llava20.7K
Visual Instruction System
A system that uses large language and vision models to generate and process visual instructions
LLM framework
A framework for training and serving large language models using JAX/Flax
LLM toolkit
An open-source toolkit for pretraining and fine-tuning large language models
qwenlm/qwen2-vl3.6K
Multimodal LM
A multimodal large language model series developed by the Qwen team to understand and process images, videos, and text.
Instruction-following model tuner
An implementation of a method for fine-tuning language models to follow instructions with high efficiency and accuracy
Large vision language model
A large vision language model with improved image reasoning and text recognition capabilities, suitable for various multimodal tasks
Language Model Runtime
An efficient C#/.NET library for running Large Language Models (LLMs) on local devices
LLM
A large language model series with various sizes and variants for text generation and understanding.
LLM inference engine
An efficient Large Language Model inference engine leveraging consumer-grade GPUs on PCs
shishirpatil/gorilla11.6K
API toolkit
Enables large language models to interact with external APIs using natural language queries
LLM toolkit
A toolkit for fine-tuning and inferring large machine learning models
LLM server
An inference and serving engine for large language models
Video generator
A deep learning framework for generating videos from text inputs and visual features.
Featured in 1 awesome list
Each link jumps to the spot where the list mentions sglang.
