FlexLLMGen
by FMInference
Running large language models on a single GPU for throughput-oriented scenarios.
AI summary
Batch processor
Generates large language model outputs in high-throughput mode on single GPUs
- stars
- 9.2K
- forks
- 553
- watching
- 112
Similar projects
Found by comparing what the projects do, not just their names.
LLM inference engine
An efficient Large Language Model inference engine leveraging consumer-grade GPUs on PCs
Server framework
A fast serving framework for large language models and vision language models.
Text Generation Toolkit
A toolkit for deploying and serving Large Language Models (LLMs) for high-performance text generation
Prompt engineering guide
Guides software developers on how to effectively use and build systems around Large Language Models like GPT-4.
lyogavin/airllm5.4K
Inference optimizer
Optimizes large language model inference on limited GPU resources
modeltc/lightllm2.7K
LLM server
A Python-based framework for serving large language models with low latency and high scalability.
google/big-bench2.9K
Language model benchmark
A benchmark designed to probe large language models and extrapolate their future capabilities through a diverse set of tasks.
LLM toolkit
A toolkit for fine-tuning and inferring large machine learning models
thudm/glm-130b7.7K
Bilingual Language Model
An open-source implementation of a large bilingual language model pre-trained on vast amounts of text data.
Machine learning automator
Automates machine learning workflows and optimizes model performance using large language models and efficient algorithms
aksnzhy/xlearn3.1K
Machine Learning Library
A high-performance machine learning package with linear models and factorization machines.
LLM
A large language model series with various sizes and variants for text generation and understanding.
ML explanations
An explanation of key concepts and advancements in the field of Machine Learning
mlabonne/llm-course40.1K
LLM tutorial
A comprehensive course and resource package on building and deploying Large Language Models (LLMs)
LLM framework
A framework for training and serving large language models using JAX/Flax
