mistral.rs
LLM inference platform
A fast and flexible LLM inference platform supporting various models and devices
Blazingly fast LLM inference.
4k stars
34 watching
310 forks
Language: Rust
last commit: 5 days ago
Linked from 1 awesome list
llmrust
hannibal046/awesome-llmRelated projects:
| Repository | Description | Stars |
|---|---|---|
 modeltc/lightllm | An LLM inference and serving framework providing a lightweight design, scalability, and high-speed performance for large language models. | 2,609 |
 alpha-vllm/llama2-accessory | An open-source toolkit for pretraining and fine-tuning large language models | 2,720 |
 opengvlab/llama-adapter | An implementation of a method for fine-tuning language models to follow instructions with high efficiency and accuracy | 5,754 |
 optimalscale/lmflow | A toolkit for finetuning large language models and providing efficient inference capabilities | 8,273 |
 llmware-ai/llmware | A framework for building enterprise LLM-based applications using small, specialized models | 6,651 |
 huggingface/text-generation-inference | A toolkit for deploying and serving Large Language Models. | 9,106 |
 nomic-ai/gpt4all | An open-source Python client for running Large Language Models (LLMs) locally on any device. | 70,694 |
 ggerganov/llama.cpp | Enables efficient inference of large language models using optimized C/C++ implementations and various backend frameworks | 67,866 |
 meta-llama/codellama | Provides inference code and tools for fine-tuning large language models, specifically designed for code generation tasks | 16,039 |
 berriai/litellm | Provides a unified API to interact with 100+ Large Language Models (LLMs) offered by various providers | 13,875 |
 lyogavin/airllm | A Python library that optimizes inference memory usage for large language models on limited GPU resources. | 5,259 |
 microsoft/flaml | Automates machine learning workflows and optimizes model performance using large language models and efficient algorithms | 3,919 |
 scisharp/llamasharp | A C#/.NET library to efficiently run Large Language Models (LLMs) on local devices | 2,673 |
| meta-llama/llama | A collection of tools and utilities for deploying, fine-tuning, and utilizing large language models. | 56,437 |
 ludwig-ai/ludwig | A low-code framework for building custom deep learning models and neural networks | 11,189 |