server
The Triton Inference Server provides an optimized cloud and edge inferencing solution.
AI summary
Inference server
An open-source software that enables deployment of AI models from multiple deep learning and machine learning frameworks on various devices
- stars
- 8.5K
- forks
- 1.5K
- watching
- 145
- awesome lists
- 3
Similar projects
Found by comparing what the projects do, not just their names.
Triton client libraries
Client libraries and examples for communicating with Triton using various programming languages
ML model server
A high-performance serving system for machine learning models in production environments.
AI inference toolkit
A toolkit for optimizing and deploying artificial intelligence models in various applications
triton-lang/triton13.7K
Deep learning compiler
An intermediate language and compiler for efficient custom Deep-Learning primitives
Binary analyzer
A dynamic binary analysis library providing tools and components for program analysis, reverse engineering, and software verification.
nvidia/tensorrt10.9K
Deep learning accelerator
Provides a set of tools and libraries for optimizing deep learning inference on NVIDIA GPUs.
Text Generation Toolkit
A toolkit for deploying and serving Large Language Models (LLMs) for high-performance text generation
Sequence model trainer
A modular framework for training and deploying sequence models at scale
Inference server
An inference server for machine learning models with support for multiple frameworks and scalable deployment options.
bentoml/bentoml7.2K
AI API service
An open-source Python framework for building model inference APIs and serving AI models in production environments.
Language model trainer
Provides a framework for training large-scale language models on GPUs with advanced features and optimizations.
ML accelerator
A cross-platform, high-performance machine learning accelerator
Neural network optimizer
A framework that transforms deep neural networks into high-performance GPU-optimized C++ code for efficient inference serving.
LLM inference engine
An efficient Large Language Model inference engine leveraging consumer-grade GPUs on PCs
fauxpilot/fauxpilot14.6K
Copilot emulator
An open-source alternative to GitHub Copilot server using NVIDIA's Triton Inference Server with the FasterTransformer backend
Featured in 3 awesome lists
Each link jumps to the spot where the list mentions server.
