CTranslate2
Inference engine
A high-performance inference engine for transformer models
Fast inference engine for Transformer models
3k stars
59 watching
308 forks
Language: C++
last commit: 3 months ago avxavx2cppcudadeep-learningdeep-neural-networksgemminferenceintrinsicsmachine-translationmklneonneural-machine-translationonednnopenmpopennmtparallel-computingquantizationthrusttransformer-models
Related projects:
| Repository | Description | Stars |
|---|---|---|
 nvidia/fastertransformer | A high-performance transformer-based NLP component optimized for GPU acceleration and integration into various frameworks. | 5,937 |
 systran/faster-whisper | A fast speech-to-text implementation using CTranslate2 and optimized for inference on CPU and GPU. | 12,989 |
 eleutherai/gpt-neox | Provides a framework for training large-scale language models on GPUs with advanced features and optimizations. | 6,997 |
 karpathy/mingpt | A minimal PyTorch implementation of a transformer-based language model | 20,474 |
 huggingface/transformers | A collection of pre-trained machine learning models for various natural language and computer vision tasks, enabling developers to fine-tune and deploy these models on their own projects. | 136,357 |
 facebookresearch/metaseq | A codebase for working with Open Pre-trained Transformers, enabling deployment and fine-tuning of transformer models on various platforms. | 6,519 |
 ukplab/sentence-transformers | Provides dense vector representations for text using transformer networks | 15,556 |
| huggingface/text-generation-inference | A toolkit for deploying and serving Large Language Models (LLMs) for high-performance text generation | 9,456 |
| nvidia/megatron-lm | A framework for training large language models using scalable and optimized GPU techniques | 10,804 |
 marella/ctransformers | Provides a unified interface to various transformer models implemented in C/C++ using GGML library | 1,823 |
 minimaxir/gpt-2-simple | A tool for retraining and fine-tuning the OpenAI GPT-2 text generation model on new datasets. | 3,398 |
 kimiyoung/transformer-xl | Implementations of a neural network architecture for language modeling | 3,619 |
| huggingface/trl | A library designed to train transformer language models with reinforcement learning using various optimization techniques and fine-tuning methods. | 10,308 |
 microsoft/megatron-deepspeed | Research tool for training large transformer language models at scale | 1,926 |
 shenweichen/deepctr-torch | A PyTorch-based package for building and training click-through rate models using various deep learning architectures. | 3,055 |