CTranslate2
Transformer model inference library
A high-performance library for efficient inference with Transformer models on CPUs and GPUs.
Fast inference engine for Transformer models
3k stars
60 watching
303 forks
Language: C++
last commit: 19 days ago avxavx2cppcudadeep-learningdeep-neural-networksgemminferenceintrinsicsmachine-translationmklneonneural-machine-translationonednnopenmpopennmtparallel-computingquantizationthrusttransformer-models
Related projects:
| Repository | Description | Stars |
|---|---|---|
 nvidia/fastertransformer | A high-performance transformer-based NLP component optimized for GPU acceleration and integration into various frameworks. | 5,886 |
 systran/faster-whisper | A fast speech recognition system built on top of the CTranslate2 transformer model | 12,506 |
 eleutherai/gpt-neox | Provides a framework for training large-scale language models on GPUs with advanced features and optimizations. | 6,941 |
 karpathy/mingpt | A minimal PyTorch implementation of a transformer-based language model | 20,175 |
 huggingface/transformers | A collection of pre-trained machine learning models for various natural language and computer vision tasks, enabling developers to fine-tune and deploy these models on their own projects. | 135,022 |
 facebookresearch/metaseq | A codebase for working with Open Pre-trained Transformers, enabling deployment and fine-tuning of transformer models on various platforms. | 6,517 |
 ukplab/sentence-transformers | Provides dense vector representations for text using transformer networks | 15,382 |
| huggingface/text-generation-inference | A toolkit for deploying and serving Large Language Models. | 9,106 |
| nvidia/megatron-lm | A framework for training large language models using scalable and optimized GPU techniques | 10,623 |
 marella/ctransformers | Provides a unified interface to various transformer models implemented in C/C++ using GGML library | 1,814 |
 minimaxir/gpt-2-simple | A tool for retraining and fine-tuning the OpenAI GPT-2 text generation model on new datasets. | 3,398 |
 kimiyoung/transformer-xl | Implementations of a neural network architecture for language modeling | 3,611 |
| huggingface/trl | A library designed to train transformer language models with reinforcement learning using various optimization techniques and fine-tuning methods. | 10,133 |
 microsoft/megatron-deepspeed | Research tool for training large transformer language models at scale | 1,895 |
 shenweichen/deepctr-torch | A PyTorch-based package for building and training click-through rate models using various deep learning architectures. | 3,023 |