bert-distillation
BERT distiller
A high-level API for distilling BERT models to create smaller, more efficient variants with reduced training time and improved inference speed.
Distillation of BERT model with catalyst framework
75 stars
4 watching
7 forks
Language: Python
last commit: over 3 years agobertcatalystdistilbertdistillationnlprubert
Related projects:
| Repository | Description | Stars |
|---|---|---|
 hobbitlong/repdistiller | A Python-based project implementing contrastive representation distillation and benchmarking recent knowledge distillation methods | 2,217 |
 autodistill/autodistill | Automatically trains models from large foundation models to perform specific tasks with minimal human intervention. | 2,022 |
 dbmdz/berts | Provides pre-trained language models for natural language processing tasks | 155 |
 thunlp-aipoet/bert-ccpoem | A BERT-based pre-trained model for Chinese classical poetry | 146 |
 zhuiyitechnology/wobert | A Word-based Chinese BERT model trained on large-scale text data using pre-trained models as a foundation | 460 |
 turkunlp/wikibert | Provides pre-trained language models derived from Wikipedia texts for natural language processing tasks | 34 |
 yoshitomo-matsubara/torchdistill | A framework for designing and running deep learning experiments without writing code | 1,409 |
 diogenes0319/fedmd_clean | An implementation of a heterogenous federated learning framework using model distillation. | 150 |
 marscod/bert-sort | Automated sorting of data using a neural network model | 9 |
 ncbi-nlp/bluebert | Pre-trained language models for biomedical natural language processing tasks | 560 |
 kamalkraj/bert-ner | An implementation of named entity recognition using Google's BERT model for the CoNLL-2003 dataset and Python. | 1,220 |
 mop/bier | This project implements a deep metric learning framework using an adversarial auxiliary loss to improve robustness. | 39 |
 ag14774/diffdist | Enables backpropagation in distributed settings and facilitates model parallelism using differentiable communication between processes | 62 |
 laurentmazare/ocaml-bert | Implementing BERT-like NLP models in OCaml using PyTorch bindings and pre-trained weights from popular sources. | 24 |
 birch-san/diffusers | A toolkit for creating and manipulating state-of-the-art diffusion models in PyTorch | 8 |