levanter
Language model trainer
A framework for training large language models that prioritizes legibility, scalability, and reproducibility
Legible, Scalable, Reproducible Foundation Models with Named Tensors and Jax
527 stars
14 watching
85 forks
Language: Python
last commit: 2 months ago
Linked from 1 awesome list
n2cholas/awesome-jaxRelated projects:
| Repository | Description | Stars |
|---|---|---|
 zjunlp/knowlm | A framework for training and utilizing large language models with knowledge augmentation capabilities | 1,251 |
 csuhan/onellm | A framework for training and fine-tuning multimodal language models on various data types | 601 |
 ibm-granite/granite-3.0-language-models | A collection of lightweight state-of-the-art language models designed to support multilinguality, coding, and reasoning tasks on constrained resources. | 232 |
 marvinteichmann/convcrf | An implementation of a convolutional Conditional Random Field model for semantic segmentation tasks. | 564 |
 umass-foundation-model/3d-llm | Developing a Large Language Model capable of processing 3D representations as inputs | 979 |
 afriemann/simple_model | A lightweight model framework for validating and serializing data. | 2 |
 korpling/salt | A flexible data model and API for representing linguistic data in a language-independent and theory-neutral way. | 15 |
 ai-hypercomputer/maxtext | A high-performance LLM written in Python/Jax for training and inference on Google Cloud TPUs and GPUs. | 1,557 |
 bilibili/index-1.9b | A lightweight, multilingual language model with a long context length | 920 |
 opennlg/openba | A pre-trained language model designed for various NLP tasks, including dialogue generation, code completion, and retrieval. | 94 |
 kohjingyu/fromage | A framework for grounding language models to images and handling multimodal inputs and outputs | 478 |
 google-deepmind/recurrentgemma | An implementation of a fast and efficient language model architecture | 613 |
 scicloj/scicloj.ml.clj-djl | Provides pre-trained machine learning models for natural language processing tasks using Clojure and the clj-djl framework. | 0 |
 ray-project/llmperf | A tool for evaluating the performance of large language model APIs | 678 |
 google/paxml | A framework for configuring and running machine learning experiments on top of Jax. | 461 |