levanter
Language model framework
A framework for building and training large language models with focus on reproducibility, scalability, and performance.
Legible, Scalable, Reproducible Foundation Models with Named Tensors and Jax
516 stars
14 watching
81 forks
Language: Python
last commit: 3 days ago
Linked from 1 awesome list
n2cholas/awesome-jaxRelated projects:
| Repository | Description | Stars |
|---|---|---|
 zjunlp/knowlm | A framework for training and utilizing large language models with knowledge augmentation capabilities | 1,239 |
 csuhan/onellm | A framework for training and fine-tuning multimodal language models on various data types | 588 |
 ibm-granite/granite-3.0-language-models | A collection of lightweight state-of-the-art language models designed to support multilinguality, coding, and reasoning tasks on constrained resources. | 214 |
 marvinteichmann/convcrf | An implementation of a convolutional Conditional Random Field model for semantic segmentation tasks. | 564 |
 umass-foundation-model/3d-llm | Developing a Large Language Model capable of processing 3D representations as inputs | 961 |
 afriemann/simple_model | A lightweight model framework for validating and serializing data. | 2 |
 korpling/salt | A flexible data model and API for representing linguistic data in a language-independent and theory-neutral way. | 15 |
 ai-hypercomputer/maxtext | A high-performance LLM written in Python/Jax for training and inference on Google Cloud TPUs and GPUs. | 1,529 |
 bilibili/index-1.9b | A lightweight, multilingual language model with a long context length | 904 |
 opennlg/openba | A pre-trained language model designed for various NLP tasks, including dialogue generation, code completion, and retrieval. | 94 |
 kohjingyu/fromage | A framework for grounding language models to images and handling multimodal inputs and outputs | 478 |
 google-deepmind/recurrentgemma | An implementation of a fast and efficient language model architecture | 607 |
 scicloj/scicloj.ml.clj-djl | Provides pre-trained machine learning models for natural language processing tasks using Clojure and the clj-djl framework. | 0 |
 ray-project/llmperf | A tool for evaluating the performance of large language model APIs | 641 |
 google/paxml | A framework for configuring and running machine learning experiments on top of Jax. | 457 |