lorax
Memory optimizer
A JAX transform that simplifies the training of large language models by reducing memory usage through low-rank adaptation.
LoRA for arbitrary JAX models and functions
134 stars
4 watching
5 forks
Language: Python
last commit: over 2 years agoLinked from 1 awesome list
n2cholas/awesome-jaxRelated projects:
| Repository | Description | Stars |
|---|---|---|
 rentruewang/koila | A lightweight wrapper around PyTorch to prevent CUDA out-of-memory errors and optimize model execution | 1,823 |
 google/jaxopt | An open-source project providing hardware accelerated, batchable and differentiable optimizers in JAX for deep learning. | 941 |
 lge-arc-advancedai/auptimizer | Automates model building and deployment process by optimizing hyperparameters and compressing models for edge computing. | 200 |
 zackzikaixiao/fedgrab | A tool for training federated learning models with adaptive gradient balancing to handle class imbalance in multi-client scenarios. | 14 |
 jiangoforit/yellowfin_pytorch | An optimizer that automatically tunes momentum and learning rate using local quadratic approximation. | 287 |
 typemonkey/turtle | A programming language designed to optimize memory usage through long-term storage of data | 5 |
 dmlc/mxnet-memonger | A tool for optimizing deep learning models to reduce memory usage without sacrificing performance | 308 |
 mapillary/inplace_abn | An optimization technique to reduce memory usage in deep neural networks during training | 1,323 |
 jshilong/gpt4roi | Training and deploying large language models on computer vision tasks using region-of-interest inputs | 517 |
 lyogavin/anima | An optimization technique for large language models allowing them to run on limited hardware resources without significant performance loss. | 9 |
 alibaba/conv-llava | This project presents an optimization technique for large-scale image models to reduce computational requirements while maintaining performance. | 106 |
 kvcache-ai/ktransformers | A flexible framework for LLM inference optimizations with support for multiple models and architectures | 771 |
 google-deepmind/optax | A gradient processing and optimization library designed to facilitate research and productivity in machine learning by providing building blocks for custom optimizers and gradient processing components. | 1,730 |
 alshedivat/fedpa | A modular JAX implementation of federated learning via posterior averaging for decentralized optimization | 50 |
 leonardgoeke/anymod.jl | A Julia framework for creating large-scale linear optimization models of energy system capacity expansion with multiple periods | 70 |