lorax
Memory optimizer
A JAX transform that simplifies the training of large language models by reducing memory usage through low-rank adaptation.
LoRA for arbitrary JAX models and functions
132 stars
4 watching
5 forks
Language: Python
last commit: 9 months ago
Linked from 1 awesome list
Related projects:
Repository | Description | Stars |
---|---|---|
rentruewang/koila | A lightweight wrapper around PyTorch to prevent CUDA out-of-memory errors and optimize model execution | 1,821 |
google/jaxopt | An open-source project providing hardware accelerated, batchable and differentiable optimizers in JAX for deep learning. | 933 |
lge-arc-advancedai/auptimizer | Automates model building and deployment process by optimizing hyperparameters and compressing models for edge computing. | 200 |
zackzikaixiao/fedgrab | A tool for training federated learning models with adaptive gradient balancing to handle class imbalance in multi-client scenarios. | 13 |
jiangoforit/yellowfin_pytorch | An optimizer that automatically tunes momentum and learning rate using local quadratic approximation. | 287 |
typemonkey/turtle | A programming language designed to optimize memory usage through long-term storage of data | 5 |
dmlc/mxnet-memonger | A tool for optimizing deep learning models to reduce memory usage without sacrificing performance | 308 |
mapillary/inplace_abn | An optimization technique to reduce memory usage in deep neural networks during training | 1,321 |
jshilong/gpt4roi | Training and deploying large language models on computer vision tasks using region-of-interest inputs | 506 |
lyogavin/anima | An optimization technique for large language models allowing them to run on limited hardware resources without significant performance loss. | 6 |
alibaba/conv-llava | This project presents an optimization technique for large-scale image models to reduce computational requirements while maintaining performance. | 104 |
kvcache-ai/ktransformers | A flexible framework for LLM inference optimizations with support for multiple models and architectures | 736 |
google-deepmind/optax | A gradient processing and optimization library designed to facilitate research and productivity in machine learning by providing building blocks for custom optimizers and gradient processing components. | 1,697 |
alshedivat/fedpa | A modular JAX implementation of federated learning via posterior averaging for decentralized optimization | 49 |
leonardgoeke/anymod.jl | A Julia framework for creating large-scale linear optimization models of energy system capacity expansion with multiple periods | 70 |