deepsparse
by neuralmagic
Sparsity-aware deep learning inference runtime for CPUs
AI summary
Inference runtime
A sparsity-aware deep learning inference runtime for CPUs that optimizes neural network performance on CPU hardware.
- stars
- 3.1K
- forks
- 175
- watching
- 57
- awesome list
- 1
Similar projects
Found by comparing what the projects do, not just their names.
Model optimizer
Enables the creation of smaller neural network models through efficient pruning and quantization techniques
Model optimizer
Tools and techniques for optimizing large language models on various frameworks and hardware platforms.
microsoft/deepspeed35.9K
Deep Learning Optimizer
A deep learning optimization library that simplifies distributed training and inference on modern computing hardware.
LLM Quantizer
An open-source software project that enables efficient and accurate low-bit weight quantization for large language models.
Deep learning implementations
Implementations of various deep learning algorithms and techniques with accompanying documentation
LLM evaluator
A framework for evaluating large language models
ludwig-ai/ludwig11.2K
Deep Learning Framework
A low-code framework for building custom deep learning models and neural networks
NLP course materials
An open-source repository containing lecture slides and course materials for an advanced natural language processing course.
Mixture-of-Experts Language Model
A high-performance mixture-of-experts language model with strong performance and efficient inference capabilities.
Batch processor
Generates large language model outputs in high-throughput mode on single GPUs
Research codebase
Provides implementations and illustrative code to accompany DeepMind research publications
Gradient booster
A high-performance gradient boosting framework for machine learning tasks
modeltc/lightllm2.7K
LLM server
A Python-based framework for serving large language models with low latency and high scalability.
LLM toolkit
A toolkit for fine-tuning and inferring large machine learning models
lyogavin/airllm5.4K
Inference optimizer
Optimizes large language model inference on limited GPU resources
Featured in 1 awesome list
Each link jumps to the spot where the list mentions deepsparse.
