DeepSpeed-MII
MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
2k stars
41 watching
174 forks
Language: Python
last commit: 19 days ago
Linked from 1 awesome list
deep-learninginferencepytorch
hannibal046/awesome-llm