lanarky
LLM Microservice Framework
A Python web framework specifically designed to build LLM microservices with built-in support for FastAPI and streaming capabilities.
The web framework for building LLM microservices
978 stars
15 watching
74 forks
Language: Python
last commit: over 1 year ago
Linked from 1 awesome list
fastapillmopsmicroservicespython3web
kyrolabs/awesome-langchainRelated projects:
| Repository | Description | Stars |
|---|---|---|
 msoedov/langcorn | A framework for serving large language models with a robust and efficient API | 909 |
 internlm/lagent | A lightweight framework for building agent-based applications using LLMs and transformer architectures | 1,924 |
 lsm/micromono | Enables seamless development of microservices with monolithic structure without changing the codebase. | 643 |
 dreadnode/rigging | A framework for leveraging language models in production code | 216 |
 bytedance/lynx-llm | A framework for training GPT4-style language models with multimodal inputs using large datasets and pre-trained models | 231 |
 ray-project/llmperf | A tool for evaluating the performance of large language model APIs | 678 |
 mikrl/django-llm | An app for building and executing large language model workflows within Django web applications | 54 |
 ai-hypercomputer/maxtext | A high-performance LLM written in Python/Jax for training and inference on Google Cloud TPUs and GPUs. | 1,557 |
 melih-unsal/demogpt | A comprehensive toolset for building Large Language Model (LLM) based applications | 1,733 |
 jackmpcollins/magentic | A tool to integrate Large Language Models into Python code seamlessly | 2,104 |
 wgryc/phasellm | A framework for managing and testing large language models to evaluate their performance and optimize user experiences. | 451 |
 llukas22/llm-rs-python | An unofficial Python interface to the Rust-based Large Language Model library | 73 |
 ailab-cvc/seed | An implementation of a multimodal language model with capabilities for comprehension and generation | 585 |
 luogen1996/lavin | An open-source implementation of a vision-language instructed large language model | 513 |
 r2d4/openlm | Library that provides a unified API to interact with various Large Language Models (LLMs) | 367 |