lanarky
LLM Microservice Framework
A Python web framework specifically designed to build LLM microservices with built-in support for FastAPI and streaming capabilities.
The web framework for building LLM microservices
976 stars
15 watching
74 forks
Language: Python
last commit: 5 months ago
Linked from 1 awesome list
fastapillmopsmicroservicespython3web
kyrolabs/awesome-langchainRelated projects:
| Repository | Description | Stars |
|---|---|---|
 msoedov/langcorn | A framework for serving large language models with a robust and efficient API | 906 |
 internlm/lagent | A lightweight framework for building agent-based applications using LLMs and transformer architectures | 1,865 |
 lsm/micromono | Enables seamless development of microservices with monolithic structure without changing the codebase. | 643 |
 dreadnode/rigging | An LLM framework that simplifies interacting with language models in production code | 209 |
 bytedance/lynx-llm | A framework for training GPT4-style language models with multimodal inputs using large datasets and pre-trained models | 229 |
 ray-project/llmperf | A tool for evaluating the performance of large language model APIs | 641 |
 mikrl/django-llm | An app for building and executing large language model workflows within Django web applications | 54 |
 ai-hypercomputer/maxtext | A high-performance LLM written in Python/Jax for training and inference on Google Cloud TPUs and GPUs. | 1,529 |
 melih-unsal/demogpt | A comprehensive toolset for building Large Language Model (LLM) based applications | 1,710 |
 jackmpcollins/magentic | A Python library that integrates Large Language Models into Python code by allowing developers to easily call LLMs as functions. | 2,048 |
 wgryc/phasellm | A framework for managing and testing large language models to evaluate their performance and optimize user experiences. | 448 |
 llukas22/llm-rs-python | An unofficial Python interface to the Rust-based Large Language Model library | 73 |
 ailab-cvc/seed | An implementation of a multimodal language model with capabilities for comprehension and generation | 582 |
 luogen1996/lavin | An open-source implementation of a vision-language instructed large language model | 508 |
 r2d4/openlm | Library that provides a unified API to interact with various Large Language Models (LLMs) | 366 |