phoenix
LLM monitoring tool
An AI observability platform designed to monitor and evaluate the performance of large language models.
AI Observability & Evaluation
4k stars
31 watching
316 forks
Language: Jupyter Notebook
last commit: almost 2 years agoLinked from 3 awesome lists
ai-monitoringai-observabilityai-roiaiengineeringdatasetshacktoberfestllm-evalllmopsml-observabilitymlopsmodel-observability
ethicalml/awesome-production-machine-learning
promptslab/awesome-prompt-engineering
vihar/awesome-oss-saasRelated projects:
| Repository | Description | Stars |
|---|---|---|
 evidentlyai/evidently | An observability framework for evaluating and monitoring the performance of machine learning models and data pipelines | 5,519 |
 openai/evals | A framework for evaluating large language models and systems, providing a registry of benchmarks. | 15,168 |
 giskard-ai/giskard | Automates the detection of performance, bias, and security issues in AI applications | 4,125 |
 ianarawjo/chainforge | An environment for battle-testing prompts to Large Language Models (LLMs) to evaluate response quality and performance. | 2,413 |
 confident-ai/deepeval | A framework for evaluating large language models | 4,003 |
 xlang-ai/openagents | An open platform for developing and deploying language agents in the wild | 4,032 |
 zabbix/zabbix | An enterprise-class monitoring solution designed to track performance and availability of IT resources and services in real-time. | 4,484 |
 pixie-io/pixie | A Kubernetes-native observability tool for monitoring cluster resources and application traffic | 5,651 |
 josh-xt/agixt | An AI platform that orchestrates instruction management and complex task execution across multiple AI providers | 2,674 |
 meta-llama/llama-stack | Provides pre-packaged building blocks for generative AI applications with standardized APIs and service-oriented design. | 5,164 |
 signoz/signoz | An observability platform for application performance monitoring and tracing. | 19,833 |
 betalgo/openai | A .NET library providing access to the OpenAI service API. | 2,912 |
 hegelai/prompttools | A set of tools for testing and evaluating natural language processing models and vector databases. | 2,731 |
 langgenius/dify | An open-source LLM app development platform that enables users to build and deploy AI-powered applications quickly and efficiently. | 54,931 |
 significant-gravitas/autogpt | A platform for building and deploying autonomous AI agents to automate complex workflows | 169,186 |