fiddler-auditor
by fiddler-labs
Fiddler Auditor is a tool to evaluate language models.
AI summary
LLM auditor
An auditing tool to identify weaknesses in large language models before deployment.
- stars
- 173
- forks
- 20
- watching
- 8
- awesome list
- 1
Similar projects
Found by comparing what the projects do, not just their names.
Model evaluator
A tool to automate the evaluation of large language models in Google Colab using various benchmarks and custom parameters.
Request converter
Tools to convert Fiddler/Charles requests to JMeter scripts and supports filtering functionality.
Model memorization analysis
A Python package for measuring memorization in Large Language Models.
LLM evaluator
An evaluation framework for large language models with Elo rating system and A/B testing capabilities
LLM benchmarker
A benchmarking framework for large language models
Model evaluator
Evaluates and compares the performance of multimodal large language models on various tasks
Model evaluator
An evaluation framework for large language models trained with instruction tuning methods
Trust Assessment Toolkit
A toolkit for assessing trustworthiness in large language models
Audit helper
Automates Foundry boilerplate setup for smart contract audits
LLM evaluation framework
Provides a comprehensive framework for evaluating Large Language Model (LLM) applications and pipelines with customizable metrics
LLM wrapper
A library that provides structured outputs for Large Language Models (LLMs) in Elixir
Evaluation pipeline
A framework for evaluating language models on NLP tasks
LLM evaluation resource
A repository of papers and resources for evaluating large language models.
Legal model evaluator
Evaluates the legal knowledge of large language models using a custom benchmarking framework.
Model auditor
An auditing toolbox to assess the fairness of black-box predictive models
Featured in 1 awesome list
Each link jumps to the spot where the list mentions fiddler-auditor.
