awesome-prompts

prompt

(The prompt is urgly and not stable now, let's improve it together!)

: A Python library for making chatbot interfaces

Embedchain

: A Python library for managing and syncing unstructured data with LLMs

: A Python library for automating selection of models, hyperparameters, and other tunable choices

: JavaScript-ish scripts to create execute prompts, extract structured data, integrated in Visual Studio Code

: A Python library for validating outputs and retrying failures. Still in alpha, so expect sharp edges and bugs

Guidance

: A handy looking Python library from Microsoft that uses Handlebars templating to interleave generation, prompting, and logical control

Haystack

: Open-source LLM orchestration framework to build customizable, production-ready LLM applications in Python

: An enterprise platform to evaluate, debug, and monitor LLM apps

LangChain

: A popular Python/JavaScript library for chaining sequences of language model prompts

LiteLLM

: A minimal Python library for calling LLM APIs with a consistent format

LlamaIndex

: A Python library for augmenting LLM apps with data

: A programming language for LLM interaction with support for typed prompting, control flow, constraints, and tools

OpenAI Evals

: An open-source library for evaluating task performance of language models and prompts

Outlines

: A Python library that provides a domain-specific language to simplify prompting and constrain generation

: A platform for debugging, testing, and monitoring LLM apps

: A platform for observability, model management, evals, and security for LLM apps

Promptify

: A small Python library for using language models to perform NLP tasks

: A paid product for testing and improving prompts

Prompttools

: Open-source Python tools for testing and evaluating models, vector DBs, and prompts

: A paid product for building, comparing, and shipping language model apps

Semantic Kernel

: A Python/C#/Java library from Microsoft that supports prompt templating, function chaining, vectorized memory, and intelligent planning

: A paid product for tracking model training and prompt engineering experiments

YiVal

: An open-source GenAI-Ops tool for tuning and evaluating prompts, retrieval configurations, and model parameters using customizable datasets, evaluation methods, and evolution strategies

Brex's Prompt Engineering Guide

: Brex's introduction to language models and prompt engineering

: An introductory course to prompt engineering

: An OpenAI researcher's review of the prompt engineering literature (as of March 2023)

: A slightly dated (Sep 2022) review of techniques for prompting language models

: A prompt engineering guide that demonstrates many techniques

and : A basic but opinionated introduction to prompt engineering and a follow up collection with many advanced methods starting with CoT

: A short course on prompt engineering for developers

: A detailed dive into the machine learning underlying GPT

: A one-hour video on various prompt engineering techniques

: A 30-minute interactive course about the Assistants API

: Short video introduction to prompt engineering

: Using few-shot prompts to ask models to think step by step improves their reasoning. PaLM's score on math word problems (GSM8K) rises from 18% to 57%

: Taking votes from multiple outputs improves accuracy even more. Voting across 40 outputs raises PaLM's score on math word problems further, from 57% to 74%, and 's from 60% to 78%

: Searching over trees of step by step reasoning helps even more than voting over chains of thought. It lifts 's scores on creative writing and crosswords

: Telling instruction-following models to think step by step improves their reasoning. It lifts 's score on math word problems (GSM8K) from 13% to 41%

: Automated searching over possible prompts found a prompt that lifts scores on math word problems (GSM8K) to 43%, 2 percentage points above the human-written prompt in Language Models are Zero-Shot Reasoners

: Automated searching over possible chain-of-thought prompts improved ChatGPT's scores on a few benchmarks by 0–20 percentage points

: Reasoning can be improved by a system that combines: chains of thought generated by alternative selection and inference prompts, a halter model that chooses when to halt selection-inference loops, a value function to search over multiple reasoning paths, and sentence labels that help avoid hallucination

: Chain of thought reasoning can be baked into models via fine-tuning. For tasks with an answer key, example chains of thoughts can be generated by language models

: For tasks with tools or an environment, chain of thought works better if you prescriptively alternate between asoning steps (thinking about what to do) and ing (getting information from a tool or environment)

: Retrying tasks with memory of prior failures improves subsequent performance

: Models augmented with knowledge via a "retrieve-then-read" can be improved with multi-hop chains of searches

: Generating debates between a few ChatGPT agents over a few rounds improves scores on various benchmarks. Math word problem scores rise from 77% to 85%