awesome-elasticsearch

Search platform

A curated collection of resources and tools for building search applications with Elasticsearch

A curated list of the most important and useful resources about elasticsearch: articles, videos, blogs, tips and tricks, use cases. All about Elasticsearch!

GitHub
5k stars
227 watching
559 forks
last commit: over 1 year ago
Linked from 1 awesome list

awesomedatabaseelastalertelasticsearchelasticsearch-clusterelasticsearch-configurationkibanalogstashlucenenosqlquery-elasticsearchsearch

General / Elastic Stack
Elasticsearch official website
Logstash is a data pipeline that helps you process logs and other event data from a variety of systems
Kibana is a data analysis tool that helps to visualize your data;
beats is the platform for building lightweight, open source data shippers for many types of data you want to enrich with Logstash, search and analyze in Elasticsearch, and visualize in Kibana

General / Books
Deep Learning for Search teaches you how to leverage neural networks, NLP, and deep learning techniques to improve search performance. (2019)
Relevant Search: with applications for Solr and Elasticsearch demystifies relevance work. Using Elasticsearch, it teaches you how to return engaging search results to your users, helping you understand and leverage the internals of Lucene-based search engines. (2016)
Elasticsearch in Action teaches you how to build scalable search applications using Elasticsearch (2015)
Elasticsearch in Action, Second edition hands-on guide to developing fully functional search engines with Elasticsearch and Kibana. (2021)

General / Certifications / Elastic Certified Engineer
Elastic Certified Engineer notes notes and exercises to prepare the certification exam
frutik/awesome-search 1,389 11 months ago I am building e-commerce search now. Below are listed some of my build blocks

General / Open-source and free products, based on Elasticsearch
Fess is an open source full featured Enterprise Search, with a web-crawler
Yelp/elastalert 8,004 over 1 year ago is a modular flexible rules based alerting system written in Python
etsy/411 973 over 2 years ago an Alert Management Web Application (credentials: user/user)
appbaseio/mirage 2,214 almost 3 years ago is a 🔎 GUI for composing Elasticsearch queries
exceptionless/Exceptionless 2,416 11 months ago is an error (exceptions) collecting and reporting server with client bindings for a various programming languages
searchkit/searchkit 4,784 12 months ago is a UI framework based on React to build awesome search experiences with Elasticsearch
appbaseio/reactivemaps is a React based UI components library for building Airbnb / Foursquare like Maps
appbaseio/reactivesearch is a library of beautiful React UI components for Elasticsearch
appbaseio/dejavu 8,399 about 1 year ago The missing UI for Elasticsearch;
Simple File Server 84 about 3 years ago is an Openstack Swift compatible distributed object store that can serve and securely store billions of large and small files using minimal resources
logagent a log shipper to parse and ship logs to Elasticsearch including bulk indexing, disk buffers and log format detection
ItemsAPI 312 almost 5 years ago simplified search API for web and mobile (based on Elasticsearch and Express.js)
Kuzzle 1,454 11 months ago An open-source backend with advanced real-time features for Web, Mobile and IoT that uses ElasticSearch as a database. ( )
SIAC 3 about 7 years ago SIAC is an enterprise SIEM built on the ELK stack and other open-source components
Sentinl 1,334 over 1 year ago Sentinl is a Kibana alerting and reporting app
Praeco 549 12 months ago Elasticsearch alerting made simple
DataStation 2,907 about 2 years ago Easily query, script, and visualize data from every database, file, and API
DocKit 297 11 months ago GUI client for elasticsearch to query, manage and visualize your data

General / Elasticsearch developer tools and utilities / Clients
Python Elasticsearch Client Official low-level elasticsearch client for python
Elasticsearch DSL High-level python client for Elasticsearch
ESORM 40 12 months ago Python Asyncio ORM based on the famous
Ruby Elasticsearch Client 1,971 11 months ago Official low-level elasticsearch client for Ruby

General / Elasticsearch developer tools and utilities / Development and debugging
Sense (from Elastic) 382 over 1 year ago A JSON aware developer console to Elasticsearch; official and very powerful
ES-mode 196 about 3 years ago An Emacs major mode for interacting with Elasticsearch (similar to Sense)
Elasticsearch Cheatsheet Examples for the most used queries, API and settings for all major version of Elasticsearch
Elasticstat 78 over 4 years ago CLI tool displaying monitoring informations like htop
Elastic for Visual Studio Code 164 over 2 years ago An extension for developing Elasticsearch queries like Kibana and Sense extention in Visual Studio Code
Elastic Builder 510 about 1 year ago A Node.js implementation of the Elasticsearch DSL
Bodybuilder 1,249 over 2 years ago A Node.js elasticsearch query body builder
enju 13 almost 3 years ago A Node.js elasticsearch ORM
Peek 25 almost 2 years ago An interactive CLI in Python that works like Kibana Console with additional features
Logstash pipeline parser 3 almost 2 years ago Python Parsing expression grammar (PEG) and Abstract syntax tree (AST) for Logstash pipeline syntax

General / Elasticsearch developer tools and utilities / Import and Export
Knapsack plugin 472 almost 4 years ago is an "swiss knife" export/import plugin for Elasticsearch
Elasticsearch-Exporter 593 almost 2 years ago is a command line script to import/export data from Elasticsearch to various other storage systems
esbulk 281 about 1 year ago Parallel elasticsearch bulk indexing utility for the command line
elasticdump 7,591 11 months ago tools for moving and saving indices
elasticsearch-loader 400 over 3 years ago Tool for loading common file types to elasticsearch including csv, json, and parquet

General / Elasticsearch developer tools and utilities / Management
Esctl 39 11 months ago High-level command line interface to manage Elasticsearch clusters
Vulcanizer 668 11 months ago Github's open sourced cluster management library based on Elasticsearch's REST API. Comes with a high level CLI tool

General / Elasticsearch plugins / Cluster
sscarduzio/elasticsearch-readonlyrest-plugin 954 11 months ago Safely expose Elasticsearch REST API directly to the public
mobz/elasticsearch-head 9,428 over 4 years ago is a powerful and essential plugin for managing your cluster, indices and mapping
Bigdesk Live charts and statistics for elasticsearch cluster
Elastic HQ Elasticsearch cluster management console with live monitoring and beautiful UI
Cerebro 5,555 over 1 year ago is an open source(MIT License) elasticsearch web admin tool. Supports ES 5.x
Kopf 2,409 almost 6 years ago Another management plugin that have REST console and shard allocation
Search Guard 66 11 months ago Elasticsearch and elastic stack security and alerting for free
ee-outliers 206 over 2 years ago ee-outliers is a framework to detect outliers in events stored in an Elasticsearch cluster
Elasticsearch Comrade 273 over 2 years ago Elasticsearch admin panel built for ops and monitoring
elasticsearch-admin 95 12 months ago Web administration for Elasticsearch

General / Elasticsearch plugins / Other
SIREn Join Plugin for Elasticsearch 184 over 4 years ago This plugin extends Elasticsearch with new search actions and a filter query parser that enables to perform a "Filter Join" between two set of documents (in the same index or in different indexes)

General / Elasticsearch plugins / Integrations and SQL support
NLPchina/elasticsearch-sql 7,004 11 months ago Query elasticsearch using familiar SQL syntax. You can also use ES functions in SQL
elastic/elasticsearch-hadoop 1,930 11 months ago Elasticsearch real-time search and analytics natively integrated with Hadoop (and Hive)
jprante/elasticsearch-jdbc 2,838 about 4 years ago JDBC importer for Elasticsearch
pandasticsearch 285 over 2 years ago An Elasticsearch client exposing DataFrame API
monstache 1,292 11 months ago Go daemon that syncs MongoDB to Elasticsearch in near realtime
jprante/elasticsearch-plugin-bundle 110 over 1 year ago A plugin that consists of a compilation of useful Elasticsearch plugins related to indexing and searching documents

General / Kibana plugins and applications
elastic/timelion 347 over 8 years ago time-series analyses application. Overview and installation guide: Timelion:
Kibana Alert App for Elasticsearch 1,334 over 1 year ago Kibana plugin with monitoring, alerting and reporting capabilities
VulnWhisperer 1,362 almost 3 years ago VulnWhisperer is a vulnerability data and report aggregator
Wazuh Kibana App 437 11 months ago A Kibana app for working with data generated by
Datasweet Formula 121 almost 3 years ago A real time calculated metric plugin

General / Kibana plugins and applications / Kibana Visualization plugins
nbs-system/mapster 74 over 8 years ago a visualization which allows to create live event 3d maps in Kibana
Kibana Tag Cloud Plugin 69 almost 9 years ago tag cloud visualization plugin based on d3-cloud plugin
LogTrail 1,398 almost 2 years ago a plugin for Kibana to view, analyze, search and tail log events from multiple hosts in realtime with devops friendly interface inspired by Papertrail
Analyze API 110 about 3 years ago Kibana 6 application to manipulate the API graphically
kbn_network 369 over 2 years ago This is a plugin developed for Kibana that displays a network node that link two fields that have been previously selected

General / Discussions and social media
/r/elasticsearch
Elasticsearch forum
Stackoverflow
Books on Amazon does not fit well into this category, but worth checking out!

General / Tutorials
Centralized Logging with Logstash and Kibana On Ubuntu 14.04 everything you need to now when you are creating your first Elasticsearch+Logstash+Kibana instance
dwyl/learn-elasticsearch 270 almost 5 years ago a getting started tutorial with a pack of valuable references
Make Sense of your Logs: From Zero to Hero in less than an Hour! by Britta Weber demonstrates how you can build Elasticsearch + Logstash + Kibana stack to collect and discover your data
$$ Elasticsearch 7 and Elastic Stack liveVideo course that teaches you to search, analyze, and visualize big data on a cluster with Elasticsearch, Logstash, Beats, Kibana, and more

General / System configuration
A Useful Elasticsearch Cheat Sheet in Times of Trouble
The definitive guide for Elasticsearch on Windows Azure
Elasticsearch pre-flight checklist
9 Tips on Elasticsearch Configuration for High Performance
Best Practices in AWS
How to Secure Elasticsearch and Kibana with NGINX, LDAP and SSL
Elasticsearch server on Webfaction using NGINX with basic authorization and HTTPS protocol
Elasticsearch Guides Useful Elasticsearch guides with best practices, troubleshooting instructions for errors, tips, examples of code snippets and more

General / System configuration / Docker and Elasticsearch
Running an Elasticsearch cluster with Docker

General / Java tuning
Elasticsearch Java Virtual Machine settings explained
Tuning Garbage Collection for Mission-Critical Java Applications
G1: One Garbage Collector To Rule Them All
Use Lucene’s MMapDirectory on 64bit platforms, please!
Black Magic cookbook
G1GC Fundamentals: Lessons from Taming Garbage Collection
JVM Garbage Collector settings investigation PDF Comparison of JVM GC
Garbage Collection Settings for Elasticsearch Master Nodes Fine tunine your garbage collector
Understanding G1 GC Log Format To tune and troubleshoot G1 GC enabled JVMs, one must have a proper understanding of G1 GC log format. This article walks through key things that one should know about the G1 GC log format

General / Scalable Infrastructure and performance
The Authoritative Guide to Elasticsearch Performance Tuning (Part 1)
Tuning data ingestion performance for Elasticsearch on Azure and not only for Azure. That's a great article about Elasticsearch Performance testing by example
Elasticsearch Indexing Performance Cheatsheet when you plan to index large amounts of data in Elasticsearch (by Patrick Peschlow)
Elasticsearch for Logging Elasticsearch configuration tips and tricks from Sanity
Scaling Elasticsearch to Hundreds of Developers by Joseph Lynch @yelp
10 Elasticsearch metrics to watch
Understanding Elasticsearch Performance
Our Experience of Creating Large Scale Log Search System Using Elasticsearch topology, separate master, data and search balancers nodes
Elasticsearch on Azure Guidance 10,325 11 months ago it is 10% on Azure and 90% of a very valuable general information, tips and tricks about Elasticsearch
How to avoid the split-brain problem in Elasticsearch

General / Scalable Infrastructure and performance / Datadog's series about monitoring Elasticsearch performance:
How to monitor Elasticsearch performance
How to collect Elasticsearch metrics
How to monitor Elasticsearch with Datadog
How to solve 5 Elasticsearch performance and scaling problems

General / Scalable Infrastructure and performance
Performance Monitoring Essentials - Elasticsearch Edition
Operator for running Elasticsearch in Kubernetes 353 11 months ago

General / Scalable Infrastructure and performance / Integrations
Apache Hive integration
Connecting Tableau to Elasticsearch (READ: How to query Elasticsearch with Hive SQL and Hadoop)
mradamlacey/elasticsearch-tableau-connector 149 almost 5 years ago

General / Scalable Infrastructure and performance / Logging
5 Logstash Alternatives and typical use cases

General / Scalable Infrastructure and performance / Alerts
ElastAlert: Alerting At Scale With Elasticsearch, Part 1 by engineeringblog.yelp.com
ElastAlert: Alerting At Scale With Elasticsearch, Part 2 by engineeringblog.yelp.com
Elastalert: implementing rich monitoring with Elasticsearch

General / Scalable Infrastructure and performance / Time series
Elasticsearch as a Time Series Data Store by Felix Barnsteiner
Running derivatives on Voyager velocity data By Colin Goodheart-Smithe
Part 1 Shewhart Control Charts via Moving Averages: - by Zachary Tong
Part 1 Implementing a Statistical Anomaly Detector: - - by Zachary Tong

General / Scalable Infrastructure and performance / Machine Learning
Classifying images into Elasticsearch with DeepDetect ( ) by Emmanuel Benazera
Elasticsearch with Machine Learning ( ) by Kunihiko Kido
Recommender System with Mahout and Elasticsearch

General / Scalable Infrastructure and performance / Use cases for Elasticsearch
Data Infrastructure at IFTTT Elasticsearch, Kafka, Apache Spark, Redhsift, other AWS services
OFAC compliance with Elasticsearch using AWS
Building a Streaming Search Platform Streaming Search on Tweets: Storm, Elasticsearch, and Redis

General / Other
LogZoom, a fast and lightweight substitute for Logstash
Graylog2/graylog2-server 7,481 11 months ago Free and open source log management (based on ES)
Fluentd vs. Logstash for OpenStack Log Management
Building a Directory Map With ELK
Structured logging with ELK - part 1
Search for Emoji with Elasticsearch
Complete Guide to the ELK Stack
Elasticsearch Engineer Interview Questions
logiq 35 over 4 years ago Simple WebUI Monitoring Tool for Logstash ver. 5.0 and up
ElasticSearch Report Engine An ElasticSearch plugin to return query results as either PDF,HTML or CSV
Elasticsearch Glossary explanations of Elasticsearch terminology, including examples, common best practices and troubleshooting guides for various issues

General / Videos / Overviews
Elasticsearch for logs and metrics: A deep dive – Velocity 2016 by Sematext Developers
Elasticsearch in action Thijs Feryn a beginner overview
Getting Down and Dirty with ElasticSearch by Clinton Gormley
How we scaled Raygun
Getting started with Elasticsearch
Speed is a Key: Elasticsearch under the Hood introduction + basic performance optimization
$$ Pluralsight: Getting Started With Elasticsearch for .NET Developers this course will introduce users to Elasticsearch, how it works, and how to use it with .NET projects
$$ Complete Guide to Elasticsearch Comprehensive guide to Elasticsearch, the popular search engine built on Apache Lucene
How Elasticsearch powers the Guardian's newsroom
Elasticsearch Query Editor in Grafana
Scale Your Metrics with Elasticsearch 2019 by Philipp Krenn (Elastic) optimization tips and tricks

General / Videos / Advanced
#bbuzz 2015: Adrien Grand – Algorithms and data-structures that power Lucene and Elasticsearch
Rafał Kuć - Running High Performance Fault-tolerant Elasticsearch Clusters on Docker and
Working with Elasticsearch - Search, Aggregate, Analyze, and Scale Large Volume Datastores O'Reilly Media
End-to-end Recommender System with Spark and Elasticsearch by Nick Pentreath & Jean-François Puget

General / Videos / Code, configuration file samples and other gists
Elasticsearch config for a write-heavy cluster reyjrar/elasticsearch.yml
chenryn/ESPL - Elastic Search Processing Language 49 about 9 years ago PEG parser sample for SPL to Elasticsearch DSL
thomaspatzke/EQUEL 56 over 8 years ago an Elasticsearch QUEry Language, based on G4 grammar parser
Technology Explained Blog
EagerElk
Tim Roes Blog

Backlinks from these awesome lists:

More related projects: