huniq

Dupfinder

Filtering out duplicates from data

Filter out duplicates on the command line. Replacement for sort | uniq optimized for speed (10x faster) when sorting is not needed.

GitHub
240 stars
5 watching
12 forks
Language: Rust
last commit: 10 months ago
Linked from 1 awesome list

clirusttools

Backlinks from these awesome lists:

Related projects:

Repository Description Stars
whitfin/runiq Efficient filtering of duplicate lines from input data 210
mhasbini/duprule Removes duplicate hashcat rule modifications by identifying unique rule changes and selecting the one with the fewest function calls. 41
nil0x42/duplicut Tools to remove duplicates from massive wordlists used in password cracking 881
duailibe/cuid Generates unique, sequential IDs with optimal performance for horizontal scaling and lookup 55
replicatedhq/unfork Tool to analyze and restore Helm charts in a Kubernetes cluster 146
patois/xray Tool for filtering and highlighting decompiler output based on regular expressions 125
hagrid29/duplicatedump Tools to dump LSASS memory without detection using custom LSA plugin and duplicated handle 199
replicatedhq/outdated A tool to identify outdated images running in a Kubernetes cluster 425
rentruewang/koila A lightweight wrapper around PyTorch to prevent CUDA out-of-memory errors and optimize model execution 1,821
nvzqz/divan A tool that allows developers to easily and efficiently compare the performance of different parts of their codebase 954
kunalsin9h/livejq An alternative to jq with continuous parsing support and filtering capabilities for handling invalid JSON without crashing. 15
darakian/ddh A tool to compare duplicate files across directories by hashing and reporting findings. 455
kornelski/dupe-krill A tool that identifies and replaces duplicate files with hardlinks to reduce storage space. 186
hakancelikdev/unimport Tool for detecting and removing unused import statements in code 239
ayrat555/fang Background task processing and queueing system for Rust applications. 623