awesome-rnn

RNN resource hub

A curated collection of resources and tools for building and learning about recurrent neural networks.

Recurrent Neural Network - A curated list of resources dedicated to RNN

GitHub
6k stars
615 watching
1k forks
last commit: almost 3 years ago
Linked from 4 awesome lists


Awesome Recurrent Neural Networks / Sharing
Share on Twitter
Share on Facebook
Share on Google Plus
Share on LinkedIn

Awesome Recurrent Neural Networks / Codes
Tensorflow Python, C++

Awesome Recurrent Neural Networks / Codes / Tensorflow
Get started ,

Awesome Recurrent Neural Networks / Codes / Tensorflow / Get started
Recurrent Neural Network Tutorial
Sequence-to-Sequence Model Tutorial

Awesome Recurrent Neural Networks / Codes / Tensorflow
Tutorials 6,006 over 1 year ago by nlintz
Notebook examples 43,425 4 months ago by aymericdamien
Scikit Flow (skflow) 3,181 about 3 years ago Simplified Scikit-learn like Interface for TensorFlow
Keras : (Tensorflow / Theano)-based modular deep learning library similar to Torch
char-rnn-tensorflow 2,643 almost 4 years ago by sherjilozair: char-rnn in tensorflow

Awesome Recurrent Neural Networks / Codes
Theano Python

Awesome Recurrent Neural Networks / Codes / Theano
tutorial on Theano Simple IPython
Deep Learning Tutorials

Awesome Recurrent Neural Networks / Codes / Theano / Deep Learning Tutorials
RNN for semantic parsing of speech
LSTM network for sentiment analysis

Awesome Recurrent Neural Networks / Codes / Theano
Pylearn2 : Library that wraps a lot of models and training algorithms in deep learning
Blocks 1,157 almost 6 years ago : modular framework that enables building neural network models
Keras : (Tensorflow / Theano)-based modular deep learning library similar to Torch
Lasagne 3,845 over 2 years ago : Lightweight library to build and train neural networks in Theano
theano-rnn 377 over 7 years ago by Graham Taylor
Passage 531 about 6 years ago : Library for text analysis with RNNs
Theano-Lights 267 about 9 years ago : Contains many generative models

Awesome Recurrent Neural Networks / Codes
Caffe 34,125 4 months ago C++ with MATLAB/Python wrappers

Awesome Recurrent Neural Networks / Codes / Caffe
LRCN by Jeff Donahue

Awesome Recurrent Neural Networks / Codes
Torch Lua

Awesome Recurrent Neural Networks / Codes / Torch
torchnet 996 over 5 years ago : modular framework that enables building neural network models
char-rnn 11,632 about 1 year ago by Andrej Karpathy : multi-layer RNN/LSTM/GRU for training/sampling from character-level language models
torch-rnn 2,504 over 2 years ago by Justin Johnson : reusable RNN/LSTM modules for torch7 - much faster and memory efficient reimplementation of char-rnn
neuraltalk2 5,511 about 7 years ago by Andrej Karpathy : Recurrent Neural Network captions image, much faster and better version of the original
LSTM 664 about 6 years ago by Wojciech Zaremba : Long Short Term Memory Units to train a language model on word level Penn Tree Bank dataset
Oxford by Nando de Freitas : Oxford Computer Science - Machine Learning 2015 Practicals
rnn 941 almost 7 years ago by Nicholas Leonard : general library for implementing RNN, LSTM, BRNN and BLSTM (highly unit tested)

Awesome Recurrent Neural Networks / Codes
PyTorch Python

Awesome Recurrent Neural Networks / Codes / PyTorch
Word-level RNN example 22,428 13 days ago : demonstrates PyTorch's built in RNN modules for language modeling
Practical PyTorch tutorials 4,523 over 3 years ago by Sean Robertson : focuses on using RNNs for Natural Language Processing
Deep Learning For NLP In PyTorch 1,940 almost 2 years ago by Robert Guthrie : written for a Natural Language Processing class at Georgia Tech

Awesome Recurrent Neural Networks / Codes
DL4J by : Deep Learning library for Java, Scala & Clojure on Hadoop, Spark & GPUs

Awesome Recurrent Neural Networks / Codes / DL4J
Documentation (Also in , , ) : ,
rnn examples 2,460 over 1 year ago

Awesome Recurrent Neural Networks / Codes / Etc
Neon : new deep learning library in Python, with support for RNN/LSTM, and a fast image captioning model
Brainstorm 1,303 about 2 years ago : deep learning library in Python, developed by IDSIA, thereby including various recurrent structures
Chainer : new, flexible deep learning library in Python
CGT (Computational Graph Toolkit) : replicates Theano's API, but with very short compilation time and multithreading
RNNLIB by Alex Graves : C++ based LSTM library
RNNLM by Tomas Mikolov : C++ based simple code
faster-RNNLM 561 over 2 years ago of Yandex : C++ based rnnlm implementation aimed to handle huge datasets
neuraltalk 5,411 almost 4 years ago by Andrej Karpathy : numpy-based RNN/LSTM implementation
gist by Andrej Karpathy : raw numpy code that implements an efficient batched LSTM
Recurrentjs 939 about 8 years ago by Andrej Karpathy : a beta javascript library for RNN
DARQN 115 about 9 years ago by 5vision : Deep Attention Recurrent Q-Network

Awesome Recurrent Neural Networks / Theory / Lectures
CS224d Stanford NLP ( ) by Richard Socher

Awesome Recurrent Neural Networks / Theory / Lectures / CS224d
Lecture Note 3 : neural network basics
Lecture Note 4 : RNN language models, bi-directional RNN, GRU, LSTM

Awesome Recurrent Neural Networks / Theory / Lectures
CS231n Stanford vision ( ) by Andrej Karpathy
Machine Learning Oxford by Nando de Freitas

Awesome Recurrent Neural Networks / Theory / Lectures / Machine Learning
Lecture 12 : Recurrent neural networks and LSTMs
Lecture 13 : (guest lecture) Alex Graves on Hallucination with RNNs

Awesome Recurrent Neural Networks / Theory / Books / Thesis / Alex Graves (2008)
Supervised Sequence Labelling with Recurrent Neural Networks

Awesome Recurrent Neural Networks / Theory / Books / Thesis / Tomas Mikolov (2012)
Statistical Language Models based on Neural Networks

Awesome Recurrent Neural Networks / Theory / Books / Thesis / Ilya Sutskever (2013)
Training Recurrent Neural Networks

Awesome Recurrent Neural Networks / Theory / Books / Thesis / Richard Socher (2014)
Recursive Deep Learning for Natural Language Processing and Computer Vision

Awesome Recurrent Neural Networks / Theory / Books / Thesis / Ian Goodfellow, Yoshua Bengio, and Aaron Courville (2016)
The Deep Learning Book chapter 10

Awesome Recurrent Neural Networks / Theory / Architecture Variants
Paper Bi-directional RNN [ ]
Paper Multi-dimensional RNN [ ]
Paper-arXiv GFRNN [ ] [ ] [ ]

Awesome Recurrent Neural Networks / Theory / Architecture Variants / Tree-Structured RNNs
Paper Kai Sheng Tai, Richard Socher, and Christopher D. Manning, , arXiv:1503.00075 / ACL 2015 [ ]
Paper Samuel R. Bowman, Christopher D. Manning, and Christopher Potts, , arXiv:1506.04834 [ ]

Awesome Recurrent Neural Networks / Theory / Architecture Variants
Paper Grid LSTM [ ] [ ]
Paper Segmental RNN [ ]
Paper Seq2seq for Sets [ ]
Paper Hierarchical Recurrent Neural Networks [ ]
Paper LSTM [ ]
Paper GRU (Gated Recurrent Unit) [ ]
Paper NTM [ ]
Paper Neural GPU [ ]
Paper Memory Network [ ]
Paper Pointer Network [ ]
Paper Deep Attention Recurrent Q-Network [ ]
Paper Dynamic Memory Networks [ ]

Awesome Recurrent Neural Networks / Theory / Surveys
Deep Learning Yann LeCun, Yoshua Bengio, and Geoffrey Hinton, , Nature 2015
LSTM: A Search Space Odyssey Klaus Greff, Rupesh Kumar Srivastava, Jan Koutnik, Bas R. Steunebrink, Jurgen Schmidhuber, , arXiv:1503.04069
A Critical Review of Recurrent Neural Networks for Sequence Learning Zachary C. Lipton, , arXiv:1506.00019
Visualizing and Understanding Recurrent Networks Andrej Karpathy, Justin Johnson, Li Fei-Fei, , arXiv:1506.02078
An Empirical Exploration of Recurrent Network Architectures Rafal Jozefowicz, Wojciech Zaremba, Ilya Sutskever, , ICML, 2015

Awesome Recurrent Neural Networks / Applications / Natural Language Processing
Paper Tomas Mikolov, Martin Karafiat, Lukas Burget, Jan "Honza" Cernocky, Sanjeev Khudanpur, , Interspeech 2010 [ ]
Paper Tomas Mikolov, Stefan Kombrink, Lukas Burget, Jan "Honza" Cernocky, Sanjeev Khudanpur, , ICASSP 2011 [ ]
Paper Stefan Kombrink, Tomas Mikolov, Martin Karafiat, Lukas Burget, , Interspeech 2011 [ ]
Paper Jiwei Li, Minh-Thang Luong, and Dan Jurafsky, , ACL 2015 [ ], [ ]
Paper Ryan Kiros, Yukun Zhu, Ruslan Salakhutdinov, and Richard S. Zemel, , arXiv:1506.06726 / NIPS 2015 [ ]
Paper Yoon Kim, Yacine Jernite, David Sontag, and Alexander M. Rush, , arXiv:1508.06615 [ ]
Paper Xingxing Zhang, Liang Lu, and Mirella Lapata, , arXiv:1511.00060 [ ]
Paper Felix Hill, Antoine Bordes, Sumit Chopra, and Jason Weston, , arXiv:1511.0230 [ ]
Paper Geoffrey Hinton, Li Deng, Dong Yu, George E. Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N. Sainath, and Brian Kingsbury, , IEEE Signam Processing Magazine 2012 [ ]
Paper Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton, , arXiv:1303.5778 / ICASSP 2013 [ ]
Paper Jan Chorowski, Dzmitry Bahdanau, Dmitriy Serdyuk, Kyunghyun Cho, and Yoshua Bengio, , arXiv:1506.07503 / NIPS 2015 [ ]
Paper Haşim Sak, Andrew Senior, Kanishka Rao, and Françoise Beaufays. , arXiv:1507.06947 2015 [ ]
Paper Oxford [ ]

Awesome Recurrent Neural Networks / Applications / Natural Language Processing / Univ. Montreal
Paper Kyunghyun Cho, Bart van Berrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio, , arXiv:1406.1078 / EMNLP 2014 [ ]
Paper Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio, , SSST-8 2014 [ ]
Paper Dzmitry Bahdanau, KyungHyun Cho, and Yoshua Bengio, , arXiv:1409.0473 / ICLR 2015 [ ]
Paper Sebastian Jean, Kyunghyun Cho, Roland Memisevic, and Yoshua Bengio, , arXiv:1412.2007 / ACL 2015 [ ]

Awesome Recurrent Neural Networks / Applications / Natural Language Processing
Paper Univ. Montreal + Middle East Tech. Univ. + Univ. Maine [ ]
Paper Google [ ]
Paper Google + NYU [ ]
Paper ICT + Huawei [ ]
Paper Stanford [ ]
Paper Middle East Tech. Univ. + NYU + Univ. Montreal [ ]
Paper Lifeng Shang, Zhengdong Lu, and Hang Li, , arXiv:1503.02364 / ACL 2015 [ ]
Paper Oriol Vinyals and Quoc V. Le, , arXiv:1506.05869 [ ]
Paper Ryan Lowe, Nissan Pow, Iulian V. Serban, and Joelle Pineau, , arXiv:1506.08909 [ ]
Paper Jesse Dodge, Andreea Gane, Xiang Zhang, Antoine Bordes, Sumit Chopra, Alexander Miller, Arthur Szlam, and Jason Weston, , arXiv:1511.06931 [ ]
Paper Jason Weston, , arXiv:1604.06045, [ ]
Paper Antoine Bordes and Jason Weston, , arXiv:1605.07683 [ ]

Awesome Recurrent Neural Networks / Applications / Natural Language Processing / FAIR
Web Jason Weston, Antoine Bordes, Sumit Chopra, Tomas Mikolov, and Alexander M. Rush, , arXiv:1502.05698 [ ] [ ]
Paper Antoine Bordes, Nicolas Usunier, Sumit Chopra, and Jason Weston, , arXiv:1506.02075 [ ]
Paper Felix Hill, Antoine Bordes, Sumit Chopra, Jason Weston, "The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations", ICLR 2016 [ ]

Awesome Recurrent Neural Networks / Applications / Natural Language Processing
Paper DeepMind + Oxford [ ]
Paper MetaMind [ ]

Awesome Recurrent Neural Networks / Applications / Computer Vision
Paper Pedro Pinheiro and Ronan Collobert, , ICML 2014 [ ]
Paper Ming Liang and Xiaolin Hu, , CVPR 2015 [ ]
Paper Wonmin Byeon, Thomas Breuel, Federico Raue1, and Marcus Liwicki1, , CVPR 2015 [ ]
Paper Mircea Serban Pavel, Hannes Schulz, and Sven Behnke, , IJCNN 2015 [ ]
Paper Shuai Zheng, Sadeep Jayasumana, Bernardino Romera-Paredes, Vibhav Vineet, Zhizhong Su, Dalong Du, Chang Huang, and Philip H. S. Torr, , arXiv:1502.03240 [ ]
Paper Xiaodan Liang, Xiaohui Shen, Donglai Xiang, Jiashi Feng, Liang Lin, and Shuicheng Yan, , arXiv:1511.04510 [ ]
Paper Sean Bell, C. Lawrence Zitnick, Kavita Bala, and Ross Girshick, , arXiv:1512.04143 / ICCV 2015 workshop [ ]
Paper Quan Gan, Qipeng Guo, Zheng Zhang, and Kyunghyun Cho, , arXiv:1511.06425 [ ]
Paper Karol Gregor, Ivo Danihelka, Alex Graves, Danilo J. Rezende, and Daan Wierstra, ICML 2015 [ ]
Paper Angeliki Lazaridou, Dat T. Nguyen, R. Bernardi, and M. Baroni, arXiv:1506.03500 [ ]
Paper Lucas Theis and Matthias Bethge, arXiv:1506.03478 / NIPS 2015 [ ]
Paper Aaron van den Oord, Nal Kalchbrenner, and Koray Kavukcuoglu, arXiv:1601.06759 [ ]
paper Univ. Toronto [ ]
paper Univ. Cambridge [ ]

Awesome Recurrent Neural Networks / Applications / Multimodal (CV + NLP)
Web UCLA + Baidu [ ] [ ], [ ]
Paper Univ. Toronto [ ] [ ]
Web Berkeley [ ] [ ]
Paper Google [ ]
[Web] Stanford
Paper Microsoft [ ]
Paper-arXiv CMU + Microsoft [ ], [ ]
Web Univ. Montreal + Univ. Toronto [ ] [ ]
Paper Idiap + EPFL + Facebook [ ]
Paper UCLA + Baidu [ ]

Awesome Recurrent Neural Networks / Applications / Multimodal (CV + NLP) / MS + Berkeley
Paper Jacob Devlin, Saurabh Gupta, Ross Girshick, Margaret Mitchell, and C. Lawrence Zitnick, , arXiv:1505.04467 (Note: technically not RNN) [ ]
Paper Jacob Devlin, Hao Cheng, Hao Fang, Saurabh Gupta, Li Deng, Xiaodong He, Geoffrey Zweig, and Margaret Mitchell, , arXiv:1505.01809 [ ]

Awesome Recurrent Neural Networks / Applications / Multimodal (CV + NLP)
Paper Adelaide [ ]
Paper Tilburg [ ]
Paper Univ. Montreal [ ]
Paper Cornell [ ]
Web Berkeley [ ] [ ]
Paper UT Austin + UML + Berkeley [ ]
Paper Microsoft [ ]
Paper UT Austin + Berkeley + UML [ ]
Paper Univ. Montreal + Univ. Sherbrooke [ ]
Paper MPI + Berkeley [ ]
Paper Univ. Toronto + MIT [ ]
Paper Univ. Montreal [ ]
Paper Zhejiang Univ. + UTS [ ]
Paper Univ. Montreal + NYU + IBM [ ]
Web Virginia Tech. + MSR [ ] [ ]
Web MPI + Berkeley [ ] [ ]
Paper Univ. Toronto [ ] [ ]
Paper Baidu + UCLA [ ] [ ]
Paper SNU + NAVER [ ]
Paper UC Berkeley + Sony [ ]
Paper Postech [ ]
Paper SNU + NAVER [ ]

Awesome Recurrent Neural Networks / Applications / Multimodal (CV + NLP) / Video QA
paper CMU + UTS [ ]
Paper KIT + MIT + Univ. Toronto [ ] [ ]

Awesome Recurrent Neural Networks / Applications / Multimodal (CV + NLP)
Paper A.Graves, G. Wayne, and I. Danihelka., arXiv preprint arXiv:1410.5401 [ ]
Paper Jason Weston, Sumit Chopra, Antoine Bordes, arXiv:1410.3916 [ ]
Paper Armand Joulin and Tomas Mikolov, , arXiv:1503.01007 / NIPS 2015 [ ]
Paper Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, and Rob Fergus, , arXiv:1503.08895 / NIPS 2015 [ ]
Paper Wojciech Zaremba and Ilya Sutskever, arXiv:1505.00521 [ ]
Paper Baolin Peng and Kaisheng Yao, , arXiv:1506.00195 [ ]
Paper Fandong Meng, Zhengdong Lu, Zhaopeng Tu, Hang Li, and Qun Liu, , arXiv:1506.06442 [ ]
Paper Arvind Neelakantan, Quoc V. Le, and Ilya Sutskever, , arXiv:1511.04834 [ ]
Paper Scott Reed and Nando de Freitas, , arXiv:1511.06279 [ ]
Paper Karol Kurach, Marcin Andrychowicz, and Ilya Sutskever, , arXiv:1511.06392 [ ]
Paper Łukasz Kaiser and Ilya Sutskever, , arXiv:1511.08228 [ ]
Paper Ethan Caballero, , arXiv:1511.6420 [ ]
Paper Wojciech Zaremba, Tomas Mikolov, Armand Joulin, and Rob Fergus, , arXiv:1511.07275 [ ]

Awesome Recurrent Neural Networks / Applications / Robotics
Paper Hongyuan Mei, Mohit Bansal, and Matthew R. Walter, , arXiv:1506.04089 [ ]
[Paper] Marvin Zhang, Sergey Levine, Zoe McCarthy, Chelsea Finn, and Pieter Abbeel, arXiv:1507.01273

Awesome Recurrent Neural Networks / Applications / Other
[Paper] Alex Graves, arXiv:1308.0850
Paper Volodymyr Mnih, Nicolas Heess, Alex Graves, and Koray Kavukcuoglu, , NIPS 2014 / arXiv:1406.6247 [ ]
Paper Wojciech Zaremba and Ilya Sutskever, , arXiv:1410.4615 [ ] [ ]
Paper Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer, , arXiv:1506.03099 / NIPS 2015 [ ]
Paper Bing Shuai, Zhen Zuo, Gang Wang, and Bing Wang, , arXiv:1509.00552 [ ]
Paper Soren Kaae Sonderby, Casper Kaae Sonderby, Lars Maaloe, and Ole Winther, , arXiv:1509.05329 [ ]
Paper Cesar Laurent, Gabriel Pereyra, Philemon Brakel, Ying Zhang, and Yoshua Bengio, , arXiv:1510.01378 [ ]
[Paper] Jiwon Kim, Jung Kwon Lee, Kyoung Mu Lee, , arXiv:1511.04491
Paper Quan Gan, Qipeng Guo, Zheng Zhang, and Kyunghyun Cho, , arXiv:1511.06425 [ ]
Paper Francesco Visin, Kyle Kastner, Aaron Courville, Yoshua Bengio, Matteo Matteucci, and Kyunghyun Cho, , arXiv:1511.07053 [ ]
[Paper] Juergen Schmidhuber, , arXiv:1511.09249

Awesome Recurrent Neural Networks / Datasets / Speech Recognition
OpenSLR (Open Speech and Language Resources)

Awesome Recurrent Neural Networks / Datasets / Speech Recognition / OpenSLR
LibriSpeech ASR corpus

Awesome Recurrent Neural Networks / Datasets / Speech Recognition
VoxForge

Awesome Recurrent Neural Networks / Datasets / Image Captioning
Flickr 8k
Flickr 30k
Microsoft COCO

Awesome Recurrent Neural Networks / Datasets / Question Answering
The bAbI Project Dataset for text understanding and reasoning, by Facebook AI Research. Contains:

Awesome Recurrent Neural Networks / Datasets / Question Answering / The bAbI Project
Paper The (20) QA bAbI tasks - [ ]
Paper The (6) dialog bAbI tasks - [ ]
Paper The Children's Book Test - [ ]
Paper The Movie Dialog dataset - [ ]
Data The MovieQA dataset - [ ]
Paper The Dialog-based Language Learning dataset - [ ]
Paper The SimpleQuestions dataset - [ ]

Awesome Recurrent Neural Networks / Datasets / Question Answering
SQuAD Stanford Question Answering Dataset : [ ]

Awesome Recurrent Neural Networks / Datasets / Image Question Answering
DAQUAR built upon by N. Silberman et al
VQA based on images
Image QA based on MSCOCO images
Multilingual Image QA built from scratch by Baidu - in Chinese, with English translation

Awesome Recurrent Neural Networks / Datasets / Action Recognition
THUMOS : Large-scale action recognition dataset
MultiTHUMOS : Extension of THUMOS '14 action detection dataset with dense multilabele annotation

Awesome Recurrent Neural Networks / Blogs
The Unreasonable Effectiveness of RNNs by
Understanding LSTM Networks in
WildML blog's RNN tutorial [ ], [ ], [ ], [ ]
RNNs in Tensorflow, a Practical Guide and Undocumented Features
Optimizing RNN Performance from Baidu's Silicon Valley AI Lab
Character Level Language modelling using RNN by Yoav Goldberg
Implement an RNN in Python
LSTM Backpropogation
Introduction to Recurrent Networks in TensorFlow by Danijar Hafner
Variable Sequence Lengths in TensorFlow by Danijar Hafner
Written Memories: Understanding, Deriving and Extending the LSTM by Silviu Pitis

Awesome Recurrent Neural Networks / Online Demos
link Alex graves, hand-writing generation [ ]
link Ink Poster: Handwritten post-it notes [ ]
link LSTMVis: Visual Analysis for Recurrent Neural Networks [ ]

Backlinks from these awesome lists:

More related projects: