SpecVQGAN
Visual sound generator
A project to train an audio generation model that uses visual cues to produce high-quality sounds from a reduced dataset of representative vectors.
Source code for "Taming Visually Guided Sound Generation" (Oral at the BMVC 2021)
353 stars
8 watching
39 forks
Language: Jupyter Notebook
last commit: 6 months ago audioaudio-generationbmvcevaluation-metricsganmelganmulti-modalpytorchtransformervasvggsoundvideovideo-featuresvideo-understandingvqvae
Related projects:
| Repository | Description | Stars |
|---|---|---|
 virtualanalogy/paraphrasis | A spectral modeling synthesizer for generating sound. | 32 |
 peihaochen/regnet | An implementation of a neural network for generating sound from video sequences | 52 |
 tyorikan/voice-recording-visualizer | A Java-based Android app for visualizing audio recordings | 550 |
 l0sg/seqgan-music | Generates polyphonic music sequences using deep learning models and adversarial training | 28 |
 salu133445/musegan | Generates polyphonic music from scratch or based on user input using a neural network architecture | 1,863 |
 chrisdonahue/wavegan | An open-source machine learning algorithm for generating raw audio waveforms from raw data | 1,334 |
 mrugalla/nel-19 | An open-source plugin for generating vibrato effects in audio signals | 77 |
 vincentherrmann/pytorch-wavenet | An implementation of WaveNet for generating audio using PyTorch | 979 |
 nicolas-van/sonant-x | A JavaScript synthesizer library used to generate sound effects and music for video games and small applications. | 236 |
 mfcc64/youtube-musical-spectrum | An audio visualizer extension that displays spectrograms of YouTube videos and microphone input on web pages. | 175 |
 ghostnan/revidia-audio-visualizer | An audio visualizer software that generates real-time visuals based on incoming audio streams | 24 |
 dasetwas/enginesound | A software tool that generates synthetic engine sounds with customizable parameters and real-time preview capabilities. | 313 |
 staskobzar/vue-audio-visual | An audio visualization plugin for the VueJS framework using HTML5 Web Audio API. | 726 |
 viktornova/nausea | A music visualizer that uses audio spectrum analysis to create real-time visualizations of an audio stream. | 10 |
 jonathanzwhite/audio-visualizer | A program for visualizing audio signals in real-time. | 19 |