Music Generation
In this post, We will take a hands-on-lab of Music Generation. This tutorial is the software lab, which is part of "introduction to deep learning 2021" offered from MIT 6.S191.
- Copyright Information
- Music Generation with RNNs
- Dataset
- Process the dataset for the learning task
- The Recurrent Neural Network (RNN) model
- Training the model: loss and training operations
- Generate music using the RNN model
- Experiment and get awarded for the best songs!
Copyright Information
Copyright 2021 MIT 6.S191 Introduction to Deep Learning. All Rights Reserved.
Licensed under the MIT License. You may not use this file except in compliance with the License. Use and/or modification of this code outside of 6.S191 must reference:
© MIT 6.S191: Introduction to Deep Learning http://introtodeeplearning.com
Music Generation with RNNs
In this portion of the lab, we will explore building a Recurrent Neural Network (RNN) for music generation. We will train a model to learn the patterns in raw sheet music in ABC notation and then use this model to generate new music.
import tensorflow as tf
import numpy as np
import os
import time
import regex as re
import subprocess
import urllib
import functools
from IPython import display as ipythondisplay
from tqdm import tqdm
import matplotlib.pyplot as plt
# Check that we are using a GPU, if not switch runtimes
# using Runtime > Change Runtime Type > GPU
assert len(tf.config.list_physical_devices('GPU')) > 0
cwd = os.getcwd()
def extract_song_snippet(text):
pattern = '(^|\n\n)(.*?)\n\n'
search_results = re.findall(pattern, text, overlapped=True, flags=re.DOTALL)
songs = [song[1] for song in search_results]
print("Found {} songs in text".format(len(songs)))
return songs
songs = []
with open(os.path.join(cwd, 'dataset', 'irish.abc'), 'r') as f:
text = f.read()
songs = extract_song_snippet(text)
# Print one of the songs to inspect it in greater detail!
example_song = songs[0]
print("\nExample song: ")
print(example_song)
We can easily convert a song in ABC notation to an audio waveform and play it back. Be patient for this conversion to run, it can take some time.
timidity
and abc2midi
on windows, it can be worked on windows. I`ve done this on windows jupyter environment
def save_song_to_abc(song, filename="tmp"):
save_name = "{}.abc".format(filename)
with open(save_name, "w") as f:
f.write(song)
return filename
def abc2wav(abc_file):
suf = abc_file.rstrip('.abc')
cmd = "abc2midi {} -o {}".format(abc_file, suf + ".mid")
os.system(cmd)
cmd = "timidity {}.mid -Ow {}.wav".format(suf, suf)
return os.system(cmd)
def play_wav(wav_file):
return ipythondisplay.Audio(wav_file)
def play_song(song):
basename = save_song_to_abc(song)
ret = abc2wav(basename + '.abc')
if ret == 0: #did not suceed
return play_wav(basename+'.wav')
return None
play_song(example_song)