README.md

MeloForm

MeloForm: Generating Melody with Musical Form based on Expert Systems and Neural Networks, by Peiling Lu, Xu Tan, Botao Yu, Tao Qin, Sheng Zhao, Tie-Yan Liu, ISMIR 2022, is a melody generation system that leverages expert system and neural networks to generate melodies with musical form.

The overall architecture of our MeloForm framework.

The expert system of MeloForm.

The neural network of MeloForm.

1. Data

We obtain LMD dataset from here and process them to be monophonic. We privode a script to parse LMD data in our experiments. We give some raw data in melody tracks in here as examples to run scripts.

python preprocess_lmd.py ./data/train/raw ./data/train/processed/para

Based on the above scripts, data samples will be generated under the ./data/train/processed/para directory.

We have provide the script to generate binarized data. The script to run is as:

data_dir=./data/train/processed/
user_dir=meloformt
bash binarize.sh $data_dir $user_dir

2. Training

We provide an example script for running.

data_dir=./data/train/processed/processed_para/ # The path of binarized data
user_dir=meloform

bash train.sh $data_dir $user_dir

3. Refinement

For refinement, we need to firstly process sythetic data from expert systems, then neural networks will refine it phrase by phrase iteratively. We provide several melodies from expert system to help test, more data will be released soon.

Firstly, run the following script to process sythetic data from expert systems:

data_dir=./data/refine/expert_system
song_id=0
out_dir=./data/refine/data_nn
python process_es.py $data_dir $song_id $out_dir

The processed results are under ./data/refine/data_nn/{song_id}, in ./data/refine/data_nn/{song_id}/template.

Finally, run

data_dir=./data/refine/data_nn
song_id=0
model_path=checkpoints/
res_dir=results/
bash meloform_refine_melody.sh $data_dir $song_id $model_path $res_dir

The final refined melody will be located in {res_dir}/out_midi/{song_id}/b1/src_res.mid

Notice that we only release the code of neural networks this time, the melody data from expert systems will be released soon.