Hi, I am trying to create a learning model with Fastai. I propose to convert an orchestral score to piano score, what we technically call ‘reduction’.
The first thing I am thinking about is the type of data I should use. Associated to this, the approach (vision, NLP, etc…).
I have a dataset It contains scores of orchestral with its reduction to piano each one.
The dataset is distributed in several folders according to the origin of the files…
- Each folder contains its folders indexed with numbers.
- another folder with a CSV with the same name and metadata.
-in each indexed subfolder, there are two MIDI files and their corresponding CSV.
- the MIDI files are an orchestration and its corresponding piano version.
- CSV contains mappings between the midi track names and some normalized names for the instrument names or e.g., a MIDI track name, a MIDI track name, a MIDI track name, a MIDI track name, a MIDI track name and a MIDI track name:
*Beethoven_classical_archives/32/beet9m2.csv and Beethoven_classical_archives/32/symphony_9_2_orch.csv.
Could someone give some advice for this task? Also if someone is interested in the project, we could work on It as a team.
I would be very gratefull!!
WARNING I have no musical ability so this may be of no help what so ever.
A VAE (encoder) takes a image and produces a small image which when run back through the VAE (decoder) produces a similar images. So a picture 28x28 is reduced to 7x7 and then returns to 28x28. This means you could compress by 4x4=16 before emailing a friend who also has the VAE (decoder).
So you have lots of MIDI streams one for each instrument and you want to compress it down to one instrument the piano and the maybe de-reduce it back to a full orchestra.
So imagine a 2D picture with each instrument on its own score and then the same 2D for piano.
Make the length of the picture equal to a musical phrase. Repeat for sub-phrase, motifs and sentences.
Run the 2D pictures through the VAE to produced the compressed piano version.
Eventually you would have a model that takes the larger score and produces the piano.
My guess is it would not work because the MIDI piano could only play so many notes at once but I suppose human composers would not flood humans with lots of instruments at once.
Thanks for your answer, it is inspiring and I think I can get some application to my idea. But now the first challenge I have to face is the labeling of the data (in my case the midi files) and it is a difficulty because the reduction from orchestra to piano involves a series of technical, harmonic and piano language decisions that makes the task quite complicated. For example, when making a reduction you have to eliminate instruments that are simply voices doubled from others and at the same time know what is structural and what is essential in terms of texture and melody. So, now I am in the mud with the data issue