One of the most time-consuming and potentially frustrating aspects of creating music which other people can interpret and play—or even as a record for your own keeping—is that of transcribing what is being played into legible and accurate sheet music. This is made harder because unlike an audio recording which (within technological margins) faithfully records every note with timing and performance nuance, sheet music is not an exact science. It is open to interpretation both during its creation and when used to play the music in the future.
Naturally, computers have made the task of getting musical ideas onto a printed manuscript much quicker, but every piece of software still has its own quirks and pitfalls, and although it may be quicker than writing the music by hand, it can still be problematic. It may even interrupt the creative flow if a composer wishes to transcribe a more improvised or off-the-cuff performance before the memory is lost.
Of course, various standards such as MIDI or simply recording into a DAW will capture those note values and other dynamics, and many pieces of software can do a reasonable first pass job of putting that data onto paper. They’re not perfect though, which is why this innovation from the makers of StaffPad could be a game changer, and one we fully expect to be seen in other music software over the coming years.
It uses artificial intelligence, as if you needed to ask, to listen to the audio of a pianist playing, and then translating that directly into notes and phrasings within the StaffPad notation software. It’s not relying on MIDI or any other digital data, simply listening for the piano being played and using its gained knowledge to create accurate notation.
The development team at StaffPad used a training system called Pianola which can generate thousands of hours of music, recorded via digitally-controlled acoustic pianos and sample libraries, which it then analyses and learns to recreate the audio input. After hours of training, the system can recognise note pitches and durations, even polyphonically such as within chords, and then generates written notation. As a final step, it cleans up the score so it is readable by others. Perhaps think of it as a “visual quantisation” which strips out some of the performance nuances that would simply lead to over-complicated scores if left in.
Clearly, the end user can still edit the generated score in StaffPad to remove errors, add missing notes, tidy or update how the AI has interpreted their performance, and so on. At very least, it could be a real boon to those whose performances are more spontaneous but who still want a written form of what they’ve played.
It’s a great addition to some of the other interpretive features already found in StaffPad, such as handwriting recognition and tempo mapping.
The Piano Capture feature is available in the latest version of StaffPad, which costs $90 and is available for iPad and Windows tablets. Find out more at the StaffPad website.
