Artificial Intelligence is on the rise in every industry, with music composition, performance and recording no exception.
In this article we look at a potentially emerging technology named MIAO. Although currently dwelling in the theoretical and early prototype phase, MIAO (MIDI Audio) could be the next big thing, particularly in the realm of musical performance and perceived realism.
MIAO: The Next Leap in Music Tech
Let’s take a look at where we are now. Music technology has come a long way, and MIDI has advanced significantly since its creation. That said, most MIDI instruments, sample libraries and synths still sound digital and repetitive.
That’s fine for some genres, styles and compositions, but not all.
Getting MIDI tracks and electronic instruments to sound like real performances often requires painstaking tweaking of parameters. Creating realism in real time is even more demanding, and often requires a lot of additional controllers in order to provide nuances such as articulation.
MIAO is envisioned to use AI to interpret MIDI signals and automatically create dynamic, realistic performances based on it.
The MIDI Problem
MIDI has been nothing short of revolutionary for musicians and the industry at large. Its strength is in control and composition, but it doesn’t fully provide for realism.
Even MIDI 2.0 and MPE, combined with advanced controllers, makes for overly complex methods of getting the desired sounds during recording and performance.
We have plugins which can add certain elements such as randomness, or ‘humanise’ sequenced tracks, but they only go so far.
MIAO could push much further.
MIAO: Turning MIDI into Realistic Audio
MIAO is an AI-based system which learns from real instrument performances and uses that knowledge to generate realistic sound from MIDI input.
You play, or feed in, a MIDI track, and MIAO plays it back like a real musician would.
MIDI is static. Even if a track is unquantised, natural, with nuanced performance, that performance will be the same every time the track is played. MIAO interprets the track and humanises it — in the same way that even the same musician playing the same piece of music twice will not perform identically both times.
MIAO is currently in research, but it clearly has real-world musical potential.
How MIAO Works
In basic terms, MIAO works as follows:
- It reads the MIDI notes, timing and dynamics already present in the data.
- Having been trained on real performance, it knows how various patterns would sound on different instruments when played by a human musician.
- It generates audio to match, including appropriate phrasing, resonance and subtle variations, without relying on samples (or, perhaps, knowing how to manipulate samples without user intervention)
It’s akin to hiring a session player and trusting them to play something appropriate.
How MIAO is Different to Current Samples and Synths
The technology, instruments and samples we have today allow musicians to produce some phenomenally realistic audio, but there are disadvantages.
- Samples are lifelike but can be repetitive and heavy to manage.
- Synths are flexible but not lifelike — which is fine for some genres and styles but not others.
MIAO would be intelligent, adaptive, and expressive. It could handle multiple instruments with one model. It is context aware, so it understands phrasing, not just volume and velocity.
Of course anything combining AI and audio processing, particularly if used for realtime performance, will require significant processing power. However, processors continue to improve and one would assume that the efficiency of this kind of AI and audio processing would do too.
MIAO: Partnering MIDI 2.0 and MPE
MIDI 2.0 was a major step up on the original MIDI standard, allowing for such things as higher resolution, per-note control, and smoother expression.
MPE, too, delivers finer control over individual notes and makes for more expressive playing.
MIAO could be used to translate that extra expression into sound that truly reflects it. It could interpret the intent of the player and then translate that to other instruments, sound spaces and so on.
What a MIAO Device Might Look Like
MIAO built in to a hardware instrument could perhaps be a digital keyboard which responds like an acoustic instrument.
It could learn your playing style and adjust tone automatically. Multiple instruments could be played as one expressive performance.
What a MIAO Plugin Might Look Like
A plugin could accept MIDI clips and generate natural, realistic playback.
Instruments, styles or emotions could be chosen and given different priorities. Timbres could be blended and then high faulty audio output.
How Might Musicians Use MIAO?
A number of uses for MIAO are already apparent.
- Composers could create realistic mockups without requiring huge sample libraries.
- Keyboard players would have access to more expressive live sound engines that breathe and react.
- Producers could export natural-sounding stems from MIDI sketches.
- Teachers and students would be able to hear realistic playback that demonstrates phrasing and tone control.
MIAO’s Creative Possibilities
As well as creating realistic performances of real world instruments, MIAO could also be used to create hybrid instruments which blend acoustic and synthetic tones.
Of course, this is already possible at a general level, but MIAO could take the concept much further, allowing for more nuanced soundscapes with less detailed manual parameter tweaking via controllers and the like by the human performer.
There’s potential for far more expansive “AI performers” which accompany or even improvise. While we already have many AI and non-AI accompaniment options, MIAO could take the concept so much further, which again could reduce the headaches associated with setup and leave the human musician to focus on their own creativity.
The Pros and Cons of MIAO: Promise vs Practicality
Why MIAO Could Change Everything
MIAO offers a completely new way of thinking about MIDI and sound generation. Instead of triggering pre-recorded samples or synthesising tones, it interprets your performance like a human would — phrasing notes, adjusting tone, and shaping dynamics in real time.
For musicians and producers, that could mean:
- Hyper-realistic sound without giant sample libraries.
- Natural expression that reflects how you actually play.
- Simpler workflows — less manual MIDI editing or “humanizing.”
- Better use of MIDI 2.0 and MPE data, which MIAO could turn into genuinely human-sounding performance.
- Creative freedom to design new hybrid instruments or expressive textures.
In short, it has the potential to make digital instruments feel alive in ways that sample packs and synths can’t.
Where MIAO Might Fall Short
Like any breakthrough, MIAO also raises new challenges:
- Less precise control: because it “interprets” your input, results may not be identical every time.
- High computing demand: real-time AI audio takes serious processing power (and may initially rely on the cloud).
- Workflow changes: you’ll be guiding musical intention, not just adjusting filters or velocities.
- Consistency issues: subtle playback differences could complicate professional scoring or re-mixing.
- Ethical and legal grey areas: the AI learns from real recordings — so ownership and style imitation questions will arise.
- Risk of dependence: if future instruments rely on proprietary AI, musicians could lose long-term access or compatibility.
- Artistic dilution: easy realism might reduce emphasis on real performance or nuanced programming.
MIAO’s technology could bring emotion and realism back to digital music, closing the gap between human and machine performance, but as with any powerful new tool, it will work best when used with intention — as a creative partner rather than a replacement for musicianship.
If developers get the balance right, MIAO could become to MIDI what sampling once was to recording: a game-changer that redefines how we make and experience music.
What’s Next for MIAO?
It’s important to note that MIAO is still very much in the research stage. That said, the quality of AI-generated sound is quickly improving.
It may not be too long before prototypes start appearing, perhaps as plugins for DAWs.
Once the technology really stabilises, elements of MIAO could appear as standard in popular DAWs, as virtual instruments, and in high-end hardware instruments.
It could be that the advances we have seen AI bring to speech synthesis could soon happen to music creation and production.
Conclusions
Some may be wary at the speed at which AI is affecting all aspects of musicianship. Any tool can be misused — overworked to create low quality factory conveyor belt music. If we look beyond the negative, MIAO could be an incredible tool which enhances rather than replaces talented human musicians.
MIDI captures what you play. MIAO could make it sound the way you meant it.
Future digital instruments may come to a point of understanding music rather than simply producing the sounds they’re fed.
For those who wish to dive deep into the research, it can be found on “Multi-modal music techniques for synthesizing high-quality audio waveforms from MIDI data“.
