Abstract:

Real-time human--AI musical collaboration presents a persistent design challenge: AI contributions tend to be either too conservative to be interesting or too unpredictable to play along with. We posit that looping (along with related practices of vamping and grooving) offers a framework that sidesteps this trade-off. These recurring structures bound variation within forms the musician already understands, and let development accrue over musical rather than sample-level timescales---a balance upon which improvisers have long relied when playing with each other.

We present SmartLooper, a system for real-time human–AI piano improvisation built on this premise. The performer first assembles a personal corpus of their own loop-based recordings, giving them complete control over the stylistic space the system inhabits. SmartLooper segments this corpus into short loops and embeds them using pretrained MIDI models. In performance, the system improvises by retrieving loops from the corpus via a stochastic walk in embedding space: successive loops are chosen for proximity so that transitions feel continuous.

To broaden the corpus without diluting its character, we augment the data with synthetic loops in the style of the corpus generated using MIDI-GPT. These are filtered down to only high-quality samples using a discriminator trained to distinguish degraded MIDI from samples in the personal corpus. The resulting dataset extension provides broader support for fluid transitions while preserving the performer's voice.

SmartLooper has been used by professional musicians in over ten hours of live performance for audiences totalling roughly a thousand listeners so far; we provide example recordings.

Post-Review Updates:

The reviewers’ excellent suggestions informed a broader revision, whose key additions we retained within the six-page camera-ready limit. We:

  • added detail on how data-collection guidance encouraged loopable segments;
  • clarified the relationship between corrupted proxy negatives, synthetic samples, and the original dataset;
  • added an important previously omitted reference; and
  • added a discussion of limitations.