2026 · MediaPipe · Tone.js

gesture.live

A gesture based electronic performance suite in the browser: MediaPipe reads your hands through the webcam and Tone.js turns the motion into chords, melodies, bass, drums and live effects.

gesture.live is an electronic performance suite that runs entirely in the browser. MediaPipe tracks your hands through the webcam and Tone.js turns that movement into sound, so a whole set can be played without touching an instrument.

The idea came from the gesture synth by Indecisive Eric (indecisiveeric.com/gesture-synth). I kept seeing videos of people making really cool music with it, and I wanted to see how much further the concept could go.

It keeps Eric's chord language, where you hold up a number of fingers to pick the chord degree, but I built a system that plays every chord with one hand. That leaves the right hand free for melodies, bass, drums and performance effects, which turns a single instrument into a full live rig.