How to Play Gesture Synth

A browser instrument for music lovers

Gesture Synth is made by Cian, an independent creator, for music lovers who want to explore harmony through movement. It turns two hands into a simple chord instrument: the left hand chooses the harmony, while the right hand changes its shape and sound.

Think of it as a place to sketch a progression, find a texture, or capture a brief performance idea. You do not need to know formal music theory to begin, but the Roman numerals and chord labels on screen make the musical choices visible as you play.

Get started with one simple chord

1. Open the Gesture Synth homepage. Your browser immediately asks for camera access; choose Allow.
2. The live camera preview appears first while the local hand-tracking model loads. Select the centered play button to unlock Web Audio.
3. Choose a key and synth sound from the controls in the upper-left corner of the stage, then stand or sit where both hands can stay in the camera frame. Clear lighting and a plain background can make the hand positions easier to see.
4. Make a left-hand gesture to choose a chord, then add the right hand to voice and shape it.

For a first sound, show one finger with your left hand and tilt it inward for a major I chord. On the right hand, extend one non-thumb finger for root position, then raise or lower that hand to change volume. Watch the chord label and quality readout as you settle into each pose.

Gesture Synth asks for camera access as soon as you open the homepage. If you allow it, the live camera preview appears before the on-device MediaPipe hand tracker finishes loading. Browser autoplay rules keep the synth silent until you select the centered play button; hand tracking continues to load in the background. You can stop camera access by closing the page, revoking browser permission, or disabling the camera on your device.

Open Gesture Synth and try the first chord.

Left hand: choose the harmony

Your left hand selects the scale degree and whether the chord is major or minor.

• One through five extended fingers select I through V.
• Extend index and pinky, with the middle and ring fingers folded, for VI.
• Add the thumb to that index-and-pinky shape for VII.
• Tilt the left hand inward for major and outward for minor.

If you are new to harmony, try I, IV, and V first. In a selected key, those three degrees are a useful place to make a familiar progression. Change the left-hand tilt after you know the shape to hear the difference between major and minor.

Open Gesture Synth to try the left-hand map.

Right hand: voice and shape the sound

Your right hand changes the chord voicing, octave, filter, and volume. Its finger count uses the four non-thumb fingers:

• One extended finger: root position.
• Two extended fingers: first inversion.
• Three extended fingers: a seventh chord.
• Four extended fingers: dominant seventh in major mode or diminished seventh in minor mode.

Use the right thumb position to move between octaves: thumb in for a higher octave and thumb out for a lower octave. Tilt the right hand inward for more filter and outward for less. Raise it for a louder sound or lower it for a softer sound.

Try changing one element at a time. For example, keep a left-hand I major chord steady while moving from one to three right-hand fingers, then change the hand height. This makes it easier to connect a physical movement with the sound and the on-screen readout.

The on-stage Gesture guide button repeats these mappings while you play.

Open Gesture Synth to add the right hand.

Make a short performance

Once hand tracking is ready and sound is enabled, select the circular record control. Only then does Gesture Synth ask for microphone permission. Recording combines the live performance display with microphone and generated synth audio. Select the stop control when you are finished; a recording also stops after five minutes.

When recording is available, the browser downloads the resulting MP4 locally. Download locally, then share wherever you choose. Before publishing a take, make sure you have the rights to any voices, sounds, or other material captured by your microphone.

Open Gesture Synth to make a take.

Tested browser boundary

Gesture Synth has been tested in Google Chrome. That is the browser testing result this guide can state.

It does not promise operation on a particular Chrome version, device, operating system, camera, microphone, extension setup, or any other browser. Camera access, hand tracking, audio, canvas capture, and MP4 recording depend on the browser and device in front of you. Read browser and device compatibility before you play, or use camera permission help when the browser blocks access.

Need help?

For help with a camera, hand-control, or recording issue, email a3011336204@gmail.com. Include the browser, device, operating system, and the exact message you saw, but do not send personal recordings or camera footage unless it is necessary and you are comfortable sharing it.