Robotics & Music — Live Adaptive Jazz Ensemble
Machines that do not just play notes —
they listen, respond, and groove.
Jazz is built on interaction. A drummer listens to the bassist. A pianist responds to the soloist. A trumpet player shapes phrases based on what the rhythm section gives back.
Most music robots are either pre-programmed machines or mechanical instruments that play fixed tracks. JazzBot is different.
"A robot that can play jazz well must be more than a mechanical sequencer. It needs musical judgment."
Jazz is a perfect testbed for musical robotics because it demands:
| Role | Musician | Purpose |
|---|---|---|
| Drums | Robot | Swing feel, fills, support |
| Piano | Robot | Chords, comping, melody |
| Bass | Human | Time feel, harmonic movement |
| Trumpet | Human | Melody, improvisation |
The drummer and pianist share the same core architecture — Jetson Orin as brain, real-time controllers for actuation, timestamped commands, and shared musical scheduling.
Fixed actuator modules for main voices, one movable accent arm for visual presence. Fast to build, validates the entire timing system.
| Voice | Mechanism | Purpose |
|---|---|---|
| Ride cymbal | Fixed actuator | Main swing pattern |
| Hi-hat | Fixed / pedal | Beats 2 & 4 |
| Snare | Fixed actuator | Backbeat, fills |
| Kick | Pedal actuator | Low pulse, groove |
| Crash | Fixed / movable | Section accents |
| Tom | Accent arm | Fills & stage presence |
| One-pad hit module | 2–4 days |
| Basic 4-piece drummer | 10–14 days |
| Good jazz drummer demo | 3 weeks |
| Polished + accent arm | 4–6 weeks |
| Human-following drummer | 8–12 weeks |
| 1-pad prototype | $100–$250 |
| Fast 4-piece drummer | $600–$1,500 |
| Good jazz drummer demo | $1,200–$3,500 |
| Polished + accent arm | $3,000–$7,000+ |
Two 7-DOF A1/OpenArm-style arms with ORCA dexterous hands, controlled by Jetson Orin. Far more complex — but capable of jazz comping.
| One ORCA hand presses keys | 2–4 weeks |
| One arm + ORCA plays notes | 4–6 weeks |
| Two arms play a demo | 10–12 weeks |
| Reliable two-hand demo | 12–16+ weeks |
| Live jazz adaptive pianist | 6–12+ months |
| A1/OpenArm arms (×2) | $7,160–$13,000 |
| ORCA hands (×2) | several thousand ea. |
| Jetson AGX Orin 64GB | ~$2,000 |
| Digital piano | $430–$700 |
| Safety, mounts, spares | $1,500–$4,000 |
| Starting budget | $15k–$35k+ |
"The arm moves early. The hand presses exactly on the beat."
The goal is not a giant AI system first. The goal is to make physical notes and drum hits happen reliably.
drummer_cmd.json { "target": "ride", "start_ms": 12000, "velocity": 0.72 }
pianist_cmd.json { "cmd": "PLAY_CHORD", "hand": "right", "start_ms": 20000, "hold_ms": 500, "notes": [ {"finger": "thumb", "key": "C4", "velocity": 65}, {"finger": "index", "key": "E4", "velocity": 70}, {"finger": "middle", "key": "G4", "velocity": 68} ] }
note_timing.json { "note": "E4", "desired_sound_ms": 15000, "hand": "right", "finger": "index", "arm_arrive_by_ms":14850, "finger_start_ms": 14965, "hold_ms": 300, "velocity": 70 }
The project began with the idea of a robotic jazz band: a robot drummer, a robot pianist, a human bassist, and a human trumpeter. The goal is not simply to automate music, but to create robots that can eventually listen and respond like bandmates.
The project architecture was defined around Jetson Orin as the high-level brain. Jetson handles timing, musical logic, planning, and future AI listening. Lower-level controllers handle real-time actuation.
Two A1/OpenArm-style 7-DOF arms with two ORCA hands. The main risk is not whether the parts can connect — it's whether ORCA fingers can press piano keys cleanly, repeatedly, and with useful timing.
The drummer is much easier than the pianist because it mainly requires timed strikes instead of precise finger placement. Fixed actuator modules for all main voices became the recommended first milestone.
The drummer can be made more human-like without becoming a full humanoid robot. A hybrid: human-like torso behind the kit, two visible robotic arms, fixed actuator modules for reliability, one movable accent arm.
At 50 hours/week, the drummer can reach a jazz demo in 3 weeks. A limited two-arm pianist prototype in three months is possible, but aggressive. Full virtuosity is not a three-month target.
JazzBot is an experimental robotics and music project exploring live robot-human jazz performance. Collaborators welcome.