JUTSU
Open sourceIt took a gesture classifier, a GPU compositor, and a reverse engineered binary to let me slash my friends in half on a video call.
Real-time hand gesture recognition that composites anime effects onto a virtual webcam, live inside Discord and OBS.
A multi-threaded Windows service that tracks your hands through MediaPipe, classifies gestures with a custom scikit-learn ensemble, and renders GPU accelerated effects onto a virtual camera device. Any video app reads it as an ordinary webcam feed.
The part I'm proudest of isn't the visuals. The service idles at roughly zero CPU and GPU with the camera off, and wakes only when something actually opens the virtual device. Neither UnityCapture nor OBS exposes an event for that, so I reverse engineered a timing based IPC side channel out of the compiled binary to detect consumers.
Gestures run on an 83 dimensional pose vector built from wrist origin normalization, joint bend cosines, and inter fingertip distances, trained on a 10x augmented dataset. Augmentation runs on the raw landmark arrays before feature extraction, so every derived angle stays geometrically consistent with the pose it came from.
- Lines of code
- 3,887
- My commits
- 30 of 30
- Contributors
- 1



