Project Atom
A gesture-controlled 3D robot inspired by Real Steel. Control an animated robot in real-time using hand gestures captured via webcam. Built with Ursina/Panda3D, MediaPipe, and OpenCV.
Computer VisionArtificial Intelligence
👊
Punch
🖐
Kick
👊👊
Box
🖐🖐
Dance
Clone from GitHub and run locally to try it out
Technologies & Tools
Python
Ursina/Panda3D
MediaPipe/TensorFlow
OpenCV
Mixamo
Sketchfab
Inspired by the movie Real Steel, this project brings gesture-controlled robotics to life. Using computer vision and 3D rendering, you can control a robot's actions with your bare hands - make a fist to punch, open your palm to kick, and more.
Gesture Controls
- •Both Fists: Boxing stance
- •Both Palms: Dance animation
- •Left/Right Fist: Punch left/right
- •Left/Right Palm: Kick left/right
Technical Stack
- •Ursina/Panda3D for 3D rendering and animation
- •MediaPipe for 21-point hand landmark detection
- •OpenCV for webcam capture and image processing
- •Python threading for real-time gesture processing
- •GLB animated 3D models with skeletal animation
Key Technical Challenges Solved
- •Real-time webcam texture updates in Panda3D using setRamImage() with BGR format
- •Non-blocking gesture detection using Python threading with Queue for frame/pose data
- •Gesture classification by comparing finger tip vs PIP joint y-coordinates
- •Dynamic material system with height-based coloring for robot parts
- •Left/right hand differentiation using MediaPipe handedness detection
Real-Time Performance: Achieves smooth 30+ FPS by running ML inference and 3D rendering in parallel threads, with lock-free queues for minimal latency between gesture detection and robot animation response.
Architecture Flow
Webcam → OpenCV → MediaPipe → Gesture Detection → Ursina/Panda3D → Robot GLB (Mixamo)
3D Assets & Tools
- •Sketchfab - Robot 3D model sourced from the Sketchfab marketplace
- •Mixamo - Character animations (punch, kick, dance, boxing) applied to the robot model
- •GLB Format - Exported animations in GLB format for Panda3D compatibility