• RACHANA JAGANNATH
• SOWJANYA RAI
• TEJASWINI R
• UMME SARA
• MRS SANGEETHA V

This project presents the development of a gesture-controlled virtual musical instrument system aimed at empowering motor-impaired individuals to play and enjoy music with ease. Using a Raspberry Pi 4 and a Pi Camera, the system captures and interprets hand gestures in real-time to simulate playing multiple instruments, including piano, guitar, drums, and violin. The gesture recognition is implemented using computer vision and machine learning techniques, leveraging the MediaPipe framework for accurate and lightweight hand tracking.Unlike conventional MIDI-integrated systems, this project uses FluidSynth and custom soundfont files to generate high-quality instrument tones without requiring external synthesizers or hardware. Each unique hand gesture corresponds to specific musical notes or actions, allowing users to control rhythm, pitch, and instrument selection through simple, intuitive movements. The system is capable of seamlessly switching between different instruments through specific hand gestures, allowing users to function as a one-person virtual band or engage in collaborative performances with others.Built with a strong focus on accessibility, this innovation empowers individuals with limited mobility to explore new avenues of musical creativity and expression.It is cost-effective, fully open-source, and scalable, making it a viable tool for use in music therapy, special education, and inclusive musical performances.

This project presents the development of a gesture-controlled virtual musical instrument system aimed at empowering motor-impaired individuals to play and enjoy music with ease. Using a Raspberry Pi 4 and a Pi Camera, the system captures and interprets hand gestures in real-time to simulate playing multiple instruments, including piano, guitar, drums, and violin. The gesture recognition is implemented using computer vision and machine learning techniques, leveraging the MediaPipe framework for accurate and lightweight hand tracking.Unlike conventional MIDI-integrated systems, this project uses FluidSynth and custom soundfont files to generate high-quality instrument tones without requiring external synthesizers or hardware. Each unique hand gesture corresponds to specific musical notes or actions, allowing users to control rhythm, pitch, and instrument selection through simple, intuitive movements. The system is capable of seamlessly switching between different instruments through specific hand gestures, allowing users to function as a one-person virtual band or engage in collaborative performances with others.Built with a strong focus on accessibility, this innovation empowers individuals with limited mobility to explore new avenues of musical creativity and expression.It is cost-effective, fully open-source, and scalable, making it a viable tool for use in music therapy, special education, and inclusive musical performances.

"GESTURE CONTROLLED MULTI INSTRUMENT FOR MOTOR IMPAIRED USERS", International Journal of Emerging Technologies and Innovative Research (www.jetir.org), ISSN:2349-5162, Vol.12, Issue 11, page no.f537-f547, November-2025, Available :http://www.jetir.org/papers/JETIR2511583.pdf

"GESTURE CONTROLLED MULTI INSTRUMENT FOR MOTOR IMPAIRED USERS", International Journal of Emerging Technologies and Innovative Research (www.jetir.org | UGC and issn Approved), ISSN:2349-5162, Vol.12, Issue 11, page no. ppf537-f547, November-2025, Available at : http://www.jetir.org/papers/JETIR2511583.pdf