Signlingo – Urdu Sign Language Recognition System

Signlingo is a real-time Urdu sign language recognition desktop application developed to help bridge communication gaps between deaf individuals and non-signing users in Pakistan. The system recognizes 48 single-hand Urdu sign language gestures, including alphabet letters, common cultural phrases such as Shukriya and Assalamualaikum, and special actions such as space, speech, and open_palm. The application uses a custom dataset of approximately 24,000 gesture samples. MediaPipe extracts 42-dimensional hand landmark features from webcam input, while a Support Vector Machine (SVM) with an RBF kernel classifies the gestures in real time. Recognized gestures are displayed as editable text and can be converted into Urdu speech using Google Text-to-Speech (gTTS). Signlingo was built as a lightweight client-side desktop application using Python and PyQt6, with OpenCV handling webcam processing and pygame supporting audio playback. The system was designed to run on standard hardware, making it suitable for educational institutions, community centers, and individual users. Testing reported an overall recognition accuracy of 99.38%, with approximately 0.028-second prediction time and a usability rating around 4.2/5. The project also includes a visual sign guide to help users learn and reference the supported gestures.

Urdu sign language has limited support in automated recognition technologies, while many existing sign language systems focus on languages such as American Sign Language and rely on computationally intensive approaches or specialized hardware. This creates accessibility challenges for deaf individuals in Pakistan, particularly where human interpreters are unavailable. Signlingo addresses the need for a lightweight, real-time and cost-effective Urdu sign language recognition solution that can operate using a standard webcam and desktop computer.

Signlingo provides a webcam-based Urdu sign language recognition pipeline. Live video is captured through OpenCV, MediaPipe detects 21 hand landmarks and converts them into a 42-dimensional feature vector, and a trained SVM classifier identifies one of 48 supported gestures. The recognized gesture is displayed as editable text and can optionally be converted into Urdu speech through gTTS. The system also includes a sign guide, informational pages, and a user-friendly PyQt6 interface, allowing users to recognize gestures as well as learn the supported signs.

Client-side desktop architecture Input Layer: 720p Webcam ↓ Computer Vision: OpenCV → MediaPipe Hand Tracking ↓ Feature Extraction: 21 hand landmarks → 42D feature vector ↓ ML Layer: StandardScaler → SVM (RBF Kernel) ↓ Recognition: 48 Urdu Sign Language Classes ↓ Output Layer: PyQt6 GUI → Editable Urdu Text → gTTS → Urdu Speech