Transcribing Sign Language
The DeepMind post introduces a sign-language-to-text model called SL2T, described as a breakthrough in quality and generality.
Sign language transcribing uses AI to convert signed input into text. In Google DeepMind’s account, this work matters because sign languages are the primary languages of Deaf communities, and access to sign-language processing can support communication in ways that spoken-language tools already do for hearing users.
The DeepMind post introduces a sign-language-to-text model called SL2T, described as a breakthrough in quality and generality. It is the first time the company is bringing sign language AI out of the lab and into consumer products, with sign-to-text dictation in Gboard and Live Transcribe on Pixel 11. The initial release starts with American Sign Language to English, and more devices and languages are planned.
The post explains that sign language transcription is different from spoken-language transcription in two important ways. Sign languages are independent, natural languages with their own grammar and vocabulary, so the task requires translation rather than a simple word-for-word mapping. It also requires the model to interpret physical movement, including simultaneous information from the hands, arms, torso, head, and face. Because of that, earlier approaches such as sign language gloves were limited, since sign languages are not just English expressed through hand motions.
SL2T was built with a user-centric and culturally informed approach, combined with large-scale training data. The model was trained on more than 100,000 hours of data covering over 50 sign languages, with about a quarter of the data in ASL. Training across diverse languages, dialects, and proficiency levels helps the model learn shared structures and perform better than single-language models in the company’s experiments.
Privacy is part of the design. Instead of sending raw camera video, the system uses on-device pose tracking to identify landmarks on the signer’s body, then sends only the geometric coordinates for translation. The original video is discarded immediately. The model also translates directly from landmarks to text, rather than using glosses, which the post says do not fully capture non-manual markers and spatial constructions.
DeepMind says SL2T is the most capable sign language translation model to date on key benchmarks such as FLEURS-ASL. It also highlights practical work aimed at making the system usable in real settings, including lowering streaming latency, avoiding hallucinations on non-signing input, supporting left-handed signers, and improving performance for one-handed signing while holding a phone.
The post closes by saying the work was shaped with the Deaf community through data collection, user studies, impact assessment, and an advisory committee. DeepMind presents SL2T as a first step toward broader support for additional sign languages, sign language generation, and fuller accessibility across digital products.
Sources:
Google DeepMind Sign Language Team
