Sign Language Has a 500-Year History - Now Your Pixel Can Finally Read It

For decades, voice dictation made smartphones dramatically easier to use for hearing people. Deaf and hard of hearing users mostly got workarounds: typing every response, tapping out replies while holding a phone in one hand, or watching everyone else benefit from speech-to-text while they scrolled past it. Google DeepMind published a detailed breakdown of their new sign-language-to-text model, called SL2T, and it's now live on the Pixel 11 inside Gboard and Live Transcribe. Starting with American Sign Language to English, users can sign into any text field whether they're searching the web, drafting a message, or asking Gemini something.

Sign Language Didn't Start With Technology — It Started With Survival

Before there were apps, phones, or AI models, Deaf people were already building their own languages, and the hearing world mostly ignored them for centuries. The earliest recorded references to sign language go back to ancient Greece. Plato, in his dialogue Cratylus, written around 360 BC, wrote about deaf people using hand and body movements to express meaning. But formal sign language as a structured, teachable system didn't begin to take shape until the 16th century.

A Spanish Benedictine monk named Pedro Ponce de León is widely credited as one of the first educators to formally teach deaf students, working with deaf children of Spanish nobility in the 1550s. He proved something the world had resisted accepting — that deaf people could be educated. In 1620, Juan Pablo Bonet built on that work and published what is considered the first book on deaf education and manual alphabet systems.

The bigger change came in 1760, when Abbé Charles-Michel de l'Épée opened the first free public school for deaf students in Paris. He's often called the "Father of the Deaf." What set his approach apart was that instead of forcing deaf students into spoken language, he learned signs from the Deaf community and built on them — meeting the language where it already existed rather than replacing it.

That school in Paris eventually sent its influence across the Atlantic. In 1817, Thomas Hopkins Gallaudet, an American educator, and Laurent Clerc, a Deaf teacher from the Paris institution, co-founded the American School for the Deaf in Hartford, Connecticut. That's where American Sign Language was born — a blend of the French Sign Language Clerc brought with him and local sign systems already in use in America, including the signed language that had developed naturally on Martha's Vineyard, where hereditary deafness was so common that nearly the entire island — hearing and deaf alike — had been signing for generations.

For a long time, even after these schools existed, sign languages were dismissed as incomplete or primitive — seen as rough gesture systems rather than real languages. That changed in 1960, when William Stokoe, a hearing linguistics professor at Gallaudet University, published Sign Language Structure — the first serious linguistic analysis of ASL. He demonstrated that ASL had its own phonology, morphology, and syntax. It wasn't English on the hands. It was a fully independent language. His work was controversial even within the Deaf community at the time, but it permanently changed how sign languages were studied, protected, and taught worldwide.

Today, there are more than 200 distinct sign languages in use globally, and none of them is universal. British Sign Language and ASL are not mutually intelligible. Sign languages develop organically within Deaf communities, shaped by geography, history, and culture — exactly the way spoken languages do. That history matters here because the technology Google DeepMind just released had to reckon with all of it.

What SL2T Actually Does And Why It's Harder Than It Sounds

Most people who hear "sign language AI" probably picture something like speech recognition, just for hands. That assumption is wrong, and it's the reason earlier attempts like sign language gloves never went anywhere meaningful.

Sign languages are independent, natural languages with their own distinct grammars and lexicons, requiring true machine translation rather than a sequential process of sign-to-word transformations. On top of that, the model has to visually understand simultaneous movements of the hands, arms, torso, head, and face — all at high frame rates. That's a computer vision problem layered on top of a translation problem.

Privacy is handled at the device level. Before anything leaves the phone, MediaPipe Holistic running entirely on-device maps the signer's body into a set of skeletal coordinates. Only those numbers travel to the server — the actual camera footage never does and is discarded the moment the pose data is extracted.

Most earlier sign language translation systems relied on a middle step — breaking signs down into written labels called glosses before converting them to text. It goes straight from the coordinate data to the final translation, which removes a bottleneck that had been quietly limiting how good these systems could get. Glosses cap what the model can express. Skipping them means translation quality can scale directly with more data, rather than hitting an artificial ceiling.

The Scale Behind It

The model didn't get there on a small dataset because Google DeepMind put together more than 100,000 hours of training data spanning 50-plus sign languages — ASL alone accounting for about a quarter of that total. Training jointly across diverse languages and proficiency levels caused the model to learn shared underlying structures — and it outperformed single-language models in internal experiments.

On the FLEURS-ASL benchmark, SL2T achieves a zero-shot score of 70 BLEURT — significantly higher than any previously reported score. Benchmarks alone don't tell the whole story. The team also worked through practical issues that tend to get skipped in academic research: Beyond accuracy scores, the team had to solve problems that only show up when real people use the thing. The model needed to stay quiet when nobody was signing, rather than generating phantom output. Left-handed users — roughly one in ten signers — needed equal accuracy, not an afterthought. And since plenty of people sign while holding their phone with the other hand, the model had to work with one hand doing all the talking.

That last one is easy to overlook in a lab setting but it matters enormously in real use. Someone signing a reply while holding their phone still needs accurate results. The model had to be built for that reality, not just for ideal conditions.

Built With the Deaf Community, Not Just For Them

One detail worth paying attention to: the concept was brought forward by Sam Sepah, a Deaf Googler. A lot of accessibility features get designed entirely by people who never use them — and the gaps always show up eventually. Here, the origin of the project came from within the community it's trying to reach.

Google established the AI Sign Language Advisory Committee (AISLAC), bringing together global Deaf organisations and subject-matter experts to directly influence development priorities. They also co-authored a joint impact report for the release of SL2T 1.0, transparently detailing the technology's capabilities and its current limitations. Publishing limitations openly is something more AI teams should practice. It sets honest expectations rather than leaving users to discover the gaps themselves after the fact.

Where It Works Right Now

SL2T powers sign-to-text dictation in Gboard and Live Transcribe on Pixel 11, starting with ASL, with more devices and additional languages expected to follow. In Live Transcribe, users can sign responses in real conversations instead of having to type back and forth. According to testers, signing in ASL is faster, more natural, and more enjoyable than typing in English. The feature requires no additional subscription.

Where Does This Actually Go From Here?

Sign language AI has been a research topic for a long time — long enough that most announcements in this space have come and gone without reaching real users. What's different about SL2T is that it shipped inside two apps that people already use every day, on a phone they already own, with no extra setup required. But the bigger question isn't whether this works on Pixel 11 with ASL. There are more than 200 sign languages in the world, and the vast majority of them have almost no digital support at all. Reaching them requires both massive data collection and the kind of deep community involvement that can't be rushed or outsourced. The AISLAC model — where Deaf organisations have a direct seat at the table, and co-author public reports on what the technology can and cannot do is the part that's worth watching as much as the model itself. Sign language didn't start as a workaround. It started as a full, living language that hearing society spent centuries either ignoring or trying to replace. The fact that a phone can now translate it in real time, privately, using only skeletal coordinate data from a front-facing camera, is genuinely strange to think about when you trace the line back to Pedro Ponce de León teaching Deaf students in 16th century Spain.