‘One of the most important updates’: Sundar Pichai announces new Gboard feature that turns sign language into text

Google CEO Sundar Pichai has announced a new Gboard feature. The feature can convert American Sign Language (ASL) into text, allowing people who use sign language to communicate with smartphones and people who do not sign. Pichai described it as “one of the most important updates” from Google’s Made by Google event.In his X post, Pichai said the update can help people who use ASL communicate more seamlessly with their phones and people who do not sign. He highlighted the sign-to-text capability as a Made by Google update and shared a demonstration of how the technology can convert signing into text for everyday smartphone communication.Sharing a demo video of the new Gboard feature, Pichai wrote: “One of the most important updates coming out of Made by Google today is our new sign-to-text feature in Gboard & Live Transcribe.Built in partnership with the Deaf community, this can help people who sign ASL communicate more seamlessly with their phone or with those who don’t sign, like in this Live Transcribe demo.”

Gboard sign language-to-text feature: How it works

The feature is powered by Google DeepMind’s sign-language-to-text model, called SL2T. It allows users to sign to their phones in situations where they would normally type.With Gboard, users can sign messages, enter search queries, draft documents or interact with Gemini without first manually translating their thoughts into written English.The technology is also being integrated into Live Transcribe, where users can sign their responses during conversations and have them converted into text. Google said testers found signing in ASL faster and more natural than typing in English.

Google SL2T model built with Deaf community

Google said SL2T was developed in partnership with the Deaf community, with Deaf perspectives included in data collection, testing, evaluation and deployment decisions.The model was trained using more than 100,000 hours of data covering more than 50 sign languages. Around a quarter of the training data comes from ASL.Rather than treating sign language as individual hand gestures that directly correspond to English words, SL2T processes movements across the hands, arms, body, head and face.

Google adds privacy measures for sign-to-text

Google said the sign-to-text system does not need to send raw camera footage to its servers for translation. An on-device MediaPipe model identifies pose landmarks, after which it sends geometric coordinates for processing. The original video can then be discarded.Google has also worked to reduce latency and improve the system’s performance for one-handed signing and left-handed users.

Pixel 11 gets Gboard sign language feature first

Google said SL2T-powered sign-to-text functionality will initially be available through Gboard and Live Transcribe on Pixel 11. The company plans to expand the feature to more devices and add support for additional sign languages.The feature is designed to allow sign language users to use signing for everyday smartphone tasks, including messaging, searches, document creation and interactions with Gemini.