Vehicle-based sign language communication systems and methods are provided herein. An example device can be configured to determine a sign language protocol used by the first user, determine a target language used by a second user, obtain a translation library based on the sign language protocol and the target language, receive spoken word input from a second user through a microphone, convert the spoken word input into sign language output using the translation library, and provide the sign language output using a sign language output device.

A user system for rendering accessibility enhanced content includes processing hardware, a display, and a memory storing software code. The processing hardware executes the software code to receive primary content from a content distributor and determine whether the primary content is accessibility enhanced content including an accessibility track. When the primary content omits the accessibility track, the processing hardware executes the software code to perform a visual analysis, an audio analysis, or both, of the primary content, generate, based on the visual analysis and/or the audio analysis, the accessibility track to include at least one of a sign language performance or one or more video tokens configured to be played back during playback of the primary content, and synchronize the accessibility track to the primary content. The processing hardware also executes the software code to render, using the display, the primary content or the accessibility enhanced content.

A system for distributing sign language enhanced content includes a computing platform having processing hardware and a system memory storing a software code. The processing hardware is configured to execute the software code to receive content including at least one of a sequence of audio frames or a sequence of video frames, perform an analysis of the content, and identify, based on the analysis, a message conveyed by the content. The processing hardware is further configured to execute the software code to generate a sign language translation of the content, the sign language translation including one or more of a gesture, body language, or a facial expression communicating the message conveyed by the content.

The present invention provides a speech recognition system for teaching assistance, which provides caption service for the hearing impaired. This system includes a speaker and a automatic speech recognition (ASR) classroom server, a listener-typist and a computer, a hearing impaired and a live screen, all are in the same classroom. Connect the ASR classroom server, the computer and the live screen with a local area network. The speaker's audio is sent to the ASR classroom server by a microphone for being converted into text caption, and then the text caption is sent to the live screen of the hearing impaired together with the speaker's audio so that the hearing impaired can read the text caption spoken by the speaker. The text caption can be corrected by the listener-typist to make it completely correct.

A system examines components of gestures of a gesture-based language for evaluating proper execution of the gesture, and also examines components of new gestures for evaluating lexical similarity with existing gestures of similar meaning or theme.

Methods and systems for controlling an occupant output system associated with a vehicle are provided. The methods and systems receive vehicle or occupant context data from a source of vehicle context data, generate occupant message data based on the vehicle or occupant context data and determine if an occupant associated with the occupant output system is vision or hearing impaired. When the occupant is determined to be vision or hearing impaired, the methods and systems decide on an output modality to assist the occupant, and generate an output for the occupant on the output device, and in the output modality, based on the occupant message data.

There is provided an information processing apparatus, an information processing method, and a program that make it possible to assist deaf and hard-of-hearing people in viewing a video when the video is being played back. The information processing apparatus includes a controller. The controller generates at least one of an oscillation signal corresponding to sound-effect caption data or an oscillation signal corresponding to vocalization caption data on the basis of a waveform of sound data using a result of analyzing caption information and sound information that are included in a video file, the sound-effect caption data being used to represent a sound effect in the form of text information, the vocalization caption data being used to represent a vocalization of a person in the form of text information, the sound-effect caption data and the vocalization caption data being included in caption data that is included in the caption information, the sound data being included in the sound information.

Systems and methods for doing presenting an avatar that speaks sign language based on sentiment of a speaker is disclosed herein. A translation application running on a device receives a content item comprising a video and an audio, wherein the audio comprises a first plurality of spoken words in a first language. The video comprises a character speaking the first plurality of spoken words in the first language. The translation application translates the first plurality of spoken words of the first language into a first sign of a first sign language. The translation application determines an emotional state expressed by the character based on sentiment analysis. The translation application generates an avatar that speaks the first sign of the first sign language where the avatar exhibits the determined emotional state. The content item and the avatar are presented for display on the device.

A method to present communications is provided. The method may include obtaining, at a device, a request from a user to play back a stored message that includes audio. In response to obtaining the request, the method may include directing the audio of the message to a transcription system from the device. In these and other embodiments, the transcription system may be configured to generate text that is a transcription of the audio in real-time. The method may further include obtaining, at the device, the text from the transcription system and presenting, by the device, the text generated by the transcription system in real-time. In response to obtaining the text from the transcription system, the method may also include presenting, by the device, the audio such that the text as presented is substantially aligned with the audio.

Disclosed is a method of providing a sign language video reflecting an appearance of a conversation partner. The method includes recognizing a speech language sentence from speech information, and recognizing an appearance image and a background image from video information. The method further comprises acquiring multiple pieces of word-joint information corresponding to the speech language sentence from joint information database, sequentially inputting the word-joint information to a deep learning neural network to generate sentence-joint information, generating a motion model on the basis of the sentence-joint information, and generating a sign language video in which the background image and the appearance image are synthesized with the motion model. The method provides a natural communication environment between a sign language user and a speech language user.