Systems and methods for doing presenting an avatar that speaks sign language based on sentiment of a speaker is disclosed herein. A translation application running on a device receives a content item comprising a video and an audio, wherein the audio comprises a first plurality of spoken words in a first language. The video comprises a character speaking the first plurality of spoken words in the first language. The translation application translates the first plurality of spoken words of the first language into a first sign of a first sign language. The translation application determines an emotional state expressed by the character based on sentiment analysis. The translation application generates an avatar that speaks the first sign of the first sign language where the avatar exhibits the determined emotional state. The content item and the avatar are presented for display on the device.

Techniques for addressing impulse sounds in an auditory prosthesis. The auditory prosthesis comprises a sound processor that is configured to convert received sound signals into output signals for use in generating stimulation for delivery to a recipient of the auditory prosthesis. The sound processor comprises an impulse-aware gain system that is configured to generate a time-variable gain for the application to the audio signal. The time-variable gain applied to the audio signal is dependent on both a level of the audio signal and the presence or absence of impulse sounds in the audio signal.

A novel system and multi-device invention that provide a means to communicate in real-time (conversationally) between two or more individuals, regardless of each individual's preferred or limited mode of transmission or receipt (by gesture, by voicein Mandarin to German to Farsi, by text in any major language, and via machine learning, eventually by dialect).

Systems and methods for conversational communication between two individuals using multiple language modes (e.g. visual language and verbal language) through the use of a worn device (for hands-free language input capability) are provided. Information may be stored in memory regarding user preferences, as well as various language databasesvisual to verbal to textural- or the system can determine and adapt to user (primary and second) preferences and modes based on direct input, and adapt. Core processing for worn device can be performed 1) off-device via cloud processing through wireless transmission, 2) on-board, or a 3) mix of both, depending on the embodiment, and location of use, for example if the user is out of range from access to a high-speed wireless network and needs to rely more on on-board processing, or to maintain conversational speed dual/real-time translation and conversion.

A communication augmentation system includes a camera, a transceiver and a computer. The camera is operation to record an image of users. The transceiver is operational to receive inbound messages from wireless devices. The inbound messages include an input content. The computer is operational to store registrations of the users, determine user locations where the users are located in response to facial profiles relative to the image, determine device locations where the wireless devices are located based on the inbound messages, associate the wireless devices with the users based on the user locations and the device locations, determine destinations of the inbound messages based on a comparison of input content to the identifiers, and transfer the input content and the destinations to the transceiver. The transceiver is further operational to transmit the input content in a plurality of outbound messages to the wireless devices based on the destinations.

Embodiments relate to performing haptic communication using frequency decomposition of speech where dominant frequencies of a speech is detected at a speech source and then sent to a signal generator to actuate actuators mapped to the dominant frequencies. The digitized version of the speech is segmented into a plurality of frames and then a predetermined number of dominant frequencies are detected from each frame. The dominant frequencies of frequencies are sent over to the signal generator so that the actuators corresponding to the dominant frequencies are activated for a time period corresponding to the frame.

A system and method to insert visual subtitles in videos is described. The method comprises segmenting an input video signal to extract the speech segments and music segments. Next, a speaker representation is associated for each speech segment corresponding to a speaker visible in the frame. Further, speech segments are analyzed to compute the phones and the duration of each phone. The phones are mapped to a corresponding viseme and a viseme based language model is created with a corresponding score. Most relevant viseme is selected for the speech segments by computing a total viseme score. Further, a speaker representation sequence is created such that phones and emotions in the speech segments are represented as reconstructed lip movements and eyebrow movements. The speaker representation sequence is then integrated with the music segments and super imposed on the input video signal to create subtitles.

The translator detects audio with the use of at least one microphone. The system analyzes the audio input to determine the spoken words. The translator determines the phonemes of the spoken words and outputs each phoneme to the user. The translator maps each phoneme to a haptic code that represents the detected phoneme. After determining the phonemes to output to the user, the system actuates multiple actuators to communicate the code to the user. The actuators contact the user to communicate the code associated with each phoneme of the audio input.

A haptic communication device includes one or more cutaneous actuators to generate haptic vibrations corresponding to actuator signals received by the one or more cutaneous actuators. A dampening member, proximate to a body of a user wearing the haptic communication device, focuses the haptic vibrations at one or more distinct locations on the body. The dampening member has one or more first openings, wherein the one or more cutaneous actuators transmit the haptic vibrations to the one or more distinct locations through the one or more first openings. A spacing member contacts the dampening member and is separated from the body by the dampening member. The spacing member has one or more second openings dimensioned to receive and secure the one or more cutaneous actuators.

Some embodiments provide display eyewear with auditory enhancement. In general, one aspect disclosed features a head-wearable apparatus comprising: a microphone; a display panel visible to the wearer; a gaze tracker configured to determine a direction of a gaze of a wearer of the head-wearable apparatus; and a controller configured to: extract speech from sound collected by the microphone from the determined direction, and present the extracted speech on the display panel.

Methods and devices to deliver a tactile speech analog to a person's skin providing a silent, invisible, hands-free, eyes-free, and ears-free way to receive and directly comprehend electronic communications. Embodiments include an alternative to hearing aids that will enable people with hearing loss to better understand speech. A device, worn like watch or bracelet, supplements a person's remaining hearing to help identify and disambiguate those sounds he or she can not hear properly. Embodiments for hearing aids and hearing prosthetics are also described.