Speech to Text Prosthetic Hearing Aid

Abstract

The invention is prosthetic hearing aid designed to assist and enrich the lives of people who are hearing impaired or have experienced a total loss of hearing by allowing them to hear or understand what is spoken to them. The invention consists of a frame assembly having left and right temples and a front, a lens assembly secured to the fame assembly, a set of microphones attached to the frame assembly, capable of detecting the sound of the spoken word, a television camera system attached to the frame assembly, that is able to track lip movement, a semi transparent viewing screen and a CPU microprocessor and appropriate electronic software coding in order to convert both the audio as well as the lip movement of the spoken word into text and also change the frequency of the spoken word.

Claims

1. I claim speech to text hearing aid prosthetic device and method that consists of a text screen, microprocessor CPU, video camera, microphone, audio speaker, audio to text software, lip recognition and tracking technology, lip reading to text software, and frequency shift technology.

2. The speech to text hearing aid prosthetic device according to claim 1, wherein said text reading screen is from the group consisting electronic viewing screens.

3. The speech to text hearing aid prosthetic device according to claim 1, wherein said text reading screen may super impose the text within the field of view of the user.

4. The speech to text hearing aid prosthetic device according to claim 1, wherein said video camera can process lip movement of a person speaking.

5. The speech to text hearing aid prosthetic device according to claim 1, contains a microphone in order to pickup and process the spoken word.

6. The speech to text hearing aid prosthetic device according to claim 1, contains a microprocessor CPU and built in software in order to convert the spoken word to text.

7. The speech to text hearing aid prosthetic device according to claim 1, contains software to convert lip movement into text.

8. The speech to text hearing aid prosthetic device according to claim 1, contains software to convert lip movement into audible speech.

9. The speech to text hearing aid prosthetic device according to claim 1, contain software coding technology to shift the frequency of the audible speech.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0017] FIG. 1 illustrates one embodiment of the invention and shows the major components including A face and lip recognition and tracking camera, B audio pickup microphone, C Micro-processor CPU, D. Speaker which may contain an amplifier and may also be driven by pitch changing technology.

[0018] FIG. 2 illustrates another angle of the invention again illustrating the major components of the invention including: A face and lip recognition and tracking camera, B audio pickup microphone, C Micro-processor CPU, D. Speaker which may contain an amplifier and may also be driven by pitch changing technology.

[0019] FIG. 3 shows how the invention is used with the optional cross hairs E aimed at the speakers mouth, while the directional microphone is aimed at the lips of the speaker and the speech to text software and the lip reading to text software uses the algorithms to determine the most accurate speech to text. The built in display G projects the text within the field of view of the invention wearer.

[0020] FIG. 4 illustrates how the lip tracking software is able to assist in the aiming towards the speaker F and thus facilitate the voice to text software and the automatic lip reading (ALR) software in order to display in text G, what is spoken.

DESCRIPTION OF THE PROFFERED EMBODIMENT

[0021] In the preferred embodiment, the invention has the physical appearance of regular eye glasses or sun glasses, and includes a video camera FIG. 1-A, directional microphones FIG. 1-B, a viewing screen FIG. 3, FIG. 4 a microprocessor CPU FIG. 1-C and associated electronics, electronic memory and computer software capable of voice to text translation, lip recognition and tracking software with the capability of zooming the video camera in towards to lips FIG. 4 in order to have a better view of the lips, and lip reading to text translation software. The invention has headphone speakers FIG. 1-D and FIG. 2-D coupled with audible amplification and or pitch shifting technology. The invention has fine cross hairs etched or printed on the glasses FIG. 3-E so that the wearer can manually aim the invention towards the lips of the speaker. In the preferred embodiment, the invention is battery powered, and can also be supplied with a standard AC power adapter. In the preferred embodiment, the invention has input and output connectors so that software can be updated and recorded video, sound and text can be downloaded. One possible example could be a micro USB jack. In the preferred embodiment, the invention also has the ability to have removable and interchangeable memory, one possible example being a micro SD memory card.

[0022] In operation, the user wearing the preferred embodiment, faces the speaker and if the speaker is facing the user, tries to keep the speakers lips within the cross hairs FIG. 3-E at the same time, the automatic lip tracking software also zooms into the speakers lips FIG. 4-F and tracks the lips, if either the speaker or the user should move their heads in any off angle. The built in microprocessor CPU and associated electronics and software then converts the audible sound to text and displays this text on the screen FIG. 3-G and FIG. 4-G within the glasses, so that the user can read what the speaker is saying. At the same time, the lip reading software is converting the lip movement to text and this also may be displayed on the screen FIG. 3-G and FIG. 4-G, either next to or above or below the audio text. Further, the built in software and algorithms can compare for any differences between the audio to text versus the lip reading to text and should any differences occur, these differences can be high lighted so that the user can make the determination, what is the most accurate text.

[0023] The invention is a great leap forward by having both audio to text as well as lip reading to text. If the invention only had audio to text, its use would be limited to areas where there is very little ambient sound and where the speaker is in very close proximity to the user. By having both lip reading to text as well as audio to text, the distance of the user to the speaker can be significantly father plus the ambient sounds has far less of an effect on the accuracy of the audio to text. The built in audio to text and lip reading to text comparison software and algorithms is able to present accurate text to the user, even in noisy environments or at a distance from the speaker. On the other hand, if the speaker turns away from the user and the speaker's lips are not visible, the audio to text can still potentially provide text of what the speaker is saying. The incorporation of lip reading software also allows the user to potentially understand what someone is saying at a greater distance than what a normal hearing person can hear. One example of this would be sports fan reading what the coach or players are saying, from a distance.

[0024] People with partial hearing loss, usually losses hearing in specific hearing ranges, rather than evenly across the general 40 HZ to 12,000 HZ hearing range. Normal speech is in the 75 Hz to 300 Hz range, which also happens to be the common area of partial hearing loss, thus mere amplification of the speakers voice will still not significantly allow the user to understand better. Instead, this invention may also incorporate frequency changing, also known as pitch shifting technology so that the frequencies that the person with partial hearing loss has lost most is shifted to a higher or lower frequency region. The result is that the user with partial hearing loss will now be able to hear all of what is spoken, though since the frequency is different than the original spoken word, the sound may seem distorted or squeaky, but clear and understandable, nevertheless.

[0025] It is also possible to incorporate language translating software into the invention so that two people speaking different languages can communicate with one another.

[0026] This invention has wide spread uses, but most importantly, it will greatly help the deaf and the elderly, who have diminished hearing, to understand what people are talking about around them. It is estimated that most elderly people experience some hearing loss. This invention will help them to hear or understand significantly better than without the invention and will significantly help those with partial or complete hearing loss to lead far richer and more productive lives.